Scan Band by Artiphon Live

• Cost of development: N/A
• Communities: Lens Studio
• Categories: Lenses & Masks

Screenshots & Video

Description

The team at Artiphon asked me to collaborate with them on an ML-powered augmented reality music experience. I am super excited to share the result: Scan Band, which runs in Snapchat, was built with Lens Studio, and allows you to scan over 500 real world objects and turn them into virtual instruments. 
Each instrument behaves differently, so you can you can play music either by tapping them on the screen, or triggering them using your finger or your body.
We combined machine learning in a few different ways to make this project possible, as well as building with Snapchat's Scan feature, the new AudioML tools in Lens Studio 4.0, and using full-body tracking in augmented reality.

How to Use it

Open the lens in Snapchat, and find real-world objects around you to Scan. A sticker suggestion will show if you point the phone at a banana, a car, or a plant or flower - for example. Once you confirm by tapping the sticker, you can play that sticker like an instrument, by tapping it or using your finger or hands/feet. Tap + hold a sticker to change the instrument.

Target Images

Scan these images to discover AR content