Stringer: Kinect instrument

This weekend, Tyler Williams, Aidan Feldman, and I formed a team for New York’s Music Hackday. Our goal was to build a musical instrument with the Kinect’s 3d camera. I bought a Kinect and we opened the box for the first time Saturday morning.

Project summary

Stringer is a virtual string instrument played with a Kinect 3d camera. A person can draw his/her own instrument in the air using gestures. Multiple people can play at once, through separate hand-tracking. Strings can be set to morph as people interact, with length determining pitch, creating a constantly changing melody.

More details

We started purely in Processing, using Daniel Shiffman’s library. We had fun working around some of its limitations, with simple, blob-like threshhold tracking. We built a prototype that might work as a sidewalk installation. Pedestrians would trigger a shifting set of strings, each playing a unique melody.

Then Tyler got the more advanced C++ libraries talking to Processing over OSC, and that was things got really interesting. We were able to track hands very accurately, so even multiple sets of people could play at once.

Later on, Aidan added drawing gestures through a simple but elegant hold-to-draw motion. The user could then draw new lines in the air.

What’s next

We tackled this as a proof of concept, and definitely have many more ideas. I also have many more videos and will update this blog post soon. A few more pictures in this Flickr set.

4 comments on ‘Stringer: Kinect instrument’

  1. [...] competitors, and was written up by Create Digital Music and Tech Crunch. More making-of information at this blog post. var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : [...]

  2. [...] not professional/significant musical advancement. Other independent projects with the Kinect, like “Stringer,” from New York’s Music Hack Day have been successfully showcased, but demonstrate only a [...]

Leave a Reply