maceq687 / 5d-instrument

Gesture controlled collaborative web browser instrument with WebGL visuals

 

5D-instrument

Gesture controlled instrument for collaborative music creation. Instrument is an online application that runs in browser. It is capable of tracking user's hand motion using a webcam.
It was made using Unity (WebGL, visuals, multiplayer(PUN)) and JavaScript (computer vision and Web Audio API)

This instrument was developed as a tool that could be used to counteract the negative effects of physical distancing caused by global pandemic. Since music is a universal language, it creates an opportunity for a seamless collaboration between people with various cultural backgrounds. Moreover, the use of body movements can enhance social engagement and help develop communication between people. The idea of this project was to create an easily accessible interactive multi-user application. In this application people can play music together using hand movements - similarly like it is done with physical instruments. The project’s main goal is to connect people locally and globally via a collaborative interactive musical instrument.
5D instrument is available online at: https://maceq687.github.io/5D-instrument/