sjcobb / ai-duet-3d

3D music animation + machine learning (in development)

AI Duet 3D

Overview

This project is in active development and is not live yet. Until release v1.0.0, when the full UI implementation is planned (currently a MIDI controller must be used to jam with the machine learning model), all progress can be tracked on the AI Duet - JavaScript Music Animation YouTube channel.

Release Notes

0.5

synth changes, bugfixes / refactoring, ui updates: https://www.youtube.com/watch?v=KGOxn0_Hj_k

0.4

neural network melody right hand + human left hand: https://www.youtube.com/watch?v=tTNFgoKaEnA

0.3

drum machine spinner: https://www.youtube.com/watch?v=2uCANP7zd4s

0.2

melody autocompletion: https://www.youtube.com/watch?v=uT5s3Svk7Fo

0.1

first 3D prototype: https://www.youtube.com/watch?v=8QR59RP-Z_8

Project Inspiration

• Original A.I. Duet: https://github.com/googlecreativelab/aiexperiments-ai-duet
• ANIMUSIC videos - Pipe Dream
• All of the amazing work by teropa:
  • Neural Arpeggiator: https://codepen.io/teropa/pen/ddqEwj
  • Neural Drum Machine: https://codepen.io/teropa/pen/JLjXGK
  • Neural Melody Autocompletion: https://codepen.io/teropa/pen/gvwwZL
• Performance RNN: https://magenta.tensorflow.org/performance-rnn
• Magenta demos: https://magenta.tensorflow.org/demos