cgfarmer4 / theconductor

Toolset for making musical applications with Unity, Max and Ableton.

The Conductor

Toolset for making musical applications with Unity, Max and Ableton.

Goals

• Create examples for interacting with music and spatial environments.
• Explore patterns and initiate discussion for creating new kinds of musical interactions.
• Provide environments and tools for others to become interested in music creation.

Components

OSC Address Router, Stepper, Slider, Dial, Position, Waves, Pads, Keys + more.

Dependencies

Ableton
Max
CNMAT-odot
Unity
SteamVR OSC package

Optional: Envelop for Live

Setup

You should already have Ableton and Max installed. All other dependencies are free and open source.

1. Download o.dot from Github and place it in your Max packages directory. Typically found in Documents/Max 7/Packages. To check, go to Options -> File Preferences in Max.

2. Download and install Unity.

3. Clone or download the zip of this project and open the folder TheConductor_Unity in the Unity Editor.

4. Run the SteamVR scene from the scenes folder. If you have errors, they will be shown in the Unity console.

5. Open Ableton and add the folder TheConductor_Max from this project to your Ableton places in the sidebar.

6. OSC defaults to running on port 8000 in Unity. You can change the ports and outgoing IP address on the [OSC] component in Unity and in each M4L patch.

7. Select the prefab in the Unity scene hierarchy that you would like to use and activate it.

8. Follow the instructions below for setting up each specific component.

Components

Envelop

Unity Attach: empty GameObject
Envelop for Live is an open source audio production framework for spatial audio composition and performance. This component contains a spatial audio representation of the decoder for the their speaker installations. You can send positional values and audio levels from Max to Unity for 3d representations.

OSC Control

Unity Attach: empty GameObject
Unity script for interacting with any OSC event. When a packet is received with the corresponding name, the callback event is called with a List<object> containing the OSC data.

Midi Receive Events

Unity Attach: GameObject
Receive named midi events in Ableton from Unity collisions. The Midi Send Events script sends when the GameObject OnCollisionEnter event is triggered.

Midi Send Events

Unity Attach: GameObject or empty GameObject
Send named midi events to Unity from Ableton. Attach any public function to the Note on / off events in the inspector.

VRControllerPosition

Unity Attach: VRGameController
Controller positions to XYZ live.dials. Map the controller positions to any value.

VRPosition

Unity Attach: GameObject with Transform
Attach to Unity GameObject and send its transform to XYZ live.dials. Useful for panning Envelop for Live sources.

VRDial

Unity Attach: GameObject with UI Canvas child
Radial UI controller to live.dial. This component does not take a name but could be easily modified to do so based on other patterns in these tools.

VRDrumPads

Unity Attach: GameObject(s)
Physics reactive cubes to Ableton drum rack.

VRMultiSlider

Unity Attach: GameObject(s)
Interactive cube height transforms to multislider. Grab or select cubes with laser pointer. Can select multiple. Scale by amount is mutliplied by the height value. Example of scaling values for other object interpretations.

VRMidiKeyboard

Unity Attach: GameObject
Virtual piano model sends midi duration and pitch. Represented by kslider.

VRStepper

Unity Attach: GameObject
Cube represenation of a stepper to live.step. Laser pointer controls on / off and touchpad controls velocity.

Deprecated

Why deprecate? VRTK discontinued development and these packages were based heavily on controls available within that sdk. They could easily be revitalized with a little finesse.

VR2DWave

Unity Attach: GameObject
Draw on 2D material to waveform~. Deprecation Reason: VRTK made it simple to snap the whiteboard pen to the controller.

VR3DWave

Unity Attach: empty GameObject
New instrument example with a wave picker, controller position and velocity to custom M4L device. Deprecation Reason: The dial menu was a VRTK feature. Could easily create new menu and new materials.

VRSlider

Unity Attach: GameObject with UI Canvas child
UI slider to to live.slider. Deprecation Reason: VRTK made it simple to select Unity UI Canvas in VR. Here are some links to similar implementations. https://github.com/thestonefox/VRTK/commit/5fa29d7f159b00d9e719455f16d1d4f0bfc6b6b1 https://github.com/Sergey-Shamov/Unity-VR-InputModule/blob/master/VRControllerInputModule.cs https://ritchielozada.com/2016/01/01/part-8-creating-a-gaze-based-input-module-for-unity/ https://forum.unity.com/threads/using-the-unity-event-system-for-vr.472259/

Credits

Many thanks to the work of others whose shoulders I stand on to build musical interfaces.

• CNMAT for providing o.dot.
• The Envelop team for all the inspiration and help.
• Icons for VR3DWave from the Noun project.
• The band Caspian for the project name. A tribute to their first album.

theconductor's Issues

Ableton is not sending any OSC message

Thanks for the quick reply on Youtube.
I'm pretty sure I followed your instructions very well,
but my Ableton is not sending any OSC message out.
How do I know this is that I downloaded OSC data monitoring program,
and tested both Unity and Ableton.
Unity sends OSC message well, it's captured on the monitoring program.
But no matter what parameter I give to Ableton, the message is not captured.
Like giving MidiEvents, MidiReceiveEvents different port, ip, pitch, velocity, name, or different Midi Output Device selection.
Also did with VRMidiKeyboard, VRDrumPads.

I did download CNMAT-odot-master and put it in the Max 8 Packages folder.
I've tested with other Ableton OSC sender, witch is Livegrabber, and it sends out OSC well.
I've been struggling with this for almost a day now :(

Is there any suggestions?

Possible use with a custom OpenGL Render using OpenTK

Background

Hiya, sadly I can't send private messages so I thought I'd ask for your thoughts here.

I am a aspiring live musician who also does graphical programming, I've been creating a live set using ableton and my push and I wish to automatically generate visuals with automations in my clips. It would be a bit finicky but creating a live set is already quite finicky and I think that auto generated visuals would be a good way to jump out of the crowd of electronic live acts.

My knowledge of unity is poor at best, and I already got quite a few things working with my own graphics renderer so I thought it'd be best to use my own, also performance is quite important due to this all running on a laptop.

Question

I was looking if you had some over view specifically for the automations part, and I also only really need automations from ableton -> C#. So basically only the getting of automation values.

Thanks in advance!

Updating suppport for SteamVR v2

Working on this!

Not receiving - Ableton

Hi was just wondering if you're using 32 or 64 bit. I can't seem to get ableton to pick up the input, despite oschelper.cs saying values are being sent.. Any ideas or suggestions?