merayen / elastic

Elastic - Synthesizer DAW

• 👋 Hi, I’m @merayen
• 👀 I’m interested in things
• 🌱 I’m currently learning stuff
• 💞️ I’m looking to collaborate on cool projects
• 📫 How to reach me.

merayen/merayen is a ✨ special ✨ repository because its README.md (this file) appears on merayen's GitHub profile. He can click the Preview link to take a look at his changes.

TODO

Write this file :thinking_face:

elastic's Issues

Piano roll notes to the backend and storing in project

The midi made in the Piano roll UI should be stored in the project and used by the DSP backend.

• Store notes and the midi CC-data in the project, on the midi_1-node
• Send the midi to the DSP backend, with the MidiData-class
• Only send initial and changes to the midi to the backend, as the backend should have a complete instance of the MidiData at all times

Unplugging MIDI-device while scanning

javax.sound.midi.MidiUnavailableException: そのようなデバイスはありません
	at com.sun.media.sound.MidiInDevice.nOpen(Native Method)
	at com.sun.media.sound.MidiInDevice.implOpen(MidiInDevice.java:59)
	at com.sun.media.sound.AbstractMidiDevice.doOpen(AbstractMidiDevice.java:154)
	at com.sun.media.sound.AbstractMidiDevice.open(AbstractMidiDevice.java:118)
	at net.merayen.elastic.backend.interfacing.platforms.oracle_java.OracleMidiInputDevice.ensureRunning(OracleMidiInputDevice.java:103)
	at net.merayen.elastic.backend.interfacing.platforms.oracle_java.OracleMidiInputDevice.onRead(OracleMidiInputDevice.java:61)
	at net.merayen.elastic.backend.interfacing.devicetypes.MidiInputDevice.read(MidiInputDevice.java:11)
	at net.merayen.elastic.backend.logicnodes.list.midi_in_1.LogicNode.onPrepareFrame(LogicNode.java:46)
	at net.merayen.elastic.backend.nodes.Supervisor.doProcessFrame(Supervisor.java:262)
	at net.merayen.elastic.backend.nodes.Supervisor.executeMessageFromUI(Supervisor.java:170)
	at net.merayen.elastic.backend.nodes.Supervisor.handleMessageFromUI(Supervisor.java:72)
	at net.merayen.elastic.backend.context.MessageHandler.executeMessagesToBackend(MessageHandler.kt:68)
	at net.merayen.elastic.backend.context.BackendContext.update(BackendContext.java:60)
	at net.merayen.elastic.system.ElasticSystem.update(ElasticSystem.java:38)
	at net.merayen.elastic.system.Test.waitFor(Test.kt:71)
	at net.merayen.elastic.system.Test.<init>(Test.kt:54)
	at net.merayen.elastic.system.Test.<init>(Test.kt:10)
	at net.merayen.elastic.system.Test$Companion.test(Test.kt:82)
	at net.merayen.elastic.Test$Companion.main(Test.kt:32)
	at net.merayen.elastic.Test.main(Test.kt)
Exception in thread "main" java.lang.RuntimeException
	at net.merayen.elastic.backend.interfacing.platforms.oracle_java.OracleMidiInputDevice.ensureRunning(OracleMidiInputDevice.java:106)
	at net.merayen.elastic.backend.interfacing.platforms.oracle_java.OracleMidiInputDevice.onRead(OracleMidiInputDevice.java:61)
	at net.merayen.elastic.backend.interfacing.devicetypes.MidiInputDevice.read(MidiInputDevice.java:11)
	at net.merayen.elastic.backend.logicnodes.list.midi_in_1.LogicNode.onPrepareFrame(LogicNode.java:46)
	at net.merayen.elastic.backend.nodes.Supervisor.doProcessFrame(Supervisor.java:262)
	at net.merayen.elastic.backend.nodes.Supervisor.executeMessageFromUI(Supervisor.java:170)
	at net.merayen.elastic.backend.nodes.Supervisor.handleMessageFromUI(Supervisor.java:72)
	at net.merayen.elastic.backend.context.MessageHandler.executeMessagesToBackend(MessageHandler.kt:68)
	at net.merayen.elastic.backend.context.BackendContext.update(BackendContext.java:60)
	at net.merayen.elastic.system.ElasticSystem.update(ElasticSystem.java:38)
	at net.merayen.elastic.system.Test.waitFor(Test.kt:71)
	at net.merayen.elastic.system.Test.<init>(Test.kt:54)
	at net.merayen.elastic.system.Test.<init>(Test.kt:10)
	at net.merayen.elastic.system.Test$Companion.test(Test.kt:82)
	at net.merayen.elastic.Test$Companion.main(Test.kt:32)
	at net.merayen.elastic.Test.main(Test.kt)

New add dialog

New add-dialog for when adding nodes. It should have textfield on top so user can search by node name, and a list below the user can navigate by arrow keys.

Dialog should pop up at the center and at a node's port, to directly connect it after adding.

Oscilloscope

Would be really nice to actually see the signals that gets created. Also for debugging purposes.

• Implement signal triggering
• Implement frequency wheel
• Implement amplitude (with auto-adjust as default setting?)

Play cursor

Implement cursor that can be moved back and forth.

• Cursor in midi view
• Cursor in arrangementview
• Store its position in topmost group_1 node
• Make cursor movable in midi view and arrangementview

Implement EQ-node's UI

Make a marvellous UI for the EQ-node

Undo feature

Really need this.

• Figure out how it should work. Should it send messages to the backend that do the opposite of what was done? Complete reload of the project for every undo? (ugh)

Messaging system overhaul

• Go over messaging system
• Implement a communicator that can be hooked to an ElasticSystem-instance
• Create unit tests for ElasticSystem and backend, testing messaging:
  • Creating projects
  • Adding nodes
  • Connecting nodes
  • Configuring nodes
• Implement error messaging (meaning backend can respond with e.g "node not found")

EasyMotion

Implement EasyMotion, that will handle all events from the keyboard.

• Implement the EasyMotion controller on Window
• Implement controllers that gets put on every UIObject that should be controlled
• Make it possible to override which keys the Control captures, which allows for text input inside Elastic
• Allow a Control to override getChildren to decide which controls are available, based on custom context (like which view is currently active)
• Create a dynamic text input field based on Control
• Track active stack of Controls based on last active control
  • Think about: Do we really need CTRL+t to enable EasyMotion? Probably not!
    Like when user uses the mouse to navigate, and then wants to continue using EasyMotion

Examples on keystrokes to add a node while in NodeView:

• a. Opens Add Node popup
• siggen. Searches for signalgenerator
• ENTER. Adds that node

Create midi-node and connect to signalgenerator-node:

• s. Search box opens
• sig. Signalgenerator-node
• ENTER. Shows node
• p. Goes into port-editing mode
• i. Chooses input-port of signalgenerator-node
• a. Opens add-node dialog
• midi. Searches for midi-node
• ENTER. Shows node

Split view horizontally, and move focus down to new pane:

• CTRL+w. Go into window-command
• s. Splits the current view horizontally and clones it
• CTRL+w
• DOWN. Go down to created view

Webservice

Make Elastic run as a daemon service, serving on a single port (perhaps HTTP?), where other services can connect to it and configure it.

• Make HTTP(?)-service
  • Implement a call that can fetch streams from Elastic, so that other apps can stream audio from it
    • Perhaps by reconfiguring output_1-node to write to a virtual device, which is the stream
• Figure out a way to serialize all the messages in Elastic
• Create documentation on all the message-classes themselves, so that the API can document itself?

Automatic cross-correlation detection

Create a node that takes two audio-inputs. It should show (and output?) the time difference of second input compared to first input.

This can be used to measure delay internally, but also externally to calibrate e.g speaker set up etc.

Importing audio

Using FFmpeg!
Should be possible to drop all files that FFmpeg can read into Elastic, as long as FFmpeg reports it contains audio.
Just take the first audio stream found in the file, not giving user any options. For now.

Better compressor

The compressor_1-node may suck, making bad sound when attack/release has low values.
Should it not adjust the amplitude linearly, but rather by the distance from the overload peak?

Exporting audio

Be able to export from an output_1-node, via ffmpeg. Only .wav-format support for example.

Crash when changing BPM in a poly_1 node

Exception in thread "main" net.merayen.elastic.util.ClassInstanceMerger$MustBeTheSameClass: sourceInstance is 'net.merayen.elastic.backend.logicnodes.list.group_1.Data' while destinationClass is 'net.merayen.elastic.backend.logicnodes.list.poly_1.Data'
	at net.merayen.elastic.util.ClassInstanceMerger$Companion.merge(ClassInstanceMerger.kt:20)
	at net.merayen.elastic.util.ClassInstanceMerger$Companion.merge$default(ClassInstanceMerger.kt:18)
	at net.merayen.elastic.util.NetListMessages.apply(NetListMessages.kt:112)
	at net.merayen.elastic.backend.nodes.Supervisor.executeMessageFromUI(Supervisor.java:86)
	at net.merayen.elastic.backend.nodes.Supervisor.handleMessageFromUI(Supervisor.java:73)
	at net.merayen.elastic.backend.nodes.Supervisor.handleResponseFromProcessor(Supervisor.java:218)
	at net.merayen.elastic.backend.context.MessageHandler.executeMessagesFromProcessor(MessageHandler.kt:76)
	at net.merayen.elastic.backend.context.BackendContext.update(BackendContext.java:61)
	at net.merayen.elastic.system.ElasticSystem.update(ElasticSystem.kt:33)
	at net.merayen.elastic.system.Test.waitFor(Test.kt:53)
	at net.merayen.elastic.system.Test.<init>(Test.kt:36)
	at net.merayen.elastic.system.Test.<init>(Test.kt:9)
	at net.merayen.elastic.system.Test$Companion.test(Test.kt:64)
	at net.merayen.elastic.Test$Companion.main(Test.kt:20)
	at net.merayen.elastic.Test.main(Test.kt)

Frequency spectrum analyzator node

We need this to verify quality of the filters and generator we make in the synth!

Crash when splitting horizontally

When dragging from lowest area of screen, with just 1 view, it tries to resize the only single panel instead of creating a new one. Fix.
Layout.java looks bad anyway, waste some time to rewrite it in Kotlin?

Exception in thread "AWT-EventQueue-0" java.lang.RuntimeException: Should not happen
	at net.merayen.elastic.ui.objects.top.viewport.Layout.getRuler(Layout.java:333)
	at net.merayen.elastic.ui.objects.top.viewport.Layout.resizeHeight(Layout.java:199)
	at net.merayen.elastic.ui.objects.top.viewport.ViewportContainer$1.onMouseDrag(ViewportContainer.java:113)
	at net.merayen.elastic.ui.util.MouseHandler.handle(MouseHandler.java:116)
	at net.merayen.elastic.ui.objects.top.viewport.ViewportContainer.onEvent(ViewportContainer.java:173)
	at net.merayen.elastic.ui.Supervisor.internalUpdate(Supervisor.kt:55)
	at net.merayen.elastic.ui.Supervisor.internalUpdate(Supervisor.kt:61)
	at net.merayen.elastic.ui.Supervisor.internalUpdate(Supervisor.kt:61)
	at net.merayen.elastic.ui.Supervisor.draw(Supervisor.kt:15)
	at net.merayen.elastic.system.UIBridge$1.onDraw(UIBridge.kt:32)
	at net.merayen.elastic.ui.SurfaceHandler$createSurface$surface$1.onDraw(SurfaceHandler.kt:45)
	at net.merayen.elastic.ui.surface.Swing$LolPanel.paintComponent(Swing.kt:142)

EasyMotion overlay

We need to give user feedback from EasyMotion.

• Create an utility UIObject that takes 1 uiobject which it "restores" translation and scaling from.
  This will allow us to create what seems like overlaying objects to actually lay deep down the tree
• Create floating markers for the current selected EasyMotion object
• Create hints overlay, toggled by ?-character

Reverb

We need reverb. It is important. Implement.

Convolution? Need to learn and implement FFT first.

Low level nodes

This idea is to create something similar to Reaktor Core, which will allow low level programming using nodes.

I think this might be a good idea and rather build the current nodes, like signalgenerator, on smaller set of components.

From an LLVM perspective, we plan to create dynamic programs when nodes are compiled anyway, so allowing for such small component programming might be a good idea.

What this would require:

• Redo whole node system and processing (LogicNodes and DSP)
• Integrate LLVM #26
• Build a system to build user interfaces (nodes too?)

Choose input and output-devices in UI and their settings

Make a view that lists all the available devices, and their mapping to nodes like midi_1 (midi_in_1 if it still exists)

LLVM-backend

• Output C-code (?)
  • C-code?
    • Probably easy to write
    • Probably well optimized
  • LLVM IR?
    • I know nothing about it
    • Too much work to write nodes due to being low-level?
    • May compile faster than C due to less overhead
    • We get to learn about LLVM, deeply!
    • Can we still run on GPUs
• Have LLVM JIT create binary
  • How is it to compile single group-nodes, not needing to JIT everything again upon node changes?
• Have Java run the binary
• Re-implement the nodes in this backend
  • Fixed buffer widths? Loops could have fixed iteration count, easily SIMD-unrolling too, if that is needed
    • How do we solve "tight" feedbacks?
      • Automatic detection on how big each group-node buffer should be?
  • How should we work with delay compensation? Like if any node has an offset at e.g +100 samples, meaning it requests 100 samples in the future
    • Implement offset on each node, telling how many samples it needs in the future vs its outputs
      • Need to calculate all these upon compilation, I guess
  • Output 1 mega method for each group node?
• Test threading?
• Device drivers, should they be built by nodes, only?
  • Reinventing programming language? Maybe not... Hmm

Create a nice logo when Elastic starts up

...and show the startup progress, like "Waiting for Java to initialize sound..."

Arpeggiator

Make it. Take inspiration from Ableton.

Merge midi_in_1-node into midi_1-node

Get rid of midi_in_1-node, as midi_1 should also take input from midi devices.

Piano roll graph overlay

Midi CC, sustain, pitch etc, shown as an overlay over the piano notes (not below as many other DAWs)

• GUI to be enabled/disabled by a button on right side?
• GUI should be drawn with a transparent background, e.g 70% opacity
• Implement curves from the beginning (Cubase got this in version 9.5), even for binary modes (should get quantified later) (one issue with this is that we need to smooth input from MIDI-devices?)

Piano roll notes UI

Make it possible to add notes to the piano roll.

• Choose tool by right-clicking the piano roll background?
• Make the background nicer, not with lines, but with fillRects, with smooth contrast colors
• Make the piano roll scrollable
• Make the piano roll zoomable with independent axis, without using UI's scale translations
• Able to select notes
• Able to move notes
• Able to resize notes on left and right side
• Able to delete notes

Signal generator curve quality

Make the audio generated by the signalgenerator_1-node better. The algorithm that samples the curve may not be that great at the curve joints, introducing high frequency noise.