corbanbrook / spectrotune

Performs pitch detection on a polyphonic audio source and outputs to MIDI

Spectrotune

Spectrotune is a Processing application which scans a polyphonic audio source (in wav, mp3, etc formats), performs pitch detection and outputs to MIDI.

Spectrotune offers adjustable options to help improve pitch detection, including:

• Pitch Class Profiling (PCP)

• FFT Bin Distance Weighting

• FFT Windowing - rectangular, hamming, hann, triangular, cosine, and blackman windows.

• FFT Linear Equalization - attenuate low freqencies and amplify high freqencies.

• Harmonic Filter - filters peak harmonics.

• Noise Smoothing - rectangle, triangle, and adjacent average smoothers.

• Parabolic Peak Interpolation.

• Adjustable Peak Threshold.

• Octave toggles - narrow the spectrum to the octaves you are interested in recording.

• MIDI octave channel segmenting - route each octave to its own MIDI channel.

Install and Basic Usage (macOS Standalone Application):

Spectrotune is now available in a standalone application for macOS which you can download from Downloads/Releases

• Open DMG file.

• Put audio files(wav, mp3, etc) in the music folder. (Note: music folder must be relative to the spectrotune.app path)

• Launch application.

• Click 'Files' tab and select the audio file you wish to open.

• Record with MIDI sequencer.

Currently only exported as an application for macOS as I dont have any other platforms to test on. If you want a standalone application for Windows or Linux please send me a request or install processing and export the application yourself.

If you wish to play with the source code you can follow the instructions below to install processing and the required libraries.

Processing Install (all platforms):

• Install processing

• Install rwmidi

• Install ControlP5

• Clone Spectrotune and place inside your Processing Sketchbook directory

  cd ~/Documents/Processing (this is where I keep mine)
  git clone https://github.com/corbanbrook/spectrotune.git

• Make sure your operating systems MIDI interface is configured properly.

• Put audio files (wav, mp3, etc) you wish to open in the 'music' folder within the route of the sketch.

• Open MIDI enabled sequencer software, ie Reason, Cubase, Fruityloops, etc.

• Open processing.

• Run.

If you are using a MIDI enabled sequencer software you should be able to record the input and pass on to any soft synth, or hardware synth.

Keyboard Commands:

Command	Result
0-7	Octave filter: toggle octaves 0 to 7 On/Off
p	Pitch Class Profile (PCP) toggle On/Off
e	Linear EQ toggle On/Off
h	Harmonic Filter toggle On/Off
m	Mute audio toggle On/Off
n	MIDI output toggle On/Off
SPACE	Pause/Play toggle
RIGHT/LEFT	Peak Threshold Increase/decrease
ESC	Quit

spectrotune's Issues

Organize code before 0.6.0 release

code needs to be organized a lot more before releasing 0.6.0.

MP3 support broken

issue with Minim wont load mp3s

Smoothing currently disabled

My smoothing patch has not been implemented into minim.. still waiting.

Implement Harmonic Product Spectrum for EnhancedPCP

Change AudioSample to an AudioPlayer

AudioBuffer loads the entire file into memory, this is encountering problems with mp3 files.
AudioPlayer will allow for streaming files as to not use as much memory and still be able to access the audio buffers through an AudioListener.

Minim has support to Listeners which listen to the AudioPlayer and build sample buffers which operations can then be performed upon.

Playable options like, play(), pause(), cue() should be used to seek.

Controller options such as mute() and unMute() should be used to toggle the output of sound.

NOTE: mute does not infact mute zero out the volume level of the samples buffer it mearly stops the audio from being sent to the audio device so spectrotune can quietly process sample buffers.

Scale Lock

pre compute a scale before pitch detection and add a scale lock option to snap detected pitches to scale.

Too many notes at one time causes a crash

FFT windowing currently disabled

Minim only supports a Hamming window and my patch still hasnt been implemented.

MIDI octave segmenting

Allow users to specify which MIDI channel an octaves notes are sent to.

Seeking does not close opened MIDI notes.

This causes opened midi notes to play forever when seeking.

Equalization currently disabled

Minim doesnt support FFT equalization as far as I know. Will need to patch custom minim to include it.

Reduce memory usage