nogtini / bach-bot

Uses time series RNN in with Keras to generate Baroque style music. Also contains useful for processing midi files for machine learning purposes.

Baroque Music Generator¶

Overview¶

This aim of this project is to use a time series neural network to generate music trained on 15 Chorales by Bach. A lot of this involves parsing information from midi files into the notes for the melody. There is not a lot of info for processing midi files...at all. So to start off, I'm only predicting the notes from the melody (as opposed to bass, rythm, or whatever the different layers are called) and without time i.e. every note is assumed to last for the same time.

Processing Midi Files¶

Some functions I wrote for parsing the notes from the melody in the track into something the network can process and then back into a midi message.

In [1]:

import numpy as np
import mido

def decodeMidi(midifile,num_layers=2):
    song = []
    for i, track in enumerate(midifile.tracks):
        song.append([])
        for msg in track:
            message = str(msg).split()
            if ' not in message and 'control_change' not in message and 'program_change' not in message:
                channel = int(str(msg).split()[1].split("=")[-1])
                note = int(str(msg).split()[2].split("=")[-1])
                velocity = int(str(msg).split()[3].split("=")[-1])
                time = int(str(msg).split()[4].split('=')[-1])
                song[i].append([channel, note, velocity, time])

    song = [x for x in song if x]
    song = [song[:num_layers]]
    return np.array(song)

def encodeMidi(song):
    file = mido.MidiFile()
    for i in range(len(song[0])):
        track = mido.MidiTrack()
        track.append(mido.Message('control_change', channel=0, control=0, value=80, time=0))
        track.append(mido.Message('control_change', channel=0, control=32, value=0, time=0))
        track.append(mido.Message('program_change', channel=0, program=50, time=0))
        for j in range(len(song[0][i])):
            note = mido.Message('note_on',channel=int(song[0][i][j][0]), note=int(song[0][i][j][1]), velocity=int(song[0][i][j][2]), time=int(song[0][i][j][3]))
            track.append(note)
        file.tracks.append(track)
    return file

def midiToNote(midifile,num_layers=2):
    song = []
    for i, track in enumerate(midifile.tracks):
        song.append([])
        for msg in track:
            message = str(msg).split()
            if ' not in message and 'control_change' not in message and 'program_change' not in message:
                note = int(str(msg).split()[2].split("=")[-1])
                song[i].append([note])

    song = [x for x in song if x]
    song = [song[:num_layers]]
    return np.array(song)

def predictionsToNotes(preds):
    song = preds[0][0]
    song = song.tolist()
    _ = 0
    noteIndex = []
    for i in song:
        best = max(song[_])
        key = song[_].index(best)
        noteIndex.append(key)
        # print(best,key)
        _ += 1
    return (noteIndex)

def notesToMidi(notes, velocity = 95, time = 116):
    file = mido.MidiFile()
    track = mido.MidiTrack()
    file.tracks.append(track)
    track.append(mido.Message('program_change', program=12, time=time))
    for i in range(len(notes)):
        track.append(mido.Message('note_on', note=notes[i], velocity=velocity, time=time))
    return(file)

Here I process the files, padding each track to 1000 messages (notes), labels are the subsequent notes.

In [2]:

from processingData import decodeMidi
import mido
import numpy as np
import os

files = os.listdir('train')
features = np.zeros(shape=(1,1000,88))
labels = np.zeros(shape=(1,1000,88))
for file in files:
    print(file)
    mid = mido.MidiFile("train/"+file)
    mid = decodeMidi(mid)
    mid = mid[0][0]
    featuresPart = []
    labelsPart = []
    for i in range(1001):
        if i < len(mid):
            onehot = [0] * 88
            onehot[mid[i][1]] = 1
            featuresPart.append(onehot)
            onehot = [0] * 88
            try:
                onehot[mid[i+1][1]] = 1
                labelsPart.append(onehot)
            except Exception:
                pass
        else:
            pad = [0] * 88
            featuresPart.append(pad)
            labelsPart.append(pad)
    featuresPart = np.array([featuresPart[:1000]])
    labelsPart = np.array([labelsPart[:1000]])
    features = np.concatenate((features,featuresPart))
    labels = np.concatenate((labels,labelsPart))

features = features[1:]
labels = labels[1:]
print(features.shape)
print(labels.shape)

01AusmeinesHerz.mid
02Ichdankdir.mid
03AchGott.mid
04EsistdasHeiluns.mid
05AnWasserflussen.mid
06Christus.mid
07Nunlob.mid
08Freueteuch.mid
09Ermuntredich.mid
10AustieferNot.mid
11Jesu.mid
12PuerNatusinBet.mid
13Alleinzudir.mid
14OHerreGott.mid
15ChristlaginTode.mid
(15, 1000, 88)
(15, 1000, 88)

Building and Training the Model¶

Use Keras' simple RNN to train the model.

In [3]:

%%capture
from keras.models import Sequential
from keras.layers import TimeDistributed, SimpleRNN, Dense

from keras.callbacks import ModelCheckpoint

model = Sequential()
model.add(SimpleRNN(input_dim  =  88, output_dim = 88, return_sequences = True))
model.add(TimeDistributed(Dense(output_dim = 88, activation  =  "softmax")))
model.compile(loss = "mse", optimizer = "rmsprop", metrics=['accuracy'])
model.fit(features, labels,
          epochs=1000,
          batch_size=256,
          callbacks=[ModelCheckpoint("Simple_RNN_3", monitor='val_acc',save_best_only=True)])
model.save("Simple_RNN_3.h5")

Generating Music from Beethoven's Moonlight Sonata¶

In [4]:

from keras.models import load_model
from processingData import decodeMidi, encodeMidi
import mido
import numpy as np
import pickle

song = mido.MidiFile("moonlightSonata.mid")
song = decodeMidi(song)
song = song[0][0]
test = []

for i in range(len(song)):
    onehot = [0] * 88
    onehot[song[i][1]] = 1
    test.append(onehot)
    onehot = [0] * 88

test = np.array([test[:1000]])
print(test.shape)

model = load_model('Simple_RNN_3.h5')
prediction = [model.predict(test)]

savePredictions = open("moonlightPrediction.pickle","wb")
pickle.dump(prediction, savePredictions)
savePredictions.close()

(1, 236, 88)

Converting Results to Midi¶

In [5]:

import pickle
from processingData import predictionsToNotes, notesToMidi
import mido
import numpy as np

pred = pickle.load(open('moonlightPrediction.pickle','rb'))
pred = predictionsToNotes(pred)
file = notesToMidi(pred,time=60)
print(file)

file.ticks_per_beat = 120
port = mido.open_output()

for msg in file.play():
    port.send(msg)

Here's what it sounds like:¶

Your browser does not support the audio element.

...and compared to the stripped, time-constant version of the original:

Your browser does not support the audio element.