dyelax / encore.ai

Generate new lyrics in the style of any artist using LSTMs and TensorFlow.

encore.ai

Generate new lyrics in the style of any artist using Deep Learning.
Winner of "Best Machine Learning Hack" at HackMIT!

Check it out at http://encore.ai.

Music Lives Forever.

From Elvis to The Beatles, Nirvana to Tupac, many of the musicians we love are no longer creating. encore.ai lets you revive the spirit of your favorite artists in the modern age.

We are impatient.

Cant wait for the next Kanye album to drop? Were you part of the public outcry over the delayed release of Frank Ocean's new record? Fret no more – encore.ai delivers your favorite artist's next single at the touch of a button.

But How Does It Work?!

We used TensorFlow to create an LSTM (Long Short Term Memory) Neural Network that reads all of the lyrics of any musical artist. The network learns the style of how the artist writes – the context of words, rhymes, line/stanza breaks, etc. Given a seed word or phrase, it will generate a new song in that artist's style.

Some Examples:

Kanye West

Never I admit I can't do I see a drug som...
I’m ’bout to re-united
I'll need have to play me alone
I'll fade back O extra until they cover from an ultralight beam

Because you do her money before my Sean harder
Now, 2 West I one away
I’m up, so quit, One:]
yeah, I will
You see, you need)
All your mic from a moment
I'm saying, violate

Let’s got no goodbyes to a dem me
now?
I been nothing funny but we go n****
Get all my racks steps

And rockin' (Grandma) watch my strippers got getting this bitch a shot, is like somebody It's old shit
And oh precious *
Cause we call that ready for not

[Hook]
It's an room 2 now, in the restaurant, clothing on.
Standing dem my money

[Kanye West:]
Let's fly off beats that I’m at these good life with you
keep the big ass, take my things and throw my shit’s is stronger

Taylor Swift

paper lying here
'Cause I swear out there ain't where
You outta be
And you flashback to when he said forever and always
Oh, and it rains in your bedroom
Everything is wrong
It rains when the sun came up, you were lookin' at me
Santa baby, forgot to mention one little thing
A ring

And I don't think it all through? All these things will change
Can you feel it now?
I fall in love with you
Give me a photograph to hang on my wall, superstar

How'd we got problems
And I don't think we can solve them
You were the prince
I used to see
The one we danced to all night long

Cause you got me a nice new apartment
In a city, wouldn't you have to make me go back there again."

Find the wrong wrong
On our last night
Ooh, ooh, love's like this
It's something I missed
Ooh, ooh, ooh, ooh
It was the best night, never would forget how he moved.

Instructions to Train:

If you want to use our model to train your own artists, follow these steps:

1. Pick an artist – it should be someone with a lot of lyrics. (Over 100,000 words).
2. Collect all of the artist's lyrics from your favorite lyrics website. Save each song as a text file in data/artist_name/. We recommend leaving newlines in as a special token so that the network will learn line and stanza breaks.
3. Train by navigating to the code directory and running python runner.py -a <artist_name> -m <model_save_name>.

• Our models were all trained for 30,000 steps.

4. Generate new songs by running
   python runner.py -a <artist_name> -l ../save/models/<model_save_name>/<ckpt_file> -t.

• Optional: If you would like to specify "prime text" – the initial text that the model will generate from – pass in a string with the -p flag.

5. Share your trained models with us so we can feature them on our website! Create an issue with a link to a public Dropbox or Google Drive containing your model's .ckpt file.

encore.ai's Issues

ValueError: Could not flatten dictionary: key Tensor(...) is not unique.

I have tried training this same model a couple of times now (30k and 60k passes) and this problem persists. When I try to generate lyrics, it outputs a random word from the lyrics pile, then shows the Traceback below.

It is Ubuntu 16.04. TensorFlow CPU (no GPU). I'm not sure why I installed things to the directories I did, but probably unrelated. I don't really know what I'm doing.

Thanks for looking!

Traceback (most recent call last):
File "runner.py", line 118, in
main()
File "runner.py", line 114, in main
LyricGenRunner(load_path, artist_name, test, prime_text)
File "runner.py", line 52, in init
self.test(prime_text)
File "runner.py", line 81, in test
sample = self.model.generate(prime=prime_text)
File "/var/www/encore.ai/code/LSTMModel.py", line 181, in generate
state = self.sess.run(self.final_state, feed_dict=feed_dict)
File "/etc/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 895, in run
run_metadata_ptr)
File "/etc/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1065, in _run
feed_dict = nest.flatten_dict_items(feed_dict)
File "/etc/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/util/nest.py", line 251, in flatten_dict_items
% (new_i))
ValueError: Could not flatten dictionary: key Tensor("MultiRNNCellZeroState/BasicLSTMCellZeroState/zeros:0", shape=(1, 256), dtype=float32) is not unique.

InvalidArgumentError (see above for traceback): indices[7,4] = 1931961714 is not in [0, 765)

I tried follow the steps and used attached 12 files with command "python runner.py -a linkin_park -m linkin_park" where two files are kept at location data/artists/linkin_park but I am getting
following error

Caused by op 'lstm_vars/embedding_lookup', defined at:
File "runner.py", line 118, in
main()
File "runner.py", line 114, in main
LyricGenRunner(load_path, artist_name, test, prime_text)
File "runner.py", line 39, in init
test=test)
File "C:\naveentoshiba\Optymyzebackup\mybackup\mywork\encore.ai-master\code\LSTMModel.py", line 35, in init
self.build_graph(test)
File "C:\naveentoshiba\Optymyzebackup\mybackup\mywork\encore.ai-master\code\LSTMModel.py", line 71, in build_graph
input_embeddings = tf.nn.embedding_lookup(self.embeddings, self.inputs)
File "C:\Anaconda3\lib\site-packages\tensorflow\python\ops\embedding_ops.py", line 327, in embedding_lookup
transform_fn=None)
File "C:\Anaconda3\lib\site-packages\tensorflow\python\ops\embedding_ops.py", line 151, in _embedding_lookup_and_transform
result = _clip(_gather(params[0], ids, name=name), ids, max_norm)
File "C:\Anaconda3\lib\site-packages\tensorflow\python\ops\embedding_ops.py", line 55, in _gather
return array_ops.gather(params, ids, name=name)
File "C:\Anaconda3\lib\site-packages\tensorflow\python\ops\array_ops.py", line 2667, in gather
params, indices, validate_indices=validate_indices, name=name)
File "C:\Anaconda3\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 2250, in gather
validate_indices=validate_indices, name=name)
File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 3271, in create_op
op_def=op_def)
File "C:\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 1650, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): indices[7,4] = 1931961714 is not in [0, 765)
[[Node: lstm_vars/embedding_lookup = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, _class=["loc:@lstm_vars/embeddings"], validate_indices=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](lstm_vars/embeddings/read, _arg_Placeholder_0_0)]]

and similar errors as I keep adding more files.

It seems like there is problem in the way I am feeding the data.

If you can help me with some working sample that would be gr8.

random100_2015-05-06T13_23_52.338201.txt
random100_2015-05-06T13_20_24.825285.txt
random100_2015-05-06T13_17_01.120800.txt
random100_2015-05-06T13_13_37.462984.txt
random100_2015-05-06T13_10_08.956876.txt
random100_2015-05-06T13_06_43.697151.txt
random100_2015-05-06T13_44_41.762102.txt
random100_2015-05-06T13_41_12.365548.txt
random100_2015-05-06T13_37_46.552626.txt
random100_2015-05-06T13_34_18.128560.txt
random100_2015-05-06T13_30_50.548291.txt
random100_2015-05-06T13_27_21.156519.txt

How do i use this?

Can i have a tutorial on what files to open and where or how to compile it through visual studio or something? anything? just a tutorial...?

Generating test data

I'm a little confused about generating test data with the code. So far it seems that Tensorflow saves a meta, index and data file related to the checkpoint file, while the github instructions say to just feed in the checkpoint file for test data once the model is trained.

From searching the web it looks like we have to use the meta file and use something like:
saver = tf.train.import_meta_graph('/tmp/model.ckpt.meta')
saver.restore(sess, "/tmp/model.ckpt")

To get the model up and running again. However looking at the code it looks like the new data is almost generated probabilistically?

My only guess is that something like this line above may be used in one of the "state" lines in the Generate function (in the LSTM model file), although that doesn't seem quite right (not exactly sure what these lines do specifically).

If anyone can help me with this I'd really appreciate it, I feel like I'm really close to making the code work :(

TensorFlow 1.0 compability?

Can you please upgrade this project to TensorFlow v1.0? So much has changed. I'm running into a wall of errors.

Add running instructions in README

Can you please add the instructions on how to run the scripts and train the network for different singers.

NameError: name 'reduce' is not defined

NameError: name 'reduce' is not defined

Missing Parentheses in call to 'print'

So i guess this is not for python 3?

File "runner.py", line 28 print 'Process data...' ^ SyntaxError: Missing parentheses in call to 'print'. Did you mean print('Process data...')?

is there a way to make it run? update coming soon?

The problem is tensorflow only running under py3 for windows, and this project only working with py2 makes it basically impossible to use under win :D

thanks for any response :)

Model not saving?

I'm now at 2000 epochs and the model still hasn't saved. How do I go about fixing this or reducing the save time?