matheusbitaraes / aihero

AI for music improvisation

AI Hero Server

An Artificial Intelligence built for generating musical melodies



View Demo · Report Bug · Request Feature

Table of Contents

1. About The Project
2. Getting Started
   • Installation
   • Model Training
   • Model Execution
3. Contact
4. Acknowledgments

About The Project

The AI Hero project proposes a blues melody generator, that tries to emulate the improvisation process of the human mind.

This work proposes an architecture composed of a genetic algorithm whose initial population is fed by generative adversarial networks (GANs) specialized in generating melodies for certain harmonic functions. The fitness function of the genetic algorithm is a weighted sum of heuristic methods for evaluating quality, where the weights of each function are assigned by the user, before requesting the melody. A data augmentation statregy for the GAN training data was proposed and experimentally validated. This experiment and two others are available in the masters thesis (in portuguese) generated by this work.

Also, this article, validating a data augmentation strategy proposal, was published as a consequence of the work.

(back to top)

Getting Started

Below is the guideline on how to run the python server locally for training or generating melodies

Installation

This project was developed using Python 3.8. So, make sure you are using a compatible version. Then, install the dependencies.

• Install dependencies

  pip install -r requirements.txt

• Install dependencies for mac - A different tensorflow needs to be used

  pip install -r requirements-mac.txt

(back to top)

Model Training

To train the Model (GAN), you should go to the src/GEN folder and run the train_script.py. All important configurations are available in src/config.json

(back to top)

Model Execution

To execute the Model as a server, you should run

python src/main.py src/config.json

Then, the server, with loaded models, will be available at port 8083 (this can be modified in the src/config.json).

Requesting a Melody

To actually request a melody, you could clone the front-end project and execute it:

Alternativelly you could make the REST requests by yourself. Here is how to do it:

Initially you should ask for the melody using a POST request, as follows:

url:

http://localhost:8083/melody?source=${source}

where source can have one of following values: train, gan or evo

body:

{
  harmony_specs:
    [
      { melodic_part: "", chord: "C:7maj", key: "C", tempo: 120 },
      { melodic_part: "", chord: "F:7maj", key: "C", tempo: 120 },
      { melodic_part: "", chord: "C:7maj", key: "C", tempo: 120 },
      { melodic_part: "", chord: "C:7maj", key: "C", tempo: 120 },
      { melodic_part: "", chord: "F:7maj", key: "C", tempo: 120 },
      { melodic_part: "", chord: "F:7maj", key: "C", tempo: 120 },
      { melodic_part: "", chord: "C:7maj", key: "C", tempo: 120 },
      { melodic_part: "", chord: "C:7maj", key: "C", tempo: 120 },
      { melodic_part: "", chord: "G:7maj", key: "C", tempo: 120 },
      { melodic_part: "", chord: "F:7maj", key: "C", tempo: 120 },
      { melodic_part: "", chord: "C:7maj", key: "C", tempo: 120 },
      { melodic_part: "", chord: "G:7maj", key: "C", tempo: 120 },
    ],
  evolutionary_specs:
    [
      {
        key: "notes_on_same_chord_key",
        name: "Notes on Same Chord",
        description: "notes_on_same_chord_key",
        transf_weights: [Array],
        bounds: [Array],
        weight: 0,
      },
      {
        key: "notes_on_beat_rate",
        name: "Notes on Beat",
        description: "notes_on_beat_rate",
        transf_weights: [Array],
        bounds: [Array],
        weight: 0,
      },
      {
        key: "note_on_density",
        name: "Note Density",
        description: "note_on_density",
        transf_weights: [Array],
        bounds: [Array],
        weight: 1,
      },
      {
        key: "note_variety_rate",
        name: "Note Variety",
        description: "note_variety_rate",
        transf_weights: [Array],
        bounds: [Array],
        weight: 0,
      },
      {
        key: "single_notes_rate",
        name: "Single Notes Rate",
        description: "single_notes_rate",
        transf_weights: [Array],
        bounds: [Array],
        weight: 1,
      },
      {
        key: "notes_out_of_scale_rate",
        name: "Notes out of Scale",
        description: "notes_out_of_scale_rate",
        transf_weights: [Array],
        bounds: [Array],
        weight: 0,
      },
    ],
}

You will then receive the melodyId in the response, which will be an MD5 hash (ex: 205f1c3b-da87-455f-94b6-a8f49cb346b9).

Then, you can query for the melody by making a GET request with

localhost:8083/melody/${melodyId}

If the melody is not ready, you will receive a response 404, if it is ready, you will receive a response with status 200 and the .mid file.

(back to top)

Contact

For more clarification, please reach out!

Matheus Bitarães - LinkedIn - [email protected]

Project Link: https://aihero.bitaraes.com.br

(back to top)

Acknowledgments

• Template created based on this template

(back to top)

aihero's Issues

Refinar modelo da GAN

• Revisar algumas tecnicas de redes profundas para ver se melhoro o treinamento da GAN.
• Marcar reunião com Fred para discutir.

Coletar mais amostras para meu dataset de teste

Refinar Algoritmo Evolucionario

Colocar GIF de evidencia da otimizacao evolucionária

Inserir mais testes automatizados

Concluir Documentação.

[ ] Apresentação em slides.
[ ] Readme do github.

Trocar tudo para EGMI: Evolutionary Generative Music Improvisation System

Validar e documentar conceito das entonações musicais

[ ] Estudo de harmonia funcional
...

Subir container da solução

[ ] Criar o docker
[ ] Subir em alguma infra e expor as rotas de treinamento e de requisição de melodias