hsiungc / orchestrate

Team Orchestrate Capstone Project (Fall 2023)

Orchestrate - Capstone Project (Fall 2023)

Public-Facing GitHub Repository & Documentation

Andy Fiegleman, Casey Hsiung, Dakota Potere-Ramos, James Meyer, Ken Trinh

Website | Demo

Individuals aspiring to create music face the challenge of either mastering multiple instruments required for their compositions or relying on digital tools for music production. Orchestrate harnesses the power of machine learning and natural language processing (NLP) to develop a prompt-responsive music generation product. This innovative platform empowers users to create and layer diverse music pieces, crafting personalized arrangements that can be utilized for jamming, sparking musical inspiration, or simply enjoying music in its own right.

Setup Instructions

To run the server and client locally, follow the below steps.

API Key Information

Setting the API keys is encouraged for a seamless experience. An OpenAI key (OPENAI_API_KEY) is required to use ChatGPT. Adding a HuggingFace key (HUGGINGFACE_API_KEY) is optional but useful to avoid rate limiting. The production deployment of Orchestrate calls model files from an S3 bucket. The localized version here makes calls to files stored in this repository.

Orchestrate Client

1. Install Node.js and npm from here.
2. Navigate to the orchestrate-client directory.
3. Run the following commands:

npm install
npm run start

4. A browser should open at the http://localhost:3000 address.
5. Following the frontend instructions will allow you to play and download a music file by submitting a text prompt.

Orchestrate Server

1. Install Flask from here.
2. Navigate to the orchestrate-server directory.
3. Run the following commands:

pip install -r requirements.txt
flask run

4. We recommend using a simple tool such as Postman to send POST requests to the Orchestrate server at http://localhost:5000/api/generate.

Citations

Hyun, L., Kim, T., Kang, H., Ki, M., Hwang, H., Park, K., ... & Kim, S. J. (2022). ComMU: Dataset for Combinatorial Music Generation. arXiv preprint arXiv:2211.09385.

Ens, J., & Pasquier, P. (n.d.). Building the MetaMIDI dataset: Linking Symbolic and Audio Musical Data. Proceedings of the 22nd ISMIR Conference. https://archives.ismir.net/ismir2021/paper/000022.pdf

Licenses & User Agreements

ComMU License
ComMU GitHub
ComMU Demo

The ComMU dataset is released under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0). It is provided primarily for research purposes and is prohibited to be used for commercial purposes.

MetaMIDI Copyright
MetaMIDI Dataset Github
MetaMIDI Dataset

Since we did not transcribe any of the MIDI files in the MetaMIDI Dataset, we provide a list of all the Copyright meta-events in the dataset to acknowledge the original authors of the files.