madprogramer / syd

A prototype Intelligent User Interface for making music recommendations

syd

Mood/Preference based Music Recommendation

School project, let's see where it'll take us...

Notes:

Only tested on MacOS Catalina. See logs for more details.

Prerequisites:

1. Download Julia 1.4 1.0 Launch Julia 1.1. Hit ] then type add "FileIO,LibSndFile, Makie, AbstractPlotting, SampledSignals,https://github.com/JuliaAudio/PortAudio.jl.git"
2. Install Homebrew 2.1 Execute brew install sox
3. In iTunes/Apple Music create a playlist called syd. 3.1 fill this playlist with any songs you want syd to be able to "see."

Usage:

1. Download v0.0.3 from Releases
2. Make sure iTunes/Apple Music is launched.
3. Run with julia syd.jl from inside src/

syd's Issues

Count Speakers

Long Term Goal: Detect number of speakers at a given time to see if multiple people are talking

Improve Levenshtein-based Correction

Try 0-padding to grab single-word commands like "Pause": ex. add *'s to get "Pause"

Profiling

Long Term Goal: Add Voice Recognition

Recommendation Pattern

Recommend at the end of a song or if a skip is requested

Add Dependencies for the Builtins

As described here https://genieframework.github.io/Genie.jl/documentation/11--Managing_External_Packages.html

Transfer code from Legacy version

Ended up modifying the legacy version for the demo, please move these files into the UI version!

Add in plug-and-play ports for text-to-speech and speech-to-text

Make files which make your formula explicit enough, such that someone who wants to replace the text-to-speech or speech-to-text provider to something different can run or read that instead.

Report Emotion Through UI (Realtime or not)

Make Legacy Code the Main Code on the Master Branch

Issue title says all, shelve the web interface etc.

Basic UI in Genie

Choose Emotion Categories to Include for Mid-Demo

Convert to a Genie Project

Music Player

Either use Apple Music or find another solution (ex. embedding YouTube videos) which allows for event-based control.

Speech Synthesis in Browser via Files Generated from the Native Synthesizer

See https://apple.stackexchange.com/questions/389744/how-to-save-text-to-speech-as-a-voice-recording/389749#389749

Add Microphone Listening for the Web Version

Set up Web Sockets/Web RTC to capture recordings in such a way that you can still respond through Julia

Emotion Recognition for Mid-Demo

• (https://discourse.julialang.org/t/speech-based-emotion-recognition/37846)
• LSTMs in KNET
• RAVDESS set
  TRAIN ONE WAY OR ANOTHER WITHIN THE NEXT WEEK!!!!

Reverse Vocabulary Look-Up Method

Right now syd matches the sentence against vocabulary (with priority being assigned to order in vocabulary).

Perhaps a trick to increasing speed/accuracy could be to run a serious of windows on a sentence, query these string windows from a search function on the vocabulary and substitute from there instead.

Genre Identification

Use Genre as the similarity metric for now

Reorganize Object Hierarchy to Reduce Convolution

Song Retrieval