GithubHelp home page GithubHelp logo

consensus-entropy's Introduction

Let's agree to disagree: Consensus Entropy Active Learning for Personalized Music Emotion Recognition

Source code implementation of active learning for our paper (also the suppl. material) presented at the 22nd International Society for Music Information Retrieval Conference (ISMIR).

-- Juan Sebastián Gómez-Cañón, Estefanía Cano, Yi-Hsuan Yang, Perfecto Herrera, and Emilia Gómez

Abstract

Previous research in music emotion recognition (MER) has tackled the inherent problem of subjectivity through the use of personalized models -- models which predict the emotions that a particular user would perceive from music. Personalized models are trained in a supervised manner, and are tested exclusively with the annotations provided by a specific user. While past research has focused on model adaptation or reducing the amount of annotations required from a given user, we propose a methodology based on uncertainty sampling and query-by-committee, adopting prior knowledge from the agreement of human annotations as an oracle for active learning (AL). We assume that our disagreements define our personal opinions and should be considered for personalization. We use the DEAM dataset, the current benchmark dataset for MER, to pre-train our models. We then use the AMG1608 dataset, the largest MER dataset containing multiple annotations per musical excerpt, to re-train diverse machine learning models using AL and evaluate personalization. Our results suggest that our methodology can be beneficial to produce personalized classification models that exhibit different results depending on the algorithms' complexity.

Installation

Clone this repository:

git clone https://github.com/juansgomez87/consensus-entropy
cd consensus-entropy

Create a virtual environment using venv and install requirements:

python3 -m venv cons-env
source cons-env/bin/activate
pip3 install -r requirements.txt

Since we made a minor change to the XGBoost library, run the following command to allow for re-training XGB models:

cp xgboost/sklearn.py ./cons-env/lib/python3.8/site-packages/xgboost/

We also use short-chunk CNN proposed by Won et al. Thank you to the authors!

Data

Note: The configuration file settings.py contains all the directories necessary for the code. Please update the strings deam_data and amg_data accordingly.

The DEAM dataset is available in this link. We already provide discretized annotations in the deam_annotations directory.

Regarding the AMG1608 dataset, please contact affige[at]gmaildotcom

Usage

Pre-trained models

You can find the pre-trained models on the DEAM dataset in the models/pretrained directory.

Models can also be pre-trained using the following command (-cv is the cross-validation split and -m is the classifier algorithm to be used):

python3 deam_classifier.py -cv CVAL_SPLIT -m MODEL_TYPE

Our experiments were ran with CVAL_SPLIT=5 and MODEL_TYPE=[gnb, sgd, xgb, cnn].

Personalization

In order to perform all the experiments, we use the amg_test.py script. There are four different input flags:

--queries: We used q=10 for each iteration.
--epochs: We used e=10 iterations.
--num_anno: We used only annotations from users that annotated over 150 excerpts.
--mode: mc - machine consensus, hc - human consensus, mix - hybrid consensus, and rand - random baseline.

You can perform all experiments using the following command:

python3 amg_test.py -q 10 -e 10 -m rand -n 150 && sleep 200 && python3 amg_test.py -q 10 -e 10 -m mc -n 150 && sleep 200 && python3 amg_test.py -q 10 -e 10 -m hc -n 150 && sleep 200 && python3 amg_test.py -q 10 -e 10 -m mix -n 150

All final personalized models will appear in models/user/{user_id} alongside a text file containing the corresponding evaluation metrics.

Publication

@InProceedings{GomezCanon2021ISMIR,
    author = {Juan Sebastián Gómez-Cañón and Estefanía Cano and Yi-Hsuan Yang and Perfecto Herrera and Emilia Gómez},
    title = {Let's agree to disagree: Consensus Entropy Active Learning for Personalized Music Emotion Recognition},
    booktitle = {Proceedings of the 22nd International Society for Music Information Retrieval Conference (ISMIR)},
    year = {2021},
    location = {Online},
    pages = {},
}

consensus-entropy's People

Contributors

juansgomez87 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.