wenwanchen / crowdpp Goto Github PK

Overview
Crowdpp has two parts of code: the code in the packages "de.fau.cs.jstk.*" are borrowed from the "jstk" package developed by the Speech Group at the University of Erlangen-Nuremberg, which is primarily used for MFCC feature extraction; the code in the packages "edu.rutgers.winlab.crowdpp.*" is developped by Chenren Xu and Sugang Li:
"edu.rutgers.winlab.crowdpp.audio" package includes audio recording, data convertion (from byte to float), feature extraction (MFCC and pitch), cross calibration (see "Cross calibration"), and speaker counting algorithm.
"edu.rutgers.winlab.crowdpp.db" package includes the functions related with sqLite databased operation.
"edu.rutgers.winlab.crowdpp.sensor" package includes the location tracking service.
"edu.rutgers.winlab.crowdpp.service" package includes the audio recording and speaker counting services.
"edu.rutgers.winlab.crowdpp.ui" package includes the Android fragments as the UI interface
"edu.rutgers.winlab.crowdpp.util" package includes the constants and utility functions (time, math, distance, file process)

** when you import the code into your workspace, please add JRE or JDK into your build path**

Cross calibration
Both "crossAsymmetricCalibration" and "crossSymmetricCalibration" funtions in "MicrophoneCalibration.java" compute the MFCC distance to determine the optinmal thresholds theta_s and theta_d for speaker distinction. For cross calibration, please have the MFCC and pitch data from the different participants' monologue audio data. Both "crossAsymmetricCalibration" and "crossSymmetricCalibration" will log the distance from same speaker and different speakers in different files. In "crossAsymmetricCalibration", the distance is computed based on the full data from one speaker and partial data (a few seconds) from the other. The thresholds derived from this function is used for semisupervised speaker counting purpose. In "crossSymmetricCalibration", the distance is computed from the same length of speech from two speakers. The thresholds from this function is cused for unsupervised speaker counting. You can determine your own thresholds based on how confident you will identify the same speaker (theta_s) and how conservative to admit a new speaker (theta_d). For theta_s and theta_d, please refer to our paper "Crowd++: Unsupervised Speaker Count with Smartphones" by Chenren Xu, Sugang Li, Gang Liu, Yanyong Zhang, Emiliano Miluzzo, Yih-Farn Chen, Jun Li, Bernhard Firner.