Improving the user's hearing ability to understand speech in noisy environments is critical to the development of hearing aid (HA) devices. For this, it is important to derive a metric that can fairly predict speech intelligibility for HA users. A straightforward approach is to conduct a subjective listening test and use the test results as an evaluation metric. However, conducting large-scale listening tests is time-consuming and expensive. Therefore, several evaluation metrics were derived as surrogates for subjective listening test results. In this study, we propose a multi-branched speech intelligibility prediction model (MBI-Net), for predicting the subjective intelligibility scores of HA users. MBI-Net consists of two branches of models, with each branch consisting of a hearing loss model, a cross-domain feature extraction module, and a speech intelligibility prediction model, to process speech signals from one channel. The outputs of the two branches are fused through a linear layer to obtain predicted speech intelligibility scores. Experimental results confirm the effectiveness of MBI-Net, which produces higher prediction scores than the baseline system in Track 1 and Track 2 on the Clarity Prediction Challenge 2022 dataset.
For more detail please check our Paper
You can download our environmental setup at Environment Folder and use the following script.
conda env create -f environment.yml
Please be noted, that the above environment is specifically used to run MBI-Net.py
. To generateSelf Supervised Learning (SSL) feature, please use python 3.6
and follow the instructions in following link.
To extract the SSL feature, please use the following code:
python Extract_SSL.py
Please use following script to train the model:
python MBI-Net.py --gpus <assigned GPU> --mode train
For, the testing stage, plase use the following script:
python MBI-Net.py --gpus <assigned GPU> --mode test
Please kindly cite our paper, if you find this code is useful.
Zezario, R.E., Chen, F., Fuh, C.-S., Wang, H.-M., Tsao, Y. (2022) MBI-Net: A Non-Intrusive Multi-Branched Speech Intelligibility Prediction Model for Hearing Aids. Proc. Interspeech 2022, 3944-3948, doi: 10.21437/Interspeech.2022-10838
We are grateful that our system received the best non-intrusive system at Clarity Prediction Challenge 2022
Self Attention, SincNet, wavLM are created by others