youshanzhang / birdsoundsdenoising

BirdSoundsDenoising: Deep Visual Audio Denoising for Bird Sounds

This directory contains BirdSoundsDenoising dataset and the code for paper BirdSoundsDenoising: Deep Visual Audio Denoising for Bird Sounds, which is accepted by In 2023 IEEE Winter Applications of Computer Vision(WACV).

If you have any questions, please email to [email protected].

Reference

If you find it is helpful, please cite it as:

Zhang, Youshan, and Li, Jialu. (2023). BirdSoundsDenoising: Deep Visual Audio Denoising for Bird Sounds. In 2023 IEEE Winter Applications of Computer Vision (WACV).

Datasets

The BirdSoundsDenoising dataset is available at this link.

In training and valid folders, the data structure is:

-------------Training/Valid

------------------Denoised_audios

------------------Images

------------------Masks

------------------Raw_audios

Raw_audios contain all noise bird sounds. Images are converted STFT noise bird sound images, and Masks are the labled clean sound areas. Denoised_audios are the denoised bird sound audios.

Results

Codes

audio2image.m: Matlab function of converting an audio to an image

image2audio.m: Matlab function of converting a denoised image to a denoised audio

Audio2Image_Image2Audio_example.m: An example to show audio2image and image2audio functions.

birdsoundsdenoising's Issues

A few questions and comments

Hello,

I hope you don't mind me adding some comments and questions. I've had a look at your paper and the dataset:

1. I am not clear on how you do the SDR calculations in the contexts of the segmentation models. 5.2.1 gives the fomula:

SDR=10log10 ||m||2
||˜m−m||2

It looks to me that m is the mask and ˜m is the predicted mask. Is that right?

2. Another way would be to calculate SDR based on the audio wave rather than in the frequency domain. However, the lengths of the original and denoised audio waves in the dataset are not always the same and so do this it would seem to be necessary to spend time correlating the signals. The report doesn't mentioned that you did that (but I may have missed it). So presumably, the SDR for the mask-based segmentation models was done for the masks rather than the reconstructed audio?

3. I think that there may be a typo in the formula for IoU in section 5.2.1. Should the denominator be A∪B ?

4. I noticed that some of the masks in the dataset appear to be empty (e.g. XC175650_left,XC462632_left, XC462680_left, XC463223_left, XC463661, XC469469_left, XC49097, XC490972_left, XC496206_left).

Thanks you

Mark