starmkk / mylink

1. Datasets

Speech ehancement datasets (sorted by usage frequency in paper)

English

Name	Source	Hours
Dataset by University of Edinburgh	https://datashare.ed.ac.uk	39h(141165.816338)
VCTK(2009)	https://datashare.ed.ac.uk	82h(297552.223038)
LibriSpeech	http://www.openslr.org	983h(3539995.721648)
Common Voice	https://commonvoice.mozilla.org	1837h(6615925.423006)
The VoxCeleb1 Dataset	https://www.robots.ox.ac.uk	x
The VoxCeleb2 Dataset	https://www.robots.ox.ac.uk	x

German

Name	Source	Hours
Common Voice	https://commonvoice.mozilla.org/	832h (2995609.871861)

Augmentation noise sources (sorted by usage frequency in paper)

Name	Source	Hours
DEMAND	https://zenodo.org	8h(28800.384000)
100 Noise	http://web.cse.ohio-state.edu	293s(293.299375)
RIRS_NOISES	https://www.openslr.org	27h(97661.178407)
QUT-NOISE	https://research.qut.edu.au	27h (98262.946746)
MUSAN	https://www.openslr.org	48h (175827.483386)
Deep Noise Suppression (DNS) Challenge - Interspeech 2020	https://github.com/breizhn/DNS-Challenge
Deep Noise Suppression (DNS) Challenge - Interspeech 2022	https://github.com/microsoft/DNS-Challenge

Audio data augmentation

Link	Language	Description
Data simulation	Python	Add reverberation, noise or mix speaker.
audio-SNR	Python	Mixing an audio file with a noise file at any Signal-to-Noise Ratio.

2. 논문

1. Transformer

   • Attention Is All You Need
   • Attention Is All You Need - 나동빈
   • GAN: Generative Adversarial Networks (꼼꼼한 딥러닝 논문 리뷰와 코드 실습)

2. Speech Enhancement

   • Improving GANs for Speech Enhancement 논문
   • Improving GANs for Speech Enhancement (페이퍼 관련 소스)
   • Self Attention GAN for Speech Enhancement 논문
   • Self Attention GAN for Speech Enhancement in Tensorflow 2 (페이퍼 관련 소스)
   • Spectral Normalization 논문
   • Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification 논문
   • To prune, or not to prune: exploring the efficacy of pruning for model compression 논문
   • CMGAN: Conformer-based Metric GAN for Speech Enhancement
   • MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement
   • MetricGAN+: An Improved Version of MetricGAN for Speech Enhancement
   • Dual-Signal Transformation LSTM Network for Real-Time Noise Suppression 논문
   • Dual-Signal Transformation LSTM Network for Real-Time Noise Suppression (페이퍼 관련 소스)
   • A Modulation-Domain Loss for Neural-Network-based Real-time Speech Enhancement
   • SDR – HALF-BAKED OR WELL DONE?
   • TasNet: time-domain audio separation network for real-time, single-channel speech separation
   • conv1d

3. Data Augmentation

   • SpecAugment 논문
   • SpecAugment: A New Data Augmentation Method for Automatic Speech Recognition - Google AI Blog
   • Colors of noise

4. End to End

   • Connectionist Temporal Classification
   • Convolution-augmented Transformer for Speech Recognition, 2020
   • Conformer: Convolution-augmented Transformer for Speech Recognition
   • EFFICIENT CONFORMER, 2021
   • Efficient Conformer (2021 페이퍼 관련 소스)
   • Openspeech seminar

5. Awesome Speech Enhancement

1.음성처리

1. 블러그

1. Speech Processing for Machine Learning: Filter backs, Mel-Frequency Cepstral Coefficients(MFCCs) and What's In-Between
2. RNN transducer
3. What is translation equivariance, and why do we use convolutions to get it?
4. Tensorflow-lite signiture 분석
5. TFLite Quantization
6. TensorFlow Custom Op으로 데이터 변환 최적화하기

2. Tensorflow

1. tf.data : TensorFlow 입력 파이프 라인 빌드
2. tf.data API로 성능 향상하기
3. 분산훈련
   • 분산 훈련 / strategy.num_replicas_in_sync
   • 케라스를 사용한 분산 훈련 / MirroredStrategy
   • 텐서플로로 분산 훈련하기
   • 학습 루프와 함께 tf.distribute.Strategy 사용하기
4. 체크포인트 훈련하기
5. tf.function으로 성능 향상하기

3. Audio recognition using Tensorflow Lite

1. tflite-support

2. 오디오 분류기 통합 -> 오디오 분류기 통합2

3. 오디오 데이터 준비 및 증강

4. 모델변환

5. 모델최적화

6. TensorFlow Lite Model Maker를 사용하여 오디오 도메인에 대한 전이 학습

7. Tensorflow and Tensorflow Lite code in the context of audio processing (MFCC, RNN)

8. Audio recognition using Tensorflow Lite in Flutter applications

9. Udacity - TensorFlow Lite e-Learning과정

10. Coursera - Device-based Models with TensorFlow Lite

11. A Definitive Guide for Audio Processing in Android with TensorFlow Lite Models

12. Android에서 커스텀 TensorFlow Lite 모델 사용

13. Tensorflow lite

• 파이킴 runForMultipleInputsOutputs
• Tensorflow Lite android 기본 개인정리
• TensorFlow Tutorial

14. Quantization

• Inside TensorFlow: TF Model Optimization Toolkit (Quantization and Pruning)
• TensorFlow Model Optimization Toolkit — Pruning API

Bitmap bitmap = Bitmap.createScaledBitmap(yourInputImage, 224, 224, true);
ByteBuffer input = ByteBuffer.allocateDirect(224 * 224 * 3 * 4).order(ByteOrder.nativeOrder());
for (int y = 0; y < 224; y++) {
    for (int x = 0; x < 224; x++) {
        int px = bitmap.getPixel(x, y);

        // Get channel values from the pixel value.
        int r = Color.red(px);
        int g = Color.green(px);
        int b = Color.blue(px);

        // Normalize channel values to [-1.0, 1.0]. This requirement depends
        // on the model. For example, some models might require values to be
        // normalized to the range [0.0, 1.0] instead.
        float rf = (r - 127) / 255.0f;
        float gf = (g - 127) / 255.0f;
        float bf = (b - 127) / 255.0f;

        input.putFloat(rf);
        input.putFloat(gf);
        input.putFloat(bf);
    }
}
int bufferSize = 1000 * java.lang.Float.SIZE / java.lang.Byte.SIZE;
ByteBuffer modelOutput = ByteBuffer.allocateDirect(bufferSize).order(ByteOrder.nativeOrder());
interpreter.run(input, modelOutput);
...

15. Recognize Flowers with TensorFlow Lite on Android

16. GitHub:

• Deep Learning for Audio with TFLite support to run on Android
• awesome-tensorflow-lite

4.Voice Filter

1. VoiceFilter-Lite: Streaming Targeted Voice Separation for On-Device Speech Recognition

. Tensorflow

1. tf.data : TensorFlow 입력 파이프 라인 빌드
2. 케라스 손실 함수 정리

2.Speech Recognition

1.toolkit

1. End-to-End Speech Processing Toolkit
2. Openspeech
3. DeepSpeec

Challenge

1.Deep Noise Suppression Challenge