Comments (4)
Hi Robin,
Thank you for your question.
- Here is the paragraph where the authors mention about the normalization:
From: https://arxiv.org/pdf/1705.02304.pdf
It's not crystal clear how they did so I had to make a guess. I used instance normalization as you pointed out and did not normalize with batch normalization. The 26D FBanks get normalized within a single frame for a given speaker. I'm not a big fan of BN because it's batch dependent.
Instance normalization looked like the most natural (and easiest) way to get the normalization working well. What would be ideal is to make sure it's what they did in the paper. Your guess is that they used a batch normalization for FBank normalization instead? In your example m
is the batch and v
the frame (just to make sure my understanding is correct).
- Regarding the naming I agree with you. It should be more
read_fbank()
thanread_mfcc()
.
PS: I'm not an expert in audio analysis.
from deep-speaker.
Hi Philip, Thanks for your quick reply!
I'm not a audio expert as well, but I doubt the IN(Instance Normalization) in each frame because I feel it would remove the difference between audio frames like [0,0,0,0,0..] and [100,100,100,100,100]. Besides, to my knowledge in computer vision field, the IN is usually used in style transfer, GAN-based image synthesize or image-to-image translation tasks. When handling classifications, the BN is mostly applied.
To verify my thoughts, I make a minor revise on your code(use 64D MFCC and do BN) and retrain the model. The test script result is SAME SPEAKER [0.73709786] DIFF SPEAKER [-0.00997655] (yours: SAME SPEAKER [0.8112024] DIFF SPEAKER [0.02534033]) . No significant improvement...
I find a similar discuss in https://www.kaggle.com/c/freesound-audio-tagging/discussion/54082
I'm also trying to contact the original paper authors to find the truth.
Best,
-Robin
from deep-speaker.
@RobinROAR thank you for making those experiments. It's very insightful. I would expect it to work better too but seems like in practice it yields similar results. I tried to contact the original paper authors but it has been unsuccessful. It's a paper by Baidu and they probably don't want to release too much information. I had to "guess" a lot of small details when implementing the paper. Anyway, let me know if they reply!
from deep-speaker.
I'll close this issue for inactivity. Please re-open it if you have something new!
from deep-speaker.
Related Issues (20)
- ModuleNotFoundError: No module named 'audio' HOT 2
- [feature request] Could it support telling the speakers count in a noisy environment HOT 1
- Can I use it for sentimental analysis. HOT 1
- Inquiry on use case for voice analysis HOT 1
- Inquiry on embedding extractions for voice comparisons HOT 16
- Unable to have tensorflow 2.x.x recognize GPU
- Which python version should I choose HOT 1
- Wrong number of layers while doing triplet training after softmax pre-training HOT 2
- LazyTripletBatcher configuration HOT 1
- transfer learning with pretrained models: is pre-training needed with new data? HOT 3
- Num of speakers HOT 3
- Problems to test-model with cli HOT 8
- pre-training optimization for RAM HOT 1
- how to save enroll file in encrypted mode and decrypted for matching how ? HOT 5
- impact of using delta, trim_silence in training HOT 2
- Embedding/Identifying a speaker talking in a different language with pretrained model HOT 1
- How to compute between model and audo file ? HOT 9
- Model Testing Issue HOT 4
- ValueError: "Tensor is not an element of this graph" when using TripletBatcherMiner during Triplet training HOT 6
- tensorflow.python.framework.errors_impl.InternalError: Blas xGEMMBatched launch failed : a.shape=[3,459,257], b.shape=[1,257,64], m=459, n=64, k=257, batch_size=3 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from deep-speaker.