Comments (2)
Het-Shah
commented on July 22, 2024
Hi, There is no specific reason for us to add MSE Loss. We were playing around with the loss functions of various implementations and were testing the ability of the module to adapt to new loss functions, this is the reason it has MSE instead of CE+KL div.
from kd_lib.
Het-Shah
commented on July 22, 2024
Closing this issue for now, if you still have doubts feel free to open it again!
from kd_lib.
Related Issues (20)
- Benchmarking KD
- Benchmarking Pruning and Quantization
- Making a pipeline for Pruning, Quantization and Knowledge Distillation
- Pip install "stable" doesn't work HOT 3
- NameError: name 'best_student_id' is not defined HOT 1
- Implement Knowledge distillation by Functional Mapping HOT 1
- RuntimeError: only batches of spatial targets supported (3D tensors) but got targets of dimension: 4 HOT 7
- Paper: Data-Distortion Guided Self-Distillation for Deep Neural Networks HOT 2
- custom dataloader for NLP dataset HOT 4
- Issue with CUDA HOT 2
- distillation of gelectra model
- Use mock data for unit tests
- Create 'main' branch and set it as default HOT 2
- Consider potential name change to 'kdlib' HOT 2
- Test BERT2LSTM with mock data
- Can I skip training the teacher network? HOT 1
- No module named 'KD_Lib.KD' HOT 8
- Is there a suitable speech enhancement ? HOT 1
- Relational KD
- Error in Documentation
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kd_lib.