Comments (3)
@svboeing ratio only cares about the first tasks and treats other tasks as additional tasks for performance boost. Please refer our NAACL paper for details.
from mt-dnn.
We implemented several variants of MTL training. The ratio is from our NAACL19 paper, attached https://arxiv.org/pdf/1809.06963.pdf. You can try it on a specific task, which may give some improvement.
from mt-dnn.
So, the purpose of the ratio parameter is, given one enormous dataset and some smaller ones, you can reduce the bigger one's impact, thus preventing it to dominate the learning process?
Also, I was wondering; if you do not use the ratio parameter, you will actually train on ALL datasets, is that correct? It's not that some of the smaller ones might get dropped and you end up only training on the big ones?
from mt-dnn.
Related Issues (20)
- How we can use mt-dnn to do Multi-Label Classification? HOT 1
- predict.py size mismatch for scoring_list.0.weight and scoring_list.0.bias error HOT 1
- Performance using ELECTRA and ROBERTA is significantly different from BERT HOT 3
- RuntimeError with SMART HOT 4
- Problem in SMART embedding HOT 1
- Prediction: How to find the task id? HOT 3
- Unable to get the complete model MT_DNN
- Output data in different tasks at the same time
- Project dependencies may have API risk issues
- question about task split and pretrain model
- mt-dnn on Windows?
- Readme.md is updated?
- Can you provide the pretrain files of Hugging Face?
- Older version of Pytorch unavailable HOT 1
- Code for "Targeted Adversarial Training for Natural Language Understanding"
- where is run_mt_dnn.sh HOT 1
- Problems with downloading datasets and weights HOT 3
- ddp error in fintune the task-specific like rte
- Pretrained weights for transfer learning STS benmark
- ERROR 409: Public access is not permitted on this storage account HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mt-dnn.