Comments (6)
Do we have many other duplicate tasks?
I think do, we just haven't found them yet :)
Although in this particular case what I meant was the explanations for positive and negative examples are better in task 288.
I think we're optimizing two objectives: more diversity for our annotations (more examples, more definitions, etc.) while reducing noise and redundancy (fewer duplications, fewer garbage examples, etc.) So, if the examples/explanations are not quite duplicate/noisy (as in, a human can read and be, at least partially, convinced), I'd say let's retain them.
from natural-instructions.
Task 288 has 6500 instances and task 1578 has almost 1900. Both are probably randomly sampled from Gigaword. Also, task 288 has better explanations. I guess we can keep 288 and drop 1578.
Also, task 1579 is "incorrect summary generation" and task 289 is verifying whether a summary is correct. I think task 289 makes more sense. An incorrect summary can be anything.
@danyaljj Should I drop those two?
from natural-instructions.
Here is a suggestion:
- I think there is value in having multiple task definitions. In one PR, we can change the
"Definition": ""
field to"Definition": [""]
. - In followup PRs, we can merge the tasks that are equivalent and based on the same underlying dataset.
What do you think?
from natural-instructions.
It sounds good to me! Although in this particular case what I meant was the explanations for positive and negative examples are better in task 288. Do we have many other duplicate tasks?
from natural-instructions.
@Palipoor wondering if you can help with merging these tasks?
from natural-instructions.
Sure! Will do tomorrow.
from natural-instructions.
Related Issues (20)
- Additional classification tasks HOT 1
- Indirect responses
- The problem with task 839 HOT 2
- Dialog Tasks HOT 2
- Data-to-text tasks
- Tasks 475 and 746 seem to be duplicates?
- Readme file has problems HOT 1
- Duplicate tasks: 1508 and 1509 HOT 3
- Ungrammatical Instructions for Some Tasks HOT 1
- Do you have a plan to release a standard evaluation protocol? HOT 8
- More crowdworker feedback HOT 10
- Task languages HOT 1
- Data leakage with tasks 1295 and 1640 HOT 4
- Sequence tagging HOT 5
- Task 183 - is it correct? HOT 1
- Spacing/tokenization issues HOT 4
- Task 199 (MNLI): is the data correct?
- Multi-class classification HOT 1
- about original dataset templates
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from natural-instructions.