Comments (5)
Hey @zouharvi,
I've managed to reproduce your issue. Let me take a deeper look into this and I'll get back to you!
from openkiwi.
I've found the cause of this (bug). Unfortunately, in the current version of OpenKiwi, we do not support learning gaps + tags at the same time with the NuQE model.
The crash in particular happens because on the forward pass of NuQE you define the model_out as:
outputs = OrderedDict()
if self.config.predict_target:
outputs[const.TARGET_TAGS] = h
if self.config.predict_gaps:
outputs[const.GAP_TAGS] = h
if self.config.predict_source:
outputs[const.SOURCE_TAGS] = h
return outputs
We're adding the same hidden state to whatever we assume the output to be. Then, when calculating loss (where your error message comes from) we do the following:
if self.config.predict_source:
output_name = const.SOURCE_TAGS
elif self.config.predict_gaps:
output_name = const.GAP_TAGS
else:
output_name = const.TARGET_TAGS
Again, assuming we only have 1 type of output.
This means if you want to predict word + gap tags you'll have to train 2 separate models to do it.
However, the good news is that we are planning on making public several changes we've made to the OpenKiwi architecture. These changes include this change, amongst many other things, such as multi-threaded dataloading, multi-gpu, etc.
We will open an issue soon™️ to detail these changes and a tentative release date.
from openkiwi.
Hi @captainvera, thank you for your response.
Can you give us a hint at how to train the gap-only model? When we drop every (2n+1)th tag (leaving only tags for gaps), the dimension error is still there (this time the difference is 1024). (the config contains predict-gaps: true
predict-target: false
.
ValueError: Expected input batch_size (2112) to match target batch_size (1088).
In your 2019 paper (Table 1, WMT18 benchmarks), did you also have two separate models for MT and gaps? We are aiming at replicating your results, including training the models.
from openkiwi.
Kiwi automatically knows which tags are gap tags. just settings predict-target: false
and wmt18-format: true
should be enough!
If you are talking about the Openkiwi paper, then yes. On our submission to the QE-shared task we were already using a slightly different version of Kiwi where we implemented target+gaps training.
from openkiwi.
Thank you, predicting gaps now works as expected. We're looking forward to the upcoming changes.
from openkiwi.
Related Issues (20)
- TypeError: cannot unpack non-iterable NoneType object HOT 1
- The prediction process is not complete by Predictor Estimator. HOT 5
- OpenKiwi always download the tokenizer files for XLMRoberta even if a local path is configured. HOT 2
- Do openKiwi have confident score? HOT 1
- Error Pre-Training Predictor: "model -> encoder -> encode_source extra fields not permitted (type=value_error.extra)" HOT 1
- some confusions
- pkgutil.iter_modules() error: 'PosixPath' object has no attribute 'startswith'
- Got exception when import kiwi
- Seems that maximum token support for a sentence is 512?
- PicklingError: Can't pickle <class 'kiwi.data.encoders.wmt_qe_data_encoder.InputFields[PositiveInt]'>: attribute lookup InputFields[PositiveInt] on kiwi.data.encoders.wmt_qe_data_encoder failed HOT 2
- Do you need to tokenize your data when using a BERT/ROBERTA model?
- Pretrain config file
- What are source_pos and target_pos in the train_config.yaml?
- Why does it need "--model" paramter when I give a specific config? HOT 2
- What languages do the OpenKiwi support?
- some problems about data without alignments HOT 11
- I suppose that the code comment should be remove. HOT 2
- Error at Predictor Training: "Predictor is not a subclass of QESystem" HOT 2
- OSError: Can't load weights for 'xlm-roberta-base'. HOT 16
- open cannot unpack non-iterable NoneType object HOT 16
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openkiwi.