Comments (5)
We don't have plans to do this in the short-term, as it complicates the input and output formatting. An alternative approach is to use subword encodings like BPE which can be wrapped around the whole system. We will release tools to do this in the near future, but in the mean time there are several that exist online (https://github.com/vteromero/byte-pair-encoding)
from opennmt.
Actually looks like BPE is coming very soon: in this PR #27
from opennmt.
@srush Thanks, unfortunately in my language pair, applying BPE is not convenient.
I just implemented it myself, and also advanced Bayesian dropout,
A Theoretically Grounded Application of Dropout in Recurrent Neural Networks and still tuning.
I am wondering why don't add this version of dropout? Paper above reports better performance than current Zaremba's dropout.
I found it can be implemented in a very simple way. Just rewrite nn.Dropout to force noise be fixed during all time steps, and tie noise tensors between clones, and also edit LSTM.lua to apply dropout to all nodes which feed to nn.Linear Nodes. I will share the result when the exp is complete.
from opennmt.
We would love a pull request for either!
from opennmt.
(added as feature request in the forum http://forum.opennmt.net/)
from opennmt.
Related Issues (20)
- Decoder.lua:446: 'for' limit must be a number HOT 1
- Error when translating with Language Model: "attempt to call method 'forwardOne' (a nil value)" HOT 2
- Meet with lua:446:'for' limit must be a number HOT 2
- Difference between using epoch and train_steps HOT 3
- Difference bteween gnmt and brnn encoder HOT 2
- WriteObject problem, bad argument #1 to 'ipairs' HOT 1
- IDEA: Process FOSS projects .po files for translation corpora
- The engine caches the data in the translation HOT 6
- How to speed up when preprocess the corpus ? HOT 1
- Embeddings: invalid line count HOT 3
- Test BLEU Score HOT 1
- share_embedding option HOT 2
- Better pre-trained PyTorch models
- Maximum length of string processed by the engine HOT 2
- unicode issue for translating Bangla to English HOT 2
- Error BPE tokenizer: tokenize.lua:87: attempt to concatenate upvalue 'line' HOT 3
- Is early stopping available in OpenNMT-py or OpenNMT-tf? HOT 1
- Is `early-stopping` available feature in OpenNMT-tf or OpenNMT-py?
- NMT, What if we do not pass input for decoder? HOT 1
- ModuleNotFoundError: No module named 'onmt.inputters' HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from opennmt.