robertostling / hnmt Goto Github PK
View Code? Open in Web Editor NEWHelsinki Neural Machine Translation system
License: GNU General Public License v3.0
Helsinki Neural Machine Translation system
License: GNU General Public License v3.0
Implement an encoder following [1].
[1] Lee, Jason, Kyunghyun Cho, and Thomas Hofmann. "Fully Character-Level Neural Machine Translation without Explicit Segmentation." arXiv preprint arXiv:1610.03017 (2016).
https://arxiv.org/pdf/1610.03017
Implement coverage in the attention mechanism, following [1].
[1] Tu, Zhaopeng, et al. "Coverage-based Neural Machine Translation." arXiv preprint arXiv:1601.04811 (2016).
http://arxiv.org/pdf/1601.04811
ERROR (theano.gof.opt): SeqOptimizer apply <theano.scan_module.scan_opt.PushOutScanOutput object at 0x7f6bb55b4198>
ERROR (theano.gof.opt): Traceback:
ERROR (theano.gof.opt): Traceback (most recent call last):
File "/home/lnmiit/.local/lib/python3.6/site-packages/Theano-0.9.0-py3.6.egg/theano/gof/opt.py", line 235, in apply
sub_prof = optimizer.optimize(fgraph)
File "/home/lnmiit/.local/lib/python3.6/site-packages/Theano-0.9.0-py3.6.egg/theano/gof/opt.py", line 87, in optimize
ret = self.apply(fgraph, *args, **kwargs)
File "/home/lnmiit/.local/lib/python3.6/site-packages/Theano-0.9.0-py3.6.egg/theano/scan_module/scan_opt.py", line 685, in apply
node = self.process_node(fgraph, node)
File "/home/lnmiit/.local/lib/python3.6/site-packages/Theano-0.9.0-py3.6.egg/theano/scan_module/scan_opt.py", line 745, in process_node
node, args)
File "/home/lnmiit/.local/lib/python3.6/site-packages/Theano-0.9.0-py3.6.egg/theano/scan_module/scan_opt.py", line 854, in push_out_inner_vars
add_as_nitsots)
File "/home/lnmiit/.local/lib/python3.6/site-packages/Theano-0.9.0-py3.6.egg/theano/scan_module/scan_opt.py", line 906, in add_nitsot_outputs
reason='scanOp_pushout_output')
File "/home/lnmiit/.local/lib/python3.6/site-packages/Theano-0.9.0-py3.6.egg/theano/gof/toolbox.py", line 391, in replace_all_validate_remove
chk = fgraph.replace_all_validate(replacements, reason)
File "/home/lnmiit/.local/lib/python3.6/site-packages/Theano-0.9.0-py3.6.egg/theano/gof/toolbox.py", line 365, in replace_all_validate
fgraph.validate()
File "/home/lnmiit/.local/lib/python3.6/site-packages/Theano-0.9.0-py3.6.egg/theano/gof/toolbox.py", line 256, in validate_
ret = fgraph.execute_callbacks('validate')
File "/home/lnmiit/.local/lib/python3.6/site-packages/Theano-0.9.0-py3.6.egg/theano/gof/fg.py", line 589, in execute_callbacks
fn(self, *args, **kwargs)
File "/home/lnmiit/.local/lib/python3.6/site-packages/Theano-0.9.0-py3.6.egg/theano/gof/toolbox.py", line 422, in validate
raise theano.gof.InconsistencyError("Trying to reintroduce a removed node")
theano.gof.fg.InconsistencyError: Trying to reintroduce a removed node
Traceback (most recent call last):
File "hnmt.py", line 1426, in
if name == 'main': main()
File "hnmt.py", line 1214, in main
'Heldout training sentences is no longer supported')
Error received when trying to train the Machine Translation System using command
python3 hnmt.py --train europarl-v7.sv-en \
--source-tokenizer word \ --target-tokenizer char \ --load-source-vocabulary vocab.sv \ --load-target-vocabulary vocab.en \ --batch-budget 32 \
--training-time 8
--log en-sv.log
--save-model en-sv.model
Averaging parameters for a single training run improves results, as does ensembling several independently trained models. We should make the decoder able to combine these two methods, perhaps it's possible to squeeze out an extra bit of performance this way. It should be quick to implement.
I would be very appreciate if you perfect the documentation.
Currently the whole training data set is loaded to RAM, but this obviously does not scale. At some point we need to fix this, if we want to train with huge corpora.
Vary the number of sentences per minibatch, so that the number of tokens (including padding) is close to optimal. The current fixed-size minibatches under-utilize the available memory when the minibatch is filled with short sentences.
Implement a two-level decoder following [1].
First a word-level decoder produces a sequence that may contain <UNK>
symbols.
Next, these <UNK>
symbols are filled in using a character-level decoder.
[1] Luong, Minh-Thang, and Christopher D. Manning. "Achieving open vocabulary neural machine translation with hybrid word-character models." arXiv preprint arXiv:1604.00788 (2016).
http://arxiv.org/pdf/1604.00788
Implement Minimum Risk Training (MRT), following [1].
[1] Shen, Shiqi, et al. "Minimum risk training for neural machine translation." arXiv preprint arXiv:1512.02433 (2015).
http://arxiv.org/pdf/1512.02433
Pretrained models would be much appreciated. Do you provide them?
If not, then you could maybe add a statement about this in the readme?
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.