opennmt / opennmt.github.io

OpenNMT website

This project is considered obsolete as the Torch framework is no longer maintained. If you are starting a new project, please use an alternative in the OpenNMT family: OpenNMT-tf (TensorFlow) or OpenNMT-py (PyTorch) depending on your requirements.

OpenNMT: Open-Source Neural Machine Translation

OpenNMT is a full-featured, open-source (MIT) neural machine translation system utilizing the Torch mathematical toolkit.

The system is designed to be simple to use and easy to extend, while maintaining efficiency and state-of-the-art translation accuracy. Features include:

• Speed and memory optimizations for high-performance GPU training.
• Simple general-purpose interface, only requires and source/target data files.
• C++ implementation of the translator for easy deployment.
• Extensions to allow other sequence generation tasks such as summarization and image captioning.

Installation

OpenNMT only requires a Torch installation with few dependencies.

1. Install Torch
2. Install additional packages:

luarocks install tds
luarocks install bit32 # if using LuaJIT

For other installation methods including Docker, visit the documentation.

Quickstart

OpenNMT consists of three commands:

1. Preprocess the data.

th preprocess.lua -train_src data/src-train.txt -train_tgt data/tgt-train.txt -valid_src data/src-val.txt -valid_tgt data/tgt-val.txt -save_data data/demo

2. Train the model.

th train.lua -data data/demo-train.t7 -save_model model

3. Translate sentences.

th translate.lua -model model_final.t7 -src data/src-test.txt -output pred.txt

For more details, visit the documentation.

Citation

A technical report on OpenNMT is available. If you use the system for academic work, please cite:

@ARTICLE{2017opennmt,
  author = {{Klein}, G. and {Kim}, Y. and {Deng}, Y. and {Senellart}, J. and {Rush}, A.~M.},
  title = "{OpenNMT: Open-Source Toolkit for Neural Machine Translation}",
  journal = {ArXiv e-prints},
  eprint = {1701.02810}
}

Acknowledgments

Our implementation utilizes code from the following:

• Andrej Karpathy's char-rnn repo
• Wojciech Zaremba's lstm repo
• Element rnn library

Additional resources

• Documentation
• Forum
• Gitter channel

opennmt.github.io's Issues

Korean/Japanese Translation of Docs

Hi,

I found that Korean/Japanese translation of documentations are not done yet. Only intro pages are translated. Can I contribute by translating remaining parts?

wrong pretrained model

The model to translate from de to en in file opennmt.github.io/Models.md produces a german output. But I expect an english one.

Google Analytics?

Remove Google Analytics

If you must have an analytics system, might I suggest either:

• Plausible: https://plausible.io (offers a 50% discount for FOSS projects)
• Goatcounter: https://goatcounter.com
• Matomo: https://matomo.org
• Fathom: https://usefathom.org

These solutions are all well-tested and open-source. This, of course, means they
can be independently audited for privacy and security. Instead of using a proprietary
system such as Google Analytics, perhaps we could work together to replace
it with something slightly more ethical?

The majority of privacy features in evergreen browsers block Google
Analytics from being run. I can not say the same for solutions such as Plausible,
which suggests they are trusted for their privacy by independent persons.

In that case, using Google Analytics could mean your statistics are incorrect.

document -phrase_table option

wrong place. sprry

For English Summarization, what is "replace_unk"?