tugot17 / paying-attention-to-attention

Presentation and code on basics of attention mechanism in seq2seq models

Paying attention to attention

Presentation and code about basics of the attention mechanism in seq2seq models.

We present an end-to-end trained solution for translating date information typed by a human into a date format recognizable for a computer. The implemented solution is using simple attention-based model. We also present a visualization of a trained attention weights.

Example translation below:

Code

Jupyter notebook presenting the human-machine date translation using attention-based model can be found here

You can run this code in google colab by clicking this link, but remeber to first download required data. You can do it by running the following command in the colab env.

!wget https://raw.githubusercontent.com/tugot17/Paying-Attention-to-Attention/master/human-machine.csv

Presentation

Compiled version of presentation can be found here

Presentation Latex code can be found in presentation_code folder

Credits

Presentation contains references to all images and papers used within it.

The following data sources and articles were used to create the helper notebook

Data

Human-machine dataset was taken from this repository

Pytorch official tutorial

The general idea was for seq2seq translation with attention mechanism was inspired by official pytorch tutorial

Authors

• tugot17

License

This project is licensed under the MIT License - see the LICENSE file for details