kyushik / attention Goto Github PK

Hard attention has such low accuracy on the validation data even after 10 epochs.
Is it just used for drawing the heat map?
Why is my accuracy affected in this case compared to my soft Attention?

From the papers I learnt that Soft Attention gives to multiple objects as compared to Hard attention. To cross check this I gave my model an image having 2 digits.

Soft Attention works well by giving attention to both the digits.

But in Hard attention I expected it to give attention to just one of the digits. Though the result is same compared to Soft Attention and prediction is affected.

Does step_size denote time stamps?

Is it 'k' in this image: