dedhiaparth98 / spatial-transformer-network

Spatial Transformer Network (STN) provides attention to a particular region to in an image, by doing transformation to the input image. The code in this repository does Affine transformation to image, but other transformation can be explored.

Spatial Transformer Network

Spatial Transformer Network (STN) provides attention to a particular region to in an image, by doing transformation to the input image. The code in this repository does Affine transformation to image, but other transformation can be explored. Detailed explanation of the concept is explained in the blog post

Visualizations

You can clone the repository and directly run the Visualization-STN-MNIST.ipynb file where you will see how the STN network applies transformation to the Input image. These transformations can be not only restrcited to the first layer but could be applied to other layers as well.

Below are the visualizations when applied to the input image directly

Custom Training and Model Design

If you wish to train the network, then you can run the Spatial Transformer Network.ipynb. The model will generate following graph

References

1. M. Jaderberg, K. Simonyan, A. Zisserman, K. Kavukcuoglu, Spatial Transformer Networks, CVPR, 2015

2. https://kevinzakka.github.io/2017/01/10/stn-part1/

3. https://kevinzakka.github.io/2017/01/18/stn-part2/

spatial-transformer-network's Issues

STN is not learning rotation

This is a great project, thanks!

However, I am trying to use this to learn rotation/scale/translation invariance on the affNist-dataset.
But it seems to me like this implementation does not really learn rotation. When I take a close look at your sample image, I also do not really see any rotation applied.

Is there smth I can do so that the network learns rotation, as the scaling and translating work great?