This is preliminary experiment for the implementation of Neural Turing Machines https://github.com/fumin/ntm/ . The goal is to come up with a working neural network to plug into a NTM as a controller.
Here are the results achieved with no momentum, plain SGD with learning rate 0.1, sigmoid non-linearities:
- 100 hidden units, errors -> training: 657/60000, testing: 389/10000
- 1000 hidden units, 20 iterations -> training: 269/60000, testing: 225/10000, training time took 2 hours on a Macbook Pro (Mid 2012) 2.3 GHz Intel Core i7.
To try out this implementation on real handwritings, run
cd mnist/try
go run try.go =img=eight.png
- The network correctly predicts that the number written is eight.
The images directly under the folder "mnist/try" are handwritings of mine, the ones under "mnist/try/mariusz" are Mariusz's, the ones under "mnist/try/yenting" are YenTing's.