kaiduohong / reptile-pytorch

Reptile-Pytorch

PyTorch implementation of the supervised learning experiments from the paper: Reptile: A Scalable Meta-Learning Algorithm: https://blog.openai.com/reptile/ , which is based on Model-Agnostic Meta-Learning (MAML): https://arxiv.org/abs/1703.03400

Ominiglot

Howto

change dataset = 'omniglot' in main.py and just run python main.py, the program will download omniglot dataset automatically. modify the value of meta_batchsz to fit your GPU memory size.

benchmark

Model	Fine Tune	5-way Acc.		20-way Acc
		1-shot	5-shot	1-shot	5-shot
MANN	N	82.8%	94.9%	-	-
Matching Nets	N	98.1%	98.9%	93.8%	98.5%
Matching Nets	Y	97.9%	98.7%	93.5%	98.7%
MAML	Y	98.7+-0.4%	99.9+-0.1%	95.8+-0.3%	98.9+-0.2%
Ours	Y	98.62%	99.52%	96.09%	98.24%

5way 1shot episode: 11580*512 finetune acc:0.990234 test acc:0.986250

5way 5shot episode: 27180*128 finetune acc:0.995625 test acc:0.995219

20way 1shot episode: 23160*128 finetune acc:0.960937 test acc:0.960898

20way 5shot episode: 11580*32 finetune acc:0.985938 test acc:0.982437

training curve

mini-Imagenet (CAN NOT WORK!)

train mini-imagenet is extremely slow, since the code train task one by one squentially.

Howto

download mini-imagenet dataset and make it looks like:

mini-imagenet/
├── images
	├── n0210891500001298.jpg  
	├── n0287152500001298.jpg 
	...
├── test.csv
├── val.csv
└── train.csv

MAML-Pytorch/
├── main.py
├── meta.py
├── Readme.md 
├── naive.md
    ...  

change dataset = 'mini-imagenet' in main.py and just run python main.py.

benchmark

Model	Fine Tune	5-way Acc.		20-way Acc
		1-shot	5-shot	1-shot	5-shot
Matching Nets	N	43.56%	55.31%	17.31%	22.69%
Meta-LSTM		43.44%	60.60%	16.70%	26.06%
MAML	Y	48.7%	63.11%	16.49%	19.29%
Ours	Y	-	-	-	-