leaderj1001 / simsiam

Exploring Simple Siamese Representation Learning

Exploring Simple Siamese Representation Learning (SimSiam)

Network

Experiments

Model	Pre-training Epochs	Batch size	Dim	Linear Evaluation	Acc (%)
ResNet-18 (Paper)	800	512	2048	O	91.8
ResNet-18 (Our)	300	512	1024	O	72.49
ResNet-18	800	256	1024	O	83.93
ResNet-18		512	2048	O	wip

• plot

Usage

• Dataset (CIFAR-10)
  • Data Link

  data
    └── cifar-10-batches-py
      ├── batches.meta
      ├── data_batch_1
      ├── data_batch_2
      ├── data_batch_3
      ├── data_batch_4
      ├── data_batch_5
      ├── readme.html
      └── test_batch
  

1. Pre-training

python main.py --pretrain True

2. DownStream Task (Linear)

python main.py --checkpoints checkpoints/checkpoint_pretrain_model.pth --pretrain False

Reference

• Paper Link
• Author: Xinlei Chen, Kaiming He
• Organization: Facebook AI Research (FAIR)

simsiam's Issues

Linear Classifier

Thanks for your implementation.

Can you please explain why the linear-classifier has 2 layers? I expected just one linear-layer.

Details regarding model training

Thanks for sharing!

How do you train this model?
How many GPUs do you require and how long does it take?
Can this be run on Google Colab for CIFAR-10?

--pretrain

type of --pretrain should not be bool
you can‘t change its value through command