alinlab / l2t-ww

Learning What and Where to Transfer (ICML 2019)

Learning What and Where to Transfer (ICML 2019)

Learning What and Where to Transfer (ICML 2019) https://arxiv.org/abs/1905.05901

Requirements

• python>=3.6
• pytorch>=1.0
• torchvision
• cuda>=9.0

Note. The reported results in our paper were obtained in the old-version pytorch (pytorch=1.0, cuda=9.0). We recently executed again the experiment commands as described below using the recent version (pytorch=1.6.0, torchvision=0.7.0, cuda=10.1), and obtained similar results as reported in the paper.

Prepare Datasets

You can download CUB-200 and Stanford Dogs datasets

• CUB-200: from http://www.vision.caltech.edu/visipedia/CUB-200-2011.html
• Stanford Dogs: http://vision.stanford.edu/aditya86/ImageNetDogs/

You need to run the below pre-processing script for DataLoader.

python cub200.py /data/CUB_200_2011
python dog.py /data/dog

Train L2T-ww

You can train L2T-ww models with the same settings in our paper.

python train_l2t_ww.py --dataset cub200 --datasplit cub200 --dataroot /data/CUB_200_2011
python train_l2t_ww.py --dataset dog --datasplit dog --dataroot /data/dog
python train_l2t_ww.py --dataset cifar100 --datasplit cifar100 --dataroot /data/ --experiment logs/cifar100_0/ --source-path logs --source-model resnet32 --source-domain tinyimagenet-200 --target-model vgg9_bn --pairs 4-0,4-1,4-2,4-3,4-4,9-0,9-1,9-2,9-3,9-4,14-0,14-1,14-2,14-3,14-4 --batchSize 128
python train_l2t_ww.py --dataset stl10 --datasplit stl10 --dataroot /data/ --experiment logs/stl10_0/ --source-path logs --source-model resnet32 --source-domain tinyimagenet-200 --target-model vgg9_bn --pairs 4-0,4-1,4-2,4-3,4-4,9-0,9-1,9-2,9-3,9-4,14-0,14-1,14-2,14-3,14-4 --batchSize 128

l2t-ww's Issues

Reproducibility issue on Stanford40-Actions

I am sincerely thankful for this shared code. I've really enjoyed the work.

While I try to reproduce your reported table, I faced an issue with the reproducibility for Stanford40-Actions (though I've confirmed CUB200, Dog and Indoor). It gives me around 53% compared to the reported score of 63.08%. Let me check with few details:

1. how do you split the training data with what percent (I tried 20% and 25%)?
2. Is there any change for hyper-parameters? (Adam optimizer for meta-networks has 1e-3 or 1e-4 learning rate? Or, the weight decay is 0 or 1e-4? (Appendix B)
3. Any other comment?

Thanks in advance.

Jin-Hwa

about folder */split/

How did you get the file like dog-test,dota-train and dot-val?
What do they represent for?
How to get it or are there any files related to other data sets？
thanks a lot!

Math details about "meta_backward"

can you give more math details about "meta_backward", I just read the code and Reverse_HG, but I couldn't understand your code.

Weight initialization in WeightNetwork and LossWeightNetwork

L2T-ww/train_l2t_ww.py

Line 80 in adf8ca3

self[-1].weight.data.zero_()

Why should the weight of linear layers in WeightNetwork be initialized as 0 here?

When I tried to run python train_l2t_ww.py --dataset cub200 --datasplit cub200 --dataroot /data/CUB_200_2011, the accuracy didn't improve with epochs at all, it keeped very low during all the training.

Questions about Bilevel optimization

Hi, I really enjoyed reading your paper, and I learned a lot!

I have a simple question. In your algorithm, when constructing the bilevel optimization framework, it seems that L_org does not distinguish between train and val. It means that the inner L_org and the outer L_org comes from the same (training) dataset. Can I learn the intuition of this setting (because most of bilevel optimization assumes exclusive train / val set for inner / outer)?

Thank you!

Multi-Source Transfer

hi, @hankook thx for nice work~but I can not find code for Experiments on Multi-Source Transfer, could you share it?

Could you specify details of TinyImageNet-ResNet32 to CIFAR100-VGG9

Hi, I would like to make sure some implemental details on TinyImageNet-ResNet32 to CIFAR100-VGG9 section including:

1. The architecture of ResNet-32 (how to modify stem conv; others could be found in Figure 3)?
2. The architecture of VGG-9 (how to modify the classifier, use BN)?
3. Any change of the other hyperparameters, the learning rate is just fixed to 0.1 even though the batch size is 2x increased, an input image is smaller, and the capacity of networks is decreased?
4. How to set aside validation splits for CIFAR-100 and STL-10?
5. Please specify the augmentation strategy for 32x32 regimes.

how can i transfer to object detection model like faster-rcnn?

Checkpoint of ResNet32 pre-trained using TinyImageNet

Could I get a checkpoint of VGG9 ResNet32 pre-trained on TinyImageNet in
https://github.com/alinlab/L2T-ww/blob/master/train_l2t_ww.py#L177 used in the paper?

Problems about meta_backward

I have the following problem with the function meta_backward:

X = X+g.mul(a[0].mul(-lr)+a[1]).sum() why should i use -lr?(https://github.com/alinlab/L2T-ww/blob/master/train/meta_optimizers.py#L134)

a_new = (a[0].mul(1-lrwd).add_(wd, a[1]).add_(p.grad.data),
a[1].mul(momentum).add_(-lrmomentum, a[0])) what does this function mean?(https://github.com/alinlab/L2T-ww/blob/master/train/meta_optimizers.py#L142-L143)

I would be very appreciated if you can help me:)