uber-research / gtn Goto Github PK

Hi,
I've been trying to reproduce the NAS results but I'm not really getting there.
I've trained a generator on CIFAR10 for 2000 iterations and then ran the architecture search with following hyperparameters:
`{
"noise_size": 128,
"inner_loop_init_lr": 0.02,
"lr": 0.0,
"final_relative_lr": 1e-1,
"inner_loop_init_momentum": 0.5,
"generator_batch_size": 128,
"meta_optimizer": "adam",
"adam_beta1": 0.9,
"adam_beta2": 0.9,
"adam_epsilon": 1e-5,
"meta_batch_size": 256,
"validation_learner_type": "enas",
"use_intermediate_losses": 16,
"num_inner_iterations": 128,
"num_meta_iterations": 800,
"dataset": "CIFAR10",
"learner_type": "enas",
"step_by_step_validation": false,
"randomize_width": false,
"use_dataset_augmentation": 1,
"generator_type": "cgtn",

"logging_period": 1,
"use_encoder": false,
"training_iterations_schedule": 0,
"gradient_block_size": 1,
"training_schedule_backwards": false,
"warmup_iterations": 0,
"enable_checkpointing": true,
"meta_learn_labels": false,
"iteration_maps_seed": true,
"load_from": "checkpoints/checkpoint_2000.pt"

}`

It works but my results are noticeably worse than yours, am I missing something?

Environment:
MacOS 10.13.6
Installed pytorch==1.2, mpi4py, fire with conda. horovod with pip

Error:
$ python train_cgtn.py .
Traceback (most recent call last):
File "train_cgtn.py", line 24, in
from models import Classifier, Generator, Encoder, sample_model
File "/Users/jayurbain/Dropbox/GTN/models.py", line 479, in
from exconv import ExConv2d
File "/Users/jayurbain/Dropbox/GTN/exconv.py", line 16, in
import custom_backward_cpp
ModuleNotFoundError: No module named 'custom_backward_cpp'

Any direction would be appreciated.
Thanks,
Jay

solved

Hello,

I found that the torchvision dataset was called, but it didn't really use the cifar10 dataset. Why the architecture search process didn't use the cifar10 dataset?