Title: Make Virtual World Real
- This project is motivated by SimGAN. Detail is described in the report.
- models.py includes SimGAN, CycleGAN and model from this work.
From left to right: virtual, refined using SimGAN, refined using model from this work.
I also try to do using CycleGAN, see the exapmle1 and example2.
Road scene dataset:
- Synthetic images: Virtual KITTI
- Real images: KITTI Object Detection
Data directory:
- Training data:
- virtual images under
./datasets/road/trainA
- real images under
./datasets/road/trainB
- virtual images under
- Test data:
- modify test.py for the test data directory:
x_list = glob('./datasets/' + dataset + '/vkitti_1.3.1_rgb/0018/morning/*.png')
- the refined images will be saved under
./test_predictions/
- modify test.py for the test data directory:
In addition, you could also download datasets as CycleGAN paper to run the models, e.g. sh ./download_dataset.sh horse2zebra
.
- Example output of horse2zebra run using SimGAN after 20 epochs with lambda_=0.1
- Example outpu of NYU hand dataset run using SimGAN after 20 epochs with lambda_=10.0
python train.py --dataset=road --channel=3 --ratio=2 --lambda_=10.0
python test.py --dataset=road --channel=3 --ratio=2 --lambda_=10.0
- I use least square GAN instead of negative log likelihood objective.
- For tensorboard run:
tensorboard --logdir=summaries
Code modify from CycleGAN-Tensorflow-PyTorch-Simple and simulated-unsupervised-tensorflow.