chjoanna / simgan-simple

SimGAN

Project Description

Title: Make Virtual World Real

• This project is motivated by SimGAN. Detail is described in the report.
• models.py includes SimGAN, CycleGAN and model from this work.

Example Results

From left to right: virtual, refined using SimGAN, refined using model from this work.

I also try to do using CycleGAN, see the exapmle1 and example2.

Data

Road scene dataset:

1. Synthetic images: Virtual KITTI
2. Real images: KITTI Object Detection

Data directory:

• Training data:
  • virtual images under ./datasets/road/trainA
  • real images under ./datasets/road/trainB
• Test data:
  • modify test.py for the test data directory: x_list = glob('./datasets/' + dataset + '/vkitti_1.3.1_rgb/0018/morning/*.png')
  • the refined images will be saved under ./test_predictions/

In addition, you could also download datasets as CycleGAN paper to run the models, e.g. sh ./download_dataset.sh horse2zebra.

• Example output of horse2zebra run using SimGAN after 20 epochs with lambda_=0.1
• Example outpu of NYU hand dataset run using SimGAN after 20 epochs with lambda_=10.0

Train

python train.py --dataset=road --channel=3 --ratio=2 --lambda_=10.0

Test

python test.py --dataset=road --channel=3 --ratio=2 --lambda_=10.0

Note

• I use least square GAN instead of negative log likelihood objective.
• For tensorboard run: tensorboard --logdir=summaries

Acknowledgments

Code modify from CycleGAN-Tensorflow-PyTorch-Simple and simulated-unsupervised-tensorflow.