sdv-dev / tgan Goto Github PK
View Code? Open in Web Editor NEWGenerative adversarial training for generating synthetic tabular data.
License: MIT License
tgan's People
Contributors
Stargazers
Watchers
Forkers
miranthajayatilake yoni-birman nofarpiterman sriharshams aravindsr ilevk bkmargetts nuptwuchen manuelalvarezc yuweishen soft-nougat csala baukebrenninkmeijer 3lv27 murugeshmarvel helgaarm diegobodner rbarnard kazukinoto seanchen607 upura naxda meismaomao hakanaku1234 yaserkl linxiyao swdev1202 nikolaospapachristou pythiac anthonysong98 krishna-nd wangxinqi94 nafe93 guptapiyush340 hape-xxxl afshinmarani deysaikat95 aprilxiaoyanliu sonalj96 zeta1999 ankit-da rasoolaghd pfoy hejiaxing97 tanmaysgs chrinide shlomi643 anwarkhan345 noguchi999 synce1234 eddiepbc smyng91 glederrey samlwh77 manofrancisco pandinosaurus wgarzon nipulseervi jeyasheelarakkini344 hypersyf yangchenghuang ivorylu yandai2018 pragyanaischool ylzhang29 jimmyiskandar amayadav aukerul-shuvo colber nicholasanthonys kaushikjas10 pingponglabs moanassar javihaus rpitty wandypt ronanstokes-db minmmm dochixson trellixvulnteam xiemeigongzi abhinav70291 chang-you-ren8 tdl77 mchavamaiota kjn12 yeonsu-hong shikha21s sevenxia395tgan's Issues
Some questions about TGAN
While reading the paper, I have a few questions about TGAN:
- Can TGAN generate labels as well or just features?
- How much of synthetic data should I set to generate? (For example, I have tabular data with 100.000rows and 6 columns)
- Could you give me the description of this dataset?
https://s3.amazonaws.com/hdi-demos/tgan-demo/census-train.csv
Handling the target variable in data generation (question)
Hi,
When I am trying to generate the synthetic data, do I need to treat the target column differently? Or would a correctly tuned generator take care of generating the 'correct' labels as well?
(e.g. in data augmentation for images, you don't change the target labels when you are transforming the image)
Thanks!
Why my model was retrained?
After 4 hours of training,
this is the log:
Total #vars=97, #params=8659360, size=33.03MB
[0327 10:19:47 @base.py:187] Setup callbacks graph ...
[0327 10:19:47 @base.py:205] Creating the session ...
[0327 10:19:47 @summary.py:38] Maintain moving average summary of 6 tensors in collection MOVING_SUMMARY_OPS.
[0327 10:19:47 @summary.py:75] Summarizing collection 'summaries' of size 9.
[0327 10:19:47 @graph.py:91] Applying collection UPDATE_OPS of 20 ops.
2019-03-27 10:19:47.567267: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
[0327 10:19:48 @base.py:205] Creating the session ...
[0327 10:19:48 @base.py:211] Initializing the session ...
[0327 10:19:48 @base.py:218] Graph Finalized.
[0327 10:19:48 @concurrency.py:37] Starting EnqueueThread QueueInput/input_queue ...
2019-03-27 10:19:48.785232: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
[0327 10:19:49 @base.py:250] Start Epoch 1 ...
0%| |0/10000[00:00<?,?it/s]2019-03-27 10:19:49.075548: W tensorflow/core/framework/allocator.cc:122] Allocation of 20160000 exceeds 10% of system memory.
2019-03-27 10:19:49.084332: W tensorflow/core/framework/allocator.cc:122] Allocation of 20160000 exceeds 10% of system memory.
2019-03-27 10:19:49.109645: W tensorflow/core/framework/allocator.cc:122] Allocation of 20160000 exceeds 10% of system memory.
2019-03-27 10:19:49.114194: W tensorflow/core/framework/allocator.cc:122] Allocation of 20160000 exceeds 10% of system memory.
2019-03-27 10:19:49.188510: W tensorflow/core/framework/allocator.cc:122] Allocation of 20160000 exceeds 10% of system memory.
[0327 10:19:50 @base.py:211] Initializing the session ...
[0327 10:19:50 @base.py:218] Graph Finalized.
[0327 10:19:50 @concurrency.py:37] Starting EnqueueThread QueueInput/input_queue ...
[0327 10:19:50 @base.py:250] Start Epoch 1 ...
23%|#######################################8 |2316/10000[11:16<37:13, 3.44i 10%|#################1 |1035/10000[11:15<1:40:24, 1.49it/s] 23%|######################################9 |2318/10000[11:17<38:45, 3.30it/s] 10%|#################2 |1038/10 23%|#########5 |2326/10000[11:19<38:23, 3.33it/s] 10%|#### |1039/10000[11:17<1:40:52, 1.48it/s] 2100%|##############################################################|10000/10000[48:32<00:00, 0.88it/s]
[0327 11:08:21 @base.py:260] Epoch 1 (global_step 10000) finished, time:48 minutes 32 seconds.
45%|############################3 |4501/10000[48:30<51:25, 1.78it/s][0327 11:08:21 @saver.py:77] Model saved to train_log/TGAN_synthesizer:KDD-2/model-10000.
[0327 11:08:21 @monitor.py:440] GAN_loss/discrim/accuracy_fake: 0.91
[0327 11:08:21 @monitor.py:440] GAN_loss/discrim/accuracy_real: 0.59
[0327 11:08:21 @monitor.py:440] GAN_loss/discrim/loss: 0.47944
[0327 11:08:21 @monitor.py:440] GAN_loss/gen/final-g-loss: 2.2348
[0327 11:08:21 @monitor.py:440] GAN_loss/gen/klloss: 0.12783
[0327 11:08:21 @monitor.py:440] GAN_loss/gen/loss: 2.107
[0327 11:08:21 @monitor.py:440] QueueInput/queue_size: 50
[0327 11:08:21 @base.py:250] Start Epoch 2 ...
83%|############################################################################################################################################8 |8284/10000[1:29:51<18:01, 1.59it/s]100%|###########################################################################################################################################################################|10000/10000[48:42<00:00, 3.59it/s]
[0327 11:57:04 @base.py:260] Epoch 2 (global_step 20000) finished, time:48 minutes 42 seconds.
[0327 11:57:04 @saver.py:77] Model saved to train_log/TGAN_synthesizer:KDD-2/model-20000.
[0327 11:57:04 @monitor.py:440] GAN_loss/discrim/accuracy_fake: 1
[0327 11:57:04 @monitor.py:440] GAN_loss/discrim/accuracy_real: 0.97
[0327 11:57:04 @monitor.py:440] GAN_loss/discrim/loss: 0.14634
[0327 11:57:04 @monitor.py:440] GAN_loss/gen/final-g-loss: 9.0766
[0327 11:57:04 @monitor.py:440] GAN_loss/gen/klloss: 0.28772
[0327 11:57:04 @monitor.py:440] GAN_loss/gen/loss: 8.7889
[0327 11:57:04 @monitor.py:440] QueueInput/queue_size: 50
[0327 11:57:04 @base.py:250] Start Epoch 3 ...
100%|#########################################################################################################################################################################|10000/10000[1:47:48<00:00, 0.31it/s]
[0327 12:07:39 @base.py:260] Epoch 1 (global_step 10000) finished, time:1 hour 47 minutes 48 seconds.
[0327 12:07:39 @saver.py:77] Model saved to train_log/TGAN_synthesizer:KDD-0/model-10000.
[0327 12:07:39 @monitor.py:440] GAN_loss/discrim/accuracy_fake: 1
[0327 12:07:39 @monitor.py:440] GAN_loss/discrim/accuracy_real: 0.98
[0327 12:07:39 @monitor.py:440] GAN_loss/discrim/loss: 0.25723
[0327 12:07:39 @monitor.py:440] GAN_loss/gen/final-g-loss: 2.6428
[0327 12:07:39 @monitor.py:440] GAN_loss/gen/klloss: 0.11113
[0327 12:07:39 @monitor.py:440] GAN_loss/gen/loss: 2.5316
[0327 12:07:39 @monitor.py:440] QueueInput/queue_size: 50
[0327 12:07:39 @base.py:250] Start Epoch 2 ...
100%|###########################################################################################################################################################################|10000/10000[46:23<00:00, 3.59it/s]
[0327 12:43:27 @base.py:260] Epoch 3 (global_step 30000) finished, time:46 minutes 23 seconds.
34%|##########################################################6 |3448/10000[35:47<1:08:53, 1.59it/s][0327 12:43:27 @saver.py:77] Model saved to train_log/TGAN_synthesizer:KDD-2/model-30000.
[0327 12:43:27 @monitor.py:440] GAN_loss/discrim/accuracy_fake: 1
[0327 12:43:27 @monitor.py:440] GAN_loss/discrim/accuracy_real: 1
[0327 12:43:27 @monitor.py:440] GAN_loss/discrim/loss: 0.10424
[0327 12:43:27 @monitor.py:440] GAN_loss/gen/final-g-loss: 7.3784
[0327 12:43:27 @monitor.py:440] GAN_loss/gen/klloss: 0.20924
[0327 12:43:27 @monitor.py:440] GAN_loss/gen/loss: 7.1691
[0327 12:43:27 @monitor.py:440] QueueInput/queue_size: 50
[0327 12:43:27 @base.py:250] Start Epoch 4 ...
32%|####################################################### |3198/10000[14:53<32:39, 3. 49%|##############################7 |4879/10000[50:41<56:52, 1.50it/s] 32%|####################1 |3200/10000[14:54<33:42, 3.36it/s] 32%|####################2 49%|#################### |4883/10000[50:44<55:06, 1.55it/s] 32%|#############1 |3210/10000[14:57<33:45, 3.35it/s] 32%|###########100%|##############################################################|10000/10000[48:27<00:00, 3.21it/s]
[0327 13:31:55 @base.py:260] Epoch 4 (global_step 40000) finished, time:48 minutes 27 seconds.
[0327 13:31:55 @saver.py:77] Model saved to train_log/TGAN_synthesizer:KDD-2/model-40000.
[0327 13:31:55 @monitor.py:440] GAN_loss/discrim/accuracy_fake: 0.94
[0327 13:31:55 @monitor.py:440] GAN_loss/discrim/accuracy_real: 1
[0327 13:31:55 @monitor.py:440] GAN_loss/discrim/loss: 0.19482
[0327 13:31:55 @monitor.py:440] GAN_loss/gen/final-g-loss: 3.2029
[0327 13:31:55 @monitor.py:440] GAN_loss/gen/klloss: 0.32491
[0327 13:31:55 @monitor.py:440] GAN_loss/gen/loss: 2.878
[0327 13:31:55 @monitor.py:440] QueueInput/queue_size: 50
[0327 13:31:55 @base.py:250] Start Epoch 5 ...
100%|############################################################|10000/10000[1:48:07<00:00, 1.59it/s]
[0327 13:55:47 @base.py:260] Epoch 2 (global_step 20000) finished, time:1 hour 48 minutes 7 seconds.
[0327 13:55:48 @saver.py:77] Model saved to train_log/TGAN_synthesizer:KDD-0/model-20000.
[0327 13:55:48 @monitor.py:440] GAN_loss/discrim/accuracy_fake: 0.96
[0327 13:55:48 @monitor.py:440] GAN_loss/discrim/accuracy_real: 1
[0327 13:55:48 @monitor.py:440] GAN_loss/discrim/loss: 0.18884
[0327 13:55:48 @monitor.py:440] GAN_loss/gen/final-g-loss: 7.0647
[0327 13:55:48 @monitor.py:440] GAN_loss/gen/klloss: 2.444
[0327 13:55:48 @monitor.py:440] GAN_loss/gen/loss: 4.6207
[0327 13:55:48 @monitor.py:440] QueueInput/queue_size: 50
[0327 13:55:48 @base.py:250] Start Epoch 3 ...
100%|##############################################################|10000/10000[50:46<00:00, 3.48it/s]
[0327 14:22:41 @base.py:260] Epoch 5 (global_step 50000) finished, time:50 minutes 46 seconds.
[0327 14:22:41 @saver.py:77] Model saved to train_log/TGAN_synthesizer:KDD-2/model-50000.
[0327 14:22:41 @monitor.py:440] GAN_loss/discrim/accuracy_fake: 0.99
[0327 14:22:41 @monitor.py:440] GAN_loss/discrim/accuracy_real: 1
[0327 14:22:41 @monitor.py:440] GAN_loss/discrim/loss: 0.13903
[0327 14:22:41 @monitor.py:440] GAN_loss/gen/final-g-loss: 4.2788
[0327 14:22:41 @monitor.py:440] GAN_loss/gen/klloss: 0.28196
[0327 14:22:41 @monitor.py:440] GAN_loss/gen/loss: 3.9968
[0327 14:22:41 @monitor.py:440] QueueInput/queue_size: 50
[0327 14:22:41 @base.py:264] Training has finished!
24%|##############7 |2420/10000[26:53<1:20:32, 1.57it/s]2019-03-27 14:22:41.924330: W tensorflow/core/kernels/queue_base.cc:277] _0_QueueInput/input_queue: Skipping cancelled enqueue attempt with queue not closed
`
The model was automatically retrained and it showed:
[0327 14:23:08 @collection.py:145] New collections created in tower : tf.GraphKeys.REGULARIZATION_LOSSES
[0327 14:23:08 @collection.py:164] These collections were modified but restored in : (tf.GraphKeys.SUMMARIES: 0->2)
[0327 14:23:08 @sessinit.py:90] WRN The following variables are in the checkpoint, but not found in the graph: global_step:0, optimize/beta1_power:0, optimize/beta2_power:0
2019-03-27 14:23:08.329825: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
25%|############### |2469/10000[27:20<1:07:26, 1.86it/s][0327 14:23:08 @sessinit.py:117] Restoring checkpoint from train_log/TGAN_synthesizer:KDD-2/model-40000 ...
25%|###############2 |2494/10000[27:34<1:05:35, 1.91it/s][0327 14:23:22 @develop.py:96] WRN [Deprecated] ModelDescBase._get_inputs() interface will be deprecated after 30 Mar. Use inputs() instead!
[0327 14:23:22 @develop.py:96] WRN [Deprecated] ModelDescBase._build_graph() interface will be deprecated after 30 Mar. Use build_graph() instead!
[0327 14:23:22 @registry.py:121] gen/LSTM/00/FC input: [100, 200]
[0327 14:23:22 @registry.py:129] gen/LSTM/00/FC output: [100, 500]
[0327 14:23:22 @registry.py:121] gen/LSTM/00/FC2 input: [100, 500]
[0327 14:23:22 @registry.py:129] gen/LSTM/00/FC2 output: [100, 1]
25%|###############2 |2495/10000[27:34<1:07:05, 1.86it/s]WARNING:tensorflow:From src/TGAN_synthesizer.py:71: calling softmax (from tensorflow.python.ops.nn_ops) with dim is deprecated and will be removed in a future version.
Instructions for updating:
dim is deprecated, use axis instead
[0327 14:23:22 @registry.py:121] gen/LSTM/01/FC input: [100, 200]
[0327 14:23:22 @registry.py:129] gen/LSTM/01/FC output: [100, 500]
[0327 14:23:22 @registry.py:121] gen/LSTM/01/FC2 input: [100, 500]
[0327 14:23:22 @registry.py:129] gen/LSTM/01/FC2 output: [100, 5]
[0327 14:23:22 @registry.py:121] gen/LSTM/01/FC3 input: [100, 5]
[0327 14:23:22 @registry.py:129] gen/LSTM/01/FC3 output: [100, 500]
[0327 14:23:22 @registry.py:121] gen/LSTM/02/FC input: [100, 200]
[0327 14:23:22 @registry.py:129] gen/LSTM/02/FC output: [100, 500]
[0327 14:23:22 @registry.py:121] gen/LSTM/02/FC2 input: [100, 500]
[0327 14:23:23 @registry.py:129] gen/LSTM/02/FC2 output: [100, 3]
[0327 14:23:23 @registry.py:121] gen/LSTM/02/FC3 input: [100, 3]
[0327 14:23:23 @registry.py:129] gen/LSTM/02/FC3 output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/03/FC input: [100, 200]
[0327 14:23:23 @registry.py:129] gen/LSTM/03/FC output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/03/FC2 input: [100, 500]
[0327 14:23:23 @registry.py:129] gen/LSTM/03/FC2 output: [100, 1]
[0327 14:23:23 @registry.py:121] gen/LSTM/04/FC input: [100, 200]
[0327 14:23:23 @registry.py:129] gen/LSTM/04/FC output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/04/FC2 input: [100, 500]
[0327 14:23:23 @registry.py:129] gen/LSTM/04/FC2 output: [100, 5]
[0327 14:23:23 @registry.py:121] gen/LSTM/04/FC3 input: [100, 5]
[0327 14:23:23 @registry.py:129] gen/LSTM/04/FC3 output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/05/FC input: [100, 200]
[0327 14:23:23 @registry.py:129] gen/LSTM/05/FC output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/05/FC2 input: [100, 500]
[0327 14:23:23 @registry.py:129] gen/LSTM/05/FC2 output: [100, 1]
[0327 14:23:23 @registry.py:121] gen/LSTM/06/FC input: [100, 200]
[0327 14:23:23 @registry.py:129] gen/LSTM/06/FC output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/06/FC2 input: [100, 500]
[0327 14:23:23 @registry.py:129] gen/LSTM/06/FC2 output: [100, 5]
[0327 14:23:23 @registry.py:121] gen/LSTM/06/FC3 input: [100, 5]
[0327 14:23:23 @registry.py:129] gen/LSTM/06/FC3 output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/07/FC input: [100, 200]
25%|###############2 |2496/10000[27:35<1:07:37, 1.85it/s][0327 14:23:23 @registry.py:129] gen/LSTM/07/FC output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/07/FC2 input: [100, 500]
[0327 14:23:23 @registry.py:129] gen/LSTM/07/FC2 output: [100, 1]
[0327 14:23:23 @registry.py:121] gen/LSTM/08/FC input: [100, 200]
[0327 14:23:23 @registry.py:129] gen/LSTM/08/FC output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/08/FC2 input: [100, 500]
[0327 14:23:23 @registry.py:129] gen/LSTM/08/FC2 output: [100, 5]
[0327 14:23:23 @registry.py:121] gen/LSTM/08/FC3 input: [100, 5]
[0327 14:23:23 @registry.py:129] gen/LSTM/08/FC3 output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/09/FC input: [100, 200]
[0327 14:23:23 @registry.py:129] gen/LSTM/09/FC output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/09/FC2 input: [100, 500]
[0327 14:23:23 @registry.py:129] gen/LSTM/09/FC2 output: [100, 1]
[0327 14:23:23 @registry.py:121] gen/LSTM/10/FC input: [100, 200]
[0327 14:23:23 @registry.py:129] gen/LSTM/10/FC output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/10/FC2 input: [100, 500]
[0327 14:23:23 @registry.py:129] gen/LSTM/10/FC2 output: [100, 5]
[0327 14:23:23 @registry.py:121] gen/LSTM/10/FC3 input: [100, 5]
[0327 14:23:23 @registry.py:129] gen/LSTM/10/FC3 output: [100, 500]
[0327 14:23:23 @registry.py:121] discrim/dis_fc0/fc input: [100, 33]
[0327 14:23:23 @registry.py:129] discrim/dis_fc0/fc output: [100, 500]
[0327 14:23:23 @registry.py:121] discrim/dis_fc0/fc_diversity input: [100, 500]
[0327 14:23:23 @registry.py:129] discrim/dis_fc0/fc_diversity output: [100, 100]
[0327 14:23:23 @registry.py:121] discrim/dis_fc1/fc input: [100, 510]
[0327 14:23:23 @registry.py:129] discrim/dis_fc1/fc output: [100, 500]
[0327 14:23:23 @registry.py:121] discrim/dis_fc1/fc_diversity input: [100, 500]
[0327 14:23:23 @registry.py:129] discrim/dis_fc1/fc_diversity output: [100, 100]
[0327 14:23:23 @registry.py:121] discrim/dis_fc2/fc input: [100, 510]
[0327 14:23:23 @registry.py:129] discrim/dis_fc2/fc output: [100, 500]
[0327 14:23:23 @registry.py:121] discrim/dis_fc2/fc_diversity input: [100, 500]
[0327 14:23:23 @registry.py:129] discrim/dis_fc2/fc_diversity output: [100, 100]
[0327 14:23:23 @registry.py:121] discrim/dis_fc_top input: [100, 510]
[0327 14:23:23 @registry.py:129] discrim/dis_fc_top output: [100, 1]
25%|###############2 |2497/10000[27:35<1:08:02, 1.84it/s][0327 14:23:24 @collection.py:145] New collections created in tower : tf.GraphKeys.REGULARIZATION_LOSSES
[0327 14:23:24 @collection.py:164] These collections were modified but restored in : (tf.GraphKeys.SUMMARIES: 0->2)
[0327 14:23:24 @sessinit.py:90] WRN The following variables are in the checkpoint, but not found in the graph: global_step:0, optimize/beta1_power:0, optimize/beta2_power:0
25%|###############2 |2498/10000[27:36<1:08:27, 1.83it/s]2019-03-27 14:23:24.578021: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
[0327 14:23:24 @sessinit.py:117] Restoring checkpoint from train_log/TGAN_synthesizer:KDD-2/model-50000 ...
25%|###############3 |2523/10000[27:50<1:07:24, 1.85it/s][0327 14:23:39 @logger.py:74] Argv: src/TGAN_synthesizer.py --batch_size 50 --z_dim 100 --num_gen_rnn 400 --num_gen_feature 300 --num_dis_layers 4 --num_dis_hidden 400 --learning_rate 0.001 --noise 0.1 --exp_name KDD-3 --max_epoch 5 --steps_per_epoch 10000 --data expdir/KDD/train.npz --gpu 0
25%|###############3 |2524/10000[27:51<1:11:50, 1.73it/s][0327 14:23:39 @develop.py:96] WRN [Deprecated] ModelDescBase._get_inputs() interface will be deprecated after 30 Mar. Use inputs() instead!
[0327 14:23:39 @input_source.py:221] Setting up the queue 'QueueInput/input_queue' for CPU prefetching ...
[0327 14:23:39 @develop.py:96] WRN [Deprecated] ModelDescBase._build_graph() interface will be deprecated after 30 Mar. Use build_graph() instead!
[0327 14:23:39 @registry.py:121] gen/LSTM/00/FC input: [50, 400]
[0327 14:23:39 @registry.py:129] gen/LSTM/00/FC output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/00/FC2 input: [50, 300]
[0327 14:23:39 @registry.py:129] gen/LSTM/00/FC2 output: [50, 1]
WARNING:tensorflow:From src/TGAN_synthesizer.py:71: calling softmax (from tensorflow.python.ops.nn_ops) with dim is deprecated and will be removed in a future version.
Instructions for updating:
dim is deprecated, use axis instead
[0327 14:23:39 @registry.py:121] gen/LSTM/01/FC input: [50, 400]
[0327 14:23:39 @registry.py:129] gen/LSTM/01/FC output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/01/FC2 input: [50, 300]
[0327 14:23:39 @registry.py:129] gen/LSTM/01/FC2 output: [50, 5]
[0327 14:23:39 @registry.py:121] gen/LSTM/01/FC3 input: [50, 5]
[0327 14:23:39 @registry.py:129] gen/LSTM/01/FC3 output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/02/FC input: [50, 400]
[0327 14:23:39 @registry.py:129] gen/LSTM/02/FC output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/02/FC2 input: [50, 300]
[0327 14:23:39 @registry.py:129] gen/LSTM/02/FC2 output: [50, 3]
[0327 14:23:39 @registry.py:121] gen/LSTM/02/FC3 input: [50, 3]
25%|###############4 |2525/10000[27:51<1:11:56, 1.73it/s][0327 14:23:39 @registry.py:129] gen/LSTM/02/FC3 output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/03/FC input: [50, 400]
[0327 14:23:39 @registry.py:129] gen/LSTM/03/FC output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/03/FC2 input: [50, 300]
[0327 14:23:39 @registry.py:129] gen/LSTM/03/FC2 output: [50, 1]
[0327 14:23:39 @registry.py:121] gen/LSTM/04/FC input: [50, 400]
[0327 14:23:39 @registry.py:129] gen/LSTM/04/FC output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/04/FC2 input: [50, 300]
[0327 14:23:39 @registry.py:129] gen/LSTM/04/FC2 output: [50, 5]
[0327 14:23:39 @registry.py:121] gen/LSTM/04/FC3 input: [50, 5]
[0327 14:23:39 @registry.py:129] gen/LSTM/04/FC3 output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/05/FC input: [50, 400]
[0327 14:23:39 @registry.py:129] gen/LSTM/05/FC output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/05/FC2 input: [50, 300]
[0327 14:23:39 @registry.py:129] gen/LSTM/05/FC2 output: [50, 1]
[0327 14:23:39 @registry.py:121] gen/LSTM/06/FC input: [50, 400]
[0327 14:23:39 @registry.py:129] gen/LSTM/06/FC output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/06/FC2 input: [50, 300]
[0327 14:23:39 @registry.py:129] gen/LSTM/06/FC2 output: [50, 5]
[0327 14:23:39 @registry.py:121] gen/LSTM/06/FC3 input: [50, 5]
[0327 14:23:39 @registry.py:129] gen/LSTM/06/FC3 output: [50, 300]
[0327 14:23:40 @registry.py:121] gen/LSTM/07/FC input: [50, 400]
[0327 14:23:40 @registry.py:129] gen/LSTM/07/FC output: [50, 300]
[0327 14:23:40 @registry.py:121] gen/LSTM/07/FC2 input: [50, 300]
[0327 14:23:40 @registry.py:129] gen/LSTM/07/FC2 output: [50, 1]
[0327 14:23:40 @registry.py:121] gen/LSTM/08/FC input: [50, 400]
[0327 14:23:40 @registry.py:129] gen/LSTM/08/FC output: [50, 300]
[0327 14:23:40 @registry.py:121] gen/LSTM/08/FC2 input: [50, 300]
[0327 14:23:40 @registry.py:129] gen/LSTM/08/FC2 output: [50, 5]
[0327 14:23:40 @registry.py:121] gen/LSTM/08/FC3 input: [50, 5]
[0327 14:23:40 @registry.py:129] gen/LSTM/08/FC3 output: [50, 300]
25%|###############4 |2526/10000[27:52<1:11:04, 1.75it/s][0327 14:23:40 @registry.py:121] gen/LSTM/09/FC input: [50, 400]
[0327 14:23:40 @registry.py:129] gen/LSTM/09/FC output: [50, 300]
[0327 14:23:40 @registry.py:121] gen/LSTM/09/FC2 input: [50, 300]
[0327 14:23:40 @registry.py:129] gen/LSTM/09/FC2 output: [50, 1]
[0327 14:23:40 @registry.py:121] gen/LSTM/10/FC input: [50, 400]
[0327 14:23:40 @registry.py:129] gen/LSTM/10/FC output: [50, 300]
[0327 14:23:40 @registry.py:121] gen/LSTM/10/FC2 input: [50, 300]
[0327 14:23:40 @registry.py:129] gen/LSTM/10/FC2 output: [50, 5]
[0327 14:23:40 @registry.py:121] gen/LSTM/10/FC3 input: [50, 5]
[0327 14:23:40 @registry.py:129] gen/LSTM/10/FC3 output: [50, 300]
WARNING:tensorflow:From src/TGAN_synthesizer.py:171: calling reduce_sum (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
[0327 14:23:40 @registry.py:121] discrim/dis_fc0/fc input: [50, 33]
[0327 14:23:40 @registry.py:129] discrim/dis_fc0/fc output: [50, 400]
[0327 14:23:40 @registry.py:121] discrim/dis_fc0/fc_diversity input: [50, 400]
[0327 14:23:40 @registry.py:129] discrim/dis_fc0/fc_diversity output: [50, 100]
[0327 14:23:40 @registry.py:121] discrim/dis_fc1/fc input: [50, 410]
[0327 14:23:40 @registry.py:129] discrim/dis_fc1/fc output: [50, 400]
[0327 14:23:40 @registry.py:121] discrim/dis_fc1/fc_diversity input: [50, 400]
[0327 14:23:40 @registry.py:129] discrim/dis_fc1/fc_diversity output: [50, 100]
[0327 14:23:40 @registry.py:121] discrim/dis_fc2/fc input: [50, 410]
[0327 14:23:40 @registry.py:129] discrim/dis_fc2/fc output: [50, 400]
[0327 14:23:40 @registry.py:121] discrim/dis_fc2/fc_diversity input: [50, 400]
[0327 14:23:40 @registry.py:129] discrim/dis_fc2/fc_diversity output: [50, 100]
[0327 14:23:40 @registry.py:121] discrim/dis_fc3/fc input: [50, 410]
[0327 14:23:40 @registry.py:129] discrim/dis_fc3/fc output: [50, 400]
[0327 14:23:40 @registry.py:121] discrim/dis_fc3/fc_diversity input: [50, 400]
[0327 14:23:40 @registry.py:129] discrim/dis_fc3/fc_diversity output: [50, 100]
25%|###############4 |2527/10000[27:52<1:10:00, 1.78it/s][0327 14:23:40 @registry.py:121] discrim/dis_fc_top input: [50, 410]
[0327 14:23:40 @registry.py:129] discrim/dis_fc_top output: [50, 1]
25%|###############4 |2538/10000[27:58<1:06:06, 1.88it/s][0327 14:23:46 @model_utils.py:64] Trainable Variables:
name shape dim
-------------------------------- ------------ -------
gen/LSTM/go:0 [1, 300] 300
gen/LSTM/lstm_cell/kernel:0 [1200, 1600] 1920000
gen/LSTM/lstm_cell/bias:0 [1600] 1600
gen/LSTM/00/FC/W:0 [400, 300] 120000
gen/LSTM/00/FC/b:0 [300] 300
gen/LSTM/00/FC2/W:0 [300, 1] 300
gen/LSTM/00/FC2/b:0 [1] 1
gen/LSTM/00/attw:0 [1, 1, 1] 1
gen/LSTM/01/FC/W:0 [400, 300] 120000
gen/LSTM/01/FC/b:0 [300] 300
gen/LSTM/01/FC2/W:0 [300, 5] 1500
gen/LSTM/01/FC2/b:0 [5] 5
gen/LSTM/01/FC3/W:0 [5, 300] 1500
gen/LSTM/01/FC3/b:0 [300] 300
gen/LSTM/01/attw:0 [2, 1, 1] 2
gen/LSTM/02/FC/W:0 [400, 300] 120000
gen/LSTM/02/FC/b:0 [300] 300
gen/LSTM/02/FC2/W:0 [300, 3] 900
gen/LSTM/02/FC2/b:0 [3] 3
gen/LSTM/02/FC3/W:0 [3, 300] 900
gen/LSTM/02/FC3/b:0 [300] 300
gen/LSTM/02/attw:0 [3, 1, 1] 3
gen/LSTM/03/FC/W:0 [400, 300] 120000
gen/LSTM/03/FC/b:0 [300] 300
gen/LSTM/03/FC2/W:0 [300, 1] 300
gen/LSTM/03/FC2/b:0 [1] 1
gen/LSTM/03/attw:0 [4, 1, 1] 4
gen/LSTM/04/FC/W:0 [400, 300] 120000
gen/LSTM/04/FC/b:0 [300] 300
gen/LSTM/04/FC2/W:0 [300, 5] 1500
gen/LSTM/04/FC2/b:0 [5] 5
gen/LSTM/04/FC3/W:0 [5, 300] 1500
gen/LSTM/04/FC3/b:0 [300] 300
gen/LSTM/04/attw:0 [5, 1, 1] 5
gen/LSTM/05/FC/W:0 [400, 300] 120000
gen/LSTM/05/FC/b:0 [300] 300
gen/LSTM/05/FC2/W:0 [300, 1] 300
gen/LSTM/05/FC2/b:0 [1] 1
gen/LSTM/05/attw:0 [6, 1, 1] 6
gen/LSTM/06/FC/W:0 [400, 300] 120000
gen/LSTM/06/FC/b:0 [300] 300
gen/LSTM/06/FC2/W:0 [300, 5] 1500
gen/LSTM/06/FC2/b:0 [5] 5
gen/LSTM/06/FC3/W:0 [5, 300] 1500
gen/LSTM/06/FC3/b:0 [300] 300
gen/LSTM/06/attw:0 [7, 1, 1] 7
gen/LSTM/07/FC/W:0 [400, 300] 120000
gen/LSTM/07/FC/b:0 [300] 300
gen/LSTM/07/FC2/W:0 [300, 1] 300
gen/LSTM/07/FC2/b:0 [1] 1
gen/LSTM/07/attw:0 [8, 1, 1] 8
gen/LSTM/08/FC/W:0 [400, 300] 120000
gen/LSTM/08/FC/b:0 [300] 300
gen/LSTM/08/FC2/W:0 [300, 5] 1500
gen/LSTM/08/FC2/b:0 [5] 5
gen/LSTM/08/FC3/W:0 [5, 300] 1500
gen/LSTM/08/FC3/b:0 [300] 300
gen/LSTM/08/attw:0 [9, 1, 1] 9
gen/LSTM/09/FC/W:0 [400, 300] 120000
gen/LSTM/09/FC/b:0 [300] 300
gen/LSTM/09/FC2/W:0 [300, 1] 300
gen/LSTM/09/FC2/b:0 [1] 1
gen/LSTM/09/attw:0 [10, 1, 1] 10
gen/LSTM/10/FC/W:0 [400, 300] 120000
gen/LSTM/10/FC/b:0 [300] 300
gen/LSTM/10/FC2/W:0 [300, 5] 1500
gen/LSTM/10/FC2/b:0 [5] 5
gen/LSTM/10/FC3/W:0 [5, 300] 1500
gen/LSTM/10/FC3/b:0 [300] 300
gen/LSTM/10/attw:0 [11, 1, 1] 11
discrim/dis_fc0/fc/W:0 [33, 400] 13200
discrim/dis_fc0/fc/b:0 [400] 400
discrim/dis_fc0/fc_diversity/W:0 [400, 100] 40000
discrim/dis_fc0/fc_diversity/b:0 [100] 100
discrim/dis_fc0/bn/beta:0 [410] 410
discrim/dis_fc1/fc/W:0 [410, 400] 164000
discrim/dis_fc1/fc/b:0 [400] 400
discrim/dis_fc1/fc_diversity/W:0 [400, 100] 40000
discrim/dis_fc1/fc_diversity/b:0 [100] 100
discrim/dis_fc1/bn/beta:0 [410] 410
discrim/dis_fc2/fc/W:0 [410, 400] 164000
discrim/dis_fc2/fc/b:0 [400] 400
discrim/dis_fc2/fc_diversity/W:0 [400, 100] 40000
discrim/dis_fc2/fc_diversity/b:0 [100] 100
discrim/dis_fc2/bn/beta:0 [410] 410
discrim/dis_fc3/fc/W:0 [410, 400] 164000
discrim/dis_fc3/fc/b:0 [400] 400
discrim/dis_fc3/fc_diversity/W:0 [400, 100] 40000
discrim/dis_fc3/fc_diversity/b:0 [100] 100
discrim/dis_fc3/bn/beta:0 [410] 410
discrim/dis_fc_top/W:0 [410, 1] 410
discrim/dis_fc_top/b:0 [1] 1
Total #vars=92, #params=3934650, size=15.01MB
[0327 14:23:46 @base.py:187] Setup callbacks graph ...
[0327 14:23:46 @summary.py:38] Maintain moving average summary of 6 tensors in collection MOVING_SUMMARY_OPS.
[0327 14:23:46 @summary.py:75] Summarizing collection 'summaries' of size 9.
[0327 14:23:46 @graph.py:91] Applying collection UPDATE_OPS of 16 ops.
25%|###############4 |2540/10000[27:59<1:08:34, 1.81it/s][0327 14:23:48 @base.py:205] Creating the session ...FC input: [100, 200]
[0327 14:23:23 @registry.py:129] gen/LSTM/06/FC output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/06/FC2 input: [100, 500]
[0327 14:23:23 @registry.py:129] gen/LSTM/06/FC2 output: [100, 5]
[0327 14:23:23 @registry.py:121] gen/LSTM/06/FC3 input: [100, 5]
[0327 14:23:23 @registry.py:129] gen/LSTM/06/FC3 output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/07/FC input: [100, 200]
25%|###############2 |2496/10000[27:35<1:07:37, 1.85it/s][0327 14:23:23 @registry.py:129] gen/LSTM/07/FC output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/07/FC2 input: [100, 500]
[0327 14:23:23 @registry.py:129] gen/LSTM/07/FC2 output: [100, 1]
[0327 14:23:23 @registry.py:121] gen/LSTM/08/FC input: [100, 200]
[0327 14:23:23 @registry.py:129] gen/LSTM/08/FC output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/08/FC2 input: [100, 500]
[0327 14:23:23 @registry.py:129] gen/LSTM/08/FC2 output: [100, 5]
[0327 14:23:23 @registry.py:121] gen/LSTM/08/FC3 input: [100, 5]
[0327 14:23:23 @registry.py:129] gen/LSTM/08/FC3 output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/09/FC input: [100, 200]
[0327 14:23:23 @registry.py:129] gen/LSTM/09/FC output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/09/FC2 input: [100, 500]
[0327 14:23:23 @registry.py:129] gen/LSTM/09/FC2 output: [100, 1]
[0327 14:23:23 @registry.py:121] gen/LSTM/10/FC input: [100, 200]
[0327 14:23:23 @registry.py:129] gen/LSTM/10/FC output: [100, 500]
[0327 14:23:23 @registry.py:121] gen/LSTM/10/FC2 input: [100, 500]
[0327 14:23:23 @registry.py:129] gen/LSTM/10/FC2 output: [100, 5]
[0327 14:23:23 @registry.py:121] gen/LSTM/10/FC3 input: [100, 5]
[0327 14:23:23 @registry.py:129] gen/LSTM/10/FC3 output: [100, 500]
[0327 14:23:23 @registry.py:121] discrim/dis_fc0/fc input: [100, 33]
[0327 14:23:23 @registry.py:129] discrim/dis_fc0/fc output: [100, 500]
[0327 14:23:23 @registry.py:121] discrim/dis_fc0/fc_diversity input: [100, 500]
[0327 14:23:23 @registry.py:129] discrim/dis_fc0/fc_diversity output: [100, 100]
[0327 14:23:23 @registry.py:121] discrim/dis_fc1/fc input: [100, 510]
[0327 14:23:23 @registry.py:129] discrim/dis_fc1/fc output: [100, 500]
[0327 14:23:23 @registry.py:121] discrim/dis_fc1/fc_diversity input: [100, 500]
[0327 14:23:23 @registry.py:129] discrim/dis_fc1/fc_diversity output: [100, 100]
[0327 14:23:23 @registry.py:121] discrim/dis_fc2/fc input: [100, 510]
[0327 14:23:23 @registry.py:129] discrim/dis_fc2/fc output: [100, 500]
[0327 14:23:23 @registry.py:121] discrim/dis_fc2/fc_diversity input: [100, 500]
[0327 14:23:23 @registry.py:129] discrim/dis_fc2/fc_diversity output: [100, 100]
[0327 14:23:23 @registry.py:121] discrim/dis_fc_top input: [100, 510]
[0327 14:23:23 @registry.py:129] discrim/dis_fc_top output: [100, 1]
25%|###############2 |2497/10000[27:35<1:08:02, 1.84it/s][0327 14:23:24 @collection.py:145] New collections created in tower : tf.GraphKeys.REGULARIZATION_LOSSES
[0327 14:23:24 @collection.py:164] These collections were modified but restored in : (tf.GraphKeys.SUMMARIES: 0->2)
[0327 14:23:24 @sessinit.py:90] WRN The following variables are in the checkpoint, but not found in the graph: global_step:0, optimize/beta1_power:0, optimize/beta2_power:0
25%|###############2 |2498/10000[27:36<1:08:27, 1.83it/s]2019-03-27 14:23:24.578021: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
[0327 14:23:24 @sessinit.py:117] Restoring checkpoint from train_log/TGAN_synthesizer:KDD-2/model-50000 ...
25%|###############3 |2523/10000[27:50<1:07:24, 1.85it/s][0327 14:23:39 @logger.py:74] Argv: src/TGAN_synthesizer.py --batch_size 50 --z_dim 100 --num_gen_rnn 400 --num_gen_feature 300 --num_dis_layers 4 --num_dis_hidden 400 --learning_rate 0.001 --noise 0.1 --exp_name KDD-3 --max_epoch 5 --steps_per_epoch 10000 --data expdir/KDD/train.npz --gpu 0
25%|###############3 |2524/10000[27:51<1:11:50, 1.73it/s][0327 14:23:39 @develop.py:96] WRN [Deprecated] ModelDescBase._get_inputs() interface will be deprecated after 30 Mar. Use inputs() instead!
[0327 14:23:39 @input_source.py:221] Setting up the queue 'QueueInput/input_queue' for CPU prefetching ...
[0327 14:23:39 @develop.py:96] WRN [Deprecated] ModelDescBase._build_graph() interface will be deprecated after 30 Mar. Use build_graph() instead!
[0327 14:23:39 @registry.py:121] gen/LSTM/00/FC input: [50, 400]
[0327 14:23:39 @registry.py:129] gen/LSTM/00/FC output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/00/FC2 input: [50, 300]
[0327 14:23:39 @registry.py:129] gen/LSTM/00/FC2 output: [50, 1]
WARNING:tensorflow:From src/TGAN_synthesizer.py:71: calling softmax (from tensorflow.python.ops.nn_ops) with dim is deprecated and will be removed in a future version.
Instructions for updating:
dim is deprecated, use axis instead
[0327 14:23:39 @registry.py:121] gen/LSTM/01/FC input: [50, 400]
[0327 14:23:39 @registry.py:129] gen/LSTM/01/FC output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/01/FC2 input: [50, 300]
[0327 14:23:39 @registry.py:129] gen/LSTM/01/FC2 output: [50, 5]
[0327 14:23:39 @registry.py:121] gen/LSTM/01/FC3 input: [50, 5]
[0327 14:23:39 @registry.py:129] gen/LSTM/01/FC3 output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/02/FC input: [50, 400]
[0327 14:23:39 @registry.py:129] gen/LSTM/02/FC output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/02/FC2 input: [50, 300]
[0327 14:23:39 @registry.py:129] gen/LSTM/02/FC2 output: [50, 3]
[0327 14:23:39 @registry.py:121] gen/LSTM/02/FC3 input: [50, 3]
25%|###############4 |2525/10000[27:51<1:11:56, 1.73it/s][0327 14:23:39 @registry.py:129] gen/LSTM/02/FC3 output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/03/FC input: [50, 400]
[0327 14:23:39 @registry.py:129] gen/LSTM/03/FC output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/03/FC2 input: [50, 300]
[0327 14:23:39 @registry.py:129] gen/LSTM/03/FC2 output: [50, 1]
[0327 14:23:39 @registry.py:121] gen/LSTM/04/FC input: [50, 400]
[0327 14:23:39 @registry.py:129] gen/LSTM/04/FC output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/04/FC2 input: [50, 300]
[0327 14:23:39 @registry.py:129] gen/LSTM/04/FC2 output: [50, 5]
[0327 14:23:39 @registry.py:121] gen/LSTM/04/FC3 input: [50, 5]
[0327 14:23:39 @registry.py:129] gen/LSTM/04/FC3 output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/05/FC input: [50, 400]
[0327 14:23:39 @registry.py:129] gen/LSTM/05/FC output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/05/FC2 input: [50, 300]
[0327 14:23:39 @registry.py:129] gen/LSTM/05/FC2 output: [50, 1]
[0327 14:23:39 @registry.py:121] gen/LSTM/06/FC input: [50, 400]
[0327 14:23:39 @registry.py:129] gen/LSTM/06/FC output: [50, 300]
[0327 14:23:39 @registry.py:121] gen/LSTM/06/FC2 input: [50, 300]
[0327 14:23:39 @registry.py:129] gen/LSTM/06/FC2 output: [50, 5]
[0327 14:23:39 @registry.py:121] gen/LSTM/06/FC3 input: [50, 5]
[0327 14:23:39 @registry.py:129] gen/LSTM/06/FC3 output: [50, 300]
[0327 14:23:40 @registry.py:121] gen/LSTM/07/FC input: [50, 400]
[0327 14:23:40 @registry.py:129] gen/LSTM/07/FC output: [50, 300]
[0327 14:23:40 @registry.py:121] gen/LSTM/07/FC2 input: [50, 300]
[0327 14:23:40 @registry.py:129] gen/LSTM/07/FC2 output: [50, 1]
[0327 14:23:40 @registry.py:121] gen/LSTM/08/FC input: [50, 400]
[0327 14:23:40 @registry.py:129] gen/LSTM/08/FC output: [50, 300]
[0327 14:23:40 @registry.py:121] gen/LSTM/08/FC2 input: [50, 300]
[0327 14:23:40 @registry.py:129] gen/LSTM/08/FC2 output: [50, 5]
[0327 14:23:40 @registry.py:121] gen/LSTM/08/FC3 input: [50, 5]
[0327 14:23:40 @registry.py:129] gen/LSTM/08/FC3 output: [50, 300]
25%|###############4 |2526/10000[27:52<1:11:04, 1.75it/s][0327 14:23:40 @registry.py:121] gen/LSTM/09/FC input: [50, 400]
[0327 14:23:40 @registry.py:129] gen/LSTM/09/FC output: [50, 300]
[0327 14:23:40 @registry.py:121] gen/LSTM/09/FC2 input: [50, 300]
[0327 14:23:40 @registry.py:129] gen/LSTM/09/FC2 output: [50, 1]
[0327 14:23:40 @registry.py:121] gen/LSTM/10/FC input: [50, 400]
[0327 14:23:40 @registry.py:129] gen/LSTM/10/FC output: [50, 300]
[0327 14:23:40 @registry.py:121] gen/LSTM/10/FC2 input: [50, 300]
[0327 14:23:40 @registry.py:129] gen/LSTM/10/FC2 output: [50, 5]
[0327 14:23:40 @registry.py:121] gen/LSTM/10/FC3 input: [50, 5]
[0327 14:23:40 @registry.py:129] gen/LSTM/10/FC3 output: [50, 300]
WARNING:tensorflow:From src/TGAN_synthesizer.py:171: calling reduce_sum (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
[0327 14:23:40 @registry.py:121] discrim/dis_fc0/fc input: [50, 33]
[0327 14:23:40 @registry.py:129] discrim/dis_fc0/fc output: [50, 400]
[0327 14:23:40 @registry.py:121] discrim/dis_fc0/fc_diversity input: [50, 400]
[0327 14:23:40 @registry.py:129] discrim/dis_fc0/fc_diversity output: [50, 100]
[0327 14:23:40 @registry.py:121] discrim/dis_fc1/fc input: [50, 410]
[0327 14:23:40 @registry.py:129] discrim/dis_fc1/fc output: [50, 400]
[0327 14:23:40 @registry.py:121] discrim/dis_fc1/fc_diversity input: [50, 400]
[0327 14:23:40 @registry.py:129] discrim/dis_fc1/fc_diversity output: [50, 100]
[0327 14:23:40 @registry.py:121] discrim/dis_fc2/fc input: [50, 410]
[0327 14:23:40 @registry.py:129] discrim/dis_fc2/fc output: [50, 400]
[0327 14:23:40 @registry.py:121] discrim/dis_fc2/fc_diversity input: [50, 400]
[0327 14:23:40 @registry.py:129] discrim/dis_fc2/fc_diversity output: [50, 100]
[0327 14:23:40 @registry.py:121] discrim/dis_fc3/fc input: [50, 410]
[0327 14:23:40 @registry.py:129] discrim/dis_fc3/fc output: [50, 400]
[0327 14:23:40 @registry.py:121] discrim/dis_fc3/fc_diversity input: [50, 400]
[0327 14:23:40 @registry.py:129] discrim/dis_fc3/fc_diversity output: [50, 100]
25%|###############4 |2527/10000[27:52<1:10:00, 1.78it/s][0327 14:23:40 @registry.py:121] discrim/dis_fc_top input: [50, 410]
[0327 14:23:40 @registry.py:129] discrim/dis_fc_top output: [50, 1]
25%|###############4 |2538/10000[27:58<1:06:06, 1.88it/s][0327 14:23:46 @model_utils.py:64] Trainable Variables:
name shape dim
-------------------------------- ------------ -------
gen/LSTM/go:0 [1, 300] 300
gen/LSTM/lstm_cell/kernel:0 [1200, 1600] 1920000
gen/LSTM/lstm_cell/bias:0 [1600] 1600
gen/LSTM/00/FC/W:0 [400, 300] 120000
gen/LSTM/00/FC/b:0 [300] 300
gen/LSTM/00/FC2/W:0 [300, 1] 300
gen/LSTM/00/FC2/b:0 [1] 1
gen/LSTM/00/attw:0 [1, 1, 1] 1
gen/LSTM/01/FC/W:0 [400, 300] 120000
gen/LSTM/01/FC/b:0 [300] 300
gen/LSTM/01/FC2/W:0 [300, 5] 1500
gen/LSTM/01/FC2/b:0 [5] 5
gen/LSTM/01/FC3/W:0 [5, 300] 1500
gen/LSTM/01/FC3/b:0 [300] 300
gen/LSTM/01/attw:0 [2, 1, 1] 2
gen/LSTM/02/FC/W:0 [400, 300] 120000
gen/LSTM/02/FC/b:0 [300] 300
gen/LSTM/02/FC2/W:0 [300, 3] 900
gen/LSTM/02/FC2/b:0 [3] 3
gen/LSTM/02/FC3/W:0 [3, 300] 900
gen/LSTM/02/FC3/b:0 [300] 300
gen/LSTM/02/attw:0 [3, 1, 1] 3
gen/LSTM/03/FC/W:0 [400, 300] 120000
gen/LSTM/03/FC/b:0 [300] 300
gen/LSTM/03/FC2/W:0 [300, 1] 300
gen/LSTM/03/FC2/b:0 [1] 1
gen/LSTM/03/attw:0 [4, 1, 1] 4
gen/LSTM/04/FC/W:0 [400, 300] 120000
gen/LSTM/04/FC/b:0 [300] 300
gen/LSTM/04/FC2/W:0 [300, 5] 1500
gen/LSTM/04/FC2/b:0 [5] 5
gen/LSTM/04/FC3/W:0 [5, 300] 1500
gen/LSTM/04/FC3/b:0 [300] 300
gen/LSTM/04/attw:0 [5, 1, 1] 5
gen/LSTM/05/FC/W:0 [400, 300] 120000
gen/LSTM/05/FC/b:0 [300] 300
gen/LSTM/05/FC2/W:0 [300, 1] 300
gen/LSTM/05/FC2/b:0 [1] 1
gen/LSTM/05/attw:0 [6, 1, 1] 6
gen/LSTM/06/FC/W:0 [400, 300] 120000
gen/LSTM/06/FC/b:0 [300] 300
gen/LSTM/06/FC2/W:0 [300, 5] 1500
gen/LSTM/06/FC2/b:0 [5] 5
gen/LSTM/06/FC3/W:0 [5, 300] 1500
gen/LSTM/06/FC3/b:0 [300] 300
gen/LSTM/06/attw:0 [7, 1, 1] 7
gen/LSTM/07/FC/W:0 [400, 300] 120000
gen/LSTM/07/FC/b:0 [300] 300
gen/LSTM/07/FC2/W:0 [300, 1] 300
gen/LSTM/07/FC2/b:0 [1] 1
gen/LSTM/07/attw:0 [8, 1, 1] 8
gen/LSTM/08/FC/W:0 [400, 300] 120000
gen/LSTM/08/FC/b:0 [300] 300
gen/LSTM/08/FC2/W:0 [300, 5] 1500
gen/LSTM/08/FC2/b:0 [5] 5
gen/LSTM/08/FC3/W:0 [5, 300] 1500
gen/LSTM/08/FC3/b:0 [300] 300
gen/LSTM/08/attw:0 [9, 1, 1] 9
gen/LSTM/09/FC/W:0 [400, 300] 120000
gen/LSTM/09/FC/b:0 [300] 300
gen/LSTM/09/FC2/W:0 [300, 1] 300
gen/LSTM/09/FC2/b:0 [1] 1
gen/LSTM/09/attw:0 [10, 1, 1] 10
gen/LSTM/10/FC/W:0 [400, 300] 120000
gen/LSTM/10/FC/b:0 [300] 300
gen/LSTM/10/FC2/W:0 [300, 5] 1500
gen/LSTM/10/FC2/b:0 [5] 5
gen/LSTM/10/FC3/W:0 [5, 300] 1500
gen/LSTM/10/FC3/b:0 [300] 300
gen/LSTM/10/attw:0 [11, 1, 1] 11
discrim/dis_fc0/fc/W:0 [33, 400] 13200
discrim/dis_fc0/fc/b:0 [400] 400
discrim/dis_fc0/fc_diversity/W:0 [400, 100] 40000
discrim/dis_fc0/fc_diversity/b:0 [100] 100
discrim/dis_fc0/bn/beta:0 [410] 410
discrim/dis_fc1/fc/W:0 [410, 400] 164000
discrim/dis_fc1/fc/b:0 [400] 400
discrim/dis_fc1/fc_diversity/W:0 [400, 100] 40000
discrim/dis_fc1/fc_diversity/b:0 [100] 100
discrim/dis_fc1/bn/beta:0 [410] 410
discrim/dis_fc2/fc/W:0 [410, 400] 164000
discrim/dis_fc2/fc/b:0 [400] 400
discrim/dis_fc2/fc_diversity/W:0 [400, 100] 40000
discrim/dis_fc2/fc_diversity/b:0 [100] 100
discrim/dis_fc2/bn/beta:0 [410] 410
discrim/dis_fc3/fc/W:0 [410, 400] 164000
discrim/dis_fc3/fc/b:0 [400] 400
discrim/dis_fc3/fc_diversity/W:0 [400, 100] 40000
discrim/dis_fc3/fc_diversity/b:0 [100] 100
discrim/dis_fc3/bn/beta:0 [410] 410
discrim/dis_fc_top/W:0 [410, 1] 410
discrim/dis_fc_top/b:0 [1] 1
Total #vars=92, #params=3934650, size=15.01MB
[0327 14:23:46 @base.py:187] Setup callbacks graph ...
[0327 14:23:46 @summary.py:38] Maintain moving average summary of 6 tensors in collection MOVING_SUMMARY_OPS.
[0327 14:23:46 @summary.py:75] Summarizing collection 'summaries' of size 9.
[0327 14:23:46 @graph.py:91] Applying collection UPDATE_OPS of 16 ops.
25%|###############4 |2540/10000[27:59<1:08:34, 1.81it/s][0327 14:23:48 @base.py:205] Creating the session ...
Why this problem occured? Please explain it to me
Generated invalid values
As you recommended me from this issue #8
I have increased steps_per_epoch to get better performance.
Here is my demo_config.json file:
[
{
"name": "KDD2",
"num_random_search": 2,
"train_csv": "data/KDD2.csv",
"continuous_cols": [0, 2, 3, 4, 5],
"epoch": 5,
"steps_per_epoch": 10000,
"output_epoch": 3,
"sample_rows": 10000
}
]
This is some raw lines of real data:
6 features and 1 label (duration: numerical, protocol_type: categorical, Src_bytes: numerical, Dst_bytes: numerical, Count: numerical, Srv_Count: numerical, label: categorical)
0 | tcp | 491 | 0 | 2 | 2 | normal
0 | udp | 146 | 0 | 13 | 1 | normal
0 | tcp | 0 | 0 | 123 | 6 | neptune
0 | tcp | 232 | 8153 | 5 | 5 | normal
0 | tcp | 199 | 420 | 30 | 32 | normal
0 | tcp | 0 | 0 | 121 | 19 | neptune
0 | tcp | 0 | 0 | 166 | 9 | neptune
0 | tcp | 0 | 0 | 117 | 16 | neptune
0 | tcp | 0 | 0 | 270 | 23 | neptune
0 | tcp | 0 | 0 | 133 | 8 | neptune
0 | tcp | 0 | 0 | 205 | 12 | neptune
0 | tcp | 0 | 0 | 199 | 3 | neptune
0 | tcp | 287 | 2251 | 3 | 7 | normal
0 | tcp | 334 | 0 | 2 | 2 | warezclient
And this is some raw lines of synthetic data:
0.002 | tcp | 15.6864237281966 | -0.000841004610062 | -0.42354229148823 | 0.441695400314227 | normal
0.002 | tcp | -183.218743680363 | -2308.39114875675 | 5.48673165692999 | -1.18766990154038 | normal
0.002 | tcp | -399.979889013889 | -2105.44411822541 | 7.0171823258696 | 1.83754950308268 | normal
0.002 | tcp | -420.751157266587 | -0.001472987532616 | 71.0798127335866 | 1.55743594353591 | neptune
0.002 | tcp | -246.378333662602 | -2064.23489803376 | -5.96447035856112 | 0.496180489358171 | normal
0.002 | tcp | -426.382924656137 | -0.001377933382988 | 269.354386076065 | 11.9486066341106 | neptune
0.002 | tcp | -425.43443563932 | -0.001369299530983 | 253.264007832 | 9.32537144826656 | neptune
0.002 | tcp | -308.170786425972 | 2792.1898662539 | -27.2248683703409 | 12.4523407478434 | normal
0.002 | tcp | -278.802092242666 | -2705.0595386708 | -24.7928237324707 | 11.2812369599101 | normal
0.002 | tcp | -246.509892156868 | 338.573147359628 | 7.0587820011061 | 0.077516127687718 | normal
0.002 | tcp | -285.097663499712 | -3018.49459199134 | 0.874986006445247 | -0.097756666917805 | normal
0.002 | tcp | -423.83391051841 | -0.001384188771248 | 186.17462931108 | 11.1207655780099 | neptune
0.001999954342842 | icmp | -425.348017265352 | -0.001310020565987 | -1.62019953864556 | 17.7264553210107 | ipsweep
As you can see, the result is bad actually. In the real data there are no negative values, but TGAN generated a lot of negative values which makes no sense. Because Src_bytes, Dst_bytes, Count, Srv_Count cannot be negative.
Extend the CLI
Currently the CLI only support the research command, used to keep backwards compatibility.
We should extend its funcionality to allow to fit and sample directly from the command line.
Fixing random seed to reproduce the same results
A question about your TGAN code. We need to avoid any randomness for reproducibility purposes. We were wondering if there is a way to avoid this? We have tried to set the random seed, but it seems like there is randomness involved somewhere that we can not control over. We will appreciate your response.
How to pass TGANModel hyper parameters as a dictionary
I am using TGAN to synthesize new data based on an exclusive dataset. I want to tune the TGAN model using a Bayesian approach by means of "hyperopt" library. For this purpose, I need run the TGAN model while the hyperparameters are passed as a dictionary such as below:
tgan = TGANModel(continuous_columns, params, max_epoch=4, steps_per_epoch= 100, batch_size=16, restore_session=False)
where params is a dictionary including some hyperparameters:
params = {'learning_rate': 0.002,
'z_dim': 50}
Is there anyway I can pass the hyperparameters using such a dictionary as an argument in the TGANModel function? Should I use JSON file for this purpose? If yes, could you please help me how?
Sampling method parameters are unclear and it crashes Google Colab
Hi,
I am trying out this excellent implementation of tabular-GAN
While the parameters for the fit are well illustrated in the docs and the Jupyter notebook example, the sample method parameters are not.
Currently, the sample method runs very slow. That may be fine for a GAN to generate data, but is there a way to control how much iterations it will run before generating the data? Can some more documentation be added for the sample method?
For example, I was just trying out your example Notebook in Google Colab and the instance crashed after using all the Memory!
Fitting was fine. I changed the number of steps/epochs to 1000. Crash happened after 44 minutes of GPU compute for sampling. Output log showed 144328it [44:19, 54.66it/s]
Is this normal?
tarfile.ReadError: unexpected end of data
I'm running tgan in my MacBook. Python version 3.7. Running a Conda environment with tgan all the dependencies installed. I fit the data and saved the model to a specified path. I ran another script to load the saved model. Verified the path and filename. While running the line "new_tgan = TGANModel.load(model_path)", I'm getting the error:
tarinfo = self.next()
File "/Users/<...>/envs/tgan_env/lib/python3.7/tarfile.py", line 2283, in next
raise ReadError("unexpected end of data")
tarfile.ReadError: unexpected end of data
Error with large number of columns
Hi @ManuelAlvarezC !
I am really intrigued to work with TGAN and it is my first project working with a GAN in general as well.
Recently I trained it with tabular data that contains 1000+ columns and it gave the error:
InvalidArgumentError: Cannot serialize protocol buffer of type tensorflow.GraphDef as the serialized size (3015923402bytes) would be larger than the limit (2147483647 bytes)
I understand I need to go with dimensionality reduction if I want to train the TGAN with this particular dataset but there are correlations among the columns that will be lost if I use technique like PCA.
Is there any way to work with TGAN with a large number of columns ?
Regards,
Nabaruna
Support num_samples < batch_size
The current implementation ends up in an infinite loop if the num_samples is lower than the batch_size.
This should be prevented by either always sampling at least one batch, or making the batch_size equal to the num_samples if the later is lower.
Number of epochs does not match your homepage docs and example
Hi,
Thanks for this excellent implementation. I am trying it out now.
Your homepage docs and the Jupyter notebook example say max_epoch=100 but in fact, in the latest install, it is set to 5.
Please reconcile the descriptions. The model is computationally heavy, so it is good to have a realistic expectation about the epochs to run before one starting fitting :-)
InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [4860259,100] rhs shape= [240,100]
Hi,
I am using Tensorflow Version: 1.14.0
My input is 40 continuous columns with 6948 rows.
And the arguments to the TGANModel are something like this:
tgan = TGANModel(continuous_columns,
output='output',
gpu='/gpu:0',
max_epoch=5,
steps_per_epoch=1000,
save_checkpoints=False,
restore_session=True,
batch_size=50,
z_dim=50,
noise=0.2,
l2norm=0.00001,
learning_rate=0.001,
num_gen_rnn=100,
num_gen_feature=100,
num_dis_layers=1,
num_dis_hidden=100,
optimizer='AdamOptimizer')
### Now I am getting this error, any suggestion what may have gone wrong and what to be done to fix this?
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [4860259,100] rhs shape= [240,100]
[[{{node 140201455674712/Assign_28}}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/naguha/anaconda3/envs/tgantest/lib/python3.7/site-packages/tensorflow/python/training/saver.py", line 1286, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/naguha/anaconda3/envs/tgantest/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 950, in run
run_metadata_ptr)
File "/home/naguha/anaconda3/envs/tgantest/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1173, in _run
feed_dict_tensor, options, run_metadata)
File "/home/naguha/anaconda3/envs/tgantest/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1350, in _do_run
run_metadata)
File "/home/naguha/anaconda3/envs/tgantest/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1370, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [4860259,100] rhs shape= [240,100]
[[node 140201455674712/Assign_28 (defined at /home/naguha/anaconda3/envs/tgantest/lib/python3.7/site-packages/tensorpack/tfutils/sessinit.py:111) ]]
Errors may have originated from an input operation.
Input Source operations connected to node 140201455674712/Assign_28:
discrim/dis_fc0/fc/W/Adam_1 (defined at /home/naguha/anaconda3/envs/tgantest/lib/python3.7/site-packages/tgan/trainer.py:56)
Thanks,
Nabaruna
Usage of KL divergence
I have some questions regarding the paper more than the code specifically. Why did you choose to add the KL divergence as additional loss metric in this way?
More specifically:
- Why KL divergence instead of any other (JS, etc.). Am i correct in understanding that your loss essentially consists of
loss = JS + KL_colwhere JS represents the original GAN loss which minimizes the JS divergence? - If i'm correct, this is done per column. I can see why you would want to model the per column distributions, but shouldn't this already be an result of minimizing the original GAN loss?
- Why did you not choose to use a improved already existing loss function like for example Wasserstein?
I've been experimenting with adapting TGAN to use WGAN-GP, but so far, my results have been worse than the original. Variable distributions and column correlations are worse. Given the general gap in GAN and WGAN-GP losses in stability and mode collapse, I was surprised your adaptation works so well.
Do you have any idea why TGAN would not work well with WGAN-GP, compared to other tasks?
Project scaffolding
Create a project scaffold using https://github.com/DAI-Lab/cookiecutter-pypackage.
Question about the c_i,j normalisation
Hello,
I have a question regarding the c_i,j normalization + clipping. The clustering of numerical variables in the multimodal case seems really interesting, but why not just estimating the probability to come from every cluster for variable X, then take the maximum probability and generate the X value given the generated cluster. I don't get the intuition behind this point of methodology.
Thanks in advance, Best. Aurélia
Random search crash at the end? due to NaN ? on classifier.fit(..)
Hi,
Thanks for the great paper and code!.
-
I'm trying to reproduce your results for census and covertype datasets, i wasn't able with the out of the box settings, so i tried your random search but ran into this issue.
This issue is about that. -
I would love to know your parameters settings for census data and covertype.
As they weren't mentioned. -
Also where can i find the preprocessing code for these datasets as the given ones
http://hdi-project-tgan.s3.amazonaws.com/index.html
Have already been processed. There's also no test split - where can i find it? it's not in the link provided.
The section: https://dai-lab.github.io/TGAN/readme.html#input-format
Only say:
The input data for this fitting process has to be a single table that satisfies the following rules:
Has no missing values.
Has columns of types int, float, str or bool.
Each column contains data of only one type.
How did you impute data if was missing? what scaling you've used, etc.. as i'm trying to reproduce.
- I didn't see the discriminator converges to "fake accuracy" 0.5 and "real accuracy" 0.5 at any of my runs. How come? tried to read here in issues, the papers, code.. how come?
Python version: 3.5.3
Operating System: 4.9.0-9-amd64 #1 SMP Debian 4.9.168-1+deb9u4 (2019-07-19) x86_64 GNU/Linux - on GCP
TensorFlow version: 1.14.0
nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2018 NVIDIA Corporation Built on Sat_Aug_25_21:08:01_CDT_2018 Cuda compilation tools, release 10.0, V10.0.130
GPU:
Description
I've ran
tgan experiments config_hpo.json results.jsons
This was the outcome:
What I Did
config_hpo.json looks:
Seems this is the line that raises the error because of NaN ?
https://github.com/DAI-Lab/TGAN/blob/51243cc705046538683d41738d1bb9cd22871f03/tgan/research/evaluation.py#L68
Thank you!
Understand parameters in exp-params.json file
I am reading parameters in exp-params.json file. There are some parameters I don't understand them completely. Could you please explain it to me?
"--noise", "0.3"
"--z_dim", "50"
"--num_gen_rnn","100"
"--num_gen_feature", "500"
"--num_dis_layers", "5"
Why restoring checkpoint is run on CPU not GPU?
This is my question, which has been posted on SOF https://stackoverflow.com/questions/55755668/why-restoring-checkpoint-is-run-on-cpu-not-gpu
Could you please explain it to me?
No such file or directory
Why i faced this error? I thought that synthetic1_4.npz is automatically generated
FileNotFoundError: [Errno 2] No such file or directory: 'expdir/KDD/synthetic1_4.npz'
Negative samples
My original data's columns contains some sparse negative columns and some zeros ....After generation it becomes like a dense negative columns and there is no zeros ...instead of zeros they are like constant negative samples...How can i overcome this sampling?
Windows 10 FileNotFoundError: [Errno 2]
Running TGAN on windows 10 and getting the following error:
Traceback (most recent call last): File "src/launcher.py", line 134, in <module> run_experiment(task) File "src/launcher.py", line 108, in run_experiment evaluate_res = pool.map(evaluate_worker, commands) File "C:\Users\David Leon\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 288, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "C:\Users\David Leon\AppData\Local\Programs\Python\Python36\lib\multiprocessing\pool.py", line 670, in get raise self._value FileNotFoundError: [Errno 2] No such file or directory: 'expdir\\census\\synthetic0_-1.npz'
I can't find the reason why the syntetic.npz file wasn't created since the training phase
had finished.
This is the error I get right after the training finished:
2019-01-15 13:52:40.119025: W tensorflow/core/kernels/queue_base.cc:277] _0_QueueInput/input_queue: Skipping cancelled enqueue attempt with queue not closed Traceback (most recent call last): File "src/TGAN_synthesizer.py", line 314, in <module> sample(args.sample, Model(), args.load, output_filename=args.output) File "src/TGAN_synthesizer.py", line 235, in sample session_init=get_model_loader(model_path), File "C:\Users\David Leon\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorpack\tfutils\sessinit.py", line 262, in get_model_loader return SaverRestore(filename) File "C:\Users\David Leon\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorpack\tfutils\sessinit.py", line 107, in __init__ model_path = get_checkpoint_path(model_path) File "C:\Users\David Leon\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorpack\tfutils\varmanip.py", line 182, in get_checkpoint_path assert tf.gfile.Exists(model_path) or tf.gfile.Exists(model_path + '.index'), model_path AssertionError: train_log/TGAN_synthesizer:census-4/model--1000 Traceback (most recent call last): File "src/TGAN_synthesizer.py", line 314, in <module> sample(args.sample, Model(), args.load, output_filename=args.output) File "src/TGAN_synthesizer.py", line 235, in sample session_init=get_model_loader(model_path), File "C:\Users\David Leon\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorpack\tfutils\sessinit.py", line 262, in get_model_loader return SaverRestore(filename) File "C:\Users\David Leon\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorpack\tfutils\sessinit.py", line 107, in __init__ model_path = get_checkpoint_path(model_path) File "C:\Users\David Leon\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorpack\tfutils\varmanip.py", line 182, in get_checkpoint_path assert tf.gfile.Exists(model_path) or tf.gfile.Exists(model_path + '.index'), model_path AssertionError: train_log/TGAN_synthesizer:census-4/model-0 Traceback (most recent call last): File "src/TGAN_synthesizer.py", line 314, in <module> sample(args.sample, Model(), args.load, output_filename=args.output) File "src/TGAN_synthesizer.py", line 235, in sample session_init=get_model_loader(model_path), File "C:\Users\David Leon\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorpack\tfutils\sessinit.py", line 262, in get_model_loader return SaverRestore(filename) File "C:\Users\David Leon\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorpack\tfutils\sessinit.py", line 107, in __init__ model_path = get_checkpoint_path(model_path) File "C:\Users\David Leon\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorpack\tfutils\varmanip.py", line 182, in get_checkpoint_path assert tf.gfile.Exists(model_path) or tf.gfile.Exists(model_path + '.index'), model_path AssertionError: train_log/TGAN_synthesizer:census-4/model-1000
Thanks in advance for your time and your help.
Error with small sample size data
Dear Sir,
I have used a sample size to be 100 rows and set the parameters like this,
import pandas as pd
from tgan.model import TGANModel
data = pd.read_csv('1111112.csv')
continuous_columns = [0, 1, 2, 4]
tgan = TGANModel(
continuous_columns,
output='output',
max_epoch=10,
steps_per_epoch=10,
save_checkpoints=True,
restore_session=True,
batch_size=16,
z_dim=20,
noise=0.2,
l2norm=0.00001,
learning_rate=0.001,
num_gen_rnn=10,
num_gen_feature=10,
num_dis_layers=1,
num_dis_hidden=10,
optimizer='AdamOptimizer'
)
#help(tgan)
tgan.fit(data)It has an error:
Traceback (most recent call last):
File "<ipython-input-140-25e5d1cea3b4>", line 25, in <module>
tgan.fit(data)
File "D:\anaco\lib\site-packages\tgan\model.py", line 690, in fit
input_queue=input_queue,
File "D:\anaco\lib\site-packages\tgan\trainer.py", line 30, in __init__
cbs = input_queue.setup(inputs_desc)
File "D:\anaco\lib\site-packages\tensorpack\utils\argtools.py", line 176, in wrapper
return func(*args, **kwargs)
File "D:\anaco\lib\site-packages\tensorpack\input_source\input_source_base.py", line 99, in setup
self._setup(input_signature)
File "D:\anaco\lib\site-packages\tensorpack\input_source\input_source.py", line 214, in _setup
self._input_placehdrs = [build_or_reuse_placeholder(v) for v in inputs]
File "D:\anaco\lib\site-packages\tensorpack\input_source\input_source.py", line 214, in <listcomp>
self._input_placehdrs = [build_or_reuse_placeholder(v) for v in inputs]
File "D:\anaco\lib\site-packages\tensorpack\graph_builder\model_desc.py", line 35, in build_or_reuse_placeholder
"Tensor {} exists but is not compatible with the signature!".format(tensor)
AssertionError: Tensor Tensor("input00value:0", shape=(200, 1), dtype=float32) exists but is not compatible with the signature!
Could you give some suggestions? Thanks.
TGAN crashing at Epoch 1
Hi,
I am facing this issue for some time and not able to fix this.
- Python version: 3.7
- Operating System: Linux
- TensorFlow version: 1.14.0
- CUDA version: 10.0
Description
I keep getting this warning and then the execution crashes at Epoch 1.
What I Did
import tensorflow as tf
if tf.test.gpu_device_name():
print('Default GPU Device: {}'.format(tf.test.gpu_device_name()))
else:
print("Please install GPU version of TF")
And it shows tf is using GPU fine.
2019-10-03 13:11:01.720688: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-10-03 13:11:01.768834: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2596780000 Hz
2019-10-03 13:11:01.771431: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x56157647a930 executing computations on platform Host. Devices:
2019-10-03 13:11:01.771460: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>
2019-10-03 13:11:01.772877: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1
2019-10-03 13:11:04.249822: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: Tesla M60 major: 5 minor: 2 memoryClockRate(GHz): 1.1775
pciBusID: 0000:04:00.0
2019-10-03 13:11:04.250926: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 1 with properties:
name: Tesla M60 major: 5 minor: 2 memoryClockRate(GHz): 1.1775
pciBusID: 0000:05:00.0
2019-10-03 13:11:04.251999: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 2 with properties:
name: Tesla M60 major: 5 minor: 2 memoryClockRate(GHz): 1.1775
pciBusID: 0000:09:00.0
2019-10-03 13:11:04.253103: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 3 with properties:
name: Tesla M60 major: 5 minor: 2 memoryClockRate(GHz): 1.1775
pciBusID: 0000:0a:00.0
2019-10-03 13:11:04.254193: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 4 with properties:
name: Tesla M60 major: 5 minor: 2 memoryClockRate(GHz): 1.1775
pciBusID: 0000:85:00.0
2019-10-03 13:11:04.255276: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 5 with properties:
name: Tesla M60 major: 5 minor: 2 memoryClockRate(GHz): 1.1775
pciBusID: 0000:86:00.0
2019-10-03 13:11:04.255566: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2019-10-03 13:11:04.256938: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2019-10-03 13:11:04.258142: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2019-10-03 13:11:04.258427: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2019-10-03 13:11:04.260019: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2019-10-03 13:11:04.261283: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2019-10-03 13:11:04.265096: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2019-10-03 13:11:04.277832: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0, 1, 2, 3, 4, 5
2019-10-03 13:11:04.277873: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2019-10-03 13:11:04.284987: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-10-03 13:11:04.285005: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 1 2 3 4 5
2019-10-03 13:11:04.285013: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N Y Y Y N N
2019-10-03 13:11:04.285018: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 1: Y N Y Y N N
2019-10-03 13:11:04.285023: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 2: Y Y N Y N N
2019-10-03 13:11:04.285028: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 3: Y Y Y N N N
2019-10-03 13:11:04.285033: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 4: N N N N N Y
2019-10-03 13:11:04.285040: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 5: N N N N Y N
2019-10-03 13:11:04.293727: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:0 with 7647 MB memory) -> physical GPU (device: 0, name: Tesla M60, pci bus id: 0000:04:00.0, compute capability: 5.2)
2019-10-03 13:11:04.296282: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:1 with 7647 MB memory) -> physical GPU (device: 1, name: Tesla M60, pci bus id: 0000:05:00.0, compute capability: 5.2)
2019-10-03 13:11:04.298803: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:2 with 7647 MB memory) -> physical GPU (device: 2, name: Tesla M60, pci bus id: 0000:09:00.0, compute capability: 5.2)
2019-10-03 13:11:04.301310: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:3 with 7647 MB memory) -> physical GPU (device: 3, name: Tesla M60, pci bus id: 0000:0a:00.0, compute capability: 5.2)
2019-10-03 13:11:04.303979: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:4 with 7647 MB memory) -> physical GPU (device: 4, name: Tesla M60, pci bus id: 0000:85:00.0, compute capability: 5.2)
2019-10-03 13:11:04.306456: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:5 with 7647 MB memory) -> physical GPU (device: 5, name: Tesla M60, pci bus id: 0000:86:00.0, compute capability: 5.2)
2019-10-03 13:11:04.310204: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x56157ab4cab0 executing computations on platform CUDA. Devices:
2019-10-03 13:11:04.310223: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): Tesla M60, Compute Capability 5.2
2019-10-03 13:11:04.310229: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (1): Tesla M60, Compute Capability 5.2
2019-10-03 13:11:04.310234: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (2): Tesla M60, Compute Capability 5.2
2019-10-03 13:11:04.310239: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (3): Tesla M60, Compute Capability 5.2
2019-10-03 13:11:04.310244: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (4): Tesla M60, Compute Capability 5.2
2019-10-03 13:11:04.310249: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (5): Tesla M60, Compute Capability 5.2
2019-10-03 13:11:04.314251: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties:
name: Tesla M60 major: 5 minor: 2 memoryClockRate(GHz): 1.1775
pciBusID: 0000:04:00.0
2019-10-03 13:11:04.315484: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 1 with properties:
name: Tesla M60 major: 5 minor: 2 memoryClockRate(GHz): 1.1775
pciBusID: 0000:05:00.0
2019-10-03 13:11:04.316567: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 2 with properties:
name: Tesla M60 major: 5 minor: 2 memoryClockRate(GHz): 1.1775
pciBusID: 0000:09:00.0
2019-10-03 13:11:04.317632: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 3 with properties:
name: Tesla M60 major: 5 minor: 2 memoryClockRate(GHz): 1.1775
pciBusID: 0000:0a:00.0
2019-10-03 13:11:04.318705: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 4 with properties:
name: Tesla M60 major: 5 minor: 2 memoryClockRate(GHz): 1.1775
pciBusID: 0000:85:00.0
2019-10-03 13:11:04.319780: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 5 with properties:
name: Tesla M60 major: 5 minor: 2 memoryClockRate(GHz): 1.1775
pciBusID: 0000:86:00.0
2019-10-03 13:11:04.319806: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0
2019-10-03 13:11:04.319820: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
2019-10-03 13:11:04.319833: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0
2019-10-03 13:11:04.319846: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0
2019-10-03 13:11:04.319859: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0
2019-10-03 13:11:04.319872: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0
2019-10-03 13:11:04.319885: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
2019-10-03 13:11:04.332488: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0, 1, 2, 3, 4, 5
2019-10-03 13:11:04.332811: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-10-03 13:11:04.332823: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 1 2 3 4 5
2019-10-03 13:11:04.332830: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N Y Y Y N N
2019-10-03 13:11:04.332835: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 1: Y N Y Y N N
2019-10-03 13:11:04.332840: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 2: Y Y N Y N N
2019-10-03 13:11:04.332845: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 3: Y Y Y N N N
2019-10-03 13:11:04.332850: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 4: N N N N N Y
2019-10-03 13:11:04.332856: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 5: N N N N Y N
2019-10-03 13:11:04.340711: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:0 with 7647 MB memory) -> physical GPU (device: 0, name: Tesla M60, pci bus id: 0000:04:00.0, compute capability: 5.2)
2019-10-03 13:11:04.341796: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:1 with 7647 MB memory) -> physical GPU (device: 1, name: Tesla M60, pci bus id: 0000:05:00.0, compute capability: 5.2)
2019-10-03 13:11:04.342889: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:2 with 7647 MB memory) -> physical GPU (device: 2, name: Tesla M60, pci bus id: 0000:09:00.0, compute capability: 5.2)
2019-10-03 13:11:04.343989: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:3 with 7647 MB memory) -> physical GPU (device: 3, name: Tesla M60, pci bus id: 0000:0a:00.0, compute capability: 5.2)
2019-10-03 13:11:04.345103: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:4 with 7647 MB memory) -> physical GPU (device: 4, name: Tesla M60, pci bus id: 0000:85:00.0, compute capability: 5.2)
2019-10-03 13:11:04.346189: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/device:GPU:5 with 7647 MB memory) -> physical GPU (device: 5, name: Tesla M60, pci bus id: 0000:86:00.0, compute capability: 5.2)
Default GPU Device: /device:GPU:0
I set the argument of GPU in TGANModel to '/GPU:0' and also tried with '/device:GPU:0'
But, it is the same warning and the crash just while running the first epoch.
I also uninstalled and re-installed Tensorflow-gpu and TGAN, just to check but of no use.
Regards,
Nabaruna
Standardation needed?
Dear contributor,
May I know is it necessary to standard or normalize the data before using the TGAN? Is there a big difference between the results with and without such standardization/normalization? Thanks.
Beato
Change `.fit()` call to more closely align with other `.fit()` calls like keras and fastai.
As the title states, the .fit() call is different now in that the batch_size, nr_epochs are set in the object initialization rather than the start of training. To more closely align this call with other standards like keras and fastai, these should be moved.
How to bring down the number of epochs in example python notebook and generate samples based on log files?
Hey Team!
Just looking to test this package out, so I lowered the nr of epochs in the example\config.json file to 1. Still, the model won't take 1 epoch into consideration and continues to run the second epoch. Am I looking at the wrong file? Should I bring down epochs somewhere else?
On the other hand, can you please advise how I can sample data by using the log files created in the output folder? I. e. can I just run the files created there and generate data based on them?
Thanks so much!
RuntimeError: dictionary changed size during iteration
I'm getting the error "RuntimeError: dictionary changed size during iteration" when importing the TGANModel
What variables give the best results?
Hi,
Great work on this execution!
Due to GPU limitations I'm currently having I was not able to run the entire launcher.py as in the repo. But I ran TGAN_synthesizer.py separately on several datasets and the results were somewhat ok.
Can I have an idea on what tunable parameters out of below gave the best results according to your experience?
tunable_variables = {
"--batch_size": [50, 100, 200],
"--z_dim": [50, 100, 200, 400],
"--num_gen_rnn": [100, 200, 300, 400, 500, 600],
"--num_gen_feature": [100, 200, 300, 400, 500, 600],
"--num_dis_layers": [1, 2, 3, 4, 5],
"--num_dis_hidden": [100, 200, 300, 400, 500],
"--learning_rate": [0.0002, 0.0005, 0.001],
"--noise": [0.05, 0.1, 0.2, 0.3]
}
For example on the Census dataset.
Thank you!
Select action while training
While training, this announcement showed up on the terminal all the time. What does it really mean?
[0402 09:23:29 @logger.py:109] WRN Log directory train_log/TGAN_synthesizer:KDD2-1 exists! Use 'd' to delete it.
[0402 09:23:29 @logger.py:112] WRN If you're resuming from a previous run, you can choose to keep it.
Press any other key to exit.
Select Action: k (keep) / d (delete) / q (quit):
Version of tensorpack?
Got the following error:
from tensorpack.utils.globvars import globalns as opt
ModuleNotFoundError: No module named 'tensorpack.utils.globvars'
It seems the newest version of tensorpack is not compatible with the version used in the implementation.
How to deal with discrete (integer) variables?
Hi,
Here's another question, for discrete variables, such as age and other count variables (integers), should we set them as continuous columns?
From my understanding of your paper, age was assigned as a continuous column. However, when I do that, the synthetic output of age would not longer be integers.
Thank you!
TypeError: softmax() got an unexpected keyword argument 'axis'
So I have Tensorflow version 1.14.0 installed, and this issue is somehow related to different versions. Can TGAN point out which packages and version they are using? Or is there a different approach to solve this issue? Thanks in advance.
[0911 08:47:31 @input_source.py:223] Setting up the queue 'QueueInput/input_queue' for CPU prefetching ...
[0911 08:47:31 @registry.py:90] 'gen/LSTM/00/FC': [200, 100] --> [200, 100]
[0911 08:47:31 @registry.py:90] 'gen/LSTM/00/FC2': [200, 100] --> [200, 1]
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-4-437682dbdfa1> in <module>()
----> 1 tgan.fit(data)
~/celgene/TGAN-master/tgan/model.py in fit(self, data)
688 trainer = GANTrainer(
689 model=self.model,
--> 690 input_queue=input_queue,
691 )
692
~/celgene/TGAN-master/tgan/trainer.py in __init__(self, model, input_queue)
34 self.tower_func = TowerFuncWrapper(model.build_graph, inputs_desc)
35 with TowerContext('', is_training=True):
---> 36 self.tower_func(*input_queue.get_input_tensors())
37
38 opt = model.get_optimizer()
~/packages/anaconda3/lib/python3.6/site-packages/tensorpack/tfutils/tower.py in __call__(self, *args)
288 ctx = get_current_tower_context()
289 assert ctx is not None, "Function must be called under TowerContext!"
--> 290 output = self._tower_fn(*args)
291 handle = TowerTensorHandle(ctx, args, output, self._input_signature)
292 self._handles.append(handle)
~/celgene/TGAN-master/tgan/model.py in build_graph(self, *inputs)
459
460 with tf.variable_scope('gen'):
--> 461 vecs_gen = self.generator(z)
462
463 vecs_denorm = []
~/celgene/TGAN-master/tgan/model.py in generator(self, z)
280 input = tf.concat([h, z], axis=1)
281 attw = tf.get_variable("attw", shape=(len(states), 1, 1))
--> 282 attw = tf.nn.softmax(attw, axis=0)
283 attention = tf.reduce_sum(tf.stack(states, axis=0) * attw, axis=0)
284
TypeError: softmax() got an unexpected keyword argument 'axis'
Column names are replaced by index
Dear all,
I used TGAN in PyPI (ver 0.1.0) and found that column names of the original DataFrame are replaced by index.
Problem
I suppose the problem is derived from the following lines:
https://github.com/DAI-Lab/TGAN/blob/master/tgan/data.py#L321-#L322
In L321, column names of the original DataFrame are assigned into self.columns. And in the next line, column names of the original DataFrame are replaced by index.
The codes seems to try reassigning column names in L410, but the trouble is that the function fit_transform is called several times. Since column names of the original DataFrame are replaced by index in L322, self.columns are assigned as index by the 2nd time when fit_transform is called.
https://github.com/DAI-Lab/TGAN/blob/master/tgan/data.py#L410
How to fix
I believe that one of the solution is to add if statement around L321 like following:
if self.columns is None:
self.columns = data.columns
It might be wrong because I can't totally understand the codes of TGAN, but I wish this issue could be helpful.
Environment
- macOS High Sierra 10.13.6
- Python 3.6.8 | anaconda3-5.0.0
Issue with the output of the model
Hello everyone. I'm new to GANs as I need them to create more tabular data for my database. I came across this implementatin (thank you btw), but I have a problem with the output at the .sample() step. I've changed the parameter to test first, here's my code :
tgan = TGANModel(continuous_columns,output='output',max_epoch=5,steps_per_epoch=1000,save_checkpoints=True,restore_session=True,batch_size=200,z_dim=200,noise=0.2,l2norm=0.00001,learning_rate=0.001,num_gen_rnn=100,num_gen_feature=100,num_dis_layers=1,num_dis_hidden=100,optimizer='AdamOptimizer')
Here's the output :
It keeps running, looks like an infinite loop. I'm a beginner, I'm wondering where I am mistaken, please enlighten me.
How to save model during experiment?
Hi,
I understand that when we are running the tgan.fit using jupyter, we can save the model by using the code after fitting:
model_path = 'models/mymodel.pkl'
tgan.save(model_path)
Model saved successfully.
However, if I run the experiment in the terminal/command line, how do I save the models for every random hyperparameter search? The purpose is to load the model so that I can regenerate new synthetic data I needed.
Thanks!
census data postprocessing issue
I have trained the census data ....after training and on restoring checkpoints it kind of stucks in generating samples...have attached the issue
example doesn't run
List of all the requirements for TGAN installation?
Hey Team,
Thanks so much for this package! :)
Can you please confirm all of the requirements for TGAN installation? I remember there being a section in readme about this, but right now it only states Python is a requirement. Sorry if I am missing something!
Thanks again,
Tia
Add support for GPU and multiGPU
We should support GPU and multiGPU training and sampling, allowing to:
- Select the desired behavior with a parameter
- Select the best available option in case none is provided
EDIT
The required changes affect two scopes of the project:
-
During installation time: When installing the package we need to make the user choose between
tensorflowandtensorflow-gpuand install it before proceding with the install ofTGAN.-
In
README.md, we need to add a new subsection in Requirements called Tensorflow, en explaining that they need to install Tensorflow on their own, selecting
between the two, and that in case of doubt they sould install the non-gpu. -
In
setup.py, we need to remove thetensorflowdependence oninstall_requires.
-
-
During execution time: When creating the instance of TGANModel, we should check at the gpu argument and depending on its value use one trainer or the other.
-
In
tgan.trainer.MultiGPUTrainer:- Change the init signature, so the arguments follow the same order that
GANTrainer(nr_gpu at the end) - Change the
nr_gpuforgpus, that will expect a list of integers. - Adapt the rest of the method to the new format of
gpus(lines 117, 120, 136, 142, 143)
- Change the init signature, so the arguments follow the same order that
-
In
tgan.model.TGANModel:- Add a method
get_gpusonTGANModelwith the following code:
from tensorflow.python.client import device_lib def get_gpus(): return [x.locality.bus_id for x in device_lib.list_local_devices() if x.device_type == 'GPU']
- In TGANModel.init change the name of the argument
gputogpusand on line 627 to:
self.gpus = gpus or self.get_gpus() if self.gpus: os.environ['CUDA_VISIBLE_DEVICES'] = ','.join(list(map(str, self.gpus)))
-
Add a method
get_trainer(self, input_queue)that will:- If
self.gpusis longer than 1 return a new instance ofMultiGPUTrainer, instantiated withself.gpus - In any other case, return a
GANTraineras in line 688
- If
-
Replace line 688 with a call to the newly created
get_trainer.
- Add a method
-
How the attention is created.
Forgive me if this is naive. You are applying softmax to the variable attw immediately after creating it without performing operations with respect to the input or any other variable. This is different from the conventional attention mechanisms that I am used to in NLP. Is there any particular reason?
Vector Dimensions
Hi, could I ask how you deal with the different dimensions vectors in Generator part? Since categorical data converted into one-hot learning vector may has larger dimensions than continuous data as input.
documentation - checkpointing and reusing models
Hi. This is very interesting software. Thank you for providing it. It seems to run stably. Congratulations, this is not easy.
How do you restart a training run from a checkpoint?
How do you generate new data from an existing model?
The graph couldn't be sorted in topological order - during tgan.fit
Hi,
When I run the experiment or tgan.fit, there will be warning messages as such below, about The graph couldn't be sorted in topological order.
Should this be of concern?
Thank you.
�[32m[1010 06:26:22 @base.py:275]�[0m Start Epoch 1 ...
0%| |0/5000[00:00<?,?it/s]2019-10-10 06:26:26.595311: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:697] Iteration = 0, topological sort failed with message: The graph couldn't be sorted in topological order.
2019-10-10 06:26:26.868907: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:697] Iteration = 1, topological sort failed with message: The graph couldn't be sorted in topological order.
2019-10-10 06:26:26.987556: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:502] layout failed: Invalid argument: The graph couldn't be sorted in topological order.
2019-10-10 06:26:27.797393: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:502] remapper failed: Invalid argument: The graph couldn't be sorted in topological order.
2019-10-10 06:26:27.850938: E tensorflow/core/grappler/optimizers/meta_optimizer.cc:502] arithmetic_optimizer failed: Invalid argument: The graph couldn't be sorted in topological order.
2019-10-10 06:26:27.897542: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:697] Iteration = 0, topological sort failed with message: The graph couldn't be sorted in topological order.
2019-10-10 06:26:27.986491: E tensorflow/core/grappler/optimizers/dependency_optimizer.cc:697] Iteration = 1, topological sort failed with message: The graph couldn't be sorted in topological order.
2019-10-10 06:26:35.334083: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0
Understand stats in Trainlog file
{"QueueInput/queue_size": 50.0, "GAN_loss/gen/loss": 0.9750093221664429, "GAN_loss/gen/klloss": 0.27809518575668335, "GAN_loss/gen/final-g-loss": 1.2531044483184814, "GAN_loss/discrim/loss": 0.6561644077301025, "GAN_loss/discrim/accuracy_real": 0.0, "GAN_loss/discrim/accuracy_fake": 1.0, "epoch_num": 5, "global_step": 1000}
Could you please clarify these metric and their values. How good or how bad it is?
Question: Running on GPU - VM instance
Hello,
I am trying to run my TGAN on a Google VM instance with 1 GPU. I am passing an argument 'gpu=0' or 'gpu='0'' to my TGANModel() but it keeps running on CPU rather than GPU. Could you please let me know what I might be doing wrong, is running on GPU supported while just passing a gpu argument or is there something else I should be doing to enable it?
Thank you
from tgan.model import TGANModel
tgan = TGANModel(continuous_columns=continuous, output='tGAN/', gpu=0, max_epoch=1, steps_per_epoch=6000, save_checkpoints=True,
restore_session=True, batch_size=256, z_dim=200, noise=0.2, l2norm=0.00001, learning_rate=0.001,
num_gen_rnn=100, num_gen_feature=100, num_dis_layers=1, num_dis_hidden=100, optimizer='AdamOptimizer')
tgan.fit(data)
tgan.save(model_path, force=True)
I have a problem when I try to use 'Random hyperparameter search'
- I prepared a dataset and a JSON file.
And I ran the code 'tgan experiments config.json results.json', but it didn't function.
There was a massage 'SyntaxError: invalid syntax' - So, I checked the API reference('https://sdv-dev.github.io/TGAN/api/tgan.research.experiments.html').
And I found the code 'tgan.research.experiments.run_experiments(config_path, output_path)' - However, when I ran the code.... I have a error massage. It was 'AttributeError: module 'tgan.research' has no attribute 'experiments''.
- I found that I have to import experiments with the code 'from tgan.research import experiments'. And I ran the code 'tgan.research.experiments.run_experiments(config_path, output_path)'
- However, I met another error massage 'ValueError: Passing negative integer to header is invalid. For no header, use header=None instead'. (I have no negative integer in my data. Data composed of positive integers only.)
- Now, I am trying to modify the code of 'run_experiments' and 'run_experiment'. But still I can not resolve this header problem.
So, please let me know how I can resolve this header problem and use 'Random hyperparameter search'.
Error: skip census, folder exist.
$ python3.6 /home/harry/Documents/GANs-demo/TGAN-master/src/launcher.py /home/harry/Documents/GANs-demo/TGAN-master/demo_config.json
[{'name': 'census', 'num_random_search': 10, 'train_csv': 'data/census-train.csv', 'continuous_cols': [0, 2, 3, 4, 5], 'epoch': 5, 'steps_per_epoch': 10000, 'output_epoch': 3, 'sample_rows': 10000}]
skip census, folder exist.
Why i faced this issue?
Here's my demo_config.json file:
[
{
"name": "census",
"num_random_search": 10,
"train_csv": "data/census-train.csv",
"continuous_cols": [0, 2, 3, 4, 5],
"epoch": 5,
"steps_per_epoch": 10000,
"output_epoch": 3,
"sample_rows": 10000
}
]
Computation time
Hi, I have a question about the computation time needed ! In my case, I have a dataset of 1450 rows and 168 features and it takes about 2 hours and 16minutes to generate samples witch 5 epochs and 100 steps per epoch. This means that I will need days in order to run the model with 10000 steps. Is this normal ?
PS : I have 8GB of RAM
vecs_denorm in model.py
Is vecs_denorm in build_graph model.py used anywhere? I only see how it is created but not where it is used. Thanks.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.