Dear Dev Team,

I am using seq_gan model for generating text sequence. I am trying to use list of valid "subdomains" as input and generate new "text" as output. I am making changes in run/run_seqgan.py, and adding dataset that I want as input - datasets/subdomains.txt.

I am getting generated output is numeric data. I used text as input, but output seems in numeric form. Am I missing something here, in terms of configuration or code changes ??

ubuntu@ip-XXXXX:~/TextGAN-PyTorch/save/20200625/subdomains/seqgan_vanilla_lt-rsgan_sl20_temp1_T0625_1930_08/samples$ vim samples_ADV_00001.txt
3197 2049 2875 3903 2649 2338 116 4714 4111 638 3665 3782 4722 2049 2467 4123 2560 2823 224 3468
1961 4136 2923 595

Installing collected packages: numpy, torch, tqdm
Found existing installation: numpy 1.16.4
Uninstalling numpy-1.16.4:
Successfully uninstalled numpy-1.16.4
Found existing installation: torch 0.4.1
Uninstalling torch-0.4.1:
Successfully uninstalled torch-0.4.1
Found existing installation: tqdm 4.31.1
Uninstalling tqdm-4.31.1:
Successfully uninstalled tqdm-4.31.1
Successfully installed numpy-1.14.5 torch-1.1.0 tqdm-4.32.1

ub16c9@ub16c9-gpu:/media/ub16c9/fcd84300-9270-4bbd-896a-5e04e79203b7/ub16_prj/TextGAN-PyTorch/run$ python3.6 run_seqgan.py 0 0
job_id: 0, gpu_id: 0
Traceback (most recent call last):
File "main.py", line 5, in
import config as cfg
File "/media/ub16c9/fcd84300-9270-4bbd-896a-5e04e79203b7/ub16_prj/TextGAN-PyTorch/config.py", line 13, in
import torch
ImportError: No module named torch

ub16c9@ub16c9-gpu:/media/ub16c9/fcd84300-9270-4bbd-896a-5e04e79203b7/ub16_prj/TextGAN-PyTorch/run$ python3.6
Python 3.6.8 (default, Dec 24 2018, 19:24:27)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.

import torch
torch.version
'1.1.0'
quit()
ub16c9@ub16c9-gpu:/media/ub16c9/fcd84300-9270-4bbd-896a-5e04e79203b7/ub16_prj/TextGAN-PyTorch/run$

Hallo, thanks for making your code available.

I am interested in using SeqGAN (or perhaps another GAN variant capable of dealing with integer data), for purposes of function approximation

I get this at the very beginning, during the Oracle instance creation:

Traceback (most recent call last):
  File "main.py", line 93, in <module>
    inst = instruction_dict[cfg.run_model](opt)
  File "/scratch/ishalyminov/data/TextGAN-PyTorch/instructor/oracle_data/leakgan_instructor.py", line 25, in __init__
    super(LeakGANInstructor, self).__init__(opt)
  File "/scratch/ishalyminov/data/TextGAN-PyTorch/instructor/oracle_data/instructor.py", line 36, in __init__
    self.oracle_samples = torch.load(cfg.oracle_samples_path)
  File "/home/ishalyminov/miniconda3/envs/textgan_pytorch/lib/python3.6/site-packages/torch/serialization.py", line 382, in load
    f = open(f, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: 'pretrain/oracle_data/oracle_samples_NUM10000_lstm.pt'

Hello,Thanks for your sharing your code.
But In your implement of seqgan, I have a problems:
That is computing reward(in uitls.rollout), why you use a mean Q-value(reward) to replace every Q-value? I think it should be every Q-value multiply log(P(y_t|Y_1:Y_{t-1})).

ub16c9@ub16c9-gpu:/media/ub16c9/fcd84300-9270-4bbd-896a-5e04e79203b7/ub16_prj/TextGAN-PyTorch/run$ python3.6 run_seqgan.py 0 0
job_id: 0, gpu_id: 0

training arguments:

if_test: 0
run_model: seqgan
dataset: oracle
model_type: vanilla
loss_type: JS
if_real_data: 0
cuda: 1
device: 0
shuffle: 0
use_truncated_normal: 0
samples_num: 10000
vocab_size: 5000
mle_epoch: 120
adv_epoch: 200
inter_epoch: 10
batch_size: 64
max_seq_len: 20
start_letter: 1
padding_idx: 0
gen_lr: 0.01
gen_adv_lr: 0.0001
dis_lr: 0.0001
clip_norm: 5.0
pre_log_step: 5
adv_log_step: 1
train_data: dataset/oracle.txt
test_data: dataset/testdata/oracle_test.txt
temp_adpt: exp
temperature: 1
ora_pretrain: 1
gen_pretrain: 0
dis_pretrain: 0
adv_g_step: 1
rollout_num: 32
gen_embed_dim: 32
gen_hidden_dim: 32
goal_size: 16
step_size: 4
mem_slots: 1
num_heads: 2
head_size: 256
d_step: 50
d_epoch: 3
adv_d_step: 5
adv_d_epoch: 3
dis_embed_dim: 64
dis_hidden_dim: 64
num_rep: 64
log_file: log/log_0611_1726.txt
save_root: save/relgan_vanilla_oracle_RSGAN_glr0.01_temp2_T0611-1726/
signal_file: run_signal.txt
tips: vanilla SeqGAN
====================================================================================================
Starting Generator MLE Training...
[MLE-GEN] epoch 0 : pre_loss = 7.8519, oracle_NLL = 10.1270, gen_NLL = 7.6967,
[MLE-GEN] epoch 5 : pre_loss = 7.1450, oracle_NLL = 9.4151, gen_NLL = 6.9945,
[MLE-GEN] epoch 10 : pre_loss = 6.7705, oracle_NLL = 9.2543, gen_NLL = 6.7171,

.....
[MLE-DIS] d_step 47: d_loss = 0.0005, train_acc = 0.9999, eval_acc = 0.5068,
[MLE-DIS] d_step 48: d_loss = 0.0018, train_acc = 0.9993, eval_acc = 0.5020,
[MLE-DIS] d_step 49: d_loss = 0.0010, train_acc = 0.9997, eval_acc = 0.5010,
Save pretrain_generator discriminator: pretrain/oracle_data/dis_pretrain_seqgan_vanilla.pt
Starting Adversarial Training...
Initial generator: oracle_NLL = 9.0884, gen_NLL = 5.9763,

ADV EPOCH 0

Traceback (most recent call last):
File "main.py", line 100, in
inst._run()
File "/media/ub16c9/fcd84300-9270-4bbd-896a-5e04e79203b7/ub16_prj/TextGAN-PyTorch/instructor/oracle_data/seqgan_instructor.py", line 72, in _run
self.adv_train_generator(cfg.ADV_g_step) # Generator
File "/media/ub16c9/fcd84300-9270-4bbd-896a-5e04e79203b7/ub16_prj/TextGAN-PyTorch/instructor/oracle_data/seqgan_instructor.py", line 120, in adv_train_generator
rewards = rollout_func.get_reward(target, cfg.rollout_num, self.dis)
File "/media/ub16c9/fcd84300-9270-4bbd-896a-5e04e79203b7/ub16_prj/TextGAN-PyTorch/utils/rollout.py", line 142, in get_reward
samples = self.rollout_mc_search(sentences, given_num)
File "/media/ub16c9/fcd84300-9270-4bbd-896a-5e04e79203b7/ub16_prj/TextGAN-PyTorch/utils/rollout.py", line 36, in rollout_mc_search
out, hidden = self.gen.forward(inp, hidden, need_hidden=True)
File "/media/ub16c9/fcd84300-9270-4bbd-896a-5e04e79203b7/ub16_prj/TextGAN-PyTorch/models/generator.py", line 44, in forward
out = self.temperature * out # temperature
RuntimeError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 10.92 GiB total capacity; 9.41 GiB already allocated; 31.19 MiB free; 81.37 MiB cached)
ub16c9@ub16c9-gpu:/media/ub16c9/fcd84300-9270-4bbd-896a-5e04e79203b7/ub16_prj/TextGAN-PyTorch/run$

I checked files are being produced by the name of samples_MLE in sample folder but they consist of numbers. I guess they are the tokens of the individual word, if this is the case can you tell me how can I map these words back to words.

training arguments:

if_test: False
run_model: maligan
k_label: 2
dataset: treasure-island
model_type: vanilla
loss_type: rsgan
mu_type: ragan
eval_type: Ra
d_type: Ra
if_real_data: True
cuda: True
device: 0
devices: 0
shuffle: False
gen_init: truncated_normal
dis_init: uniform
n_parent: 1
eval_b_num: 8
lambda_fq: 1.0
lambda_fd: 0.0
d_out_mean: True
freeze_dis: False
freeze_clas: False
use_all_real_fake: False
use_population: False
samples_num: 10000
vocab_size: 7102
mle_epoch: 150
clas_pre_epoch: 10
adv_epoch: 2000
inter_epoch: 15
batch_size: 64
max_seq_len: 24
start_letter: 1
padding_idx: 0
gen_lr: 0.01
gen_adv_lr: 0.0001
dis_lr: 0.0001
clip_norm: 5.0
pre_log_step: 10
adv_log_step: 20
train_data: dataset/treasure-island.txt
test_data: dataset/testdata/treasure-island_test.txt
temp_adpt: exp
evo_temp_step: 1
temperature: 1
ora_pretrain: True
gen_pretrain: False
dis_pretrain: False
adv_g_step: 1
rollout_num: 16
gen_embed_dim: 32
gen_hidden_dim: 32
goal_size: 16
step_size: 4
mem_slots: 1
num_heads: 2
head_size: 256
d_step: 5
d_epoch: 3
adv_d_step: 5
adv_d_epoch: 3
dis_embed_dim: 64
dis_hidden_dim: 64
num_rep: 64
use_nll_oracle: True
use_nll_gen: True
use_nll_div: True
use_bleu: True
use_self_bleu: False
use_clas_acc: True
use_ppl: False
log_file: log/log_0310_1756_25.txt
save_root: save/20210310/treasure-island/maligan_vanilla_dt-Ra_lt-rsgan_mt-ra_et-Ra_sl24_temp1_lfd0.0_T0310_1756_25/
signal_file: run_signal.txt
tips:
====================================================================================================
Traceback (most recent call last):
File "D:\other\code\PyCode\courseWork\models\TextGAN-PyTorch\main.py", line 167, in
inst = instruction_dictcfg.run_model
File "D:\other\code\PyCode\courseWork\models\TextGAN-PyTorch\instructor\real_data\maligan_instructor.py", line 25, in init
super(MaliGANInstructor, self).init(opt)
File "D:\other\code\PyCode\courseWork\models\TextGAN-PyTorch\instructor\real_data\instructor.py", line 72, in init
self.ppl = PPL(self.train_data, self.test_data, n_gram=5, if_use=cfg.use_ppl)
AttributeError: 'MaliGANInstructor' object has no attribute 'train_data'`

When I`m trying to start training on my real dataset, this error happens
Dataset files are already in correct folders

I notice that generator the porject use is only LSTMGenerator, i wonder if a basic transformer can be a generator?
Hope your response, thanks!

Hi
I run sentigan with my dataset. It has 48000 recoeds. After 150 epochs for MLE training and 1000 epochs for ADV training, sentigan generate repetitive samples. Why?

I have trained the model on the oracle synthetic data but now I want to generate samples on the trained model, how can I do it?

I'm wondering if you could give a few tips into transforming this code to be able to generate captions for images. By that i mean to use image features as well.

I am writing to double check whether instructor.oracle_data means that a certain algorithm will run with synthesized dataset instead of a real dataset.

@williamSYSU I have a small question. I am trying to understand how you handle Out of vocabulary tokens, since we can a max cap on the vocabulary size if we deal with another dataset. I can see that in the current code, there is only 2 special tokens: start_token and padding_token, and the two datasets COO image and EMNLP both has a very limited size of vocabulary (around 6K) so you are getting all the unique tokens in the main.py file. How should we handle a dataset with larger vocabulary base?

From the original code of RelGAN (https://github.com/weilinie/RelGAN), D is initialized with truncated_normal (similar with G). In your code, you are using Uniform (I saw a commit that you did so). Is it a preference or I am missing something? I am trying to figure out where I got wrong in the NLL_gen curve (#20), so I want to ask about this.

Thanks!

Do you have any plans to port the attention RMC models over? Even though RelGAN_D with rmc attention is not working well with the current GAN settings (claimed from RelGAN paper author), I think it is still a good idea to port it over.

I am sure can help you debug and check it after it is ported.

Hi,
I am facing a problem in training RelGAN. From their paper, the NLL score on COO Image dataset should first go up and then gradually go down during adversarial training, yet seems like the NLL score returned by the current code continuously go up. However, the BLEU scores match the one described in the paper. I am not sure what I am missing hope you can help.

Thanks!

what is the output of program exactly? my output is some numbers? How can I convert numbers to sentences?

The figure of the sentiGAN show that the discriminator of sentiGAN is a multi-class classifier, but in your code, I think it is a binary classifier. So the difference between sentiGAN and seqGAN is just the penalty based objective?

Dear author, thank you very much for your contribution.
But I noticed that at real_data/SentiGAN_instructor.py line 143.
rewards = rollout_func.get_reward(target, cfg.rollout_num, self.dis) see mingling haven't select different index in discriminator, where

reward 函数是这样
def get_reward(self, sentences, rollout_num, dis, current_k=0):
"""
get reward via Monte Carlo search
:param sentences: size of batch_size * max_seq_len
:param rollout_num:
:param dis:
:param current_k: current training gen
:return: reward: [batch_size]
"""
batch_size = sentences.size(0)
rewards = torch.zeros([rollout_num * self.max_seq_len, batch_size]).float()
if self.gpu:
rewards = rewards.cuda()
idx = 0
for i in range(rollout_num):
for given_num in range(1, self.max_seq_len + 1):
samples = self.rollout_mc_search(sentences, given_num)
out = dis.forward(samples)
out = F.softmax(out, dim=-1)
reward = out[:, current_k + 1]
rewards[idx] = reward
idx += 1
rewards = torch.mean(rewards.view(batch_size, self.max_seq_len, rollout_num), dim=-1)
return rewards

Seems like this reward doesn't include multi-class classification score?

I am running the codes on macOS without NVIDIA GPU Graphic Card available. I wonder how I can run the scripts in CPU-only mode. What sections should I modify?

Thanks in advance!

Hi,
Please help me how to run sentiGAN on the real dataset?
What is cat_train_data?
What is cat_test_data?
I know sentigan need to some data for MLE training and some data for advertial traing.

Thank you

I think the title explains what I need to understand.

I am trying to learn GANs so I started to build the models on my own taking the SeqGAN model as a base. So what i did was change the Discriminator to an BiLSTM model.

When I ran these models, they produced variable length sentences from the data that i gave as input. However even though I used pack->lstm->unpack inside the generator I cannot see variable length sequences when I sample the generator. How did you achieve this?

This class doesn't have any function init_param(). I think it can't inherit from LSTMGenerator as its initialization gets other params

Hello, I have the same issue as #6 but I don't understand the solution.
Thanks in advance for your time

Thanks for your coding.I have some small problems,could you help me?
When I use my dataset，it happens.
`> training arguments:

if_test: False
run_model: seqgan
k_label: 2
dataset: comment
model_type: vanilla
loss_type: rsgan
if_real_data: True
cuda: True
device: 0
shuffle: False
gen_init: truncated_normal
dis_init: uniform
samples_num: 10000
vocab_size: 930
mle_epoch: 150
clas_pre_epoch: 10
adv_epoch: 2000
inter_epoch: 15
batch_size: 64
max_seq_len: 16
start_letter: 1
padding_idx: 0
gen_lr: 0.01
gen_adv_lr: 0.0001
dis_lr: 0.0001
clip_norm: 5.0
pre_log_step: 10
adv_log_step: 20
train_data: dataset/comment.txt
test_data: dataset/testdata/comment_test.txt
temp_adpt: exp
temperature: 1
ora_pretrain: True
gen_pretrain: False
dis_pretrain: False
adv_g_step: 1
rollout_num: 16
gen_embed_dim: 32
gen_hidden_dim: 32
goal_size: 16
step_size: 4
mem_slots: 1
num_heads: 2
head_size: 256
d_step: 5
d_epoch: 3
adv_d_step: 5
adv_d_epoch: 3
dis_embed_dim: 64
dis_hidden_dim: 64
num_rep: 64
use_nll_oracle: True
use_nll_gen: True
use_nll_div: True
use_bleu: True
use_self_bleu: True
use_clas_acc: True
use_ppl: False
log_file: log/log_1216_1650_47.txt
save_root: save/20191216/comment/seqgan_vanilla_lt-rsgan_sl16_temp1_T1216_1650_47/
signal_file: run_signal.txt
tips:
====================================================================================================
Traceback (most recent call last):
File "C:/Users/Administrator/Desktop/TextGAN-PyTorch-master/main.py", line 134, in
inst = instruction_dictcfg.run_model
File "C:\Users\Administrator\Desktop\TextGAN-PyTorch-master\instructor\real_data\seqgan_instructor.py", line 23, in init
super(SeqGANInstructor, self).init(opt)
File "C:\Users\Administrator\Desktop\TextGAN-PyTorch-master\instructor\real_data\instructor.py", line 74, in init
self.ppl = PPL(self.train_data, self.test_data, n_gram=5, if_use=cfg.use_ppl)
AttributeError: 'SeqGANInstructor' object has no attribute 'train_data'`

When I use my dataset，it happens
could you help me?

training arguments:

if_test: False
run_model: seqgan
k_label: 2
dataset: obama
model_type: vanilla
loss_type: rsgan
if_real_data: True
cuda: True
device: 0
shuffle: False
gen_init: truncated_normal
dis_init: uniform
samples_num: 10000
vocab_size: 8077
mle_epoch: 150
clas_pre_epoch: 10
adv_epoch: 2000
inter_epoch: 15
batch_size: 64
max_seq_len: 148
start_letter: 1
padding_idx: 0
gen_lr: 0.01
gen_adv_lr: 0.0001
dis_lr: 0.0001
clip_norm: 5.0
pre_log_step: 10
adv_log_step: 20
train_data: dataset/obama.txt
test_data: dataset/testdata/obama_test.txt
temp_adpt: exp
temperature: 1
ora_pretrain: True
gen_pretrain: False
dis_pretrain: False
adv_g_step: 1
rollout_num: 16
gen_embed_dim: 32
gen_hidden_dim: 32
goal_size: 16
step_size: 4
mem_slots: 1
num_heads: 2
head_size: 256
d_step: 5
d_epoch: 3
adv_d_step: 5
adv_d_epoch: 3
dis_embed_dim: 64
dis_hidden_dim: 64
num_rep: 64
use_nll_oracle: True
use_nll_gen: True
use_nll_div: True
use_bleu: True
use_self_bleu: True
use_clas_acc: True
use_ppl: False
log_file: log/log_0125_1829_54.txt
save_root: save/20200125/obama/seqgan_vanilla_lt-rsgan_sl148_temp1_T0125_1829_54/
signal_file: run_signal.txt
tips:
====================================================================================================
Traceback (most recent call last):
File "main.py", line 134, in
inst = instruction_dictcfg.run_model
File "/home/drriyahi/Desktop/obama/instructor/real_data/seqgan_instructor.py", line 23, in init
super(SeqGANInstructor, self).init(opt)
File "/home/drriyahi/Desktop/obama/instructor/real_data/instructor.py", line 72, in init
self.ppl = PPL(self.train_data, self.test_data, n_gram=5, if_use=cfg.use_ppl)
AttributeError: 'SeqGANInstructor' object has no attribute 'train_data'

can i run sentigan model on different datasets from exist in the repo ? for e.g if i have one dataset which is labeled data as binary classification and it's raw text how can i run this

training arguments:

if_test: 1
run_model: leakgan
dataset: atc3
model_type: vanilla
loss_type: JS
if_real_data: 1
cuda: 1
device: 0
shuffle: 0
gen_init: normal
dis_init: uniform
samples_num: 10000
vocab_size: 442
mle_epoch: 8
adv_epoch: 200
inter_epoch: 10
batch_size: 64
max_seq_len: 55
start_letter: 1
padding_idx: 0
gen_lr: 0.0015
gen_adv_lr: 0.0001
dis_lr: 5e-05
clip_norm: 5.0
pre_log_step: 1
adv_log_step: 1
train_data: dataset/oracle.txt
test_data: dataset/testdata/oracle_test.txt
temp_adpt: exp
temperature: 1
eta: 2
learn_temperature: True
learn_eta: True
ora_pretrain: 1
gen_pretrain: 0
dis_pretrain: 0
adv_g_step: 1
rollout_num: 4
gen_embed_dim: 32
gen_hidden_dim: 32
goal_size: 16
step_size: 4
mem_slots: 1
num_heads: 2
head_size: 256
d_step: 5
d_epoch: 3
adv_d_step: 5
adv_d_epoch: 3
dis_embed_dim: 64
dis_hidden_dim: 64
num_rep: 64
log_file: log/log_0617_1636_39.txt
save_root: save/20200617/atc3/leakgan_vanilla_lt-JS_sl55_temp1_T2/
signal_file: run_signal.txt
tips: vanilla LeakGAN
====================================================================================================
Traceback (most recent call last):
File "main.py", line 119, in
inst = instruction_dictcfg.run_model
File "/content/TextGAN-master/instructor/real_data/leakgan_instructor.py", line 26, in init
super(LeakGANInstructor, self).init(opt)
File "/content/TextGAN-master/instructor/real_data/instructor.py", line 33, in init
self.train_data = GenDataIter(cfg.train_data)
File "/content/TextGAN-master/utils/data_loader.py", line 39, in init
dataset=GANDataset(self.read_data(samples)),
File "/content/TextGAN-master/utils/data_loader.py", line 56, in read_data
inp, target = self.load_data(samples)
File "/content/TextGAN-master/utils/data_loader.py", line 92, in load_data
samples_index = tokens_to_tensor(tokens, self.word_index_dict)
File "/content/TextGAN-master/utils/text_process.py", line 135, in tokens_to_tensor
return torch.LongTensor(tensor)
ValueError: expected sequence of length 55 at dim 1 (got 0)

Thank you for developing great software.

At the present, the newest stable version of torch is 1.3.1.
With the 1.3.1, an error occurs in dataloader.py when trying to run some script in run directory.
Works well with torch version 1.2.

$ python run_seqgan.py 0 0
job_id: 0, gpu_id: 0
====================================================================================================
> training arguments:
>>> if_test: 0
>>> run_model: seqgan
>>> k_label: 2
>>> dataset: oracle
>>> model_type: vanilla
>>> loss_type: rsgan
>>> if_real_data: 0
>>> cuda: 1
>>> device: 0
>>> shuffle: 0
>>> gen_init: normal
>>> dis_init: uniform
>>> samples_num: 10000
>>> vocab_size: 5000
>>> mle_epoch: 120
>>> clas_pre_epoch: 10
>>> adv_epoch: 200
>>> inter_epoch: 15
>>> batch_size: 64
>>> max_seq_len: 20
>>> start_letter: 1
>>> padding_idx: 0
>>> gen_lr: 0.01
>>> gen_adv_lr: 0.0001
>>> dis_lr: 0.0001
>>> clip_norm: 5.0
>>> pre_log_step: 10
>>> adv_log_step: 1
>>> train_data: dataset/oracle.txt
>>> test_data: dataset/testdata/oracle_test.txt
>>> temp_adpt: exp
>>> temperature: 1
>>> ora_pretrain: 1
>>> gen_pretrain: 0
>>> dis_pretrain: 0
>>> adv_g_step: 1
>>> rollout_num: 16
>>> gen_embed_dim: 32
>>> gen_hidden_dim: 32
>>> goal_size: 16
>>> step_size: 4
>>> mem_slots: 1
>>> num_heads: 2
>>> head_size: 256
>>> d_step: 5
>>> d_epoch: 3
>>> adv_d_step: 4
>>> adv_d_epoch: 2
>>> dis_embed_dim: 64
>>> dis_hidden_dim: 64
>>> num_rep: 64
>>> log_file: log/log_1204_0925_58.txt
>>> save_root: save/20191204/oracle/seqgan_vanilla_lt-rsgan_sl20_temp1_T1204_0925_58/
>>> signal_file: run_signal.txt
>>> tips: SeqGAN experiments
====================================================================================================
Creating Oracle...
NLL_Oracle Groud Truth: 5.9067
Starting Generator MLE Training...
Traceback (most recent call last):
  File "main.py", line 127, in <module>
    inst._run()
  File "/content/textgan/instructor/oracle_data/seqgan_instructor.py", line 52, in _run
    self.pretrain_generator(cfg.MLE_train_epoch)
  File "/content/textgan/instructor/oracle_data/seqgan_instructor.py", line 101, in pretrain_generator
    '[MLE-GEN] epoch %d : pre_loss = %.4f, %s' % (epoch, pre_loss, self.cal_metrics(fmt_str=True)))
  File "/content/textgan/instructor/oracle_data/instructor.py", line 180, in cal_metrics
    self.gen_data.reset(self.gen.sample(cfg.samples_num, 4 * cfg.batch_size))
  File "/content/textgan/utils/data_loader.py", line 64, in reset
    self.loader.dataset = data
  File "/usr/local/lib/python3.6/dist-packages/torch/utils/data/dataloader.py", line 270, in __setattr__
    'initialized'.format(attr, self.__class__.__name__))
ValueError: dataset attribute should not be set after DataLoader is initialized

Thanks,

Hi,

Could you please advise what's the way to train LeakGAN on a custom, real dataset?

Currently, I'm at the step of providing the Oracle with real samples from my dataset. I see they are already vectorized - which vocabulary is it done with?

I'll probably have more questions down the road, so it would be great if you could bear with me :)

I could not understand the way you are sampling data from the generator. Basically, how are you creating batches from num_samples?

def sample(self, num_samples, batch_size, start_letter=cfg.start_letter):
    """
    Samples the network and returns num_samples samples of length max_seq_len.
    :return samples: num_samples * max_seq_length (a sampled sequence in each row)
    """
    num_batch = num_samples // batch_size + 1 if num_samples != batch_size else 1
    samples = torch.zeros(num_batch * batch_size, self.max_seq_len).long()

    # Generate sentences with multinomial sampling strategy
    for b in range(num_batch):
        hidden = self.init_hidden(batch_size)
        inp = torch.LongTensor([start_letter] * batch_size)
        if self.gpu:
            inp = inp.cuda()

        for i in range(self.max_seq_len):
            out, hidden = self.forward(inp, hidden, need_hidden=True)  # out: batch_size * vocab_size
            next_token = torch.multinomial(torch.exp(out), 1)  # batch_size * 1 (sampling from each row)
            samples[b * batch_size:(b + 1) * batch_size, i] = next_token.view(-1)
            inp = next_token.view(-1)
    samples = samples[:num_samples]

    return samples

Whenever I trained my own custom dataset with seqGAN, this error is coming,

Starting Generator MLE Training...
Traceback (most recent call last):
  File "main.py", line 169, in <module>
    inst._run()
  File "/content/drive/My Drive/Colab Notebooks/2/TextGAN-PyTorch/instructor/real_data/seqgan_instructor.py", line 41, in _run
    self.pretrain_generator(cfg.MLE_train_epoch)
  File "/content/drive/My Drive/Colab Notebooks/2/TextGAN-PyTorch/instructor/real_data/seqgan_instructor.py", line 85, in pretrain_generator
    pre_loss = self.train_gen_epoch(self.gen, self.train_data.loader, self.mle_criterion, self.gen_opt)
  File "/content/drive/My Drive/Colab Notebooks/2/TextGAN-PyTorch/instructor/real_data/instructor.py", line 107, in train_gen_epoch
    return total_loss / len(data_loader)
ZeroDivisionError: division by zero

How to make my own dataset.txt file?

Even if I use run_relgan.py with
use_nll_oracle = int(True)
use_nll_gen = int(True)
use_nll_div = int(True)
use_bleu = int(True)
use_self_bleu = int(True)
use_ppl = int(False)

I don't get Bleu, nor self-Bleu in my output. See below:

[ADV] epoch 5: g_loss: 0.6944, d_loss: 0.6920, NLL_oracle = 11.3019, NLL_gen = 8.549, NLL_div = 8.4708 g_loss: 0.6944, d_loss: 0.6920, temperature: 1.0000: 100%|████████████████████████████████| 6/6 [00:23<00:00, 3.86s/it]

Once I restore a model by directly changing pretrained_gen_path and pretrained_dis_path in config.py. How do I then only generate samples and not have the model train again ?

Describe the bug

python3 run_relgan.py 1 1
No modifications were made to the base config.

Environment

• OS: [e.g., Ubuntu 18.04]
• PyTorch = 1.7.0
• Python 3.7
• Numpy 1.14.5
• CUDA 10.1
• nltk 3.4
• tqdm 4.32.1

Error traceback、

/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/models/relational_rnn_general.py:174: UserWarning: Output 0 of SplitWithSizesBackward is a view and is being modified inplace. This view is an output of a function that returns multiple views. Inplace operators on such views are being deprecated and will be forbidden starting from version 1.8. Consider using `unsafe_` version of the function that produced this view or don't modify this view inplace. (Triggered internally at  /pytorch/torch/csrc/autograd/variable.cpp:491.)
  q *= (self.key_size ** -0.5)
[W python_anomaly_mode.cpp:104] Warning: Error detected in SplitWithSizesBackward. Traceback of forward call that caused the error:
  File "main.py", line 169, in <module>
    inst._run()
  File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/instructor/real_data/relgan_instructor.py", line 42, in _run
    self.pretrain_generator(cfg.MLE_train_epoch)
  File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/instructor/real_data/relgan_instructor.py", line 86, in pretrain_generator
    pre_loss = self.train_gen_epoch(self.gen, self.train_data.loader, self.mle_criterion, self.gen_opt)
  File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/instructor/real_data/instructor.py", line 103, in train_gen_epoch
    pred = model.forward(inp, hidden)
  File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/models/generator.py", line 51, in forward
    out, hidden = self.lstm(emb, hidden)  # out: batch_size * seq_len * hidden_dim
  File "/home/goscien/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/models/relational_rnn_general.py", line 339, in forward
    logit, memory = self.forward_step(inputs[:, idx_step], memory)
  File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/models/relational_rnn_general.py", line 307, in forward_step
    next_memory = self.attend_over_memory(memory_plus_input)
  File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/models/relational_rnn_general.py", line 265, in attend_over_memory
    attended_memory = self.multihead_attention(memory)
  File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/models/relational_rnn_general.py", line 171, in multihead_attention
    q, k, v = torch.split(qkv_transpose, [self.key_size, self.key_size, self.value_size], -1)
  File "/home/goscien/.local/lib/python3.7/site-packages/torch/functional.py", line 121, in split
    return tensor.split(split_size_or_sections, dim)
  File "/home/goscien/.local/lib/python3.7/site-packages/torch/tensor.py", line 476, in split
    return super(Tensor, self).split_with_sizes(split_size, dim)
 (function _print_stack)
Traceback (most recent call last):
  File "main.py", line 169, in <module>
    inst._run()
  File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/instructor/real_data/relgan_instructor.py", line 42, in _run
    self.pretrain_generator(cfg.MLE_train_epoch)
  File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/instructor/real_data/relgan_instructor.py", line 86, in pretrain_generator
    pre_loss = self.train_gen_epoch(self.gen, self.train_data.loader, self.mle_criterion, self.gen_opt)
  File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/instructor/real_data/instructor.py", line 105, in train_gen_epoch
    self.optimize(optimizer, loss, model)
  File "/media/goscien/17561A2624DB95F7/python/TextGAN-PyTorch/instructor/real_data/relgan_instructor.py", line 143, in optimize
    loss.backward(retain_graph=retain_graph)
  File "/home/goscien/.local/lib/python3.7/site-packages/torch/tensor.py", line 221, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/goscien/.local/lib/python3.7/site-packages/torch/autograd/__init__.py", line 132, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [64, 2, 2, 768]], which is output 0 of PermuteBackward, is at version 1; expected version 0 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!

I run python3 run_seqga.py 1 1 normal
I searched for a long time, but I didn't get the answer I wanted. What file do I need to modify to solve this error? thank you

My GPU model is 2080Ti .
But it will raise an error:RuntimeError: CUDA out of memory. Tried to allocate 1.43 GiB (GPU 4; 10.76 GiB total capacity; 4.20 GiB already allocated; 147.69 MiB free; 4.28 GiB reserved in total by PyTorch)
Can you tell me how to solve it? Can only reduce the batchsize?

Where can I see the sample? I want to use my own data, but I don't know what the sample of the data set looks like. I can't access the public data set provided in the article

when i run run_seqgan.py 1 1,the CPU usage rate reaches 100%
Please tell me how to solve this problem,thanks

1.On category text generation, the k_label is set to 2 then the model generats two types of text. So there are two scores of each metric in the outputs.Could you ask me how to calculate the final score?
output:
[ADV] epoch 220: temp = 1.2257, d_loss: 0.0367, BLEU-[2, 3, 4, 5] = [[0.473, 0.271, 0.174, 0.134], [0.481, 0.242, 0.159, 0.13]], NLL_gen = [0.9063, 0.8691], NLL_div = [0.4943, 0.4795], Self-BLEU-[2, 3, 4] = [[0.952, 0.884, 0.777], [0.957, 0.889, 0.796]], [PPL-F, PPL-R] = [0, 0], clas_acc = [0.719, 0.5251]
[ADV] epoch 240: temp = 1.2485, d_loss: 0.0239, BLEU-[2, 3, 4, 5] = [[0.478, 0.263, 0.175, 0.14], [0.509, 0.281, 0.183, 0.145]], NLL_gen = [0.9665, 0.9309], NLL_div = [0.4787, 0.4686], Self-BLEU-[2, 3, 4] = [[0.943, 0.864, 0.785], [0.966, 0.882, 0.784]], [PPL-F, PPL-R] = [0, 0], clas_acc = [0.7362, 0.5099]
[ADV] epoch 260: temp = 1.2706, d_loss: 0.0236, BLEU-[2, 3, 4, 5] = [[0.473, 0.283, 0.188, 0.149], [0.499, 0.262, 0.175, 0.142]], NLL_gen = [1.0321, 0.9985], NLL_div = [0.4645, 0.4584], Self-BLEU-[2, 3, 4] = [[0.952, 0.88, 0.778], [0.98, 0.923, 0.82]], [PPL-F, PPL-R] = [0, 0], clas_acc = [0.7467, 0.494]
the score in the paper:

I am new to the TextGAN repo. Could anyone tell me the usage of the testing data in emnlp? For example, in realGAN with real dataset of emanlp news, to my understanding, we use training data to train the model. But, I could not figure out how we use the testing data and how to evaluate the performance on the test data? Do you have the ground truth fo the testing data? Any help is highly appreciated. Many thans.

It looks like the initial hidden layer in both LSTM Generator and RelGAN_G are chosen so they are the same every time. Wouldn't that reduce the diversity of generated samples? What makes sure that the samples are diverse and different?

I have no idea why this error happend. Maybe someone have the same question before?

training arguments:

if_test: False
run_model: seqgan
k_label: 2
dataset: oracle
model_type: vanilla
loss_type: rsgan
if_real_data: False
cuda: True
device: 0
shuffle: False
gen_init: truncated_normal
dis_init: uniform
samples_num: 10000
vocab_size: 5000
mle_epoch: 150
clas_pre_epoch: 10
adv_epoch: 2000
inter_epoch: 15
batch_size: 64
max_seq_len: 20
start_letter: 1
padding_idx: 0
gen_lr: 0.01
gen_adv_lr: 0.0001
dis_lr: 0.0001
clip_norm: 5.0
pre_log_step: 10
adv_log_step: 20
train_data: dataset/oracle.txt
test_data: dataset/testdata/oracle_test.txt
temp_adpt: exp
temperature: 1
ora_pretrain: True
gen_pretrain: False
dis_pretrain: False
adv_g_step: 1
rollout_num: 16
gen_embed_dim: 32
gen_hidden_dim: 32
goal_size: 16
step_size: 4
mem_slots: 1
num_heads: 2
head_size: 256
d_step: 5
d_epoch: 3
adv_d_step: 5
adv_d_epoch: 3
dis_embed_dim: 64
dis_hidden_dim: 64
num_rep: 64
use_nll_oracle: True
use_nll_gen: True
use_nll_div: True
use_bleu: True
use_self_bleu: True
use_clas_acc: True
use_ppl: False
log_file: log/log_0731_1038_46.txt
save_root: save/20200731/oracle/seqgan_vanilla_lt-rsgan_sl20_temp1_T0731_1038_46/
signal_file: run_signal.txt
tips:
====================================================================================================
Starting Generator MLE Training...
[MLE-GEN] epoch 0 : pre_loss = 7.8660, NLL_oracle = 9.9152, NLL_gen = 7.7255, NLL_div = 7.7996
[MLE-GEN] epoch 10 : pre_loss = 7.1071, NLL_oracle = 9.2863, NLL_gen = 7.0592, NLL_div = 7.1416
[MLE-GEN] epoch 20 : pre_loss = 6.7752, NLL_oracle = 9.1259, NLL_gen = 6.7668, NLL_div = 6.7878
[MLE-GEN] epoch 30 : pre_loss = 6.5874, NLL_oracle = 9.0911, NLL_gen = 6.5832, NLL_div = 6.5767
[MLE-GEN] epoch 40 : pre_loss = 6.4855, NLL_oracle = 9.0727, NLL_gen = 6.473, NLL_div = 6.4514
[MLE-GEN] epoch 50 : pre_loss = 6.4151, NLL_oracle = 9.0601, NLL_gen = 6.3907, NLL_div = 6.3667
[MLE-GEN] epoch 60 : pre_loss = 6.3669, NLL_oracle = 9.0677, NLL_gen = 6.3345, NLL_div = 6.3069
[MLE-GEN] epoch 70 : pre_loss = 6.3282, NLL_oracle = 9.0544, NLL_gen = 6.2672, NLL_div = 6.2518
[MLE-GEN] epoch 80 : pre_loss = 6.2894, NLL_oracle = 9.0483, NLL_gen = 6.2203, NLL_div = 6.2093
[MLE-GEN] epoch 90 : pre_loss = 6.2711, NLL_oracle = 9.0562, NLL_gen = 6.1969, NLL_div = 6.1857
[MLE-GEN] epoch 100 : pre_loss = 6.2481, NLL_oracle = 9.0306, NLL_gen = 6.1654, NLL_div = 6.1537
[MLE-GEN] epoch 110 : pre_loss = 6.2363, NLL_oracle = 9.0364, NLL_gen = 6.1292, NLL_div = 6.1452
[MLE-GEN] epoch 120 : pre_loss = 6.2178, NLL_oracle = 9.0427, NLL_gen = 6.1125, NLL_div = 6.1228
[MLE-GEN] epoch 130 : pre_loss = 6.2085, NLL_oracle = 9.0212, NLL_gen = 6.1075, NLL_div = 6.108
[MLE-GEN] epoch 140 : pre_loss = 6.2119, NLL_oracle = 9.022, NLL_gen = 6.1081, NLL_div = 6.1021
[MLE-GEN] epoch 149 : pre_loss = 6.1958, NLL_oracle = 9.0302, NLL_gen = 6.085, NLL_div = 6.0905
Traceback (most recent call last):
File "main.py", line 131, in
inst._run()
File "/home/oneseven/text/instructor/oracle_data/seqgan_instructor.py", line 43, in _run
torch.save(self.gen.state_dict(), cfg.pretrained_gen_path)
File "/home/oneseven/Python-3.6.2/lib/python3.6/site-packages/torch/serialization.py", line 369, in save
with _open_file_like(f, 'wb') as opened_file:
File "/home/oneseven/Python-3.6.2/lib/python3.6/site-packages/torch/serialization.py", line 234, in _open_file_like
return _open_file(name_or_buffer, mode)
File "/home/oneseven/Python-3.6.2/lib/python3.6/site-packages/torch/serialization.py", line 215, in init
super(_open_file, self).init(open(name, mode))
PermissionError: [Errno 13] Permission denied: 'pretrain/oracle_data/gen_MLE_pretrain_seqgan_vanilla_sl20_sn10000.pt'

Hi
I run sentigan with my dataset. It has 48000 recoeds. After 150 epochs for MLE training and 1000 epochs for ADV training, sentigan generate repetitive samples. Why?

Whenever I attached my own custom dataset, this error is coming

Starting Generator MLE Training...
Traceback (most recent call last):
  File "main.py", line 169, in <module>
    inst._run()
  File "/content/drive/My Drive/Colab Notebooks/2/TextGAN-PyTorch/instructor/real_data/seqgan_instructor.py", line 41, in _run
    self.pretrain_generator(cfg.MLE_train_epoch)
  File "/content/drive/My Drive/Colab Notebooks/2/TextGAN-PyTorch/instructor/real_data/seqgan_instructor.py", line 85, in pretrain_generator
    pre_loss = self.train_gen_epoch(self.gen, self.train_data.loader, self.mle_criterion, self.gen_opt)
  File "/content/drive/My Drive/Colab Notebooks/2/TextGAN-PyTorch/instructor/real_data/instructor.py", line 107, in train_gen_epoch
    return total_loss / len(data_loader)
ZeroDivisionError: division by zero

The loss in the code (batchPGLoss) for sentigan is as follows:

loss = -torch.sum(pred * (1 - reward))

why two negative signs? if (1-reward) =Penalty, and we are then negating, doesn't that mean we are minimizing the negated penalty*prediction or maximizing the penalty*prediction. In the paper the author mentions to minimize the following.

The negative sign in the code confuses me, aren't we supposed to minimize torch.sum(pred * (1 - reward))?