chin-gyou / improving-variational-encoder-decoders-in-dialogue-generation

Tensorflow Implementation of Improving Variational Encoder-Decoders in Dialogue Generation

Improving Variational Encoder-Decoders in Dialogue Generation

Description

This repository hosts the improved VEDs(Variational Encoder-Decoders) model for generative dialog modeling as described by Shen and Su et al. 2018

Creating Datasets

1. Download the DailyDialog Corpus as released by Li, Su and Shen et al. (2017) which can be found : http://yanran.li/dailydialog.html
2. Create the dictionary from the corpus and Serialize the dicitonary and corpus.(we give a demo convert_text2dict.py for creating pkl file)
3. Download Word2Vec trained by GoogleNes: https://drive.google.com/file/d/0B7XkCwpI5KDYNlNUTTlSS21pQmM.
4. Changing dataproducer.py to generate tfrecord from the serialized corpus text(We use TFRecord for fast and stable training process)

Model Training

We use Tensorflow1.0 and Python2.7 for convenient.

1. Create a new "Checkpoints" directories inside it.
2. Change the parameters in main.py according to your GPU memeory size.
3. Read the core code file WS_vhred.py
4. Change and Run main.py

References

Improving Variational Encoder-Decoders in Dialogue Generation. Xiaoyu Shen, Hui Su, Shuzi Niu, Vera Demberg. 2018. AAAI https://arxiv.org/abs/1802.02032
The DailyDialog Corpus: DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset. Yanran Li, Hui Su, Xiaoyu Shen, Wenjie Li, Ziqiang Cao, Shuzi Niu. 2017. IJCNLP. https://arxiv.org/abs/1710.03957.

improving-variational-encoder-decoders-in-dialogue-generation's Issues

Can't train model with dialogue length limit=500 on even NVIDIA Volta GPU

Hi,

I want to train my dataset with dataproducer.py limit set to 500. It seems like https://github.com/snakeztc/NeuralDialog-CVAE can easily handle that and even bigger (limit=1000) with batch size 128. I'm using NVIDIA Volta GPU.

There'es some inefficiency going on here that I can't seem to solve.
Do you have any idea how can I go about solving this?

Thanks.

RuntimeError: select(): index 19 out of range for tensor of size [19, 32, 30000] at dimension 0

I am new about pytorch, what's wrong about this problem?
Traceback (most recent call last): File "/home/yangjingying/PycharmProjects/VariationalSeq2Seq-master/main.py", line 269, in <module> save_every=save_every, sample_every=sample_every, print_every=print_every, learning_rate=learning_rate) File "/home/yangjingying/PycharmProjects/VariationalSeq2Seq-master/main.py", line 142, in train criterion, kl_anneal_weight, p_teach_force) File "/home/yangjingying/PycharmProjects/VariationalSeq2Seq-master/main.py", line 100, in _train_step loss.backward() File "/home/yangjingying/anaconda2/lib/python2.7/site-packages/torch/tensor.py", line 102, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph) File "/home/yangjingying/anaconda2/lib/python2.7/site-packages/torch/autograd/__init__.py", line 90, in backward allow_unreachable=True) # allow_unreachable flag RuntimeError: select(): index 19 out of range for tensor of size [19, 32, 30000] at dimension 0 Closing remaining open files:./data/train.h5...done