GithubHelp home page GithubHelp logo

textgail's Introduction

TextGAIL: Generative Adversarial Imitation Learning for Text Generation

This is the repository for the paper: TextGAIL: Generative Adversarial Imitation Learning for Text Generation

Requirements

TorchFly (https://github.com/qywu/TorchFly) is needed for all dependencies. It is included in this repository.

Datasets

To run the code, You need to download the following datasets, and store them under the folder data.

The datasets used in the paper can be found below:

Then run the pre-processing script to get train.jsonl, valid.jsonl, test.jsonl for each dataset. The details can be found in the data folder.

Training

Pre-trained Weights

We use GPT-2 small (117M) and RoBERTa base (117M) as the pre-trained model weights, which are automatically downloaded and loaded.

Scripts

To obtain the best MLE models, please run scripts in the Conditional/MLE folder or Conditional/MLE first.

cd Conditional/MLE
#  choose the task.name in {CommonGEN, DailyDialog}
python main.py --config config/config.yaml task.name=CommonGEN

To train TextGAIL, please specify the MLE model after warm-up training. Then simply run the following command.

python main.py --config config/config.yaml task.name=CommonGEN task.weights_path="../../../MLE/outputs/CommonGEN/Checkpoints/iter_252_model_state.pth"

All hyper-paramters are stored in YAML format and will be automatically loaded by the script. You can adjust them according to your task.

Evaluation

Please run Generation.ipynb to have the geenerated outputs before the evaluation.

Conditional generation tasks and unconditional generation tasks have different evaluation metrics. Therefore, please refer to the Evaluation folder for details.

We have provided generated outputs along with the generated outputs for evaluation.

Citation

@inproceedings{wu2021textgail,
  author    = {Qingyang Wu and
               Lei Li and
               Zhou Yu},
  title     = {TextGAIL: Generative Adversarial Imitation Learning for Text Generation},
  booktitle = {The Thirty-Fifth {AAAI} Conference on Artificial Intelligence, {AAAI} 2021},
  pages     = {online},
  publisher = {{AAAI} Press},
  year      = {2021},
}

Contact

Feel free to contact me ([email protected]) if you have any questions.

textgail's People

Contributors

mrazizi avatar qywu avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.