This repository contains the code for our EMNLP 2020 paper:
Blank Language Models
Tianxiao Shen*, Victor Quach*, Regina Barzilay, and Tommi Jaakkola (*: Equal contribution)
Given partially specified text with one or more blanks, BLM will fill in the blanks with a variable number of tokens consistent with the context, making it ideal for text editing and rewriting.
Input: They also have ___ which ___ .
Output: They also have ice cream which is really good .
We have an online demo built using streamlit, available here
Or try locally by running:
streamlit run app.py
Our code is based on the PyTorch Lightning framework.
It has been tested in PyTorch 1.6.0, PyTorch Lightning 1.0.7
We use the Transformer implementation from https://github.com/jadore801120/attention-is-all-you-need-pytorch
If you use our work, please cite:
@inproceedings{shen2020blank,
title = "{Blank Language Models}",
author = "Shen, Tianxiao and
Quach, Victor and
Barzilay, Regina and
Jaakkola, Tommi",
booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing",
month = nov,
year = "2020",
address = "Online",
publisher = "Association for Computational Linguistics"
}