jind11 / titlestylist

Source code for our "TitleStylist" paper at ACL 2020

TitleStylist

Source code for our "TitleStylist" paper at ACL 2020: Jin, Di, Zhijing Jin, Joey Tianyi Zhou, Lisa Orii, and Peter Szolovits. "Hooks in the Headline: Learning to Generate Headlines with Controlled Styles." ACL (2020).. If you use the code, please cite the paper:

@inproceedings{jin2020hooks,
  author    = {Di Jin and Zhijing Jin and Joey Tianyi Zhou and Lisa Orii and Peter Szolovits},
  title     = {Hooks in the Headline: Learning to Generate Headlines with Controlled
               Styles},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational
               Linguistics, {ACL} 2020, Online, July 5-10, 2020}, pages = {5082--5093},
  publisher = {Association for Computational Linguistics}, year = {2020},
  url       = {https://www.aclweb.org/anthology/2020.acl-main.456/}
}

Here is a talk that introduces our work.

Requirements

Python packages

• Pytorch
• fairseq
• blingfire

In order to install them, you can run this command:

pip install -r requirements.txt

Bash commands

In order to evaluate the generated headlines by ROUGE scores, you need to install the "files2rouge" package. To do so, run the following commands (provided by this repository):

pip install -U git+https://github.com/pltrdy/pyrouge
git clone https://github.com/pltrdy/files2rouge.git     
cd files2rouge
python setup_rouge.py
python setup.py install

Usage

1. All data including the combination of CNN and NYT article and headline pairs, and the three style-specific corpora (humor, romance, and clickbait) mentioned in the paper have been placed in the folder "data".

2. Please download the pretrained model parameters of MASS from this link, unzip it, and put the unzipped files into the folder "pretrained_model/MASS".

3. To train a headline generation model that can simultaneously generated a facutal and a stylistic headline, you can run the following command:

./train_mix_CNN_NYT_X.sh --style YOUR_TARGET_STYLE

Here the arugment YOUR_TARGET_STYLE specifies any style you would like to have, in this paper, we provide three options: humor, romance, clickbait.

After running this command, the trained model parameters will be saved into the folder "tmp/exp".

4. If you want to evaluate the trained model and generate headlines (both factual and stylistic) using this model, please run the following command:

./evaluate_mix_CNN_NYT_X.sh --style YOUR_TARGET_STYLE --model_dir MODEL_STORED_DIRCTORY

In this command, the argument MODEL_STORED_DIRCTORY specifies the directory which stores the trained model.

5. If you want to train and evaluate the headline generation model for more than one style, run the following command:

./train_mix_CNN_NYT_multiX.sh
./evaluate_mix_CNN_NYT_multiX.sh --model_dir MODEL_STORED_DIRCTORY

Extension

For the humorous style, although we used humorous novels, you can also try the following datasets:

• 16000 One-Liners (16K humorous)
• Pun of the Day (16K humorous)
• Short Jokes (231K humorous)
• Plaintext Jokes (208K humorous)

We suggest that the large dataset Short Jokes is likely to generate good headlines.

titlestylist's Issues

Can not get your result

After doing the data preprocessing, training and evaluation for humorous headline generation provided by this repo, i get the following result. The BLEU score is 9.0, much lower than you have written in your paper(13.3).
Is there anything wrong?

[Errno 2] No such file or directory: 'data/CNN_NYT/processed/dict.src.txt'

where is pretrained_model/MASS/dict.txt

Hi, thanks for your outstanding work.
Here I want to preprocess my own dataset using data_process.sh, but the srcdict and tgtdict files are missing in L24 and L25 below. Could you tell me where to find them?

TitleStylist/data_process.sh

Lines 17 to 26 in 752a1b8

	fairseq-preprocess \
	--user-dir mass --task masked_s2s \
	--source-lang src --target-lang tgt \
	--trainpref $DATA_DIR/train \
	--validpref $DATA_DIR/valid \
	--testpref $DATA_DIR/test \
	--destdir $OUT_DIR \
	--srcdict pretrained_model/MASS/dict.txt \
	--tgtdict pretrained_model/MASS/dict.txt \
	--workers 20

Error: module name (mass) is not globally unique

I ran sh train_mix_CNN_NYT_X.sh, and met this error.

Traceback (most recent call last):
  File "train.py", line 337, in <module>
    cli_main()
  File "train.py", line 300, in cli_main
    parser = options.get_training_parser()
  File "/root/anaconda3/envs/titlestyle/lib/python3.6/site-packages/fairseq/options.py", line 34, in get_training_parser
    parser = get_parser("Trainer", default_task)
  File "/root/anaconda3/envs/titlestyle/lib/python3.6/site-packages/fairseq/options.py", line 210, in get_parser
    utils.import_user_module(usr_args)
  File "/root/anaconda3/envs/titlestyle/lib/python3.6/site-packages/fairseq/utils.py", line 461, in import_user_module
    "something unique and try again.".format(module_path, module_name)
ImportError: Failed to import --user-dir=/home/baojunshan/TitleStylist/mass because the corresponding module name (mass) is not globally unique. Please rename the directory to something unique and try again.

I also changed the folder name but did not help.

Do you have the pretrained model?

I downloaded MASS model
Trying thing like that
./evaluate_mix_CNN_NYT_X.sh --style clickbait --model_dir ./pretrained_model
but get error: OSError: Model file not found: ./pretrained_model/checkpoint_best.pt
I agree with that, but where can I get the checkpoint for correct model?

Thank you and have a nice day!%)

4090 bug

Hello author, I would like to ask whether this model can run on the 4090 machine? When I tried to reproduce this paper, there was a bug that I can't solve at present. I guess this may be due to the problem that my pytorch version is too new. I would like to ask if anyone can run this code on the 4090 machine? Here's a failure case I'm having, it happens during the evaluation phase

can't generate headlines

when i use the command'./evaluate_mix_CNN_NYT_X.sh --style YOUR_TARGET_STYLE --model_dir MODEL_STORED_DIRCTORY' ,there is a problem'TypeError: init() takes 1 positional argument but 6 were given',is it a code wrong or else?