xrc10 / formal-sty-trans

source code for "Formality Style Transfer with Hybrid Textual Annotations"

Overview

This repo contains source code for "Formality Style Transfer with Hybrid Textual Annotations"

Install

The software is built upon fairseq, follow the instruction on this link for installation

Data

The preprocessed data could be downloaded from this link.

Usage

To replicate our experiment on the GYAFC dataset, see the pipeline.sh script for a working example.

The general usage is similar to fairseq with the same syntax for arguments. The only difference is to make sure the --task is style-transfer and arch is sty-transformer.

Training with supervised paired data: train.py

usage: train.py [-h] DIR
                --task style_transfer
                --arch sty_transformer [--criterion CRIT] [--max-epoch N]
                [--max-update N] [--clip-norm NORM] [--sentence-avg]
                [--mt-loss-weight] [--classify-loss-weight]
                [--self-recon-loss-weight] [--cycle-recon-loss-weight]
                [--max-source-positions] [--max-target-positions]
                [--encoder-embed-dim] [--encoder-ffn-embed-dim]
                [--decoder-embed-dim] [--decoder-ffn-embed-dim]
                [--encoder-attention-heads] [--decoder-attention-heads]
                [--encoder-layers] [--decoder-layers]
                [--optimizer OPT] [--lr LR_1,LR_2,...,LR_N]
                [--momentum M] [--weight-decay WD]
                [--lr-scheduler LR_SCHEDULER] [--lr-shrink LS] [--min-lr LR]
                [--save-dir DIR]
                [--restore-file RESTORE_FILE] [--save-interval N]
                [--save-interval-updates N] [--keep-interval-updates N]
                [--no-save] [--no-epoch-checkpoints] [--validate-interval N]

Generate with trained model: generate.py

usage: generate.py [-h] DIR
                   --task style_transfer  
                   [--no-progress-bar] [--log-interval N]
                   [--remove-bpe [REMOVE_BPE]] [--beam N]
                   [--nbest N]
                   [--output-path OUTPUT_PATH]

formal-sty-trans's Issues

AttributeError: 'NoneType' object has no attribute 'items'

  File "train.py", line 469, in <module>
    main(args)
  File "train.py", line 181, in main
    trg_plain_epoch_iter=trg_plain_epoch_iter,
  File "train.py", line 241, in train
    for k, v in log_output.items():

I get this error while training (I ran pipeline.sh). log_output is None. Can someone suggest what is wrong?

Hello, I got this error while training this model with GYAFC dataset but not working. How should I give processed dataset to the terminal to train the model?

Namespace(adaptive_softmax_cutoff=None, arch='sty_transformer', attention_dropout=0.0, clip_norm=25, cnn_dropout=0.5, cnn_filters=100, criterion='cross_entropy', data='/content/formal-sty-trans/pro_data.txt', decoder_attention_heads=8, decoder_embed_dim=512, decoder_embed_path=None, decoder_ffn_embed_dim=2048, decoder_layers=6, decoder_learned_pos=False, decoder_normalize_before=False, device_id=0, distributed_backend='nccl', distributed_init_method=None, distributed_port=-1, distributed_rank=0, distributed_world_size=1, dropout=0.1, encoder_attention_heads=8, encoder_embed_dim=512, encoder_embed_path=None, encoder_ffn_embed_dim=2048, encoder_layers=6, encoder_learned_pos=False, encoder_normalize_before=False, fp16=False, keep_interval_updates=-1, left_pad_source='True', left_pad_target='False', log_format=None, log_interval=1000, lr=[0.25], lr_scheduler='reduce_lr_on_plateau', lr_shrink=0.1, max_epoch=0, max_sentences=None, max_sentences_valid=None, max_source_positions=1024, max_target_positions=1024, max_tokens=6000, max_update=0, min_loss_scale=0.0001, min_lr=1e-05, momentum=0.99, no_epoch_checkpoints=False, no_progress_bar=False, no_save=False, no_token_positional_embeddings=False, optimizer='nag', pre_train_max_epoch=0, raw_text=False, relu_dropout=0.0, restore_best=False, restore_file='checkpoint_last.pt', save_dir='checkpoints', save_interval=1, save_interval_updates=0, seed=1, sentence_avg=False, share_all_embeddings=True, share_decoder_input_output_embed=False, skip_invalid_size_inputs_valid_test=False, source_lang=None, target_lang=None, task='translation', train_subset='train', update_freq=[1], valid_subset='valid', validate_interval=1, weight_decay=0.0)
Traceback (most recent call last):
File "train.py", line 469, in
main(args)
File "train.py", line 32, in main
task = tasks.setup_task(args)
File "/content/formal-sty-trans/formal-sty-trans/fairseq/tasks/init.py", line 19, in setup_task
return TASK_REGISTRY[args.task].setup_task(args)
File "/content/formal-sty-trans/formal-sty-trans/fairseq/tasks/translation.py", line 57, in setup_task
args.source_lang, args.target_lang = data_utils.infer_language_pair(args.data)
File "/content/formal-sty-trans/formal-sty-trans/fairseq/data/data_utils.py", line 21, in infer_language_pair
for filename in os.listdir(path):
NotADirectoryError: [Errno 20] Not a directory: '/content/formal-sty-trans/pro_data.txt'