thunlp / prompt-transferability

On Transferability of Prompt Tuning for Natural Language Processing

On Transferability of Prompt Tuning for Natural Language Processing (Prompt Transferability)

This is the source code of "On Transferability of Prompt Tuning for Natural Language Processing", an NAACL 2022 paper [pdf].

Overview

Prompt tuning (PT) is a promising parameter-efficient method to utilize extremely large pre-trained language models (PLMs), which can achieve comparable performance to full-parameter fine-tuning by only tuning a few soft prompts. However, PT requires much more training time than fine-tuning. Intuitively, knowledge transfer can help to improve the efficiency. To explore whether we can improve PT via prompt transfer, we empirically investigate the transferability of soft prompts across different downstream tasks and PLMs in this work. We find that (1) in zero-shot setting, trained soft prompts can effectively transfer to similar tasks on the same PLM and also to other PLMs with a cross-model projector trained on similar tasks; (2) when used as initialization, trained soft prompts of similar tasks and projected prompts of other PLMs can significantly accelerate training and also improve the performance of PT. Moreover, to explore what decides prompt transferability, we investigate various transferability indicators and find that the overlapping rate of activated neurons strongly reflects the transferability, which suggests how the prompts stimulate PLMs is essential. Our findings show that prompt transfer is promising for improving PT, and further research shall focus more on prompts' stimulation to PLMs.

Reproduce results in the paper

• Prompt-Transferability-1.0 provides the original codes and details to reproduce the results in the paper.
• Prompt-Transferability-2.0-latest refactors the Prompt-Transferability-1.0 and provides more user-friendly codes for users. In this README.md, we mainly demostrate the usage of the Prompt-Transferability-2.0-latest.

Setups

• python==3.8.0

We recommend to create a new Anaconda environment to manage the required packages.

conda create -n prompt_transfer python=3.8.0
conda activate prompt_transfer
bash requirements.sh

Note: If the system shows error about the torch, please find the proper version that can matches your CPUs or GPUs.

User can also directly create the environment via Prompt-Transferability-2.0-latest/environment.yml.

conda env create -f environment.yml

Dataset

Excute Prompt-Transferability-2.0-latest/download_data.sh to download datasets.

cd Prompt-Transferability-2.0-latest
bash download_data.sh

Usage

You can easily use PromptHub for various perposes, including prompt training, evaluation, cross-task transfer, cross-model transfer, and activated neuron. The Colab notebook and the example script also demonstrate the usages. Or, you can run the bash file to run a quick example.

cd Prompt-Transferability-2.0-latest
bash example/train.sh

The script of train.sh is:

DATASET=sst2
LEARNINGRATE=1e-2
EPOCH=3

python example/train.py \
        --output_dir outputs \
        --dataset $DATASET \
        --learning_rate $LEARNINGRATE \
        --num_train_epochs EPOCH \
        --save_total_limit 1 \
        --evaluation_strategy epoch \
        --save_strategy epoch \
        --load_best_model_at_end true \
        --metric_for_best_model combined_score

The above code train.py shows an example that includes prompt training, evaluation, activated neuron analysis on a specific dataset.

from prompt_hub.hub import PromptHub
from prompt_hub.training_args import PromptTrainingArguments

OUTPUT=outputs
DATASET_1=sst2
DATASET_2=rotten_tomatoes
MODEL=roberta-base
PRMOPTLEN=100
LEARNINGRATE=1e-2


# Training config
args = PromptTrainingArguments(
  output_dir= OUTPUT, 
  dataset= DATASET_1, 
  backbone= MODEL, 
  prompt_len= PRMOPTLEN,
  learning_rate= LEARNINGRATE
)
trainer = PromptHub(args=args)

# Prompt training and evaluation
trainer.train_prompt()
trainer.eval_prompt()

# Cross-task evaluation
cross_task_eval_results = trainer.cross_task_eval(MODEL, DATASET_1, DATASET_1)


# Activated neuron
activated_neuron_before_relu, activated_neuron_after_relu = trainer.activated_neuron(args.backbone, args.dataset)

• OUTPUT: Output directory.
• DATASET_1: Source (training) dataset. This framework supports all datasets here. We select sst2 as the example.
• DATASET_1: Target dataset. This framework supports all datasets here. We select rotten_tomatoes as the example.
• MODEL: Backbone models. This framework supports Bert, Roberta, GPT, and T5 v1.1, currently. We select roberta-base as the example.
• PROMPTLEN: The length of prompt. We set 100 here.
• LEARNINGRATE: Learning rate. We set 1e-2 here.

Detailed Usage

Prompt Tuning

Users can use the well-trained prompts in Prompt-Transferability-2.0-latest/task_prompt_emb or re-train the prompts by your own as the following instruction.

Step 1: initialization of arguments and trainer

We first need to define a set of arguments or configurations, including backbone (backbone model), dataset, prompt_len (the length of soft prompt), etc. Then we instantiate a PromptHub object and provide the parameters.

from prompt_hub.training_args import PromptTrainingArguments

# Training config
OUTPUT=outputs
DATASET_1=sst2
DATASET_2=rotten_tomatoes
MODEL=roberta-base
PRMOPTLEN=100
LEARNINGRATE=1e-2

args = PromptTrainingArguments(
  output_dir= OUTPUT, 
  dataset= DATASET_1, 
  backbone= MODEL, 
  prompt_len= PRMOPTLEN,
  learning_rate= LEARNINGRATE
)
trainer = PromptHub(args=args)

For a complete list of arguments, please refer to Prompt-Transferability-2.0-latest/prompt_hub/training_args.py and HuggingFace transformers.training_arguments for more details.

Step 2: prompt training

Then we start training a soft prompt. (Optional) You can pass the parameters to overwrite the default configurations in the arguments you passed in.

# Optional arguments to overwrite default parameters
# trainer.train_prompt('roberta-large', 'sst2') 
trainer.train_prompt() 

Step 3: prompt evaluation

With the prompt (trained on specific dataset and utilized backbone model), we excute the following code to evaluate its performance.

# Optional arguments to overwrite default parameters
# eval_results = trainer.eval_prompt('roberta-base', 'sst2') 
eval_results = trainer.eval_prompt()

Cross-Task Transfer

Prompt can directly transfer among tasks. Here, we provide an example to transfer the prompt trained from SST2 dataset to Rotten Tomatoes dataset.

cross_task_eval_results = trainer.cross_task_eval('roberta-base', 'sst2', 'rotten_tomatoes')

Cross-Model Transfer

Prompt can utilize a well-trained projector to transfer among different backbones.

Step 1: cross-model Training

We first use SST2 datase and ttrain a projector that can transfer (from roberta-base to roberta-large).

trainer.cross_model_train(source_model='roberta-base', target_model='roberta-large', task='sst2')

Step 2: cross-model evaluation

Then, we utilize it to transfer the prompt among different models.

cross_model_eval_results = trainer.cross_model_eval(source_model='roberta-base', target_model='roberta-large', task='sst2')

Transferability Indicators (Activated neuron)

Prompt can be seen as a paradigm to manipulate PLMs (stimulate artificial neurons) knowledge to perform downstream tasks. We further observe that similar prompts will activate similar neurons; thus, the activated neurons can be a transferability indicator.

Definition of Neurons: the output values between 1st and 2nd layers of feed-forward network FFN (in every layer of a PLM) [Refer to Section 6.1 in the paper]

Step 1: Acquire task-specific neurons

Given a model and the trained task-specific prompt, you can obtain the activated neurons values.

activated_neuron_before_relu, activated_neuron_after_relu = trainer.activated_neuron('roberta-base', 'sst2')

Step 2: Similarity/Transferability between two tasks

You can caculate the similarity/transferability between two prompts via actiaved neurons.

cos_sim = trainer.neuron_similarity(backbone='roberta-base', task1='sst2', task2='rotten_tomatoes')

Step 3: Masked Neurons

To further demonstrate the importance of task-specific neurons, we mask them and find the model performance on the corresponding task will degrade. Visualization of activated neurons is also supported.

eval_metric, mask = trainer.mask_activated_neuron(args.backbone, args.dataset, ratio=0.2)
trainer.plot_neuron()

Citations

Please cite our paper if it is helpful to your work!

@inproceedings{su-etal-2022-transferability,
    title = "On Transferability of Prompt Tuning for Natural Language Processing",
    author = "Su, Yusheng  and
      Wang, Xiaozhi  and
      Qin, Yujia  and
      Chan, Chi-Min  and
      Lin, Yankai  and
      Wang, Huadong  and
      Wen, Kaiyue  and
      Liu, Zhiyuan  and
      Li, Peng  and
      Li, Juanzi  and
      Hou, Lei  and
      Sun, Maosong  and
      Zhou, Jie",
    booktitle = "Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",
    month = jul,
    year = "2022",
    address = "Seattle, United States",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.naacl-main.290",
    doi = "10.18653/v1/2022.naacl-main.290",
    pages = "3949--3969"
}

Contact

Yusheng Su

Mail: [email protected]; [email protected]

prompt-transferability's Issues

How is the performance evaluated when there is no test dataset available?

Hello, Thank you for your work. I have some questions arising from reviewing the paper and its code. Thank you in advance for your response.

1. In cases where a test dataset is not provided (e.g., similar to SQuAD and NQ_Open), how was the performance measured?
2. When examining the code, it seems that the validation dataset is being called during testing. Is my understanding correct?
   (Note : I have reviewed version 1 of the code.)

Regards.

Report Bug in cross-model transfer

crossPrompt.py line317
mask_logits = logits[:, 0] should be modified to mask_logits = logits[:, 100] because there are 100 prompt tokens in the front.

Is T5 also learned during Prompt Tuning?

Hello, Thank you for the good work.

Is T5 also learned during Prompt Tuning?
When I check the learnable layers, I found that not only the prompt but also the T5 model is learned.
I am confused about whether PLM is frozen or not during training because I believe the previous prompt tuning (https://arxiv.org/abs/2104.08691) is that PLM is frozen and only the soft prompt is unfrozen.

▶ Command

gpus=0
BACKBONE=T5
DATASET=SST2

CUDA_VISIBLE_DEVICES=$gpus python3 train.py --config config/${DATASET}Prompt${BACKBONE}.config --gpu $gpus

▶ Add Code lines
I add below codes on line 159 Prompt-Transferability-1.0/tools/train_tool.py (link)

    # Check encoder-decoder requires_grad T/F
    for name, param in model.named_parameters():
        print(name, param.requires_grad)

▶ Results

Sincerely,
Jeewoo Sul

There is a minor issue in 'Prompt_Transferability_1.0/valid.sh'

Hello! Thank you for your good work! I think there is minor typo in valid.sh.

original code :
CUDA_VISIBLE_DEVICES=$gpus python3 valid.py --config config/{DATASET}Prompt${BACKBONE}.config \

fixed code :
CUDA_VISIBLE_DEVICES=$gpus python3 valid.py --config config/${DATASET}Prompt${BACKBONE}.config \

Thank you!

Report bugs in "Prompt-Transferability-2.0-latest/prompt_hub/hub.py "

• line 244
  model, template, verbalizer, plm, tokenizer, model_config, tokenizer_wrapper_class, model_type = self.get_model(args.backbone, processor, args)
  args should be changed to self.args
  model, template, verbalizer, plm, tokenizer, model_config, tokenizer_wrapper_class, model_type = self.get_model(self.args.backbone, processor, self.args)
• The code seems to be only suitable for models like Roberta that use mask to predict labels. For models like t5, I think change PromptForClassification into PromptForGeneration on line 100 might work.

More details on the rules for naming the trained prompts in the "task_prompt_emb" folder

Hello! Thank you greatly for your extensive work and code!

I'm thinking about using the pretrained prompt embeddings directly, which are just stored in the Prompt-Transferability-2.0-latest/task_prompt_emb folder. Great thanks for that!

It's just the names of each sub-folders that confuse me, as I don't quite understand your naming protocol.

For example, if I'm looking for the prompt embeddings trained on the Deontology dataset using RoBERTa model, then ethicsdeontologyPromptRoberta should probably be the right folder.
However, for some datasets and some models, there could be multiple folders. For example, in the case of RoBERTa on IMDB, there are some confusing folders:

• IMDB_base_emotionPromptRoberta
• IMDB_base_nliPromptRoberta
• IMDB_trainedPromptRoberta
• IMDBPromptRoberta

Such confusion makes it a little hard for me to quickly get the right prompt embedding.
So, if it's okay, may I ask what are the differences between folders like that?
Or, in another word, may I ask for a short description of your naming protocol when you save the prompts?

Again, thank you for your work and code. All the best wishes!

question about paper

文中的跨任务cross task是不是也可以叫做跨域cross domain了呢？比如SA情感分类中，movie到restaurant的zero-shot transfer，他们属于同一种任务SA，但是他们的distribution不一样，所以也可是说是cross domain了。若理解有误，见谅

Questions about ON score

Hi,

I have a question about ON score.
It seems like it is kind of cosine similarity between activated neurons.
Then did you get the score by averaging similarities over all inputs?
For example, if my input(aka batch size) is 3 for each task and layer is 12 and hidden dimension is 768, I can get outputs (12,3,786).
So I can get 3 cosine similarities per layer because input is 3, and should I just use the mean value of them?

Also, for the last appendix(D.4), you grouped the layers 1-3/4-6/7-9/10-12.
I'd like to know if you have the results per layer, because in my experiment with GPT3 (24 layers), the ON score in layer 21/22 is very low while 23/24 is very high.

Thanks!