hzfinfdu / mpmp

ACL'2023: Multi-Task Pre-Training of Modular Prompt for Few-Shot Learning

Multi-Task Pre-Training of Modular Prompt for Few-Shot Learning

Implementation of Multi-Task Pre-Training of Modular Prompt for Few-Shot Learning (https://arxiv.org/abs/2210.07565).

Release

Oct 22, 2022. Released the first version of MPMP and the 45 datasets on Google Drive.

Installation

python 3.7 is recommended.

pip install transformers==4.11.3
pip install datasets
pip install cma
pip install fastNLP
pip install sklearn
pip install bayesian-optimization
pip install scipy

Training

Please modify the meta path to the directory that contains the datasets in dataconfig.py.

Use the default parameters in run.py to reproduce the training of MPMP.

sh run.sh

mpmp's Issues

预训练模型和推理脚本

你好，可以分享下预训练模型和推理脚本吗，谢谢

About task_id

In the pre-train stage, the sample should use corresponding task router (using task_id param) to get fused prompt embedding, but in the code, I found task_id set to 0 always in https://github.com/Hzfinfdu/MPMP/blob/master/Deep/modeling_bert.py#L548, is that any thing wrong with it ?

by the way, in the inference stage, what task_id number should the new tasks (e.g. unseen task category) use ?