About task_id about mpmp

[Hzfinfdu]

Hi, our model implements an inform_model function to get task_id from the batch in this part of our code.
The current task_id is included in the batch here.

In the inference stage, or we say downstream adapting stage, the model re-initialize an only router here(i.e. with only 1 task) and always set the task_id to 0 (and that is why in modeling_bert the task_id is by default 0 so that we do not need to explicitly operate the task_id). Optimizing this router (or along with the prompts) yields a downstream composition of the pretrained prompts.
btw, if we have some correlation prior in between the downstream task and the pretraining tasks, we can also initialize the router with a pretrained router. In this case, the task_id is also 0.