Why expect Z in Adapter? about tasknet

[sileod]

Hi, great question

It is used here:

tasknet/src/tasknet/utils.py

Line 210 in c9f4360

if adapt_task_embedding:

But actually, it would be cleaner to have it in adapt_model_to_task
I'll try to do it for the next release

The general idea is to have a shared encoder, one classifier per task (unless some task share all their labels), and task embedding per task
The task embedding is randomly dropped at 10% rate to work without using it, but it allows the model to "see" the task it should do and it improves results, so it is best to add it alongsides the classifier. It's actually the core of the Adapter