Discussion: About Knowledge Editing about memit

[Robin-WZQ]

Just for discussion @guanghuixu. For question 1:

In MEMIT, authors said "In each individual layer l, we wish to store a large batch of u ≫ 1 memories. " u here means the new associations that we want the model to learn.

On the next line, authors said "assuming that the layer contains previously-stored memories that should be preserved." I believe i means the memorized associations that we want the model to preserve.

I guess the reason why author write like that is to emphasize the batch of associations updating. As we can see on author's previous work ROME (eq 2), they use :

$the \ W \ that \ minimized\ ||WK-V||^2_F$

where I believe it shares a same meaning with:

$W_0 = argmin_W\ \sum||\bar{W}k_i-m_i||^2 \ \ (equation\ 9\ in\ MEMIT)$