How to implement the unconcat version? about uno

[DonkeyShot21]

The branch has been merged into main.

[DonkeyShot21]

Hi,

Thanks for your interest in our work.

I don't remember exactly how I implemented it but for sure you will need to:

1. do not concat here
   

   UNO/main_discover.py

   Line 155 in 1d7811f

   logits = torch.cat([outputs["logits_lab"], outputs["logits_unlab"]], dim=-1)

2. the shape of this tensor will also change
   

   UNO/main_discover.py

   Line 164 in 1d7811f

   targets = torch.zeros_like(logits)

3. change the slicing in this loop
   

   UNO/main_discover.py

   Line 167 in 1d7811f

   # generate pseudo-labels with sinkhorn-knopp and fill unlab targets

4. use the swapped prediction only for the unlab samples
   

   UNO/main_discover.py

   Line 179 in 1d7811f

   # compute swapped prediction loss

5. here add cross-entropy on the labeled samples, something like F.cross_entropy(logits_lab, targets_lab)
   

   UNO/main_discover.py

   Line 189 in 1d7811f

   loss = (loss_cluster + loss_overcluster) / 2

Something like this. It should not be too hard. Remember that the repo now contains code for UNO v2 that performs much better than the original version, so results are gonna be different.

[ziyunli-2023]

what's the main difference between v1 and v2? I note that you fix cross_entropy loss.
And I also have another problem? do you think multiple heads(unlabel heads) help a lot for the performance?

[ziyunli-2023]

Thank you so much for the fast reply! I will implement it following your guides

[DonkeyShot21]

The main difference is in the augmentations. I recommend you use the new version. With UNO v2 multiple heads do not help as much as before, so if you want to speed up computation you can decrease the number of heads.

[JosephKJ]

Hi Enrico,

The main difference is in the augmentations.

But, I find that both transforms.py are identical in master and v2 branch. Had the new changes been ported to both the branches alike?

Kindly clarify when you are free. Thanks in advance!

~ Joseph

[JosephKJ]

Great! Thanks!