doubtedsteam / mpanet Goto Github PK

Hello, excuse me, I'm very interested in this visual part of the code, can you share it? Thank you

您好，我按照您的代码训练时，报了以下错误：
FileNotFoundError: [Errno 2] No such file or directory: './SYSU-MM01/exp/rand_perm_cam.mat'
SYSU-MM01数据集的exp文件夹下没有rand_perm_cam.mat文件，我该怎么办呢？

Hello, I want to know where is the 'rand_perm_cam.mat', is it generated by your code?

In the file 'MPANet/models/baseline.py', the 'train_forward' function calculates the final loss composed of multiple components. In the following code snippet, I notice that 'v_cls_loss' and 'i_cls_loss' are added twice.

        if self.mutual_learning:
            # cam_ids = kwargs.get('cam_ids')
            # sub = (cam_ids == 3) + (cam_ids == 6)
            
            logits_v = self.visible_classifier(feat[sub == 0])
            v_cls_loss = self.id_loss(logits_v.float(), labels[sub == 0])
            **loss += v_cls_loss * self.weight_sid**
            logits_i = self.infrared_classifier(feat[sub == 1])
            i_cls_loss = self.id_loss(logits_i.float(), labels[sub == 1])
            **loss += i_cls_loss * self.weight_sid**

            logits_m = torch.cat([logits_v, logits_i], 0).float()
            with torch.no_grad():
                self.infrared_classifier_.weight.data = self.infrared_classifier_.weight.data * (1 - self.update_rate) \
                                                 + self.infrared_classifier.weight.data * self.update_rate
                self.visible_classifier_.weight.data = self.visible_classifier_.weight.data * (1 - self.update_rate) \
                                                 + self.visible_classifier.weight.data * self.update_rate

                logits_v_ = self.infrared_classifier_(feat[sub == 0])
                logits_i_ = self.visible_classifier_(feat[sub == 1])

                logits_m_ = torch.cat([logits_v_, logits_i_], 0).float()
            logits_m = F.softmax(logits_m, 1)
            logits_m_ = F.log_softmax(logits_m_, 1)
            mod_loss = self.KLDivLoss(logits_m_, logits_m) 

            **loss += mod_loss * self.weight_KL + (v_cls_loss + i_cls_loss) * self.weight_sid**
            metric.update({'ce-v': v_cls_loss.data})
            metric.update({'ce-i': i_cls_loss.data})
            metric.update({'KL': mod_loss.data})

Did you do it on purpose with double 'self.weight_sid'?

Hello，where can i get the features without dual path for TSNE ，like ‘ features/sysu/query-sysu-test-nodual-nore-adam-16x8-grey_model_150.mat’

I noticed that the Baidu Pan link of checkpoints has failed.
Can you provide it again?

I am interested in the visualization of learned features in this paper.

Could you shared the code with me? Thank you!

I have tried multiple trainings with MPANet
But it is consistently getting about 66% of Rank-1 in single shot and all mode of sysu dataset

Is there any method to reproduce the Rank-1 accuracy(70.58%) that is written in your MPANet paper?

by the way this is my environment settings
torch == 1.13.0
torchvision == 0.14.0
ignite == 1.1.0
apex == 0.1

Hi~
It's a nice job. The proposed CC loss seems to narrow the modality gap, which is often done by triplet loss. So, how about replacing CC with the triplet loss?

Thanks~

How to get the visualization result like Fig.3?

I was trying to get results but I don't have rand_perm_cam.mat file in my SYSU-dataset. Could you share your file or share its format.

@DoubtedSteam
Hi~
How can I eval the model and get the result in your paper after training?

Thanks!

请问论文中的结果是按照这个配置训练出来的吗？
https://github.com/DoubtedSteam/MPANet/blob/main/configs/SYSU.yml
我在自己机器上测试基本要比论文低5个点左右。