关于SYSU测试集的问题 about mambacd

[ChenHongruixuan]

Hi,

Thank you so much for your interest to our work. Some of these datasets have validation sets and some do not. For the datasets with validation sets, we choosed the model with the best accuracy on the validation set. For those without, we choosed the best model based on their accuracy on the test set. Here the above write-up was used because of the need for a consistent code form. Of course, you can also take the N-fold way to get better results.

Best,

[docShen]

Hi,

Thank you so much for your interest to our work. Some of these datasets have validation sets and some do not. For the datasets with validation sets, we choosed the model with the best accuracy on the validation set. For those without, we choosed the best model based on their accuracy on the test set. Here the above write-up was used because of the need for a consistent code form. Of course, you can also take the N-folder way to get better results.

Best,

thanks!

[ChenHongruixuan]

My pleasure~

[pingcong]

非常感谢这篇MambaCD的工作，我在复现代码的时候发现，代码并没有用到验证集，而是在每个固定的iteration进行了测试集的评估。如： 在sh中指定的是： --test_dataset_path '<dataset_path>/SYSU/test' 是测试集的路径，而不是验证集 在训练脚本中： if (itera + 1) % 10 == 0: print(f'iter is {itera + 1}, overall loss is {final_loss}') if (itera + 1) % 500 == 0: self.deep_model.eval() rec, pre, oa, f1_score, iou, kc = self.validation() if kc > best_kc: torch.save(self.deep_model.state_dict(), os.path.join(self.model_save_path, f'{itera + 1}_model.pth')) best_kc = kc best_round = [rec, pre, oa, f1_score, iou, kc] self.deep_model.train()

print('The accuracy of the best round is ', best_round)

看上去像是在测试集中找到最好的performance，请问论文中报告的performance是否是用这种方式找到的呢？ 非常感谢

我自己在SYSU数据集上复现，为什么精度很差呀，是下载的源码跟权重，弹出这个_IncompatibleKeys(missing_keys=['outnorm0.weight', 'outnorm0.bias', 'outnorm1.weight', 'outnorm1.bias', 'outnorm2.weight', 'outnorm2.bias', 'outnorm3.weight', 'outnorm3.bias'], unexpected_keys=['classifier.norm.weight', 'classifier.norm.bias', 'classifier.head.weight', 'classifier.head.bias'])
Backbone_VSSM load_pretrained

[pingcong]

您好，这篇MambaCD的工作很棒，想在自己数据集上测验下效果，对我们非常有用。下载git链接上对应的预训练权重文件，进行推理时候复现的时候出现了IncompatibleKeys(missing_keys=['outnorm0.weight', 'outnorm0.bias', 'outnorm1.weight', 'outnorm1.bias', 'outnorm2.weight', 'outnorm2.bias', 'outnorm3.weight', 'outnorm3.bias'], unexpected_keys=['classifier.norm.weight', 'classifier.norm.bias', 'classifier.head.weight', 'classifier.head.bias'])
Backbone_VSSM load_pretrained，导致在论文截图的那几张影像上表现都很差，有什么注意事项吗，