fxxjuses / micformer

implement of "Multimodal Information Interaction for Medical Image Segmentation."

Multimodal Information Interaction for Medical Image Segmentation.

Welcome to reproduce our code!!!

Our article is now publicly available on ArXiv（[2404.16371] Multimodal Information Interaction for Medical Image Segmentation (arxiv.org)）. This repository provides the training code for the MM-WHS dataset. If you need to reproduce our results, you can use the training scripts provided in this repository.

Dataset

The dataset used in this paper is the MM-WHS dataset, which can be found at Multi-Modality Whole Heart Segmentation Challenge. Additionally, the data preprocessing method used in this paper can be performed through the registration method described in the text.

We also provided our dataset processing script in prepocess.py, which we can run by changing the file path to get the same data as the article.

python prepocess.py

Run

In addition to providing the MicFormer code, this repository also includes training and testing code for state-of-the-art methods. These include VT-Unet, Swin-Unet, SwinUneter, nnFormer, and MedNeXt.

Citations

@misc{fan2024multimodal,
      title={Multimodal Information Interaction for Medical Image Segmentation}, 
      author={Xinxin Fan and Lin Liu and Haoran Zhang},
      year={2024},
      eprint={2404.16371},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

micformer's Issues

Training details

Dear author, thank you for your excellent work! Can you complete the training details of the readme file, including the dataset format and location?

RuntimeError: ones needs to be contiguous

Traceback (most recent call last):
File "MicFormer/train_mmwhs_noPad.py", line 414, in
main(arguments)
File "MicFormer/train_mmwhs_noPad.py", line 200, in main
loss_.backward()
File "/home/linda/anaconda3/envs/yyf1/lib/python3.7/site-packages/torch/_tensor.py", line 307, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)
File "/home/linda/anaconda3/envs/yyf1/lib/python3.7/site-packages/torch/autograd/init.py", line 156, in backward
allow_unreachable=True, accumulate_grad=True) # allow_unreachable flag
RuntimeError: ones needs to be contiguous

代码请教

你好，很抱歉打扰您。您的论文给了我很大的启发。我非常欣赏你的实验，但是我在尝试复现这个算法的时候遇到了一些麻烦。1.请问from MMWHS_pre.Multi_modal.SymCFNet.models.MICFormer_self import Head 是不是应该改为 from models.MICFormer_self import Head 2. 在你提供的数据预处理代码中做了一个裁剪CT图像中非零像素的区域，请问CT图像CT值范围比较大，直接以0来判断是否会丢失很多重要信息，而一些值为-1000等这种负值其实是黑的没用的，这种是不是没有处理呢？

data

Your paper is very meaningful, but I don't quite understand the format of your dataset placement, can you give me a specific example?

多模态分割的一些问题请教

您好，非常出色的工作。我想请问下文中使用的多模态数据集，这里的多模态是同一个病人的CT和MRI图像。那么这里金标准是基于CT还是MRI勾画的呢。假如是只有CT有金标准的，那么MRI是不是只是作为辅助模态来提供信息帮助CT分割呢。所以，这里是不是送进模型训练之前要先将MRI配准到CT上。 那么这里肯定是不精确对齐吧 ， 这个有考虑到，是怎么解决的呢 ？我不知道是不是我理解的这样多模态分割，就是辅助模态帮助主模态分割。感谢

MM-WHS数据集问题请教

大佬您好！我是刚接触Multimodal Medical Image Segmentation的新手，想请教下大佬多模态数据集除了MM-WHS还有其他数据集吗？

模型输出的维度与标签维度不一致

您好，我输入的维度如上[b c d w h], 但输出的维度中的d维度发生了变化，导致作Loss损失的时候，跟标签d维度对不上，请问如何解决？