Slice-Wise Knowledge Transfer for Auto Annotation on Serial Medical Imaging Using Optical-Flow-Based Extrapolation
Preview Implementation of the paper
- Yiqin Zhang
- Qingkui Chen (Corresponding Author)
- Meiling Chen
- Zhengjie Zhang
- Chen Huang
- Zhibing Fu
- Binchan Wang
The First Author's Email: [email protected]
Corresponding Author's Email: [email protected]
Affiliations:
-
School of Optical-electrical and Computer Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China
- Yiqin Zhang
- Qingkui Chen
- Meiling Chen
- Zhibing Fu
-
School of Health Science and Engineering, University of Shanghai for Science and Technology, Shanghai, 200093, China
- Zhengjie Zhang
- Binchan Wang
-
Department of Gastrointestinal Surgery, Shanghai General Hospital, Shanghai Jiao Tong University, Shanghai, 201600, China
- Chen Huang
Deep-Learning-Based Medical imaging analysis faces significant challenges due to the high cost of annotation, which has limited its dataset size compared to other areas of computer vision. The majority of current research concentrates on advancing neural network architectures. However, there is a notable lack of studies focused on augmenting and enhancing medical imaging datasets specifically. From an intuitive perspective, we examined the commonalities and distinctions between medical and conventional imaging, concluding that optical flow-based annotation augmentation is highly suitable for medical data. We further refined the optical flow and remapping processes in line with the unique aspects of medical annotation, leading to a robust and substantial expansion of our few-shot datasets. Our experiments across various few-shot datasets confirm that, without additional manual annotation, multiple famous neural networks can achieve a notable boost in accuracy.
Please see ./requirement/*.
- Ubuntu 22.02 on WSL2
- Ampere or above GPU
- MMengine 0.10.4
- MMsegmentation 1.2.2
- PyTorch 2.5.0+ (current nightly)
- CuDNN 9.1.0+
- Our Medical Image Analysis Toolkit
All experiments configurations are available in ./Project/configs and based on mmengine framework. So you may be familar to openmim project to better reproducibility. OpenMIM docs are available in mmenging and mmsegmentation.
I (The first author) am a contributor to the openmim open source project. If you have any questions, please feel free to reach out to me via email: [email protected].
Please access our Medical Image Analysis Toolkit, where our previous work and this work are included.
| Criterion | DATransUnet | DconnNet | LM Net | MedNext | Resnet50 | Swin Transformer | Swin UMamba | VMamba | ||
|---|---|---|---|---|---|---|---|---|---|---|
| Dice | 71.36 | 67.33 | 66.31 | 70.14 | 65.65 | 60.51 | 59.79 | 58.49 | ||
| Pre. | 83.25 | 76.14 | 70.39 | 73.52 | 71.53 | 65.56 | 74.34 | 66.73 | ||
| Rec. | 66.55 | 64.40 | 63.47 | 70.06 | 61.97 | 57.30 | 53.52 | 54.99 | ||
| - | - | - | - | - | - | - | - | - | - | - |
| 3 | 1 | Dice | 70.79 | 73.22 | 67.21 | 74.40 | 69.66 | 60.55 | 63.19 | 67.50 |
| Pre. | 81.43 | 81.88 | 69.84 | 80.91 | 77.21 | 71.40 | 73.61 | 75.17 | ||
| Rec. | 65.31 | 69.75 | 64.99 | 71.45 | 65.31 | 55.21 | 58.23 | 63.30 | ||
| Imp. | -1.21 | +5.66 | +0.62 | +4.35 | +4.35 | +1.26 | +2.46 | +8.58 | ||
| - | - | - | - | - | - | - | - | - | - | - |
| 3 | 5 | Dice | 73.13 | 72.27 | 70.69 | 65.67 | 69.22 | 58.50 | 71.85 | 60.92 |
| Pre. | 76.40 | 81.44 | 74.24 | 72.08 | 76.02 | 60.83 | 76.74 | 70.74 | ||
| Rec. | 71.20 | 69.20 | 69.41 | 66.19 | 65.52 | 57.40 | 70.99 | 56.74 | ||
| Imp. | -0.14 | +5.01 | +4.72 | -3.26 | +3.88 | -2.21 | +10.64 | +2.73 | ||
| - | - | - | - | - | - | - | - | - | - | - |
| 5 | 3 | Dice | 72.48 | 71.90 | 70.65 | 69.49 | 63.92 | 65.04 | 53.72 | 62.18 |
| Pre. | 84.74 | 81.22 | 75.80 | 75.79 | 78.99 | 71.73 | 69.11 | 71.94 | ||
| Rec. | 67.55 | 69.60 | 66.88 | 66.99 | 57.84 | 61.63 | 47.92 | 57.65 | ||
| Imp. | +1.20 | +4.95 | +4.38 | -0.48 | +0.53 | +5.01 | -5.63 | +3.85 | ||
| - | - | - | - | - | - | - | - | - | - | - |
| 7 | 1 | Dice | 74.37 | 66.49 | 69.71 | 71.23 | 68.65 | 65.54 | 54.94 | 60.91 |
| Pre. | 89.30 | 82.11 | 75.95 | 81.92 | 82.18 | 72.30 | 70.62 | 72.95 | ||
| Rec. | 69.24 | 61.64 | 65.32 | 69.52 | 63.11 | 62.38 | 48.13 | 55.48 | ||
| Imp. | +3.92 | +0.79 | +3.60 | +2.99 | +4.93 | +5.62 | -4.66 | +3.04 | ||
| - | - | - | - | - | - | - | - | - | - | - |
| 9 | 1 | Dice | 75.01 | 69.26 | 67.22 | 71.89 | 69.91 | 65.47 | 52.45 | 63.50 |
| Pre. | 83.25 | 78.02 | 72.37 | 80.89 | 73.83 | 69.88 | 64.24 | 69.32 | ||
| Rec. | 71.04 | 66.19 | 64.24 | 69.12 | 68.46 | 62.48 | 47.93 | 62.30 | ||
| Imp. | +2.71 | +1.87 | +1.22 | +2.73 | +4.35 | +4.82 | -7.68 | +4.97 | ||
| - | - | - | - | - | - | - | - | - | - | - |
N/A
