zyrant / lsnet

[TIP2023][LSNet]Lightweight Spatial Boosting Network for Detecting Salient Objects in RGB-Thermal Images

LSNet

This project provides the code and results for 'LSNet: Lightweight Spatial Boosting Network for Detecting Salient Objects in RGB-Thermal Images', IEEE TIP, 2023. IEEE link

Requirements

Python 3.7+, Pytorch 1.5.0+, Cuda 10.2+, TensorboardX 2.1, opencv-python
If anything goes wrong with the environment, please check requirements.txt for details.

Architecture and Details

Results

Data Preparation

• Download the RGB-T raw data from baidu, pin: sf9y / Google drive
  
• Download the RGB-D raw data from baidu, pin: 7pi5 / Google drive
  

Note that the depth maps of the raw data above are foreground is white.

Training & Testing

modify the train_root train_root save_path path in config.py according to your own data path.

• Train the LSNet:

  python train.py

modify the test_path path in config.py according to your own data path.

• Test the LSNet:

  python test.py

Note that task in config.py determines which task and dataset to use.

Evaluate tools

• You can select one of toolboxes to get the metrics CODToolbox / PySODMetrics

Saliency Maps

• RGB-T baidu pin: fxsk / Google drive
  
• RGB-D baidu pin: 6352 / Google drive
  

Note that we resize the testing data to the size of 224 * 224 for quicky evaluate.
please check our previous works APNet and CCAFNet.

Pretraining Models

• RGB-T baidu pin: wnoa / Google drive
  
• RGB-D baidu pin: wnoa / Google drive
  

Citation

    @ARTICLE{Zhou_2023_LSNet,
                author={Zhou, Wujie and Zhu, Yun and Lei, Jingsheng and Yang, Rongwang and Yu, Lu},
                journal={IEEE Transactions on Image Processing}, 
                title={LSNet: Lightweight Spatial Boosting Network for Detecting Salient Objects in RGB-Thermal Images}, 
                year={2023},
                volume={32},
                number={},
                pages={1329-1340},
                doi={10.1109/TIP.2023.3242775}}      

Acknowledgement

The implement of this project is based on the codebases bellow.

• BBS-Net
  
• Knowledge-Distillation-Zoo
  
• Fps/speed test MobileSal
• Evaluate tools CODToolbox / PySODMetrics
  

If you find this project helpful, Please also cite codebases above.

Contact

Please drop me an email for any problems or discussion: https://wujiezhou.github.io/ ([email protected]) or [email protected].

lsnet's Issues

训练问题

hi, 请问在训练代码那里，验证集是使用训练集还是测试集中的某一个来计算best mae，例如vt821还是？

where is the LENetmobilenetv2_KD?

In "test.py", line 9, from LSNet import LENetmobilenetv2_KD. However, i cannot find LENetmobilenetv2_KD in LSNet.py.