z-zheng / farseg

Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery (CVPR 2020) https://arxiv.org/pdf/2011.09766.pdf

Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery

Zhuo Zheng, Yanfei Zhong, Junjue Wang and Ailong Ma

This is an official implementation of FarSeg in our CVPR 2020 paper Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery.

---

News

• 2024/03, source code of FarSeg++ is released.
• 2023/10, UV6K dataset is publcily available.
• 2023/07, FarSeg++ is accepted by IEEE TPAMI.

Citation

If you use FarSeg or FarSeg++ in your research, please cite the following paper:

@inproceedings{zheng2020foreground,
  title={Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery},
  author={Zheng, Zhuo and Zhong, Yanfei and Wang, Junjue and Ma, Ailong},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={4096--4105},
  year={2020}
}
@article{zheng2023farseg++,
  title={FarSeg++: Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery},
  author={Zheng, Zhuo and Zhong, Yanfei and Wang, Junjue and Ma, Ailong and Zhang, Liangpei},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2023},
  volume={45},
  number={11},
  pages={13715-13729},
  publisher={IEEE}
}

Getting Started

Install SimpleCV

pip install --upgrade git+https://github.com/Z-Zheng/SimpleCV.git

Requirements:

• pytorch >= 1.1.0
• python >=3.6

Prepare iSAID Dataset

ln -s </path/to/iSAID> ./isaid_segm

Evaluate Model

1. download pretrained weight in this link

2. move weight file to log directory

mkdir -vp ./log/isaid_segm/farseg50
mv ./farseg50.pth ./log/isaid_segm/farseg50/model-60000.pth

3. inference on iSAID val

bash ./scripts/eval_farseg50.sh

Train Model

bash ./scripts/train_farseg50.sh

farseg's Issues

Training script seems to be stuck

I followed the procedure provided in the repo but the training script seems to be stuck at:

Defaults for this optimization level are:
enabled                : True
opt_level              : O1
cast_model_type        : None
patch_torch_functions  : True
keep_batchnorm_fp32    : None
master_weights         : None
loss_scale             : dynamic
Processing user overrides (additional kwargs that are not None)...
After processing overrides, optimization options are:
enabled                : True
opt_level              : O1
cast_model_type        : None
patch_torch_functions  : True
keep_batchnorm_fp32    : None
master_weights         : None
loss_scale             : dynamic

After this, it doesn't seems to show any output. How to check if the model is running?

Questions about the Foreground-Scene Relation Module in the paper

When I reproduced the network in this paper, I encountered the following problems: Scenne embedding u in the module is realized by C6 through 1x1 convolution layer, and C6 is a two-dimensional tensor (None, C) obtained by C5 through global average pooling calculation. The input requirements of the convolution layer are (N, H, W, C), and the dimension of C6 does not meet the requirements. What's the matter, please? Did I get it wrong?

关于论文中的相关问题请教

您好，读了您的论文非常受教，但是还有几个问题没看明白，想专门请教下：

1. 在 Scale-aware projection for scene embedding 这个章节
   （1）projection functions 是用来做维度对齐，那么在 这个模块，去掉 \eta function，那么维度如何对齐呢？
   （2）在这个小节的结论中，With F-A optimization, the gain in mIoU from scale-aware projection is larger. It indicates that geospatial scene representation is related to scale and foregrounds. 为什么 这个 loss 上的 optimization 模块 可以 体现 geospatial scene representation 与 scale and foregrounds 相关的呢？

2. 文中的 foreground-relative contexts 和 geospatial scene-relevant context 和 图3 的 "relevant context features" 三者是否是同一个指代？这个 context 具体是指什么，它有公式表示或者具体作图表示么？

3. 在 segmentation 的时候，focal loss 需要 norm 而 检测的时候不需要呢？关于梯度消失的角度进行解释，easy 样本 梯度消失应该不影响训练吧，所以这个梯度消失为什么会影响效果？

distributed train, what should I do?

subprocess.CalledProcessError: Command '['/home/ps/anaconda3/envs/farseg/bin/python', '-u', 'apex_train.py', '--local_rank=1', '--config_path=configs.isaid.farseg50', '--model_dir=./log/isaid_segm/farseg50', '--opt_level=O']' returned non-zero exit status 1.

Visualization

@Z-Zheng thanks for open-sourcing the code base, which code to use to save the segmentation data on the original image with different color schema and class names

Thanks in advance