junjue-wang / lovecs

[RSE 2022] Cross-sensor domain adaptation for high-spatial resolution urban land-cover mapping: from airborne to spaceborne imagery

Cross-sensor domain adaptation for high-spatial resolution urban land-cover mapping: from airborne to spaceborne imagery

by Junjue Wang, Ailong Ma, Yanfei Zhong, Zhuo Zheng, and Liangpei Zhang

[Paper], [BibTex], [Product]

---

This is an official implementation LoveCS in our RSE 2022 paper. The referenced methods in this paper can be found in our LoveDA repo.

Highlights:

1. A Cross-Sensor Land-cOVEr framework (LoveCS) is proposed.
2. LoveCS advances cross-sensor domain adaptation.
3. LoveCS learns divergence between sensors from structure and optimization.
4. The effectiveness of LoveCS was evaluated in three cities of China.
5. High-resolution city-scale mapping can be achieved within 9 hours on one GPU.

Requirements:

• pytorch >= 1.7.0
• python >=3.6

pip install --upgrade git+https://github.com/Z-Zheng/ever
pip install git+https://github.com/qubvel/segmentation_models.pytorch

1. Flexible Cross-sensor Normalization

Cross-sensor normalization can help you encode the source and target domain statistics separately.

1.You can replace the batch normalizations with cross-sensor normalizations as follows:

from module.csn import replace_bn_with_csn
from module.semantic_fpn import SemanticFPN
# Semantic-FPN (https://arxiv.org/pdf/1901.02446.pdf) as an example 
model = SemanticFPN(dict())
# Replace the BNs with CSNs
model = replace_bn_with_csn(model)

2.Model Forward

from module.csn import change_csn
model = change_csn(model, source=True)
source_outputs = model(source_images)
model = change_csn(model, source=False)
target_outputs = model(target_images)

2.Model Backward

from torch.nn import CrossEntropyLoss
loss_cal = CrossEntropyLoss()
src_loss = loss_cal(source_outputs, src_labels)
tgt_loss = loss_cal(target_outputs, pse_labels)
total_loss = tgt_loss + src_loss
total_loss.backward()

2. LoveCS framework

LoveCS_train.py is a training example and LoveCS_eval.py is an evaluation example. You can configure your domain adaptation dataset (i.e. LoveDA) and use the following scripts for training and evaluation.

#!/usr/bin/env bash
config_path='st.lovecs.2CZ.lovecs'
python LoveCS_train.py --config_path=${config_path}


config_path='st.lovecs.2CZ.lovecs'
ckpt_path='./log/sfpn.pth'
python LoveCS_eval.py --config_path=${config_path} --ckpt_path=${ckpt_path}

Citation

If you use LoveCS in your research, please cite our RSE 2022 paper.

    @article{WANG2022113058,
    title = {Cross-sensor domain adaptation for high spatial resolution urban land-cover mapping: From airborne to spaceborne imagery},
    journal = {Remote Sensing of Environment},
    volume = {277},
    pages = {113058},
    year = {2022},
    issn = {0034-4257},
    doi = {https://doi.org/10.1016/j.rse.2022.113058},
    url = {https://www.sciencedirect.com/science/article/pii/S0034425722001729},
    author = {Junjue Wang and Ailong Ma and Yanfei Zhong and Zhuo Zheng and Liangpei Zhang},
    }

LoveCS can be used for academic purposes only, and any commercial use is prohibited.