fmenat / multiviewrs-models

List of multi-view fusion learning models proposed for remote sensing (RS) multi-view data. Extension from our JSTAR paper.

Remote Sensing based Multi-view Models

List of multi-view fusion learning models proposed for remote sensing (RS) multi-view data. 📡 🌎 📡

This is a complementary source used in the following paper:

Common Practices and Taxonomy in Deep Multi-view Fusion for Remote Sensing Applications

Name	Reference	Description	Code
MV CNN	Xu et al. 2018	Feature-level fusion with 2D CNN.	https://github.com/Hsuxu/Two-branch-CNN-Multisource-RS-classification << Not available
V-FuseNet	Audebert et al. 2018	Dense fusion with 2D CNN and central model.	https://github.com/nshaud/DeepNetsForEO
Multi3Net	Rudner et al. 2019	Feature-level fusion with 2D CNN.	https://github.com/FrontierDevelopmentLab/multi3net
UNet-CLSTM	Rustowicz et al. 2019	Decision-level fusion with 2D CNN and convolutional-LSTM	https://github.com/roserustowicz/crop-type-mapping
HRWN	Zhao et al. 2020	Input-level fusion with 2D CNN and pixel graph constraints.	https://github.com/xudongzhao461/HRWN
FusAtNet	Mohla et al. 2020	Feature-level fusion with 2D CNN and cross attention.	https://github.com/ShivamP1993/FusAtNet
LFMC from SAR	Rao et al	Input-level fusion with LSTM	https://github.com/kkraoj/lfmc_from_sar
CCR-Net	Wu et al. 2021	Feature-level fusion with 2D CNN and cross view-reconstruction.	https://github.com/danfenghong/IEEE_TGRS_CCR-Net
MV PSE-TAE	Ofori-Ampofo et al. 2021	Multiple fusion strategies with PSE-TAE.	https://github.com/ellaampy/CropTypeMapping
MDL-RS	Hong et al. 2021	Multiple fusion strategies with NN.	https://github.com/danfenghong/IEEE_TGRS_MDL-RS
CMGFNet	Hosseinpour et al. 2022	Dense fusion with 2D CNN and gated attention.	https://github.com/hamidreza2015/CMGFNet-Building_Extraction
S2FL	Hong et al. 2021	Feature-level fusion with feature contrains.	https://github.com/danfenghong/ISPRS_S2FL
CFCNN	He et al. 2021	Feature-level fusion with 2D and 1D CNN.	https://github.com/SysuHe/MultiSourceData_CFCNN
MV NN	Danilevicz et al. 2021	Feature-level fusion with tabular NN and 2D CNN.	https://github.com/mdanilevicz/maize_early_yield_prediction
SEnSeI	Francis et al. 2022	Sensor invariant model based on 2D CNN	https://github.com/aliFrancis/SEnSeI
ASF2N	Gao et al. 2022	Feature-level fusion with 2D CNN and attention.	https://github.com/zhonghaocheng/ELSEVIER_IJAEOG_AS2F2N << Empty code
IP-CNN	Zhang et al. 2022	Feature-level fusion with 2D CNN and view-reconstruction.	https://github.com/HelloPiPi/IP-CNN-code
MV CNN	Lu et al. 2022	Feature-level fusion with 2D CNN and adaptive attention.	https://github.com/GeoX-Lab/UnifiedDL-UFZ-extraction
SE$^2$Net	Fang et al. 2022	Feature-level fusion with 2D CNN.	https://github.com/likyoo/Multimodal-Remote-Sensing-Toolkit
EndNet	Hong et al. 2022	Feature-level fusion with 2D CNN and view-reconstruction.	https://github.com/danfenghong/IEEE_GRSL_EndNet
MAHiDFNet	Wang et al. 2022	Dense feature fusion with 2D CNN.	https://github.com/SYFYN0317/-MAHiDFNet
AM$^3$Net	Wang et al. 2022	Feature-level fusion with 2D CNN and cross attention.	https://github.com/Cimy-wang/AM3Net_Multimodal_Data_Fusion
AMM-FuseNet	Ma et al. 2022	Feature-level fusion with 2D CNN and attention.	https://github.com/oktaykarakus/ReSIF/tree/main/AMM-FuseNet
MCANet	Li et al. 2022	Dense fusion with 2D CNN and cross attention.	https://github.com/yisun98/SOLC
ChangeFormer	Bandara et al. 2022	Dense fusion with transformer and attention.	https://github.com/wgcban/ChangeFormer
CMAFF	Qingyun et al. 2022	Dense fusion with 2D CNN and cross attention.	https://github.com/DocF/CMAFF
OmbriaNet	Drakonakis et al. 2022	Feature fusion with 2D CNN and skip-connections	https://github.com/geodrak/OMBRIA
DCSA-Net	Wang et al. 2022	Hybrid fusion with 2D CNN and attention.	https://github.com/Julia90/DCSA-Net
Siamese U-Net	Cummings et al. 2022	Dense fusion with 2D CNN and skip-connections	https://github.com/solcummings/earthvision2021-weakly-supervised
SatViT	Fuller et al.	Input-level fusion with ViT (with self-supervised training)	https://github.com/antofuller/SatViT
ELECTS	Russwurm et al. 2023	Input-level fusion with LSTM.	https://github.com/marccoru/elects
MV CNN	Ferrari et al. 2023	Multiple fusion strategies with 2D CNN (encoder-decoder)	https://github.com/felferrari/deforestation-from-data-fusion
AFCF3D-Net	Ye et al. 2023	Input-level fusion with 3D CNN.	https://github.com/wm-Githuber/AFCF3D-Net
UnCRtainTS	Ebel et al 2023	Input fusion with 2D CNN and attention.	https://github.com/PatrickTUM/UnCRtainTS
MFT	Roy et al. 2023	Feature-level fusion with transformer modules (one source - LIDAR- is used as a query over the main source - optical)	https://github.com/AnkurDeria/MFT
OOD Fusion	Gawlikowski et al. 2023	Input-level, Feature-level, and Decision-level fusion with CNN and weighted average aggregation	https://github.com/JakobCode/OOD_DataFusion
PRESTO	Tseng et al. 2023	Input-level fusion with transformer modules (self-supervised pretraining)	https://github.com/nasaharvest/presto
Cross-HL	Roy et al. 2023	Feature-level fusion with directed attention in transformer layers	https://github.com/AtriSukul1508/Cross-HL
SCT Fusion	Hoffman et al. 2023	Dense fusion with tranformer layers and class tokens in all	https://git.tu-berlin.de/rsim/sct-fusion
MMST-ViT	Lin et al. 2023	Feature-level fusion with transformer layers	https://github.com/fudong03/MMST-ViT
DiffusionSat	Khanna et al. 2023	Multi-modal diffusion generative model	https://github.com/samar-khanna/DiffusionSat
SSL4EO-S12	Wang et al. 2024	Self-supervised model	https://github.com/zhu-xlab/SSL4EO-S12
EarthGPT	Zhang et al. 2024	Feature-level fusion with transformer layers	https://github.com/wivizhang/EarthGPT
ContextFormer	Benson et al. 2024	Feature-level fusion with transformer	https://github.com/vitusbenson/greenearthnet
SEnSeIv2	Francis 2024	Sensor-invariant model	https://github.com/aliFrancis/SEnSeIv2
OmniSat	Astruc et al. 2024	Feature-level fusion with transformer layers and pre-training	https://github.com/gastruc/OmniSat

Feel free to ask me to update some content!

---

Some abbrevations

Abbrevation	name
CNN	convolutional neural network
LSTM	long-short term memory
NN	neural network
PSE-TAE	pixel set encoder - temporal attention encoder