meiqihu / remote-sensing-chatgpt

RS-ChatGPT: Human-Interactive Remote Sensing Task Solving with ChatGPT

Introduction

Remote Sensing ChatGPT(RS-ChatGPT) is an open source tool for solving remote sensing tasks with ChatGPT in an interactive way.ChatGPT acts as an expert to response to users' linguistic resquests based on the input remote sensing image. It supports various interpretation tasks that are trained on remote sensing datasets. To help ChatGPT better understand remote sensing knowledge, image captioning and scene understanding of remote sensing image are set as prefix.RS-ChatGPT is developed based on the Visual ChatGPT project.

Generally, RS-ChatGPT includes four steps in implementation:

• Prompt Template Generation
• Task Planning
• Task Execution
• Response Genration

Updates

Initial release：2023.08.23

• TODO: Provide an interactive interface

The code

Requirements

Please Refer to requirements.txt

Usage

Clone the repository:git clone https://github.com/HaonanGuo/BFSeg-Efficient-Building-Footprint-Segmentation-Framework.git

->Run rs_chatgpt.py

Supported Function

Function	Description	Method	Pretrain Dataset	Model Weights
Image Captioning	Describe the remote sensing image	BLIP	BLIP Dataset	weight
Scene Classification	Classify the type of scene	ResNet	AID Dataset	weight
Object Detection	Detect RS object from image	YOLO v5	DOTA	weight
Instance Segmentation	Extract Instance Mask of certain object	SwinTransformer+UperNet	iSAID	weight
Landuse Classification	Extract Pixel-wise Landuse Classification	HRNet	LoveDA	weight
Object Counting	Count the number of certain object in an image	SwinTransformer+UperNet	iSAID	weight
Edge Detection	Extract edge of remote sensing image	Canny	None	None

More funtions to be updated~

Citation

Please cite the repo if you use the data or code in this repo.

@misc{RS-ChatGPT,
  author = {Haonan Guo and Bo Du and Liangpei Zhang },
  title = {RS-ChatGPT: Human-Interactive Remote Sensing Task Solving with ChatGPT},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/HaonanGuo/Remote-Sensing-ChatGPT}},
}

Acknowledgments

• Visual ChatGPT
• YOLOv5
• BLIP

Help

Remote Sensing ChatGPT is an open source project that welcome any contribution and feedback. Please contact us with: [email protected]