gmoncrieff / hyper-iap

Classification of alien invasive plants from hyperspectral data from point localities

---

Detection of invasive plants using hyperspectral imagery

Description

This package is intended for running deep learning classifiers on hyperspectral data for mapping invasive alien plants. Models are pretrained using self-supervision and/or noisy land cover label data and then fine-tuned using point labels.

Currently this package is under heavy development

Development roadmap

Current and planned data sources, models and module features are indicated below

Data sources

• Sentinel 2
• Hyperspectral

Models

• TempCNN
• ViT
• Masked Autoencoder

Module features

• Logging with W&B
• Hyperparameter tuning with W&B sweeps

Getting started

First, install dependencies

# clone project   
git clone https://github.com/GMoncrieff/hyper-iap

# install project   
cd hyper-iap  
pip install -r requirements.txt

Next, train classifiers using the command line.

python train.py --model_class=vit.simpleVIT   

For a full list of command line options run

python train.py --help

Imports

You can also import individual modules and incorporate them into python workflows

from lightning import Trainer, seed_everything
from hyperiap.models.vit import simpleVIT
from hyperiap.datasets.xarray_module import XarrayDataModule
from hyperiap.litmodels.litclassifier import LitClassifier

xmod = XarrayDataModule()
model = LitClassifier(simpleVIT(data_config=xmod.config()))
trainer = Trainer(limit_train_batches=5, limit_val_batches=3, max_epochs=2)
trainer.fit(model, datamodule=xmod)
trainer.validate(datamodule=xmod)

Acknowledgements

The module builds on contributions and implementations from :

• TempCNN - Pelletier et al., 2019
• BreizhCrops - Rußwurm et al., 2019
• ViT - Dosovitskiy et al., 2021
• SpectralFormer - Hong et al., 2021
• Full Stack Deep Learning
• xbatcher
• zen3geo

The land cover labels used from pre-training from

• The South African Department of Forestry, Fisheries and the Environment

hyper-iap's Issues

add config.yaml

pass accuracy variable name as argument

implement xarray data module for point data

Data module that allows point data to be read from xarray. Data from neighboring pixels must already be stacked into each pixel in the extraction and data prep process as neighbor data will not be available for each point at this stage

implement xarray module for raster label data

adding of dimension data can be done in the data module as all neighbors will be present in this dataset

add wandb logging

add github actions

pytorch performance upgrade

torch.compile
fp16
maybe also amp optimizers

label smoothing option

calculate num classes

add wandb sweep for hyperparam tuning

pass relevant params for command line

set data downloaded location