rodekruis / ada-collection

Tools for Automated Mapping and Damage Assessment

ada-collection

Tools for Automated Mapping and Damage Assessment.

This repo wraps two packages:

• abd_model - forked from automated-building-detection
• ada_tools - tools for pre- and post-processing of remote sensing images and vector data

The damage assessment framework & model is at caladrius:ada-0.1 automatically installed via Docker (caladrius.Dockerfile).

Important

This readme is intended for the general public.

For 510-specific instructions during emergencies, see the ADA SOP.

Getting started

Get pre-trained models and images

1. Download pre-trained building detection model: neat-fullxview-epoch75.pth:

• architecture: AlbuNet (U-Net-like encoder-decoder with a ResNet, ResNext or WideResNet encoder)
• training: xBD dataset, 75 epochs
• performance: IoU 0.79, MCC 0.75

2. Download pre-trained building damage classification model: caladrius_att_effnet4_v1.pkl

• architecture: pseudo-siamese network with two ImageNet pre-trained EffNet-B4 models + attention
• training: xBD dataset, 75 epochs
• performance: up to F1 score 0.79, AUC 0.984 (see performance paper)

3. [OPTIONAL] pre- and post-disaster satellite images

Your workspace should then look like

    <workspace>
    ├── ...
    ├── neat-fullxview-epoch75.pth    # pre-trained building detection model
    ├── caladrius_att_effnet4_v1.pkl  # pre-trained building damage classification model
    ├── images                        # satellite images
    │   ├── pre-event                 # before the disaster
    │   └── post-event                # after the disaster
    └── ...

Using Docker

1. Install Docker.
2. Download the latest Docker Image

docker pull jmargutti/ada-collection

3. Create a docker container and connect it to a local directory (<path-to-your-workspace>)

docker run --name ada-collection -dit -v <path-to-your-workspace>:/workdir --ipc=host --gpus all -p 5000:5000 rodekruis/ada-collection

4. Access the container

docker exec -it ada-collection bash

Manual Setup

1. Install Python 3.7 and pip.
2. Install Anaconda.
3. Create and activate a new Anaconda environment.

conda create --name abdenv python=3.7 
conda activate abdenv

4. From root directory, move to ada_tools and install

cd ada_tools
pip install .

Note: Make sure libboost/boost is installed.

5. Move to abd_model and install

cd ../abd_model
pip install .

6. Get caladrius:ada-0.1 (damage assessment framework) and install

git clone --branch ada-0.1 https://github.com/rodekruis/caladrius.git
cd caladrius
./caladrius_install.sh

End-to-end example

1. Get satellite images of typhoon Mangkhut from Maxar Open Data

load-images --disaster typhoon-mangkhut --dest <workspace>/images

• Alternatively, load images from Azure blob storage

load-images-azure --disaster typhoon-mangkhut --dest <workspace>/images

• Set the CONNECTION_STRING and CONTAINER_NAME environmental variables corresponding to your Azure account

2. Prepare images for building detection

abd cover --raster <workspace>/images/pre-event/*.tif --zoom 17 --out <workspace>/abd/cover.csv
abd tile --raster <workspace>/images/pre-event/*.tif --zoom 17 --cover <workspace>/abd/cover.csv --out <workspace>/abd/images --format tif --no_web_ui --config ada-tools/config.toml

3. Detect buildings

abd predict --dataset <workspace>/abd --cover <workspace>/abd/cover.csv --checkpoint <workspace>/neat-fullxview-epoch75.pth --out <workspace>/abd/predictions --metatiles --keep_borders --config ada-tools/config.toml

4. Generate vector file with buildings and filter noise

abd vectorize --masks <workspace>/abd/predictions --type Building --out <workspace>/abd/buildings.geojson --config ada-tools/config.toml
filter-buildings --data <workspace>/abd/buildings.geojson --dest <workspace>/abd/buildings-clean.geojson

5. Prepare images for building damage classification

prepare-data --data <workspace>/images --buildings <workspace>/abd/buildings-clean.geojson --dest <workspace>/caladrius

6. Classify building damage

CUDA_VISIBLE_DEVICES="0" python caladrius/caladrius/run.py --run-name run --data-path <workspace>/caladrius --model-type attentive --model-path <workspace>/caladrius_att_effnet4_v1.pkl --checkpoint-path <workspace>/caladrius/runs --batch-size 2 --classification-loss-type f1 --output-type classification --inference

7. Generate vector file with buildings and damage labels

final-layer --builds <workspace>/abd/buildings-clean.geojson --damage <workspace>/caladrius/runs/run-input_size_32-learning_rate_0.001-batch_size_32/predictions/run-split_inference-epoch_001-model_inception-predictions.txt --out <workspace>/buildings-predictions.geojson --thresh 1

8. Take your favorite GIS application and visualize <workspace>/buildings-predictions.geojson in a nice map

ada-collection's Issues

[BUG] docker image not available

Who notified us about the bug?

Julian

Describe the bug

The docker image is not available (nor is any other docker image by rodekruis on docker for that matter)

To Reproduce

Steps to reproduce the behavior:

1. Go to docker
2. See error

Expected behavior

docker image

Screenshots