adrianamusic / aerial_wildlife_detection

Tools for detecting wildlife in aerial images using active learning

AIDE: Annotation Interface for Data-driven Ecology

AIDE is two things in one: a tool for manually annotating images and a tool for training and running machine (deep) learning models. Those two things are coupled in an active learning loop: the human annotates a few images, the system trains a model, that model is used to make predictions and to select more images for the human to annotate, etc.

More generally, AIDE is a modular Web framework for labeling image datasets with AI assistance. AIDE is configurable for a variety of tasks, but it is particularly intended for ecological applications, such as the acceleration wildlife surveys that use aerial images.

AIDE is primarily developed by Benjamin Kellenberger, supported by the Microsoft AI for Earth program.

Contents

• Highlights
• News
• Demo
• Installation and launching AIDE
  • Migration from AIDE v1
  • New installation
    • With Docker
    • Manual installation
• AI models in AIDE
  • Using a built-in AI model
  • Writing your own AI model
• Publications and References
• Contributing

Highlights

• Powerful: AIDE explicitly integrates humans and AI models in an annotation loop.
• Fast: AIDE has been designed with speed in mind, both in terms of computations and workflow.
• Flexible: The framework allows full customizability, from hyperparameters to models to annotation types to libraries. It provides:
  • Support for image classification, point annotations, and bounding boxes (object detection)
  • A number of AI models and Active Learning criteria built-in
  • Interfaces for custom AI models and criteria, using any framework or library you want (see how to write your own model).
• Fully featured: Beyond image labeling and model training, AIDE has management and graphical user/machine performance evaluation tools built-in, right in the web browser, allowing for advanced, manual label quality checks.
• Modular: AIDE is separated into individual modules, each of which can be run on separate machines for scalability. It even supports on-the-fly addition of computational workers for computationally intensive model training!

News

• November 19, 2020: We now have an official publication on AIDE: Kellenberger, Benjamin, Devis Tuia, and Dan Morris. "AIDE: Accelerating image‐based ecological surveys with interactive machine learning." Methods in Ecology and Evolution, in press. DOI: 10.1111/2041-210X.13489. Please cite this manuscript whenever you use AIDE in your work. Thank you very much!
• August 17, 2020: AIDE v2 now officially replaces the old version as the new master! If you have a v1 project running, you may want to migrate it according to the instructions below. For the legacy v1 code, see here.
• July 17, 2020: AIDE now supports watching image folders for changes! Every project can be configured to automatically register newly added images and, optionally, remove all data for deleted images. See "Project Settings" > "General" and the new entry watch_folder_interval in the settings.ini file.
• June 11, 2020: The built-in RetinaNet now adapts to the project's label classes by adding or removing neurons whenever classes change! Check out the new, GUI-enhanced options in the project configuration page under "AI model" > "Settings".
• June 10, 2020: Thanks to the outstanding work of contributor Jarek, AIDE V2 finally supports one-line installation and setup through Docker! See below for instructions.

Demo

A demo of AIDE can be accessed here.

This demo allows exploring the annotation front-end with a number of example datasets, including:

• Image labels on the Snapshot Serengeti camera traps dataset
• Points on the VGG Penguins dataset
• Bounding boxes on the NOAA Arctic Seals aerial imagery
• Semantic segmentation on the Chesapeake Land Cover satellite imagery

Installation and launching AIDE

Migration from AIDE v1

If you have AIDE v1 already running and want to upgrade its contents to AIDE v2, see here.

New installation

With Docker

AIDE now comes with both Docker and Docker Compose support! The current scripts are not 100% tested yet, but they make installation a breeze!

Here's how to install and launch AIDE with Docker on the current machine:

1. Download and install Docker as well as Docker Compose
2. If you want to use a GPU (and only then), you have to install the NVIDIA container toolkit:

    distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
    curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
    curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
    sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
    sudo systemctl restart docker

3. Clone the AIDE repository: git clone https://github.com/microsoft/aerial_wildlife_detection.git && cd aerial_wildlife_detection/
4. Important: modify the docker/settings.ini file and replace the default super user credentials (section [Project]) with new values. Make sure to review and update the other default settings as well, if needed.
5. Install:

       cd docker
       sudo docker-compose build
       cd ..

6. Launch:
   • With Docker:

       sudo docker/docker_run_cpu.sh     # for machines without a GPU
       sudo docker/docker_run_gpu.sh     # for AIWorker instances with a CUDA-enabled GPU (strongly recommended for model training)

   • With Docker Compose (note that Docker Compose currently does not provide support for GPUs):

       cd docker
       sudo docker-compose up

Manual installation

See here for instructions on configuring an instance of AIDE.

After that, see here for instructions on launching an instance of AIDE.

AI models in AIDE

Using a built-in AI model

AIDE ships with a set of built-in models that can be configured and customized:

• Image labels: ResNet
• Points: weakly-supervised detector
• Bounding boxes: RetinaNet
• Semantic segmentation: U-Net%E5%92%8CTiramisu

See this page for instructions on how to use one of the built-in models.

Writing your own AI model

AIDE is fully modular and supports custom AI models, as long as they provide a Python interface and can handle the different annotation and prediction types appropriately. We greatly welcome contributions and are happy to help in the implementation of your custom models! For example, check out the awesome TensorFlow implementation of YOLOv3 by contributor Colin Torney!

See here for instructions on using custom models.

Publications and References

Please cite the following paper if you use AIDE in your work:

Kellenberger, Benjamin, Devis Tuia, and Dan Morris. "AIDE: Accelerating image‐based ecological surveys with interactive machine learning." Methods in Ecology and Evolution, in press. DOI: 10.1111/2041-210X.13489.

Thank you.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.