microsoft / satellite-imagery-labeling-tool

This is a lightweight web-interface for creating and sharing vector annotations over satellite/aerial imagery scenes.

page_type	description	languages	products
sample	A lightweight web-interface for creating and sharing vector labels (lines or polygons) over satellite/aerial imagery scenes.	javascript python	azure azure-maps

Spatial imagery labeling toolkit

This is a lightweight web-interface for creating and sharing vector labels (lines or polygons) over satellite/aerial imagery scenes. This project includes the following tools:

• Project builder - Allows you to specify an area of interest, select the imagery layers that can be used for reference, break the area up into chunks and generate tasks files that can be assigned to those who will be doing the labeling. For more details, see the Project builder documentation.
  • Easily break up an area of interest into smaller subsets that can be assigned to individuals via a task file.
• Labeler - Loads a task file for an labeling project (annotation). For more details, see the Labeler documentation.
  • Automatically caches data in local browser storage (does not leave your machine) to help prevent data lose from accidental browser refresh or closer. Makes it easy to come back later and continue where you left off. Supports multiple labeling projects by seperating the data by the projects task name. Learn more.
  • Import existing data from local geospatial files. Supports: GeoJSON, GeoJSONL, KML, KMZ, GeoRSS, GPX, GML, CSV (comma, tab, or pipe delimited).
  • Easily import existing data from Open Street Maps via the Overpass turbo API or a custom data service. Avoids loading in any data for areas that already have drawn features.
• Project viewer - View and merge the results from the project. See stats and easily identify which areas have little to no labels. For more details, see the Project viewer documentation.

Also see the Documentation and the Spatial imagery ML Jupyter notebook example.

NOTE: Version 1 of this project uses a different file schema that is not compatible with version 2. You can download version 1 of this project here.

Learn more about Microsoft AI

Additional Features

• Multi-delete (lasso delete).
• Shift imported data to align with your imagery.
• Support for color themes to align with user preferences.
• Local settings saved so users only have to set their preference once and have those presist across multiple sessions.

Unlock enhanced functionality with Azure Maps

• High resolution global satellite imagery.
• Map labels and points of interest.
• Location search.
• Enhanced accessibility (screen reader) capabilities.

Potential Roadmap

The following are some ideas to take this project further:

• Consider integrating Azure Planetary Computer services. Import imagery layers and/or vector data.
• Add an orthogonalize polygon/squaring option in the labeler to help make drawn features look nicer.
• Magic wand drawing tool that uses a flood fill algorithm on the imagery to generate polygons based on the likeness of nearby pixels.
• Potentially add an option to use fill patterns for features to visualize the secondary class while color is used for the primary class. Alternatively use a fill pattern for the primary class as an alternative to color for better accessibility support for those who are color blind.

Have a feature request? Use the Issues tab of this project repo.

Related projects

Here are some related projects we recommend checking out:

• satellite-image-deep-learning
• TorchGeo
• GlobalMLBuildingFootprints

List of third party javascript libraries/versions

This project has directly imported and hosted versions of the following third party javascript libraries. This project purposely avoids using NPM and other developer tools and frameworks to keep things simple, so that anyone can easily get this project up and running locally (many users will be data and AI people, not developers).

• Azure Maps Image Exporter 0.0.2 - A helper library for generating screenshots of an Azure Maps control.
• Azure Maps Bring Data Into View Control 0.0.2 - Adds a button to the map that makes it easy to focus back to where your data is.
• JSZip v3.10.1 - Library for reading and writing zip files.
• localForage 1.10.0 - Makes it easy to store data locally in the browser using a single API. For more information, see the Labeler's Auto save documentation.
• Marked 4.0.18 - Adds support for markdown syntax, used by the instructions panel.
• osmtogeojson 3.0.0 - A library for converting OSM data into GeoJSON objects.
• TurfJS 6.5.0 - A library for performing advanced spatial calculations.

License

This project is licensed under the MIT License.

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

satellite-imagery-labeling-tool's Issues

Feature request: Selecting default map layer

For labeling tasks with custom layers, users will often want to start on the custom layer instead of the Azure Maps Layer. An option in the project configuration that allows to set a default layer might streamline the process a little.

Availability of data/pre/annotations/pre_labels.geojson

To work through the example jupyter notebook are there plans to add data/pre/annotations/pre_labels.geojson and the other files? Cheers

[Question] Labeling a dataset of a lot of TIFs

Greetings,

I want to use this awesome tool to label a lot of TIFs that I have in a Google Storage bucket. I have deployed a TiTiler instance that is authenticated to serve TIFs from the bucket. However, I can only serve a single TIF with TiTiler by passing it a URL of the image (like setting url=gs://my-bucket/path/to/the/image.tif in the manual request to TiTiler). I think I need to create a MosaicJSON or TileJSON for my images, which I can do, but I am not sure how to add that to this tool. In the GeoTiff example docs there is an example using a Tile layer, but I couldn't get that to work, is there a way of adding multiple images using a MosaicJSON or TileJSON endpoint of TiTiler? FWIW, I can view the images using the /cog/map endpoint of TiTiler.

Using http://localhost:8000/mosaicjson/{z}/{x}/{y}.jpg?tileMatrixSetId=WebMercatorQuad&tile_scale=1&url=gs://<MY-BUCKET>/mosaic.json should work but it doesn't.

Unable to label a feature as a different secondary class by selecting it on the side-menu in the labeling tool

Summary

Unable to label a feature as a different secondary class by selecting it on the side-menu in the labeling tool

Overview

In the labeling tool, I selected one of the secondary class properties that I had defined in the project builder and drew a shape around a building. When I saved the labeled task files and loaded them onto the project viewer, the shapes were not labeled as the secondary class property that I selected before drawing them. The secondary class options were 0, 1, 2, 3, 4, and 5, but no matter what I selected each shape remained labeled as 0, which seemed to be the default since it was the first secondary class that I added. The only way I was able to successfully apply a different secondary class in the labeler was to draw the shapes first, click the rectangle select button, and then click the shapes and edit the secondary class that way (rather than before/as I drew them).

Steps to reproduce

1. Open the project builder.
2. On Step 1: Project details, fill in a project name (i.e. “testing”) and click next.
3. On Step 2: Add layers, keep the default layers and click next.
4. On Step 3: Create focus area, draw a focus area of the University of Massachusetts Amherst (or anywhere). Adjust the grid size and focus area so that they align with each other.
5. On Step 4: Define classes, change the Primary class display name and Primary class property name to “building”.
6. Under primary classes, add the primary class “building” and press the green plus(+) sign to add it.
7. Click Capture secondary class.
8. Change the Secondary class display name to “Damage severity” and the Secondary class property name to “damage_severity”.
9. Under Secondary classes, add the following secondary classes: 0, 1, 2, 3, 4, and 5.
10. Click next to Step 5: Complete and download the project files.
11. Open the project labeler.
12. Under Import data, load one of the task files from your downloaded project files.
13. Under Damage severity on the right-hand side menu, select “5” as the secondary class.
14. Select the draw-polygon option and trace one of the buildings.
15. Select a Damage severity of “3” and trace another building.
16. Navigate to the Save page and save your labeled task file.
17. Put the output file that you just saved into the results folder of the project builder output folder and zip the project builder folder.
18. Open the project viewer.
19. Load the zipped project builder folder onto the project viewer.
20. On the left-hand side, change the View to Damage severity.
21. Notice on the left that two labeled features are listed as having a damage severity of 0.
22. Click on the labeled buildings.
23. Notice that the the damage_severity remains at 0 for both buildings.

Expected result:

Before drawing a shape around a building in the labeler, I selected one of the secondary class options that I had previously defined in the project builder to label the building as. Once labeled and loaded into the project viewer, I expected to see the features labeled as the secondary classes that I selected for them.

Actual result:

The features were not labeled as the secondary classes that I selected for them, but instead remained labeled as the default secondary class that was already selected before I changed the selection.

Helpful screenshots

Project builder configuration:

Labeling features with secondary class selected on side-menu:

Results of labeled features in project viewer:

Adding a new layer does not work in Project Builder

Hi. I have deployed the application on Kubernetes like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: silt-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: silt
  template:
    metadata:
      labels:
        app: silt
    spec:
      containers:
        - name: nginx
          image: nginx:latest
          resources:
            requests:
              memory: 256Mi
              cpu: 100m
            limits:
              memory: 256Mi
              cpu: 100m
          ports:
            - containerPort: 80
          command: ["/bin/sh", "-c"]
          args:
            - |
              apt update
              apt install -y git
              SILT_DIR=/usr/share/nginx/html/satellite-imagery-labeling-tool
              if [ ! -d $SILT_DIR ]; then
                git clone -b v2.0.3 --depth 1 https://github.com/microsoft/satellite-imagery-labeling-tool.git $SILT_DIR
              fi
              nginx -g "daemon off;"

I can access the application, but in the second step of project creation workflow the (+Add new layer(s) button does not work. No dialogue window is opened, and I can't see any relevant errors in the developer console. I do not have an Azure account, and intend to use custom TIF tiles (Based on discussion in previous issues the application should work without an Azure account).

Appreciate any pointers, thanks in advance.

Documentation of TiTiler XYZ tiles step

I am getting an error at step 1 of the Satellite Workflow, after following the instructions, most likely a silly mistake on my behalf or something missing in the documentation.

Using the AI for Good Tiler instance and two COG TIF files, one from Maxar in the previous example(which I assume is a COG), and a different Sentinel2 image which is definitely a COG I get the following error:

{"detail":[{"type":"int_parsing","loc":["path","z"],"msg":"Input should be a valid integer, unable to parse string as an integer","input":"{z}","url":"https://errors.pydantic.dev/2.3/v/int_parsing"},{"type":"int_parsing","loc":["path","x"],"msg":"Input should be a valid integer, unable to parse string as an integer","input":"{x}","url":"https://errors.pydantic.dev/2.3/v/int_parsing"},{"type":"int_parsing","loc":["path","y"],"msg":"Input should be a valid integer, unable to parse string as an integer","input":"{y}","url":"https://errors.pydantic.dev/2.3/v/int_parsing"}]}

Here are some variations of the URLs that I tried - encoded/unencoded/with the '$' variable and without
https://titiler.aiforgood.ai/cog/tiles/WebMercatorQuad/{z}/{x}/{y}.png?url=$https%3A%2F%2Fmaxar-opendata.s3.amazonaws.com%2Fevents%2FMaui-Hawaii-fires-Aug-23%2Fard%2F04%2F122000330002%2F2023-08-12%2F10300100EB15FF00-visual.tif

https://titiler.aiforgood.ai/cog/tiles/WebMercatorQuad/{z}/{x}/{y}.png?url=https%3A%2F%2Fmaxar-opendata.s3.amazonaws.com%2Fevents%2FMaui-Hawaii-fires-Aug-23%2Fard%2F04%2F122000330002%2F2023-08-12%2F10300100EB15FF00-visual.tif

https://titiler.aiforgood.ai/cog/tiles/WebMercatorQuad/%7Bz%7D/%7Bx%7D/%7By%7D.png?url=https://sentinel-cogs.s3.us-west-2.amazonaws.com/sentinel-s2-l2a-cogs/36/Q/WD/2020/7/S2A_36QWD_20200701_0_L2A/TCI.tif

Can you provide an example URLin the documentation that uses the AI for Good Titiler instance?

Feature request: lasso delete

I am finding the delete tool quite tricky to use - example below I cannot delete this line. A lasso to delete would be very handy

Test Service URL not catching bad URL if titiler running

I can insert a wrong URL, hit test, and there is no error. Note that titiler is running on 8000 and the image can be downloaded at http://localhost:5005/data/m_2808060_sw_17_060_20191215.tif so I am using URL http://localhost:8000/cog/tiles/{z}/{x}/{y}.jpg?url=http://localhost:5005/data/m_2808060_sw_17_060_20191215.tif and then breaking it in the test below:

However this functionality does appear to be working if I give a clearly wrong URL:

I dont know if this might be somehow due to using python to serve the file, I am running python3 -m http.server 5005 and have the tif at data/m_2808060_sw_17_060_20191215.tif

I have also tried adding to labeler_settings.js but the layer is not shown - actually that appears to be a caching issue, it does appear in an incognito window

Cant upload a local image as a layer

Can't upload a local image as a layer using both the bounds and rotation and/or the corners.

Instead the image appears as the entire map.

Way to remove basemaps so only my layer is visible?

When using the labeler tool, I am using a WMS service added as a layer but sometimes the layer doesn't load for whatever reason..

Problem is Azure Maps Satellite base layer always does load so I'm worried labelers will label the wrong imagery..

Is there a way to make a project only load imagery from the WMS service ?

update training-example with 0.4.1 and 0.5.0 releases

the training-example is a valuable community resource for developers who want a reference working baseline for semantic segmentation.
at the moment it is ~2 releases behind torchgeo 0.4.1 and 0.5.0.

some notes:

• need to update pip installs and imports e.g., lightning
• del sample["bbox"] needs clarifications between 0.4.1 and 0.5.0
• SegmentationDataModule should extend GeoDataModule (see related discussion)
• params in SemanticSegmentationTask need update (may be different for 0.4.1 and 0.5.0)

valuable example. would be very very helpful to update.

see Adam's suggestion to open an issue.

FWIW, will be happy to interact, give feedback and testing. please feel free to contact directly.

Adjust brightness and contrast of image only

Hi, great work on this! I need to adjust the brightness and contrast (B&C) of my COG, but using the existing sliders also changes the B&C of the map. Not sure if this is (a) intended (b) a WIP or (c) a bug?
Cheers

Drawn shapes not visibile when using GeoTIFF

I can see the Number of shapes counter increasing but the shapes are not visible while drawing or after drawing.

Steps to reproduce:

• Add GeoTIFF layer as described here: https://github.com/microsoft/satellite-imagery-labeling-tool/blob/main/docs/GeoTiffs.md
• Using the labeler tool attemps to draw a shape
• Observe count increases but shapes are not visible

Maybe something to do wtih z index ?

Consider using Azure Maps for the map control

Consider using Azure Maps for the map control. It performs much better than leaflet and will be a lot more usable. You can also put the NAIP imagery below map labels if desired.

[Feature] [Labeler] Copy & Paste Annotations

I have a project that requires labeling lots of features of identical size and shapes within close proximity.

In the Labeler, I would like to be able to Select a Polygon/Rectangle/Circle, and select a UI "copy" element (or even better, a CTRL+C shortcut) and then "paste" (ideally, a CTRL+V shortcut) onto a different section of the map.

Rename `src/overpassScripts/*.txt` to `src/overpassScripts/*.overpassql`

*.overpassql is now the recommended file extension for Overpass QL: OSM Wiki: Overpass QL § File extension
(Disclaimer: I added that section after some discussion.)

I propose to rename all Overpass QL files in this repo to *.overpassql for consistency.

Context: GitHub's syntax highlighting engine requires that 200 repositories use the same file extension before they consider supporting that syntax and file extension: github-linguist/linguist#5890

On adding a result the project viewer does not open the project

Hi
I have created a project and can successfully view it in the project viewer. I then annotated a single task and place it in the results dir of the project, then zip it all up. However the project viewer will then not open the project, and there is no error message. The generated geojson is below - no issues loading in qgis however:

{"type":"FeatureCollection","features":[
{"type":"Feature","properties":{"source":"Drawn|Azure Maps Satellite","class":"house","secondary_class":null,"task_name":"Robins_house_0"},"geometry":{"type":"Polygon","coordinates":[[[-0.26219,51.389004]]]}},
{"type":"Feature","properties":{"source":"Drawn|Azure Maps Satellite","class":"house","secondary_class":null,"task_name":"Robins_house_0"},"geometry":{"type":"Polygon","coordinates":[[[-0.261835,51.388774],[-0.261975,51.388854],[-0.261915,51.388891],[-0.261975,51.388929],[-0.261908,51.388962],[-0.261747,51.388875],[-0.261734,51.388883],[-0.261674,51.388849],[-0.261754,51.388799],[-0.261774,51.388808],[-0.261835,51.388774]]]}},
{"type":"Feature","properties":{"source":"Drawn|Azure Maps Satellite","class":"house","secondary_class":null,"task_name":"Robins_house_0"},"geometry":{"type":"Polygon","coordinates":[[[-0.261788,51.388766],[-0.261687,51.388829],[-0.261519,51.388728],[-0.26162,51.388661],[-0.261788,51.388766]]]}},
{"type":"Feature","properties":{"source":"Drawn|Azure Maps Satellite","class":"house","secondary_class":null,"task_name":"Robins_house_0"},"geometry":{"type":"Polygon","coordinates":[[[-0.261922,51.38836],[-0.261751,51.388439],[-0.261643,51.388358],[-0.261707,51.388326],[-0.261724,51.388341],[-0.261798,51.388314],[-0.261784,51.388299],[-0.261821,51.38828],[-0.261922,51.38836],[-0.261922,51.38836]]]}},
{"type":"Feature","properties":{"task_name":"Robins_house_0"},"geometry":{"type":"Polygon","coordinates":[[]]}}]}