GithubHelp home page GithubHelp logo

grhalbgott / index-calculator Goto Github PK

View Code? Open in Web Editor NEW
3.0 1.0 1.0 608 KB

A python program which calculates a chosen index for Sentinel-2 and Landsat 8 images and gives multiple output options.

License: GNU General Public License v3.0

Python 100.00%
indices remote-sensing satellite-imagery command-line-tool python landsat-8 sentinel-2

index-calculator's Introduction

Index Calculator

Calculate different indices of Sentinel-2 and Landsat 8 raster images and, if desired, cut them to a specific area of interest (AOI, recommended since they are clipped beforehand and can therefore strongly decrease execution times!).

Test picture

Following indices are available so far:

Index Sentinel-2 Landsat 8
Atmospherically Resistant Vegetation Index (ARVI) x x
Green Chlorophyll Vegetation Index (GCI) x x
Green Normalized Difference Vegetation Index (GNDVI) x
Normalized Burn Ratio (NBR) x x
Normalized Burn Ratio 2 (NBR2) x x
Normalized Difference Build-up Index (NDBI) x x
Normalized Difference Moisture Index (NDMI) x x
Normalized Difference Red-Edge Vegetation Index (NDRE) x
Normalized Difference Snow Index (NDSI) x x
Normalized Difference Vegetation Index (NDVI) x x
Normalized Difference Water Index (NDWI) x x
Red-Edge Inflection Point (REIP) x
Soil-Adjusted Vegetation Index (SAVI) x x
Structure Intensive Pigment Vegetation Index (SIPI) x x

If you want any indices to be implemented as well, please don't hesitate to write an issue.

Output options

The following outputs can be automatically generated and saved to ./results/:

  • Plot the resulting array with the most suitable ranges (as found in literature)
  • Save the plot as figure
  • Save the resulting array as txt-file
  • Export the resulting array as GIS-ready tif-file (raster)
  • Generate a histogram and include descriptive statistics (min, max, mean, std.dev)

As before, if there is any type of output which you would want to get as well, please write an issue.

Data

Required data to calculate indices are multispectral raster images with specific bands needed for specific indices. So far both the Sentinel-2 and Landsat 8 satellite platforms with their respective multispectral sensoring systems are implemented and can be used as input datasets.
Note: indices for Landsat 8 datasets can only be calculated with a spatial resolution of 30 meters, Sentinel-2 offers the possibility to calculate with a spatial resolution of 10, 20 and 60 meters.

The datasets can be acquired through different ways, the following two are only exemplarily shown:

How to acquire Sentinel-2 raster data
  1. Navigate to Copernicus Open Access Hub by ESA registration form and set up an account (skip if already registered)
  2. Log in on Copernicus Open Access Hub. Without logging in you cannot download the required data
  3. Specify the search area in the map with right-click (move map with left-click and zoom in with mouse wheel)
  4. Click on the three stripes left of the search box to open the advanced search (upper left corner of screen)
  5. Select Sentinel-2 and put following statement in the box for the cloud cover: [0 TO 2]
  6. If you want to search for data in a specific time period, put the required dates in "sensing period"
  7. Click on the search button (upper right of search box) and wait until the results are displayed
  8. Search for an image with full extent (no black parts) and minimal cloud cover
  9. Hover over the entry and click on the eye icon ("View product details") which appears along with other icons on the lower right side of the entry
  10. Check in the quick look window if the data seems suitable

    If the images you are looking for are offline, take a look at troubleshooting - Sentinel-2 data offline for some help on that problem.

  11. In the Inspector, click on the download-arrow in the lower right corner to download the complete ZIP-file
  12. When downloaded, extract the ZIP-file and put the new folder in the ./data/raster/ folder (./data/raster/S*)
How to acquire Landsat 8 raster data
  1. Navigate to USGS EROS registration system and set up an account (skip if already registered)
  2. Log in on USGS login form. Without logging in you cannot download the required data
  3. Go to USGS Earth Explorer
  4. Specify the search area in the map by zooming in to the area of interest
  5. Click on the "Use Map" button on the left side. This bounds the area of interest in a rectangle which can be used as search criteria
  6. Click on "Data Sets" on top of the left box and select Landsat -> Landsat Collection 2 (Level 1 or 2) -> Landsat 8/9 OLI/TIRS...
  7. If you want to search for data in a specific time period or with specific cloud coverage, click on "Additional criteria" on top of the left boy and put the required dates in "Date Product Generated"
  8. Click on "Results" and wait until the results are displayed
  9. Search for an image with minimal cloud cover and with the right footprint (leftmost icon on each image)
  10. If the data seems suitable, you can click on the download icon of the according image. A popup pops up and enables you to choose the data you want to download
  11. Click on "Product options" and on the download button on top to download the whole data set
  12. When downloaded, extract the ZIP-file and put the new folder in the ./data/raster/ folder (./data/raster/L*)

Exemplary multispectral raster data for both satellites (S* is Sentinel-2, L* is Landsat 8) can be found here and should be put into ./data/raster/ (unzip so as the name of the file is the first folder like ./data/raster/L*/tif-files).

If you want to clip the raster data, you will need a shapefile with your AOI. It has to be manually put into ./data/shapes/.
An example AOI to test the program can be found in the HeiBOX folder mentioned above as well (unzip without creating a new folder like ./data/shapes/aoi.shp).

Getting Started

To run the program every machine having at least Python 3.9 installed is suitable. Your Python environment should additionally include following libraries:

  • numpy
  • matplotlib
  • fiona
  • rasterio

The rest should be included in the python installation. If you are using Linux or Mac, you should check the relative paths beforehand.

Installing

Create your virtual python environment and clone the repository. Open the terminal and navigate to the cloned/downloaded folder index-calculator. If you don't have the required packages installed already, call this first:

$ pip install -r ./requirements.txt

You should then be ready to execute the program.

Usage

If ready, call the program without any arguments to access the help within the terminal with information on how to use the arguments:

$ python src/main.py

usage: main.py [-h] -i Index name [-c Clip] [-sat Satellite] [-r Resolution] [-ov Optional value]
[-tif Save raster] [-gp Generate plot] [-sp Save plot] [-txt Save as txt] [-stat Statistics]

Calculate an index with Sentinel-2 satellite imagery.
You can use the following options to adapt the calculation to your needs. Have fun!

required arguments:
  -i Index name       String | Choose which index gets calculated.
                      Check the README for a list of possible indices.

optional arguments:
  -c Clip             String | Clip raster to shapefile with shapefile. Use the name and file-type only
                      (like aoi.shp). Default value: None
  -sat Satellite      String | You can use different satellite datasets (sentinel2/s2 or landsat8/l8).
                      Default value: s2
  -r Resolution       Integer | When using Sentinel-2 datasets, the indices can be calculated with
                      different resolutions (10, 20, 60 (m)). Default value: highest resolution possible
  -ov Optional value  Float | Some indices need additional values like the L-value in SAVI (0.5).
                      Default value: as in literature
  -tif Save raster    Boolean | Do you want to export the results/ndarray as tif-file locally
                      to ./results/? Use true/false. Default: false
  -gp Generate plot   Boolean | Do you want to generate a plot? Use true/false. Default: true
  -sp Save plot       Boolean | Do you want to save the plot locally to ./results/? Use true/false.
                      Default: false
  -txt Save as txt    Boolean | Do you want to save the results/ndarray as txt-file locally
                      to ./results/? Use true/false. Default: false
  -stat Statistics    Boolean | Do you want to generate statistics (histogram & descriptive) for
                      the results and save them locally to ./results/? Use true/false. Default: false

Exiting program, call again with arguments to run.

Cleaning up

If finished with multiple analyses, you can empty ./results/ and delete temporary used files from ./data/.
Make sure to save any results you want to keep to another location BEFORE executing the following command!

To do a cleanup, call:

$ python src/cleanup.py

Footnote:

If you have any questions, wishes or ideas, feel free to ask me in the issues section, I'm looking forward to it. Have fun!

index-calculator's People

Contributors

grhalbgott avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar

Forkers

nilick

index-calculator's Issues

VARI

Visible Atmospherically Resistant Index
VARI = (Green - Red) / (Green + Red - Blue)

Gitelson, A., et al. "Vegetation and Soil Lines in Visible Spectral Space: A Concept and Technique for Remote Estimation of Vegetation Fraction." International Journal of Remote Sensing 23 (2002): 2537−2562.

Argument: satellite

Sentinel-2, Landsat 8 and MODIS/AVHHR are suitable
look up what makes sense next to Sentinel-2 and Landsat

BUG: GCI

Statistics show wrong values
Histogram seems wrong as well

REIP isn't working as expected

the value range is wrong and the image seems wrong as well

Possible solution:

  • calculation is wrong (but it is not, proven by using QGIS Raster Calculator)
  • range of values, NoData values

BUG: reip

Statistics show wrong values
Histogram seems wrong as well

NDBI

Normalized Difference Built-up Index
NDBI = (SWIR - NIR) / (SWIR + NIR)

Zha, Y., J. Gao, and S. Ni. "Use of Normalized Difference Built-Up Index in Automatically Mapping Urban Areas from TM Imagery." International Journal of Remote Sensing 24, no. 3 (2003): 583-594.

Change plot of NDWI

-0.8 to 0.8 with three colors (RdYlGn) but with higher contrast (Green - White - Blue) -> viridis?

Testing

In_raster1 = np.array(0, 0, 0, 0, 1, 2, 3)
In_raster2 = np.array(0, 0, 0, 0, 1, 2, 3)

Assert out_raster = zahlen verändert je nach forme

ARVI

Atmospherically Resistant Vegetation Index
ARVI = (NIR – (2 * RED) + BLUE) / (NIR + (2 * RED) + BLUE)

BUG: SAVI

Statistics show wrong values
Histogram seems wrong as well?

New file indices.py

Put all index-calculation in a new file and use utils for:

  • read_raster
  • cut_raster
  • function caller (which function for which index)

GCI

Green Chlorophyll Vegetation Index
GCI = NIR / GREEN – 1

SIPI

Structure Intensive Pigment Vegetation Index
SIPI = (NIR – BLUE) / (NIR – RED)

HOW TO INDEX

  1. write function in indices.py
  2. include in index_calculator() in utils.py
  3. Check max resolution in resolution_handler() in utils.py
  4. define how to plot in plottype_handler() in utils.py
  5. Update Readme
  6. Commit and close issue

BUG: SIPI

Statistics show wrong values
Histogram seems wrong as well

Merge index calculator functions

the only thing that changes between Landsat and Sentinel is the data structure. Therefore the calculations can be the same but there has to be another variable "data structure/path"

Utils function index_calculator manages then which calculations are possible

Range for plot

extra argument for range from to (-f 0 -t 0.4)
parse into plot function and give to index_plot

MNDWI

Modified Normalized Difference Water Index
MNDWI = (Green − MIR)/(Green + MIR)

Hanqiu Xu (2006) Modification of normalised difference water index (NDWI) to enhance open water features in remotely sensed imagery, International Journal of Remote Sensing, 27:14, 3025-3033.

BAI

Burn Area Index
BAI = 1/((0.1 -RED)^2 + (0.06 - NIR)^2)

Chuvieco, E., M. Pilar Martin, and A. Palacios. "Assessment of Different Spectral Indices in the Red-Near-Infrared Spectral Domain for Burned Land Discrimination." Remote Sensing of Environment 112 (2002): 2381-2396.

NBR2

Three things are required:

  1. Name of index in full and as abbreviation:
  • NBR2
  • Normalized Burn Ratio 2
  1. Formula (including bands), maybe even reference to according literature
  • (SWIR1 – SWIR2) / (SWIR1 + SWIR2)
  • Sentinel-2: NBR2 = (B11 – B12) / (B11 + B12)
  • Landsat 8-9, NBR2 = (Band 6 – Band 7) / (Band 6 + Band 7)
  1. How the plot should look like (output range and cmap)
  • RdYlGn
  • -1 to 1

GNDVI

Green Normalized Difference Vegetation Index
GNDVI = (NIR – GREEN) / (NIR + GREEN)

SAVI

Soil-Adjusted Vegetation Index
SAVI = ((NIR - Red) / (NIR + Red + L)) x (1 + L)
default L = 0,5
but you can put in you own value

Automatic analysis

After calculating the index, save some further analytic data into results/ like:

  • histogram
  • statistics (min, max, mean, std.dev)
  • other?

NBR

Three things are required:

  1. Name of index in full and as abbreviation:
  • NBR
  • Normalized Burn Ratio
  1. Formula (including bands), maybe even reference to according literature
  • (NIR - SWIR) / (NIR + SWIR)
  • Sentinel 2: NBR = (B8A – B12) / (B8A + B12)
  • Landsat 8-9, NBR = (Band 5 – Band 7) / (Band 5 + Band 7
  1. How the plot should look like (output range and cmap)
  • RdYlGn, -1 to 1

Automate getting aoi if no aoi is specified

OSM-boundaries or gadm.org, check where an API is available and use it to get an aoi (Name of city) if no aoi is specified (if os.path(glob.glob("./data/shapes/*)) not True: get aoi OR if aoi is specified through cmd, get from api

NDSI

Normalized Difference Snow Index
NDSI = (Green - SWIR)/(Green + SWIR)

Riggs, G., D. Hall, and V. Salomonson. "A Snow Index for the Landsat Thematic Mapper and Moderate Resolution Imaging Spectrometer." Geoscience and Remote Sensing Symposium, IGARSS '94, Volume 4: Surface and Atmospheric Remote Sensing: Technologies, Data Analysis, and Interpretation (1994), pp. 1942-1944.

NDRE

Normalized Difference Red Edge Vegetation Index
NDRE = (NIR – RED EDGE) / (NIR + RED EDGE)

Update Readme

Include output options
check for grammar and language

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.