GithubHelp home page GithubHelp logo

sertit / eoreader Goto Github PK

View Code? Open in Web Editor NEW
265.0 7.0 21.0 312.91 MB

Remote-sensing opensource python library reading optical and SAR sensors, loading and stacking bands, clouds, DEM and spectral indices in a sensor-agnostic way.

Home Page: https://eoreader.readthedocs.io/en/latest/

License: Apache License 2.0

Python 100.00%
sar sentinel-2 sentinel-1 cosmo-skymed terrasar radarsat sentinel-3 landsat rasterio geopandas

eoreader's Introduction

pypi Conda Tests Gitter Apache DOI stars Conda

eoreader_logo EOReader

EOReader is a remote-sensing opensource python library reading optical and SAR constellations, loading and stacking bands, clouds, DEM and spectral indices in a sensor-agnostic way.

Managed optical constellations

Sentinel-2 SAFE and Theia Sentinel-3 OLCI and SLSTR Landsat 1 to 9 Harmonized Landsat-Sentinel PlanetScope, SkySat and RapidEye Pleiades and Pleiades-Neo SPOT-6/7 and 4/5 Vision-1 Maxar (WorldViews, GeoEye) SuperView-1 GEOSAT-2

Managed SAR constellations

Sentinel-1 COSMO-Skymed 1st and 2nd Generation TerraSAR-X, TanDEM-X and PAZ SAR RADARSAT-2 and RADARSAT-Constellation ICEYE SAOCOM Capella

Features

EOReader implements sensor-agnostic features:

EOReader works mainly with:

Python Quickstart

Optical

EOReader allows you ta load and stack spectral bands, spetrcal indices, DEM and cloud bands agnostically from every handled optical constellation:

from eoreader.reader import Reader
from eoreader.bands import *

# Sentinel-2 path
s2_path = "S2B_MSIL1C_20181126T022319_N0207_R103_T51PWM_20181126T050025.SAFE"

# Create the reader object and open satellite data
reader = Reader()

# The reader will recognize the constellation from its product structure
s2_prod = reader.open(s2_path)

# Load some bands and index
bands = s2_prod.load([NDVI, GREEN, CLOUDS])

# Create a stack with some bands
stack = s2_prod.stack([RED, GREEN, BLUE], stack_path="s2_rgb_stack.tif")

EOReader aligns spectral bands from every handled sensor in order to make any call to a band generic:
Optical Band Mapping

SAR

In the same way, you can import and stack radar band from any handled SAR constellation, with the same pattern.

from eoreader.reader import Reader
from eoreader.bands import *

# Sentinel-1 GRD path
s1_path = "S1B_EW_GRDM_1SDH_20200422T080459_20200422T080559_021254_028559_784D.zip"

# Create the reader object and open satellite data
reader = Reader()

# The reader will recognize the constellation from its product structure
s1_prod = reader.open(s1_path)

# Load some bands and index
bands = s1_prod.load([VV, VH])

# Create a stack with some bands
stack = s1_prod.stack([VV_DSPK, VH_DSPK], stack_path="s1_stack.tif")

⚠️SNAP and SAR

SAR products need ESA SNAP free software to be orthorectified and calibrated. Ensure that you have the folder containing your gpt executable in your PATH. If you are using SNAP 8.0, be sure to have your software up-to-date (SNAP version >= 8.0).

Documentation

The API documentation can be found here.

Examples

Available notebooks provided as examples:

Installation

Pip

You can install EOReader via pip:

pip install eoreader

EOReader mainly relies on geopandas and rasterio (through rioxarray).

On Windows and with pip, you may face installation issues due to GDAL. The well known workaround of installing from Gohlke's wheels also applies here. Please look at the rasterio page to learn more about that.

Conda

You can install EOReader via conda:

conda config --env --set channel_priority strict
conda install -c conda-forge eoreader

Context

As one of the Copernicus Emergency Management Service Rapid Mapping and Risk and Recovery Mapping operators, SERTIT needs to deliver geoinformation (such as flood or fire delineation, landslides mapping, etc.) based on multiple EO constellations.

In rapid mapping, it is always important to have access to various sensor types, resolutions, and satellites. Indeed, SAR sensors are able to detect through clouds and during nighttime (which is particularly useful during flood and storm events), while optical sensors benefit from of multi spectral bands to better analyze and classify the crisis information.

As every minute counts in the production of geoinformation in an emergency mode, it seemed crucial to harmonize the ground on which are built our production tools, in order to make them as sensor-agnostic as possible.

This is why SERTIT decided to decouple the sensor handling from the extraction algorithms: the latter should be able to ingest semantic bands (i.e. RED or VV) without worrying about how to load the specific sensor band or in what unit it is.
The assumption was made that all the spectral bands from optical sensors could be mapped between each other, in addition to the natural mapping between SAR bands.

Thus, thanks to EOReader, these tools are made independent to the constellation:
✅ the algorithm (and its developer) can focus on its core tasks (such as extraction) without taking into account the sensor characteristics (how to load a band, which band correspond to which band number, …)
✅ new sensor addition is effortless (if existing in EOReader) and requires no algorithm modification
✅ maintenance is simplified and the code quality is significantly improved
✅ testing is also simplified as the sensor-related parts are tested in EOReader library

However, keep in mind that the support of all the constellations used in CEMS is done in the best effort mode, especially for commercial data. Indeed, we may not have faced every product type, sensor mode or order configuration, so some details may be missing. If this happens to you, do not hesitate to make a PR or write an issue about that !

Talks

Press Release

Articles

Blog

License

EOReader is licensed under Apache License v2.0. See LICENSE file for details.

Authors

EOReader has been created by ICube-SERTIT.

Credits

EOReader is built on top of amazing libs, without which it couldn't have been coded:

eoreader's People

Contributors

dependabot[bot] avatar floriandeboissieu avatar jsetty avatar jteulade avatar parusinski-unistra avatar pre-commit-ci[bot] avatar remi-braun avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

eoreader's Issues

Missing ogr2ogr in PATH: error in ‘wgs84_extent’ when processing Sentinel-1 products

Assigning CRS to a GeoDataFrame without a geometry column is not supported. Use GeoDataFrame.set_geometry to set the active geometry column.
Traceback (most recent call last):
File "d:\SAR_DATA\process\eoreader-main\eoreader\products\sar\s1_product.py", line 187, in wgs84_extent
raise InvalidProductError(
eoreader.exceptions.InvalidProductError: Cannot determine the WGS84 extent of S1A_IW_GRDH_1SDV_20200308T121318_20200308T121343_031584_03A39B_5C41

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\Users\10992.conda\envs\DL1\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\10992.conda\envs\DL1\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "c:\Users\10992.vscode\extensions\ms-python.python-2022.5.11221002\pythonFiles\lib\python\debugpy_main
.py", line 45, in
cli.main()
File "c:\Users\10992.vscode\extensions\ms-python.python-2022.5.11221002\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 444, in main
run()
File "c:\Users\10992.vscode\extensions\ms-python.python-2022.5.11221002\pythonFiles\lib\python\debugpy/..\debugpy\server\cli.py", line 285, in run_file
runpy.run_path(target_as_str, run_name=compat.force_str("main"))
File "C:\Users\10992.conda\envs\DL1\lib\runpy.py", line 268, in run_path
return _run_module_code(code, init_globals, run_name,
File "C:\Users\10992.conda\envs\DL1\lib\runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "C:\Users\10992.conda\envs\DL1\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "d:\SAR_DATA\process\eoreader-main\K.py", line 15, in
s1_prod.wgs84_extent()
File "C:\Users\10992.conda\envs\DL1\lib\site-packages\methodtools.py", line 72, in call
return self.call(*args, **kwargs)
File "d:\SAR_DATA\process\eoreader-main\eoreader_init
.py", line 31, in wrapper
return func(*args, **kwargs)
File "d:\SAR_DATA\process\eoreader-main\eoreader\products\sar\s1_product.py", line 196, in wgs84_extent
raise InvalidProductError(ex) from ex
eoreader.exceptions.InvalidProductError: Cannot determine the WGS84 extent of S1A_IW_GRDH_1SDV_20200308T121318_20200308T121343_031584_03A39B_5C41

ENH: Support Skysat

Skysat is a sensor potentially used in CEMS.

It would be great to support it.

image

ENH: Replace custom spectral indices by spyndex library

Now, all the spectral indices are hard coded in the index file.
It could be way better to use an external and promising library named spyndex to replace the file.

It would also avec the benefit of allowing (one way or another) the user to compute custom indices.

DOC: try to rationalize documentation links

For now, documentation links have no policy, every source is equal (provider sources, constructor sources, others...)

It would be great to use less sources to track them easily, among only:

Discard others, such as PCI...

ENH: Load bands as radiance

For now, we can load bands "as is" and converted to reflectance.

It would be great to allow the user to load bands as radiance too, as asked in #50.

It may be also pertinent to differentiate TOA and BOA reflectances.

[Optical] Do not use SNAP

Sentinel-3

Do not geocode S3 data with SNAP, to remove this dependency for optical data.

Reproject

⚠️ Geocoding is not as simple as initially thought

The goal is to apply lon/lat/alt arrays to every rasters (maybe through GCPs ? or -geoloc in gdalwarp ?)

Take a look at the ongoing @rioxarray #329 discussion.
And maybe at this thread

This solution using gdal warp -geoloc (script) exists (from this thread and this gdal documentation) but uses gdal CLI, which is not really wanted...

Rad2Refl

rad2refl: here

    public static float radianceToReflectance(float rad, float sza, float e0) {
        return (float) ((rad * Math.PI) / (e0 * Math.cos(sza * RAD_PER_DEG)));
    }

ENH: Support of SPOT-4/5

In order to ensure the ability for EOReader to process VHR time series, it would be great to support SPOT-4 and SPOT-5 data

  • Support SPOT-4
  • Support SPOT-5

ENH: Open custom band

Allow the user to open a custom band not referenced in EOReader (or call it by its number, ie. Oa01 for Sentinel-3 OLCI or F1 for Sentinel-3 SLSTR)

  • Enable to reading not by band but by number (at user's risk !)
  • Complete the mapping with bands with str identifiers (for not referenced bands) for every satellite ?
  • Ensure that calling the band by the referenced name or by its number still works

ENH: Update Sentinel-2 products to support format update

From what I see in this article:

From what we have heard at CNES, the changes will include :

  • A change in the coding of the reflectances : an offset will be added to keep the zero as no-data value, and still enable to code for zeroi or negative reflectances in the products.
  • The vector masks should be replaced by raster masks, which is probably not a simple change that you can implement with two lines of code.

The change should be put in production before the end of October

It should be easily tackled and should simplify the code (raster masks and GML removal)

ENH: Support SAOCOM-1

SAOCOM is an argentinan SAR sensor also used in CEMS.

It would be great to support it.

ENH: Allow the user to chose the default CRS

We should allow the user to set his preferred CRS, through maybe an environment variable such as EOREADER_CRS.

Options could be:

  • UTM
  • WGS84
  • DEFAULT (use the crs as delivered)
  • Any other valid CRS identified by its code

BUG: Create fallbacks for broken Sentinel-2 products

Manage partially broken Sentinel-2, downloaded from AWS:

  • it happens that MTD_MSIL2A.xml exists but is empty (see S2A_MSIL2A_20170331T103021_N0001_R108_T32UMV_20190508T053047.SAFE\MTD_MSIL2A.xml)
  • it also happens that the detector footprints have empty geometries (see S2A_MSIL2A_20170331T103021_N0001_R108_T32ULU_20190508T053054.SAFE\GRANULE\L2A_T32ULU_A009257_20170331T103340\QI_DATA\MSK_DETFOO_B04.gml)

Create fallbacks for nodata and existing but empty datatake mtd.

OPTIM: Speeding up EOReader

It could be really useful to use external libraries to speed up EOReader and ensure the clusterization works properly:

  • Optimize the code using libraries such as numba, xarray-geospatial...
  • Use WarpedVRT instead of reprojecting data (VHR...)
  • Use directly S3 files when possible (wait for cloudpathlib to work properly with fsspec or maybe use universal_pathlib) (-> small optim, most of the time S3 files are used, with the Path converted to string -> but better for the code's reading)
  • Hope that one day SNAP will be able to process S3-stored products (right now, it seems still impossible)
  • Make Dask really work (even if #4 is closed)
    • Ensure reading and writing is accelerated
    • Ensure everything is as lazy as possible (don't call .data !)
    • Ensure that everything has proper chunks
    • Check if reprojection, geocoding, index computation can be daskified and/or optimized:
      • Spyndex #47
      • geocode S3 #55,
      • reprojection: #115
      • Don't load data into memory before computing spectral indices
      • Use chunks when reading data
  • Enforce the window reading (#25)
  • Don't use SNAP for SAR data (#2)

ENH: Dask support

We need to implement and test the support of Dask, which could accelerate a lot our processes

ENH: Metadata management

  • Read metadata/namespaces only once and store it as a private member. Keep accessing it through the read_mtd function
  • Decouple metadata from product name (in order not to fail if the products are renamed)

ENH: Support SuperView

SuperView 1 and 2 are VHR sensors potentially used in CEMS.

  • Implement SuperView-1
  • Implement SuperView-2

It would be great to support them.

image

[OPTIM] SAR: Interpolate na only on user's request

For now, nan are always interpolated when saving SAR images to disk.

However this is only useful for a very few SAR products (only for some COSMO for now) and this takes some computational effort!

So add a keyword (i.e. SAR_INTERPOLATE_NA) and change default behaviour.

BUG: CSG + calibration step

For some reason, applying a calibration step in SNAP to some CSG products outputs an empty raster.
The workaround is to use the cplx_no_calib_preprocess_default.xml graph instead.

import os
from eoreader.reader import Reader
from eoreader.bands import VV
from eoreader.env_vars import PP_GRAPH

os.environ[PP_GRAPH]="path/to/eoreader/data/cplx_no_calib_preprocess_default.xml"

path = r"CSG_SCS_1254349-1052847"
prod = Reader().open(path)
prod.load(VV)

This may be worth investiguating (when having more CSG data?)

Issue on page /api/eoreader.html

I have an issue after trying to opening a sentinel 1 image:
Code:
l5_path = r"/media/michael/Sicherheit/S1B_IW_GRDH_1SDV_20210731T171709_20210731T171734_028041_035850_C4F8.zip"
os.environ[DEM_PATH] = os.path.join("/media/michael/Sicherheit", "s1b-iw-grd-vv-20210731t171709-20210731t171734-028041-035850-001.tiff")
eoreader = Reader()
l5_prod = eoreader.open(l5_path)
band_dict = l5_prod.load([VV])

Failure:
gpt: opção inválida: -q
Traceback (most recent call last):
File "/home/michael/anaconda3/envs/opencv_tut/lib/python3.7/site-packages/eoreader/products/sar/sar_product.py", line 442, in get_band_paths
exact_name=True,
File "/home/michael/anaconda3/envs/opencv_tut/lib/python3.7/site-packages/sertit/files.py", line 986, in get_file_in_dir
f"File with pattern {glob_pattern} not found in {directory}"
FileNotFoundError: File with pattern *20210731T171709_S1_IW_GRD_VV.tif not found in /tmp/tmpl5_4fb7c/tmp_20210731T171709_S1_IW_GRD

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/michael/anaconda3/envs/opencv_tut/lib/python3.7/site-packages/eoreader/products/sar/sar_product.py", line 787, in _pre_process_sar
misc.run_cli(cmd_list)
File "/home/michael/anaconda3/envs/opencv_tut/lib/python3.7/site-packages/sertit/misc.py", line 354, in run_cli
raise RuntimeError(f"Exe {cmd[0]} has failed.")
RuntimeError: Exe gpt has failed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/michael/PycharmProjects/opencv_tut/ship.py", line 255, in
band_dict = l5_prod.load([VV])
File "/home/michael/anaconda3/envs/opencv_tut/lib/python3.7/site-packages/eoreader/products/product.py", line 652, in load
band_dict = self._load(bands, resolution, size)
File "/home/michael/anaconda3/envs/opencv_tut/lib/python3.7/site-packages/eoreader/products/sar/sar_product.py", line 680, in _load
bands = self._load_bands(band_list, resolution=resolution, size=size)
File "/home/michael/anaconda3/envs/opencv_tut/lib/python3.7/site-packages/eoreader/products/sar/sar_product.py", line 622, in _load_bands
band_paths = self.get_band_paths(bands, resolution)
File "/home/michael/anaconda3/envs/opencv_tut/lib/python3.7/site-packages/eoreader/products/sar/sar_product.py", line 451, in get_band_paths
all_band_paths = self._pre_process_sar(resolution)
File "/home/michael/anaconda3/envs/opencv_tut/lib/python3.7/site-packages/eoreader/products/sar/sar_product.py", line 789, in _pre_process_sar
raise RuntimeError("Something went wrong with SNAP!") from ex
RuntimeError: Something went wrong with SNAP!

DEP: Drop Python 3.7 support

Python 3.7 is gradually abandonned by the community. References:

However, ArcGis Pro still uses Python 3.7 😓

So, drop Python 3.7 support as soon as ArcGis Pro supports Python 3.8 ?

OPTIM: Use pyresample to geocode Sentinel-3 products

For now, Sentinel-3 products are geocoded thanks to GDAL, using GCPs selected among the lat/lon files.
This is not optimal.

It is better to use the library pyresample to geocode the rasters, to ensure that the process is validated by someone else.
It may also time-optimize the geocoding.

BUG: Check and fix the gain/offset to compute the reflectance of optical products

Check for other optical products if the gain and offset for reflectance computation is correctly handled:

  • Sentinel-2 L2A & L1C
    • PB < 04.00: ✅ divide by 10000
    • PB > 04.00: ✅ Use coefficients in the metadata (should be bias = -1000 and gain =1/10000)
  • Sentinel-2 Theia
    -> ✅ divide by 10000
  • Landsats
    -> ✅ Use the coefficients in metadata
  • Sentinel-3 OLCI and SLSTR
    -> ✅ Use the coefficients in metadata
  • DIMAP (Pleiades, Pleiades-Neo, SPOT-6 and 7):
    -> ✅ divide by 10000 (gain and offset are used to compute the radiance)
    -> ⚠️ Check if the product is delivered in Reflectance Radiometric Processing (should me, most of the time)
  • Maxar (GeoEYE, WV)
  • PlanetScope
    • ⚠️ Multiply DN by reflectanceCoefficient from metadata
  • Vision-1
    -> ⚠️ provided in TOA radiance (uint16) !
    Divide by 100 to get scaled radiance, then apply the formula to convert to reflectance

✅ : was OK
⚠️: to fix !

DOC: Updates

It could be cool to create one page for every supported sensor.

This could be useful to:

  • Show what product type is not used for a specific sensor
  • Warn the user about some specificity
  • ...

[EDIT]: Not really useful until a user needs it

ENH: STAC compliance

It could be nice to make eoreader compliant with STAC item format, by:

  • Changing the attributes names (i.e. extent to bbox, footprint to geometry)
  • Creating a to_stac() function for every product
  • Create a stac attribute implementing all STAC relevant features but keep legacy EOReader functions (footprint, extent...)
  • Changing band names
  • Add informations to the bands (centrale wavelength, bandwidth, gsd, ...)

SAOCOM L1A products are already calibrated

I'm trying to read SAOCOM data without success.

Here is a minimal example to reproduce my error:

import os
from pathlib import Path

from eoreader.reader import Reader
from eoreader.bands import *


data_dir = os.path.abspath(os.path.join("..", "..", "data", "raw", "saocom"))
saocom_file = "S1A_OPER_SAR_EOSSP__CORE_L1A_OLVF_20220317T145008"

saocom_prod = Reader().open(data_dir, saocom_file)

print(saocom_prod)

eoreader.SaocomProduct 'S1A_OPER_SAR_EOSSP__CORE_L1A_OLVF_20220317T143007'
Attributes:
condensed_name: 20200801T103425_SAOCOM_VV_HH_VH_HV_SM_L1A
path: /root/data/raw/saocom
constellation: SAOCOM-1
sensor type: SAR
product type: L1A
default resolution: 10.0
acquisition datetime: 2020-08-01T10:34:25.550874
band mapping:
VV: VV
VV_DSPK: VV_DSPK
HH: HH
HH_DSPK: HH_DSPK
VH: VH
VH_DSPK: VH_DSPK
HV: HV
HV_DSPK: HV_DSPK
needs extraction: True
orbit direction: ASCENDING

It appears to correctly read the data/metadata, but it fails at getting some of the bands:

ok_bands = saocom_prod.get_existing_bands()
saocom_prod.load(bands=[VV])

Here is the traceback of the error:
https://pastebin.com/SwL9D9Ju

I need help to load the data as an xarray.

Thanks!

ENH: [SAR] Don't use SNAP

Calibration

RADARSAT-2

Sentinel-1

https://sentinels.copernicus.eu/documents/247904/685163/S1-Radiometric-Calibration-V1.0.pdf/66e69a62-11ae-4160-916b-f2b97cb8a350?t=1432307754000

TerraSAR-X

COSMO Skymed

Terrain-Correction

RADARSAT-2

Use RPCs: https://gis.stackexchange.com/questions/303233/error-in-orthorectification-of-radarsat-2-scenes-with-gdal-and-rpc-dem-informa

Should be implementable like DIMAP orthorectification:

        kwargs = {"RPC_DEM": dem_path, "RPC_DEM_MISSING_VALUE": 0}
        # TODO:  add "refine_gcps" ? With which tolerance ? (ie. '-refine_gcps 500 1.9')
        #  (https://gdal.org/programs/gdalwarp.html#cmdoption-gdalwarp-refine_gcps)

        # Reproject
        # WARNING: may not give correct output resolution
        out_arr, dst_transform = warp.reproject(
            src_arr,
            rpcs=rpcs,
            src_crs=WGS84,
            dst_crs=self.crs(),
            resolution=self.resolution,
            src_nodata=0,
            dst_nodata=0,  # input data should be in integer
            num_threads=MAX_CORES,
            resampling=Resampling.bilinear,
            **kwargs,
        )

We just need to extract RPCs from product.xml

Sentinel-1

Use GCPS, retrievable with opening directly the .SAFE product

TerraSAR-X

Simply use TDX1_SAR__MGD_SE___SM_S_SRA_20201016T231611_20201016T231616.xml

COSMO Skymed

Use GCPS, retrievable with opening directly band from tyhe .h5 file: COSMO/CSKS4_DGM_B_HI_04_HH_RA_FF_20201008224042_20201008224049.h5://S01/SBI

LinearToFromdB

TODO

Despeckle

TODO

Specific S1

ThermalNoiseRemoval

TODO

Apply-Orbit-File

TODO

Remove Remove-GRD-Border-Noise

TODO

ENH: Refine the Despckle Graph

Replace the Despeckle GPT graph by a more usual Refined Lee, for example:

  <node id="Speckle-Filter">
    <operator>Speckle-Filter</operator>
    <sources>
      <sourceProduct refid="Calibration"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <sourceBands>Sigma0_VV</sourceBands>
      <filter>Refined Lee</filter>
      <filterSizeX>3</filterSizeX>
      <filterSizeY>3</filterSizeY>
      <dampingFactor>2</dampingFactor>
      <estimateENL>true</estimateENL>
      <enl>1.0</enl>
      <numLooksStr>1</numLooksStr>
      <windowSize>5x5</windowSize>
      <targetWindowSizeStr>3x3</targetWindowSizeStr>
      <sigmaStr>0.9</sigmaStr>
      <anSize>50</anSize>
    </parameters>

ENH: Store more data as private members to speed up the processes

We should store all the data that are time-consuming (at least not immediate) and lightweight (floats or even vectors) in order to speed up the recomputation.
These would be computed only if needed.

Hereunder is a premilinary list (for product):

  • extent
  • footprint
  • mean_az, mean_el
  • crs
  • default_transform

And why not:

  • condensed_name
  • datetime
  • date

Other product_classes could have others to add.

They could be accessed as regular members through a @property -> this will break things !

ENH: Rationalize get_band_path functions

It could be great to rationalize all the functions that gravite around get_band_path, and see which one needs to be public or private.
The actual issue is that this function orthorectifies, reprojects in UTM or geocodes the arrays under the hood, which can be time consuming and not desired.

So it could be better to disambiguate these function names:

  • get_band_path: Main function that orthorectifies, reprojects in UTM or geocodes the arrays under the hood.
  • get_existing_band_paths: Get all the paths for all the existing bands and relies on get_band_path
  • get_default_band_path: Get the path of the default band and relies on get_band_path
  • _get_raw_band_paths: Get the raw band file path, SAR only. This may need to be made public and generalized to optical data ?

Other sensor specific function that may need to be grouped or generalized:

  • _get_preprocessed_band_path: Get the path of preprocessed Sentinel-3 data
  • _get_utm_band_path: Get the reproject/orthorectified the VHR bands to UTM

Other functions managing paths but which don't pose any problem:

  • _get_clean_band_path: Get the path for cleaned data, optical only.
  • _get_cloud_band_path: Get the path for cloud data written on disk, optical only.

If no clearer way of managing this is found, at least an update in the documentation is needed.

ENH: Enabling the support of windows and AOI when loading bands

When loading bands, enable the support of:

Loading a window

Loading an AOI

  • determine its extent
  • load only its window
  • then crop the windowed array by the AOI

⚠️ Implementation limits

➡️Pay attention to non orthorectified or non projected bands.
✔️Ortho/project them entirely first to mimic the orthorectification of the whole VHR stacks before reading a band.
➡️What file to write on disk, only the whole files ?
✔️Always load, process nodata or clean pixels for the whole file, and then read a window (to complicated to manage a file per window)

BUG: Fix multi-tile Planet data

Some PlanetScope data may come as a bundle of several tiles:
2022-08-26_09h36_51

For now, only the first one is processed, so add a mosaicing process before anything (create VRT of the stacks)

Maybe it can be difficult to manage the metadata (name etc.) -> create synthetic metadata?
How to manage cloud cover ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.