geotiff / geoblaze

Blazing Fast JavaScript Raster Processing Engine

GeoBlaze

A blazing fast javascript raster processing engine

---

GeoBlaze is a geospatial raster processing engine written purely in javascript. Powered by geotiffjs, it provides tools to analyze GeoTIFFs. Using geoblaze, you can run computations ranging from basic statistics (min, max, mean, mode) to band arithmetic and histogram generation in either a web browser or a node application.

Learn more by reading our documentation

Getting Started

1. Add GeoBlaze to your project

  npm install geoblaze

2. Load a GeoTIFF and run a calculation

import geoblaze from 'geoblaze';

const url = 'http://url-to-geotiff';

async function getMean () {
  const georaster = await geoblaze.parse(url);
  const mean = await geoblaze.mean(georaster);
  return mean;
}

This is a contrived example, but using geoblaze is really this simple!

Contributing

We would love to have your support improving geoblaze. Here are some ways to contribute:

Issues

If you find a 🐛, please report it as an issue! To avoid overwhelming our developers, please first scan the list of bugs to make sure it hasn't already been reported. Make sure to add steps to reproduce, so we can quickly find and fix the problem.

We are also accepting feature requests! Our goal is to make geoblaze as useful to the community as possible. Towards that goal, we would love suggestions for additional features to improve the tool.

Please tag issues with the appropriate label.

Pull Requests

We are always accepting new PRs. To get started, see the developer's guide below

We squash and rebase all PRs. Please squash all of your commits into one and rebase off dev.

Submitting a PR

1. Make sure the branch is rebased and passing tests.

2. Answer all of the questions in the PR template.

3. Add the "needs review" label to your PR.

4. Upon review, the label will be adjusted. If the label is switched to "needs development", this means there are comments that need to be addressed. After addressing the comments, move the label back to "needs review"

5. Once comments have been addressed, a reviewer will move the label to "can deploy". After a final check, the PR will be merged and deployed.

Developer's Guide

Setup

1. Fork the repository

2. Clone your fork

  git clone [email protected]:[your-username]/geoblaze.git

3. Install dependencies

  cd geoblaze
  npm install

Linting

GeoBlaze uses eslint for linting. We suggest using an eslint plugin for your text editor. Here are some common options:

• For Sublime: SublimeLinter-eslint
• For Atom: linter-eslint

Our style guide extends the eslint:recommended set of standards and is therefore relatively minimal. As the project expands in scope and number of contributors, we will start to consider more linting rules. Please feel free to suggest some in a PR!

To test for linting errors:

  npm run lint

Testing

GeoBlaze uses flug as its testing framework.

To run tests:

1. Run the setup script. This script will download all of the necessary test data from s3.

  npm run setup

2. Run tests. Each test file will automatically spin up a server using srvd.

  npm run test

Resources:

• GeoBlaze: Cloud-Optimized Raster Analysis: https://www.youtube.com/watch?v=v3e2ODCK9Co&t=6135s

Used By

geoblaze's Issues

node-fetch doesn't support blobs

which are what the Load File option uses

Implement RasterWindow for Identify

We absolutely don't want to load all of the raster pixel values into JavaScript memory for a simple identify function. We should use a rasterWindow, which you can be seen used here: https://www.npmjs.com/package/geotiff

Using RasterWindow will also decrease the odds of maxing out the system memory and crashing the browser.

ObservableHQ Notebook for Raster Band Math?

I think it would be awesome if we could create ObservableHQ Notebook examples of raster band math. It'd also help users understand how to use it. You could also push it to app.geotiff.io like I do in https://beta.observablehq.com/@danieljdufour/create-an-rgb-geotiff if the GeoTIFF file isn't too big to embed in the url!

Add histogram feature

Enhance speed of average calculations

I think it'll be faster if we have an if statement or ternary expression in the reduce function rather than having filter then reduce. That's because we're currently iterating over all the values twice. Not sure though. We can write performance tests too to find out!

DONE

Support Cloud Optimized GeoTIFFs

parseFloat for GDAL_NO_DATA

Here's an example of a raster that uses a float for their GDAL_NO_DATA value

It's a Rwanda raster for world pop.

https://github.com/GeoTIFF/geotiff.io/blob/master/data/RWA_ppp_v2b_2020.tif

Convert from snake case to camel case

Since it is the standard convention in javascript, we should more to camel case.

intersect polygon should yield location

need to return row index and column index along with value and band_index.

pass in blank geometries to tools

already did sum, have to do others, too

Handle FeatureCollection

Here's example of geojson:
http://bl.ocks.org/d/17662f9239fd6e4c38008c48125af756

{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"properties": {},
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-112.1484375,
-55.37911044801048
],
[
101.6015625,
-55.37911044801048
],
[
101.6015625,
68.9110048456202
],
[
-112.1484375,
68.9110048456202
],
[
-112.1484375,
-55.37911044801048
]
]
]
}
}
]
}

Clip One Raster by Another Raster

Params:

Clip By This Raster:

• raster loaded
• values (exact or range)

Raster To Clip

(No Data Value? or take from GeoTIFF metadata)

Explore Rearchitecting to Classes

Not necessarily gonna rearchitect

Code complexity will become an issue as we increase the number of features we support. Converting to classes may improve readability

This is a research task - if this refactor would significantly hurt performance, we should abort

Improve performance for getting all values as flat array

Enhance Speed of Basic Stats Tools using Spatial Gridding/Indexing

Refactor Testing

With features getting more complex, testing will need to be re-architected to be more efficient at getting high coverage. Specifically, we need:

• a common access point for geometries
• coverage of bbox, single and multi-part geometries, feature collections, union geometries
• coverage for all possible geometry input formats
• a standard for how these tests are asserted

Make tool results consistent (assuming multiple bands)

Currently, tools check to see if there is only one band and returns just those results if that is the case. This will be unsustainable in the future; it is better to have the user handle it on the front end

Identify Doesn't Work with SRTM Degree Tiles

Here's the link to the file:

ftp://ftp.glcf.umd.edu/glcf/SRTM/Degree_Tiles/n038/SRTM_f03_n038e077/SRTM_f03_n038e077.tif.gz

get performance metrics

Need to write tests (probably in a separate repository) that tests and compares GDAL, GeoDjango, rasterio, and geoblaze.

Speed up Identify Calculation

Identify runs slow on mobile devices probably because readRasters takes a while to run (not 100% sure on this though).

We could speed this up by doing the following:

• converting the values stored in the geotiff.js object to a regular array when the size of the object isn't too big. This would mean that you load the values into JavaScript and when you run an identify, you directly grab the element in the array that corresponds with the user-designated cell.

We can use the Browser's performance API, to track how long operations take.

Add travis-ci build badge when updated master with travis-ci .yml

Expand Point Support

gio-convert-geometry should handle points when passed in as object with x and y as properties

add documentation

using documentation.js that MapBox GL JS and Turf use

Improve flexibility handling geojson geometries

Create Documentation for intersect_polygon

Should probably rename to something a little more intuitive

Handle parallel lines in polygon intersect calculations

Get Band Arithmetic to Support No Data Values

Write Function To Write TIFF

check out UTIF
https://github.com/photopea/UTIF.js?utm_source=tuicool&utm_medium=referral

Convert Weird Data Format to GeoTIFF

http://ec.europa.eu/eurostat/web/gisco/geodata/reference-data/population-distribution-demography/geostat

Pass in null geom for packages

Throw Errors

Throw errors in gio.js. Could you throw an errors in addition to logging to console with console.error? We need this in order to capture errors and alert the users.

We can then catch errors in promises with .catch (the opposite of .then).

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/catch

Hide interface when click on map on Mobile

Allow null geometry for histograms

Add polygon option to area calculations

Don't use deprecated turf pkgs

use namespaced

[USER REQUEST] Describe 3rd Param for Histogram

Currently, documentation doesn't describe that third options parameter for histogram. A user spent a long time and had to go through our source code to figure out what params to pass in with options. We need to update documentation.

Create automated testing

Is your feature request related to a problem? Please describe.

We need to set up automated testing, so we can automatically test pull requests

Describe the solution you'd like

We probably should create a .circleci folder and create a config.yml inside of this folder. This config.yml configures the Circle CI tests. Here's an example of a config.yml file: https://github.com/FirstDraftGIS/cooccurrences/blob/master/.circleci/config.yml

Describe alternatives you've considered

TravisCI also provides from testing for Open Source projects. I'd be happy with Travis CI, too! :-) Here's an example of a .travis.yml file: https://github.com/DanielJDufour/date-extractor/blob/master/.travis.yml

Additional context

N/A

Polygon calculations miss the last row of pixels in the polygon

New Prod Release

@DanielJDufour DanielJDufour Merge pull request #24 from sdpeyton/temp …

Add Code Editor to Docs?

• Should we add code editor to docs, so people can actually edit example code on the fly???

• We probably should only have a few rasters loaded for all the samples though...

https://github.com/ajaxorg/ace

Add polygon check to gio-convert-geometry

Added Verbiage to Read Me

It'd be nice of the Readme had the following categories, if not more:

• Basic Description
• Installation npm install geotiff-io
• Usage Examples
• Table of high-level features (e.g., identify, viewshed, etc.)
• How To run the test
• Also link to documentation

Here's an example:
https://github.com/DanielJDufour/date-extractor

Make Documentation look better

Documentation https://docs.geoblaze.io/ looks ugly (I did it)

We need to make it look better, something like https://turfjs.org

Perhaps we can borrow orange color from http://geoblaze.io/

Clip Raster by Vector

Is your feature request related to a problem? Please describe.
A user should be able to clip a raster by a vector. In other words, the clip function should clip the raster by the geometry's bounding box and set all the remaining values outside of the geometry to No Data. It should also update the metadata for the raster.

Describe the solution you'd like
A clear and concise description of what you want to happen.
Create a clip module that takes in a georaster and geovector and returns the result as a georaster. The approach could include:

1. Calculating bbox of geometry
2. Getting insides of the geometry
3. Iterating through rows of pixels, creating a new row starting at xmin of geometry, and adding NoData where outside geometry and current pixel value where inside the geometry

Describe alternatives you've considered
None at the moment

Additional context
You'll need to borrow some logic from https://github.com/GeoTIFF/geoblaze/blob/master/src/intersect-polygon/intersect-polygon.module.js and potentially split out some of the code, creating a separate getInsides module

Load Zip File with tiff or tiffs in it

ift.tt/2gKrEzF

Set Up Automated Testing on Travis CI

We want to run automated tests after each PR created.

You want to place a .travis.yml file in the root directory and configure it to run npm test. Here's an example for a Python package.

https://github.com/DanielJDufour/date-extractor/blob/master/.travis.yml

https://travis-ci.org/

Add Debug Mode

We should add a debug mode in the settings, also as a url param, that will add a lot of alerts to the page with relevant info. For example, after we run an identify, the alert should say how much each part of that process took.

Improve Leaflet.CanvasLayerField

Histogram Quantile is missing the last quantile