Comments (15)
Yep, this issue seems to be fixed. Thanks!
from coastsat.
oh yeah, when trying to divide 0 by 0 it returns a NaN, good catch @conlin-matt I didn't think an image could have 0s at the same pixel for different bands. could you post a JPEG of that image (generated with SDS_preprocess.save_jpg(metadata, settings) ? the image must have some very dark areas...
I think replacing nans with zeros should be fine... another option could be to add those pixels to the cloud_mask so they are ignored
from coastsat.
I have corrected this bug in the latest update to the toolbox...if you pull the latest changes it shouldn't happen any more.
The problem was that if you accept an image that has no mapped shoreline it would then crash when trying to write a .geojson file as there are no coordinates for that shoreline.
from coastsat.
ok perfect.
Did you also figure out why it could not map shorelines (the message: "Could not map shoreline for this image" that appears in your terminal)?
from coastsat.
No, that issue has stumped me so far. The warning only seems to appear for S2 images. When the "find_wl_contours2" function (i.e. line 734-735 of SDS_shoreline.py) is run on an image which results in the error, I get the ValueError shown in the screenshot. So it looks like it has something to do with an inability to detect ranges for different classes in the thesholding maybe? I should also note that the older, less robust thresholding method (implemented in find_wl_contours1) works on the image that generates the error.
from coastsat.
could you post the image @conlin-matt ?
I think the error is that Otsu's threholding function doesn't work if there are NaNs in the histogram, but I don't see how you could get NaNs in there...
what you can do also is that you can put pdb.set_trace()
before it calls threshold_otsu and it will stop the code there and then you can see what's in int_all and check if there are any NaNs or inf values...
from coastsat.
Github doesn't support uploads of TIFFs, so I haven't uploaded the image. After some debugging this morning, here's what I found:
You are correct that this issue appears when "int_all" has NaNs in it. This happens when the SWIR1 and green bands of the image both contain a value of 0 in the same pixel (i.e. im_ms[:,:,4] and im_ms[:,:,1] in your code both have 0s for the same pixels). Here is what happens when this occurs:
- SDS_shorelines.extract_shorelines calls SDS_shorelines.find_wl_contours2 (line 734-735 of SDS_shorelines)
- SDS_shorelines.find_wl_contours2 calls SDS_tools.nd_index (line 248 of SDS_shorelines)
- In SDS_tools.nd_index you have a calculation on lines 181-182 where you np.divide the two bands. But when you run np.divide(0,0) the result is a nan. So the variable im_mwi which results from the call to this function has NaNs in it, which propagate into int_all, causing the error.
Perhaps a fix to this could be to replace nans with zeros in im_mwi?
from coastsat.
Yep, looks like there are definitely some black spots in the image...
from coastsat.
the black spots located on the clouds are part of the cloud mask (I know the cloud mask is terrible for Sentinel-2 images!)... what I am more worried about is what's happening on the top-left corner of the image, it seems that the pixels are a bit funny (blueish colours) at the edge of the no data region. My guess would be that the 0 division is happening there, if you don't mind sending me the coordinates of your polygon I can also investigate that 2019-01-01 S2 image.
from coastsat.
Gotcha. Polygon coordinates are below:
polygon = [[[-80.680891,28.699524], [-80.587325,28.526645], [-80.510662,28.537539], [-80.620483,28.698629], [-80.680891,28.699524]]]
from coastsat.
the image below shows in red the pixels that have an intensity of 0 for ALL BANDS.
I added to the cloud mask the pixels that have 0s in the G, NIR and SWIR bands in S2 images. I just pushed this fix into the development branch. Let me know if it works.
Also @conlin-matt , the area that you are mapping is quite large, if you see that processing the shorelines is too slow you can split the polygon in 2/3 smaller polygons to speed up the processing (more images to manually check though).
from coastsat.
Yep, that fix seems to be working for me. Glad we were able to figure these things out!
from coastsat.
cool! did you run into this issue also with Landsat images or it was only S2?
from coastsat.
It only ever popped up with S2 images for me.
from coastsat.
issue solved
from coastsat.
Related Issues (20)
- ee attribute error: module 'ee' has no attribute 'Initialize' HOT 3
- s2cloudless datatype not compatible for gdal translation HOT 4
- S2cloudless and KML format HOT 4
- Can Coastsat be used to map seawater/freshwater interface? HOT 1
- Issues in accessing Data from Google Account ? «Earth Engine client library not initialized. Run `ee.Initialize()`» HOT 1
- Animation is not ordered in a date sequence HOT 3
- multi reference shoreline .geojson not proper working HOT 1
- Download issue with L9 and S2 cloud mask HOT 4
- Error in preprocess L5 cloud mask HOT 5
- UnavailableInvalidChannel: HTTP 404 NOT FOUND for channel earthengine-api <https://conda.anaconda.org/earthengine-api> HOT 2
- IndexError: list index out of range - when plotting seasonal averages and long term trends HOT 2
- Doesn't plot final seasonal and monthly trend figures HOT 1
- After opening the clash proxy software, some errors message appears HOT 1
- Tidal Correction data error:''value' must be an instance of str or bytes, not a float' HOT 7
- Quick question about water level correction HOT 1
- Problem with GEE HOT 2
- white water border in pixel classification HOT 11
- Axis limits cannot be NaN or Inf HOT 1
- CoastSAT seasonal and monthly average graphs HOT 8
- tidal correction plus setup and runup correction! HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from coastsat.