GithubHelp home page GithubHelp logo

Shoreline extraction issues about coastsat HOT 15 CLOSED

kvos avatar kvos commented on June 1, 2024
Shoreline extraction issues

from coastsat.

Comments (15)

conlin-matt avatar conlin-matt commented on June 1, 2024 1

Yep, this issue seems to be fixed. Thanks!

from coastsat.

kvos avatar kvos commented on June 1, 2024 1

oh yeah, when trying to divide 0 by 0 it returns a NaN, good catch @conlin-matt I didn't think an image could have 0s at the same pixel for different bands. could you post a JPEG of that image (generated with SDS_preprocess.save_jpg(metadata, settings) ? the image must have some very dark areas...

I think replacing nans with zeros should be fine... another option could be to add those pixels to the cloud_mask so they are ignored

from coastsat.

kvos avatar kvos commented on June 1, 2024

I have corrected this bug in the latest update to the toolbox...if you pull the latest changes it shouldn't happen any more.

The problem was that if you accept an image that has no mapped shoreline it would then crash when trying to write a .geojson file as there are no coordinates for that shoreline.

from coastsat.

kvos avatar kvos commented on June 1, 2024

ok perfect.
Did you also figure out why it could not map shorelines (the message: "Could not map shoreline for this image" that appears in your terminal)?

from coastsat.

conlin-matt avatar conlin-matt commented on June 1, 2024

No, that issue has stumped me so far. The warning only seems to appear for S2 images. When the "find_wl_contours2" function (i.e. line 734-735 of SDS_shoreline.py) is run on an image which results in the error, I get the ValueError shown in the screenshot. So it looks like it has something to do with an inability to detect ranges for different classes in the thesholding maybe? I should also note that the older, less robust thresholding method (implemented in find_wl_contours1) works on the image that generates the error.

Screen Shot 2019-06-20 at 3 29 20 PM

from coastsat.

kvos avatar kvos commented on June 1, 2024

could you post the image @conlin-matt ?
I think the error is that Otsu's threholding function doesn't work if there are NaNs in the histogram, but I don't see how you could get NaNs in there...
what you can do also is that you can put pdb.set_trace() before it calls threshold_otsu and it will stop the code there and then you can see what's in int_all and check if there are any NaNs or inf values...

from coastsat.

conlin-matt avatar conlin-matt commented on June 1, 2024

Github doesn't support uploads of TIFFs, so I haven't uploaded the image. After some debugging this morning, here's what I found:

You are correct that this issue appears when "int_all" has NaNs in it. This happens when the SWIR1 and green bands of the image both contain a value of 0 in the same pixel (i.e. im_ms[:,:,4] and im_ms[:,:,1] in your code both have 0s for the same pixels). Here is what happens when this occurs:

  1. SDS_shorelines.extract_shorelines calls SDS_shorelines.find_wl_contours2 (line 734-735 of SDS_shorelines)
  2. SDS_shorelines.find_wl_contours2 calls SDS_tools.nd_index (line 248 of SDS_shorelines)
  3. In SDS_tools.nd_index you have a calculation on lines 181-182 where you np.divide the two bands. But when you run np.divide(0,0) the result is a nan. So the variable im_mwi which results from the call to this function has NaNs in it, which propagate into int_all, causing the error.

Perhaps a fix to this could be to replace nans with zeros in im_mwi?

from coastsat.

conlin-matt avatar conlin-matt commented on June 1, 2024

Yep, looks like there are definitely some black spots in the image...

2019-01-01-16-05-27_S2

from coastsat.

kvos avatar kvos commented on June 1, 2024

the black spots located on the clouds are part of the cloud mask (I know the cloud mask is terrible for Sentinel-2 images!)... what I am more worried about is what's happening on the top-left corner of the image, it seems that the pixels are a bit funny (blueish colours) at the edge of the no data region. My guess would be that the 0 division is happening there, if you don't mind sending me the coordinates of your polygon I can also investigate that 2019-01-01 S2 image.

from coastsat.

conlin-matt avatar conlin-matt commented on June 1, 2024

Gotcha. Polygon coordinates are below:

polygon = [[[-80.680891,28.699524], [-80.587325,28.526645], [-80.510662,28.537539], [-80.620483,28.698629], [-80.680891,28.699524]]]

from coastsat.

kvos avatar kvos commented on June 1, 2024

the image below shows in red the pixels that have an intensity of 0 for ALL BANDS.

S2_image_canaveral

I added to the cloud mask the pixels that have 0s in the G, NIR and SWIR bands in S2 images. I just pushed this fix into the development branch. Let me know if it works.

shoreline_detection

Also @conlin-matt , the area that you are mapping is quite large, if you see that processing the shorelines is too slow you can split the polygon in 2/3 smaller polygons to speed up the processing (more images to manually check though).

from coastsat.

conlin-matt avatar conlin-matt commented on June 1, 2024

Yep, that fix seems to be working for me. Glad we were able to figure these things out!

from coastsat.

kvos avatar kvos commented on June 1, 2024

cool! did you run into this issue also with Landsat images or it was only S2?

from coastsat.

conlin-matt avatar conlin-matt commented on June 1, 2024

It only ever popped up with S2 images for me.

from coastsat.

kvos avatar kvos commented on June 1, 2024

issue solved

from coastsat.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.