snap-contrib / snapista Goto Github PK

Good morning.

I cannot install Snapista in any way. I have created several conda environments with different python versions but whenever I type conda install -c terradue snapista in the terminal, a problem with the python version occurs. As I said before, I have tried several Python versions according to the message which pops up on my screen, but I have not been able to install it. I attach two screenshots.

Best regards,
Mario

With Python 3.7:

With Python3.5:

I am trying to use snapista to build a graph using the Binning Operator, which needs to define aggregators and variables, but I cannot find the way to define aggregators using the aggregatorConfigs and variables using variableConfigs

Could you please provide some guidance with that or illustrate it with one example?

Thanks in advance

Good afternoon,

I am trying to build up a graph which uses the operator SliceAssembly. The problem is that I do not know how to define two sources at the same time for the SliceAssembly operator

For instance:
step_prev1 = 'ThermalNoiseRemoval'
step_prev2 = 'ThermalNoiseRemoval(2)'

sliceop = Operator('SliceAssembly')
g.add_node(operator = sliceop, node_id = 'SliceAssembly', source = step_prev1, source = step_prev2)

This gives an error as I cannot establish two sources for the SliceAssembly operator.

Many thanks in advance.

Hello !

First thank you for developing and maintaining this snapista package. To install the package I had to first run conda install -c terradue -c conda-forge snap=8.0.0, it may be useful to precise it in the README.

I am applying the operations listed below on Sentinel-1 data (~1.5GB) and it took ~10mins on a machine with 8 CPUs. Is it what I sould expect ?

• Manifest name: S1A_IW_GRDH_1SDV_20220113T015151_20220113T015216_041436_04ED43_BAD7.SAFE
• Operations
  • Read
  • Apply-Orbit-File"
  • Remove-GRD-Border-Noise
  • Calibration
  • Write

I am currently adding the Terrain-Flattening and Terrain-Correction to see how long it takes

Thanks

Good afternoon!

Are there alternative installation options for snapista? I have tried installing it on two separate machines with no success. Here is what I tried:

1. On my personal macOS machine, conda doesn't find the package at all. So instead I tried cloning the repository and installing it manually, but there is no requirements file. I tried figuring out the requirements one by one by failing to import snapista in the Python console, and the second requirement is snappy, which is super unexpected because I thought that snapista is a replacement for non-pythonic and performance degrading snappy. At this point I gave up, at least for now, to install it on this machine.
2. On my Ubuntu workstation, conda did find snapista, but even an environment that I just used for snappy was for some reason incompatible. I tried creating a separate environment for just snapista, and was very surprised to see that it was installing a brand new version of snap itself, which is weird, because I obviously already have a latest install and just need a Python wrapper for gpt. I installed it anyway, and it even kind of worked for a bit: I managed to play around with different operators. But the moment I tried to initialize a Graph object, an exception was raised that gpt could not be found. Which is also super weird, because I now have two gpts: one that I had before and one that I assume was installed with snapista into a conda environment. The original gpt is definitely on my PATH, so I am lost.

Can you please help me install snapista?

Hi,

the Terrain-Correction operator is not working properly. One thing is a non-existent element in the Graph leading to

Error: [NodeId: terrain-correction] Operator 'RangeDopplerGeocodingOp': Unknown element 'outputComplex'

However, when removing element outputComplex from the graph (I exported it into a file and run with gpt) the next error occurs which I did not further investigate:

java.lang.ArithmeticException: / by zero
        at javax.media.jai.PlanarImage.XToTileX(PlanarImage.java:832)
        at org.esa.snap.core.gpf.internal.OperatorImageTileStack.createLocks(OperatorImageTileStack.java:190)
        at org.esa.snap.core.gpf.internal.OperatorContext.setTargetImages(OperatorContext.java:750)
        at org.esa.snap.core.gpf.internal.OperatorContext.initializeOperator(OperatorContext.java:531)
        at org.esa.snap.core.gpf.internal.OperatorContext.getTargetProduct(OperatorContext.java:298)
        at org.esa.snap.core.gpf.Operator.getTargetProduct(Operator.java:385)
        at org.esa.snap.core.gpf.graph.NodeContext.initTargetProduct(NodeContext.java:77)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:199)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
        at org.esa.snap.core.gpf.graph.GraphContext.initNodeContext(GraphContext.java:182)
        at org.esa.snap.core.gpf.graph.GraphContext.initOutput(GraphContext.java:166)
        at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:85)
        at org.esa.snap.core.gpf.graph.GraphContext.<init>(GraphContext.java:58)
        at org.esa.snap.core.gpf.graph.GraphProcessor.executeGraph(GraphProcessor.java:118)
        at org.esa.snap.core.gpf.main.DefaultCommandLineContext.executeGraph(DefaultCommandLineContext.java:86)
        at org.esa.snap.core.gpf.main.CommandLineTool.executeGraph(CommandLineTool.java:547)
        at org.esa.snap.core.gpf.main.CommandLineTool.runGraph(CommandLineTool.java:391)
        at org.esa.snap.core.gpf.main.CommandLineTool.runGraphOrOperator(CommandLineTool.java:287)
        at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:188)
        at org.esa.snap.core.gpf.main.CommandLineTool.run(CommandLineTool.java:121)
        at org.esa.snap.core.gpf.main.GPT.run(GPT.java:60)
        at org.esa.snap.core.gpf.main.GPT.main(GPT.java:37)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.esa.snap.runtime.Launcher.lambda$run$0(Launcher.java:55)
        at org.esa.snap.runtime.Engine.runClientCode(Engine.java:189)
        at org.esa.snap.runtime.Launcher.run(Launcher.java:51)
        at org.esa.snap.runtime.Launcher.main(Launcher.java:31)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.exe4j.runtime.LauncherEngine.launch(LauncherEngine.java:84)
        at com.install4j.runtime.launcher.UnixLauncher.start(UnixLauncher.java:66)
        at install4j.org.esa.snap.runtime.Launcher_gpt.main(Unknown Source)

When I re-create the workflow with SNAP GUI it runs fine. So, I guess there is a version conflict when building the graph as elements of the operator seem to be outdated? I'm using snapista version 0.2.0 and snap version 8.0.0 with Python 3.7 installed with conda from terradue repo.

The links to installation and "getting started" in the https://snap-contrib.github.io/snapista/ page appear to point to the wrong place. The links in the TOC are correct.

Good afternoon,

I have an error when I run the graphs by means of the g.run() function.

Running Graph 1
Processing the graph
Executing processing graph
INFO: org.esa.snap.core.gpf.operators.tooladapter.ToolAdapterIO: Initializing external tool adapters
Unable to find calibration element in original product metadata
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: GDAL 2.4.0 found on system. JNI driver will be used.
done.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 2.4.0 set to be used by SNAP.
INFO: org.esa.snap.core.util.EngineVersionCheckActivator: Please check regularly for new updates for the best SNAP experience.
INFO: org.esa.s2tbx.dataio.gdal.GDALVersion: Installed GDAL 2.4.0 set to be used by SNAP.
INFO: org.hsqldb.persist.Logger: dataFileCache open start

Error: [NodeId: Calibration] Unable to find calibration element in original product metadata

If I save the graph in xml format and I run it in SNAP the process works perfectly.

On the other hand this is how the graph looks like (attached):
Change the extension to .xml

20170114.txt

Thanks in advance.

Do I miss something?

my image: Do I have to use the manifest.safe?
img_ = '/codede/Sentinel-2/MSI/L2A/2018/08/10/S2A_MSIL2A_20180810T102021_N0208_R065_T32UND_20180810T152927.SAFE/manifest.safe'

g = Graph()
g.add_node(operator=Operator('Read',
                            formatName='SENTINEL-2-MSI-MultiRes-UTM32N',
                            file=img_,
                            ),
          node_id='read',
          )
g.add_node(operator=Operator('Resample', referenceBandName='B2'),
          node_id='resample',
          source='read',
          )
g.add_node(operator=Operator('Write', 
                             file='/media/data_storage/MaCro/raster/snapista_test.tif',
                            formatName='GeoTIFF-BigTIFF'),
          node_id='write',
          source='resample',
          )
g.run()

The error message isn't very informative:

Processing the graph
Executing processing graph
done.
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
<timed eval> in <module>

~/anaconda3/envs/snapista/lib/python3.8/site-packages/snapista-0.1.2-py3.8.egg/snapista/graph.py in run(self, gpt_options)
    336         if rc != 0:
    337 
--> 338             raise Exception("Graph execution failed (exit code {})".format(rc))
    339 
    340         return rc

Exception: Graph execution failed (exit code 1)

g.view() gives me:

<graph>
  <version>1.0</version>
  <node id="read">
    <operator>Read</operator>
    <sources/>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <bandNames/>
      <copyMetadata>true</copyMetadata>
      <file>/codede/Sentinel-2/MSI/L2A/2018/08/10/S2A_MSIL2A_20180810T102021_N0208_R065_T32UND_20180810T152927.SAFE/manifest.safe</file>
      <formatName>SENTINEL-2-MSI-MultiRes-UTM32N</formatName>
      <geometryRegion/>
      <maskNames/>
      <pixelRegion/>
    </parameters>
  </node>
  <node id="resample">
    <operator>Resample</operator>
    <sources>
      <sourceProduct refid="read"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <bandResamplings/>
      <downsamplingMethod>First</downsamplingMethod>
      <flagDownsamplingMethod>First</flagDownsamplingMethod>
      <referenceBandName>B2</referenceBandName>
      <resampleOnPyramidLevels>true</resampleOnPyramidLevels>
      <resamplingPreset/>
      <targetHeight/>
      <targetResolution/>
      <targetWidth/>
      <upsamplingMethod>Nearest</upsamplingMethod>
    </parameters>
  </node>
  <node id="write">
    <operator>Write</operator>
    <sources>
      <sourceProduct refid="resample"/>
    </sources>
    <parameters class="com.bc.ceres.binding.dom.XppDomElement">
      <clearCacheAfterRowWrite>false</clearCacheAfterRowWrite>
      <deleteOutputOnFailure>true</deleteOutputOnFailure>
      <file>/media/data_storage/MaCro/raster/snapista_test.tif</file>
      <formatName>GeoTIFF-BigTIFF</formatName>
      <writeEntireTileRows>false</writeEntireTileRows>
    </parameters>
  </node>
</graph>

The current conda package depends on python<3.10, so it can't be installed in up-to-date conda environments. Note that Python 3.9 has been out of full support since 2022.

Is this project still maintained? The latest commit was in 2021.

Hi,
I've tried to build snapista forDebian but I was running into a build time test issue which boils down to:

$ python3
Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from snappy import GPF
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'GPF' from 'snappy' (/usr/lib/python3/dist-packages/snappy/__init__.py)
>>> from snappy import jpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'jpy' from 'snappy' (/usr/lib/python3/dist-packages/snappy/__init__.py)

I've reported this on snappy but they said: GPF and jpy don't sound like they have anything to do with this package, I don't think there have ever been things named like that.

Do you have any hint what to do to get GPF and jpy?

Kind regards, Andreas.

I have experienced an issue when running a graph with BandMath where the expression contains '<' and/or '>' symbols.

Indeed in the url: https://snap-contrib.github.io/snapista/examples/bandmaths/
it also shows same issue:

when you see the expression using print it shows it correctly
'S9_BT_in < 265 ? 0 : F1_BT_in > 315 and (F1_BT_in - F2_BT_in) > 15 ? 1 : 0'

but when creating the graph or viewing the graph it adds 'amp' after &
'S9_BT_in & amp;lt; 265 ? 0 : F1_BT_in & amp;gt; 315 and (F1_BT_in - F2_BT_in) & amp;gt; 15 ? 1 : 0' (I had to add and whitespace character otherwise it was not visible in here....)

@fabricebrito, could you please fix it? Otherwise, GPT complains and cannot use these types of operations.
Probably it is the way you escape these characters which seems differently where you do a graph or simply show the operator.
Thanks in advance