Comments (6)
https://github.com/geoHeil/GeoSpark/tree/shadingJTS here is a prototype built with the instructions of http://stackoverflow.com/questions/21021485/how-to-configure-maven-shade-plugin-in-a-multi-module-project
Why did you not use the parent pom
in the modules?
from sedona.
When not minimizing the resulting jar, the contents are fine. Whilst at it I would suggest to shade guava as well in order to not clash with i.e. geotools but mostly geospark
from sedona.
Hi @geoHeil , thanks for your comments! You gave me lots of great suggestion.
I have several questions about this issue: 1. In the new PR, it seems you create another module to make the shading work. This module packaged all GeoSpark and Babylon together. This is not expected. I will update JTSplus repository to avoid this conflict. 2. I am not sure why Guava is packaged into GeoSpark. I think I should totally exclude it. Will this solve the Guava conflict?
from sedona.
@jiayuasu , to my knowledge there are 2-3 Problems
- JTS geospark is using JTSPlus which is a mashup of JTS and additional functionality
- guava, spark is using guava as well, this simply should be shaded
- Geotools C dependencies (JAI) are not included / not included in the right version. (very tricky) http://stackoverflow.com/questions/43910006/geotools-jai-fatjar-causing-problems-in-native-dependencies
What I want to achieve is: Read ESRi ASCII Grid files into geoSpark. Currently I use geotools to read the raster data and create polygons from it.
When having both geospark and geotools in the project, the wrong dependencies are selected. However when shading is applied as outlined above it works.
But it only works if the right assembly strategy (fat-jar) is applied when creating the spark job: http://stackoverflow.com/questions/43910006/geotools-jai-fatjar-causing-problems-in-native-dependencies do you know why the MergeStrategy.first
is working (but has the downside that you do not know what first will be. The more elaborate strategy does seem to fail.
from sedona.
@geoHeil , I will do two things to help you out. 1. I will rename all JTSplus root package name. 2. Remove Guava dependency.
Guava dependency is introduced because GeoSpark 0.6.2 depends SerNetCDF which package lots of unrelated packages.
After this, I will push a snapshot version for you to test.
from sedona.
@geoHeil , in GeoSpark 0.7.0,
-
I have integrated the latest GeoTools (17.0) into GeoSpark. I also removed all duplicated classes under JTS root folder. Only JTSplus version JTS exists in GeoSpark fat jar.
-
I changed SerNetCDF to an optional dependency. Therefore, you won't see any unrelated libraries in the GeoSpark fat jar such as Guava.
Thanks,
Jia
from sedona.
Related Issues (20)
- Preview and Generate Open Graph Meta Tags - Missing Image HOT 1
- Retrieve WKT CRS ID using ShapefileReader HOT 2
- Package Conflict when I use ST_AsGeoJSON HOT 2
- Unable to use sedona.global.charset in ShapefileReader HOT 4
- Add `markdown-link-check` with `pre-commit` HOT 2
- Add documentation on `pre-commit` usage
- We should add `ruff-pre-commit` a `pre-commit` hook for `Ruff` for Python linting and formatting HOT 1
- Add some `pre-commit` hooks that target the `R` language
- Apache Sedona on Fabric, does not read parquet from Lakehouse HOT 7
- Images should have alternate text (alt text)
- Specify custom transformation parameters/wkt string from CoordinateSystem A til CoordinateSystem B. HOT 3
- AWS Glue official tutorial HOT 5
- Hidden requirement for geopandas in apache-sedona[spark] 1.5.2 HOT 4
- after latest update of sedona snow on snowflake side all functions are gone HOT 2
- datatype geometry is not supported when 'create table xxx (geom geometry)' HOT 3
- AttributeError: 'sedona' has no attribute 'read' HOT 2
- St_isempty(geometry) finds non null geometries but does not find null geometries. HOT 2
- ST_Snap example code does not work HOT 2
- Flink Sedona,geomTbl.execute().print() happen error: HOT 2
- Sedona fails to write Delta Lake on Databricks 15.3 Beta: ClassCastException HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sedona.