Comments (9)
- The unit of the radius should always be same with the input data. GeoSpark always assume objects are on a planar space. Thus, you don't need to specify unit for the radius.
In this regard, if you want to have accurate results, you'd better use GeoSpark CRS transformation function first to transform your input data to meter-based CRS such as epsg:3857.
- It is not a bad idea to sort the joined points. However, this will lead to unnecessary time overhead for users who are not interested in sorted results. Therefore, I leave this option to users. If you want to sort, you can simply write an additional MAP to sort the data.
from sedona.
- So, I'm trying to do like this
new PointRDD(ss.sparkContext, inputLocationCSV, pointRddOffset, pointRDDSplitter, true, StorageLevel.MEMORY_ONLY, "epsg:4326", "epsg:3857")
Got this error
Exception in thread "main" java.lang.Exception: [JoinQuery][DistanceJoinQuery]one input RDD doesn't perform necessary CRS transformation. Please check your RDD constructors. at org.datasyslab.geospark.spatialOperator.JoinQuery.executeDistanceJoinUsingIndex(JoinQuery.java:150) at org.datasyslab.geospark.spatialOperator.JoinQuery.DistanceJoinQuery(JoinQuery.java:647) at com.geo.SpatialKNNMaterialInterpolator$$anonfun$testDistanceJoinQueryUsingIndex$1$1.apply$mcVI$sp(SpatialKNNMaterialInterpolator.scala:81) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:166) at com.geo.SpatialKNNMaterialInterpolator$.testDistanceJoinQueryUsingIndex$1(SpatialKNNMaterialInterpolator.scala:79) at com.geo.SpatialKNNMaterialInterpolator$.main(SpatialKNNMaterialInterpolator.scala:88) at com.geo.SpatialKNNMaterialInterpolator.main(SpatialKNNMaterialInterpolator.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
(My input data are lat/lon and I want to use distance for Circle in metres)
What is the right sourse epsg for lat/lon ( lets say from Google Map's Mercator)?
- Sorry for that question)
from sedona.
@deil87
Since you are doing a DistanceJoin which involves two Spatial RDDs.
You need to make sure you did CRS transformation on both Spatial RDDs. It seems that you just did it on one Spatial RDD.
from sedona.
@jiayuasu
As I understand what I'm doing.... I only use one input csv and it is used as source and target RDD for distanceJoin.
val objectRDD = new PointRDD(ss.sparkContext, inputLocationCSV, pointRddOffset, pointRDDSplitter, true, StorageLevel.MEMORY_ONLY, "epsg:4326", "epsg:3857")
val queryWindowRDD = new CircleRDD(objectRDD,0.01)
objectRDD.spatialPartitioning(GridType.RTREE)
queryWindowRDD.spatialPartitioning(objectRDD.grids)
objectRDD.buildIndex(IndexType.RTREE,true)
objectRDD.indexedRDD.persist(StorageLevel.MEMORY_ONLY)
queryWindowRDD.spatialPartitionedRDD.persist(StorageLevel.MEMORY_ONLY)
val result = JoinQuery.DistanceJoinQuery(objectRDD, queryWindowRDD ,true,true)
from sedona.
There is a bug in GeoSpark due to the newly added CRS transformation function: CircleRDD forgets to copy CRS information when the user constructs a new CircleRDD from another SpatialRDD.
This bug has been fixed just now and pushed into 0.7.1-snapshot. You need to refresh your GeoSpark dependency coordinate and make sure the IDE downloads the latest version.
from sedona.
Good new! Thanks!
How soon latest version will appear in maven central repo? Seems like old build is still there.
(geospark-0.7.1-snapshot.jar 2017-06-02 00:59 13031304)
from sedona.
I think it has been synced to Maven Central several hours ago. Please try it out.
from sedona.
@jiayuasu
For some reason I can't see fixed build.
http://central.maven.org/maven2/org/datasyslab/geospark/0.7.1-snapshot/
Could you please check one more time?
from sedona.
@deil87 It seems that the auto-sync between sonatype and maven central got some problems. Please try 0.7.1-snapshot2
from sedona.
Related Issues (20)
- Preview and Generate Open Graph Meta Tags - Missing Image HOT 1
- Retrieve WKT CRS ID using ShapefileReader HOT 2
- Package Conflict when I use ST_AsGeoJSON HOT 2
- Unable to use sedona.global.charset in ShapefileReader HOT 4
- Add `markdown-link-check` with `pre-commit` HOT 2
- Add documentation on `pre-commit` usage
- We should add `ruff-pre-commit` a `pre-commit` hook for `Ruff` for Python linting and formatting HOT 1
- Add some `pre-commit` hooks that target the `R` language
- Apache Sedona on Fabric, does not read parquet from Lakehouse HOT 7
- Images should have alternate text (alt text)
- Specify custom transformation parameters/wkt string from CoordinateSystem A til CoordinateSystem B. HOT 3
- AWS Glue official tutorial HOT 5
- Hidden requirement for geopandas in apache-sedona[spark] 1.5.2 HOT 4
- after latest update of sedona snow on snowflake side all functions are gone HOT 2
- datatype geometry is not supported when 'create table xxx (geom geometry)' HOT 3
- AttributeError: 'sedona' has no attribute 'read' HOT 2
- St_isempty(geometry) finds non null geometries but does not find null geometries. HOT 2
- ST_Snap example code does not work HOT 2
- Flink Sedona,geomTbl.execute().print() happen error: HOT 2
- Sedona fails to write Delta Lake on Databricks 15.3 Beta: ClassCastException HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sedona.