GithubHelp home page GithubHelp logo

geomet / geomet Goto Github PK

View Code? Open in Web Editor NEW
167.0 7.0 32.0 371 KB

GeoMet - Pure Python conversion library for common geospatial data formats

License: Apache License 2.0

Python 99.10% Shell 0.90%
geojson wkt wkb ewkt ewkb gis esri geopackage geospatial spatial

geomet's Introduction

GeoMet geomet

Pure-Python conversion library for common geospatial data formats. Supported formats include:

Install

Install the latest version from PyPI:

$ pip install geomet

Functionality

Converion functions are exposed through idiomatic load/loads/dump/dumps interfaces.

GeoMet is intended to cover all common use cases for dealing with 2D, 3D, and 4D geometries (including 'Z', 'M', and 'ZM').

Geometry WKT/EWKT WKB/EWKB GeoPackage Binary EsriJSON
Point
LineString
Polygon
MultiPoint
MultiLineString
MultiPolygon
GeometryCollection

Example usage

Coverting a 'Point' GeoJSON object to WKT:

>>> from geomet import wkt
>>> point = {'type': 'Point', 'coordinates': [116.4, 45.2, 11.1]}
>>> wkt.dumps(point, decimals=4)
'POINT (116.4000 45.2000 11.1000)'

Converting a 'Point' GeoJSON object to WKB:

>>> from geomet import wkb
>>> wkb.dumps(point)
b'\x00\x00\x00\x10\x01@]\x19\x99\x99\x99\x99\x9a@F\x99\x99\x99\x99\x99\x9a@&333333'
>>> wkb.dumps(point, big_endian=False)
b'\x01\x01\x10\x00\x00\x9a\x99\x99\x99\x99\x19]@\x9a\x99\x99\x99\x99\x99F@333333&@'

Converting a 'Point' GeoJSON object to GeoPackage Binary:

>>> from geomet import geopackage
>>> geopackage.dumps(point)
b'GP\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\xe9@]\x19\x99\x99\x99\x99\x9a@F\x99\x99\x99\x99\x99\x9a@&333333'
>>> geopackage.dumps(point, big_endian=False)
b'GP\x00\x01\x00\x00\x00\x00\x01\xe9\x03\x00\x00\x9a\x99\x99\x99\x99\x19]@\x9a\x99\x99\x99\x99\x99F@333333&@'

Converting a 'LineString' GeoJSON object to WKT:

>>> linestring = {'type':'LineString',
...               'coordinates': [[0.0, 0.0, 10.0], [2.0, 1.0, 20.0],
...                               [4.0, 2.0, 30.0], [5.0, 4.0, 40.0]]}
>>> wkt.dumps(linestring, decimals=0)
'LINESTRING (0 0 10, 2 1 20, 4 2 30, 5 4 40)'

Converting a 'LineString' GeoJSON object to WKB:

>>> wkb.dumps(linestring)
b'\x00\x00\x00\x10\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00@$\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x00?\xf0\x00\x00\x00\x00\x00\x00@4\x00\x00\x00\x00\x00\x00@\x10\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x00@>\x00\x00\x00\x00\x00\x00@\x14\x00\x00\x00\x00\x00\x00@\x10\x00\x00\x00\x00\x00\x00@D\x00\x00\x00\x00\x00\x00'
>>> wkb.dumps(linestring, big_endian=False)
b'\x01\x02\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00$@\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x004@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00>@\x00\x00\x00\x00\x00\x00\x14@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00D@'

Converting a 'LineString' GeoJSON object to GeoPackage Binary:

>>> geopackage.dumps(linestring)
b'GP\x00\x00\x00\x00\x00\x00\x00\x00\x00\x10\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00@$\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x00?\xf0\x00\x00\x00\x00\x00\x00@4\x00\x00\x00\x00\x00\x00@\x10\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x00@>\x00\x00\x00\x00\x00\x00@\x14\x00\x00\x00\x00\x00\x00@\x10\x00\x00\x00\x00\x00\x00@D\x00\x00\x00\x00\x00\x00'
>>> geopackage.dumps(linestring, big_endian=False)
b'GP\x00\x01\x00\x00\x00\x00\x01\x02\x10\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00$@\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\xf0?\x00\x00\x00\x00\x00\x004@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00>@\x00\x00\x00\x00\x00\x00\x14@\x00\x00\x00\x00\x00\x00\x10@\x00\x00\x00\x00\x00\x00D@'

Converting 'Point' WKT to GeoJSON:

>>> wkt.loads('POINT(10 20)')
{'type': 'Point', 'coordinates': [10.0, 20.0]}

Coverting 'GeometryCollection' WKT to GeoJSON:

>>> wkt.loads('GEOMETRYCOLLECTION(POINT(10 20),POLYGON(((0 0), (10 30), (30 10), (0 0)))')
{'type': 'GeometryCollection', 'geometries': [{'type': 'Point', 'coordinates': [10.0, 20.0]}, {'type': 'Polygon', 'coordinates': [[[0.0, 0.0]], [[10.0, 30.0]], [[30.0, 10.0]], [[0.0, 0.0]]]}]}

EWKT/EWKB are also supported for all geometry types. This uses a custom extension to the GeoJSON standard in order to preserve SRID information through conversions. For example:

>>> wkt.loads('SRID=4326;POINT(10 20)')
{'type': 'Point', 'coordinates': [10.0, 20.0], 'meta': {'srid': '4326'}}
>>> wkt.dumps({'type': 'Point', 'coordinates': [10.0, 20.0], 'meta': {'srid': '4326'}, 'crs': {'properties': {'name': 'EPSG4326'}, 'type': 'name'}})
'SRID=4326;POINT (10.0000000000000000 20.0000000000000000)'
>>> wkb.loads('\x00 \x00\x00\x01\x00\x00\x10\xe6@$\x00\x00\x00\x00\x00\x00@4\x00\x00\x00\x00\x00\x00')
{'meta': {'srid': '4326'}, 'type': 'Point', 'coordinates': [10.0, 20.0]}
>>> wkb.dumps({'type': 'Point', 'coordinates': [10.0, 20.0], 'meta': {'srid': '4326'}, 'crs': {'properties': {'name': 'EPSG4326'}, 'type': 'name'}})
'\x00 \x00\x00\x01\x00\x00\x10\xe6@$\x00\x00\x00\x00\x00\x00@4\x00\x00\x00\x00\x00\x00'

GeoPackage binary supports encoding of SRID and envelope information. If your geopackage has an envelope specified, then it will be added into the resulting GeoJSON in a key called 'bbox':

>>> gpkg = b'GP\x00\x03\x00\x00\x00\x00\xf0\x9e\xa0\xa7\x05;#@hZ\xbd\x93\x83GC@\xf0\x9e\xa0\xa7\x05;#@hZ\xbd\x93\x83GC@\x01\x01\x00\x00\x00\xf0\x9e\xa0\xa7\x05;#@hZ\xbd\x93\x83GC@'
>>> geopackage.loads(gpkg)
>>> {'type': 'Point', 'coordinates': [9.615277517659223, 38.55870291467437], 'bbox': (9.615277517659223, 38.55870291467437, 9.615277517659223, 38.55870291467437)}

In the same way, if a 'bbox' key is present on a dumps-ed geometry, it will be added to the header of the GeoPackage geometry:

>>> polygon = {'type': 'Polygon', 'coordinates': [[[20.0, 20.0], [34.0, 124.0], [70.0, 140.0], [130.0, 130.0], [70.0, 100.0], [110.0, 70.0], [170.0, 20.0], [90.0, 10.0], [20.0, 20.0]]], 'bbox': (20.0, 170.0, 10.0, 140.0)}
>>> geopackage.dumps(polygon)
b'GP\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x004@\x00\x00\x00\x00\x00@e@\x00\x00\x00\x00\x00\x00$@\x00\x00\x00\x00\x00\x80a@\x00\x00\x00\x00\x03\x00\x00\x00\x01\x00\x00\x00\t@4\x00\x00\x00\x00\x00\x00@4\x00\x00\x00\x00\x00\x00@A\x00\x00\x00\x00\x00\x00@_\x00\x00\x00\x00\x00\x00@Q\x80\x00\x00\x00\x00\x00@a\x80\x00\x00\x00\x00\x00@`@\x00\x00\x00\x00\x00@`@\x00\x00\x00\x00\x00@Q\x80\x00\x00\x00\x00\x00@Y\x00\x00\x00\x00\x00\x00@[\x80\x00\x00\x00\x00\x00@Q\x80\x00\x00\x00\x00\x00@e@\x00\x00\x00\x00\x00@4\x00\x00\x00\x00\x00\x00@V\x80\x00\x00\x00\x00\x00@$\x00\x00\x00\x00\x00\x00@4\x00\x00\x00\x00\x00\x00@4\x00\x00\x00\x00\x00\x00'

If an integer SRID identifier is present in a 'meta' key (like 'meta': {'srid': 4326}), then the SRID will be included in the GeoPackage header.

History

This library was originally created as the result of a bug report related to another project: https://bugs.launchpad.net/openquake-old/+bug/1073909. The source of this issue was largely due to a dependency on GEOS, which is written in C/C++. Depending on GEOS requires any data conversion bug fixes to happen upstream, which takes time and effort. Ultimately, this was the inspiration to create a more lightweight, pure-Python conversion library as an alterntive tool for reliably converting data between various geospatial formats.

The name "GeoMet" was inspired by "met", the German word for mead. It is also a shortened version of the word "geometry".

Limitations

Outputing "empty" geometries to binary formats is not supported

Attempting to output an empty geometry to a binary format will result in an exception: ValueError: Empty geometries cannot be represented in WKB. Reason: The dimensionality of the WKB would be ambiguous. There are a few reasons for this this limitation:

  • Any EMTPY geometry (e.g., POINT EMPTY, MULTIPOLYGON EMPTY, etc.) cannot be converted into binary format because binary formats such as WKB require an explicit dimension type (2d, Z, M, or ZM). This means that some objects cannot be reliably converted to and from different formats in a bijective manner.
  • The GeoJSON standard does have a way of representing empty geometries; however, details are minimal and the dimensionality of such an object remains ambiguous.
  • Representing some geometry types (such as points and lines) as "empty" is deeply flawed to begin with. For example, a point can represent any location in 2d, 3d, or 4d space. However, a point is infinitesimally small (it has no size) and it can't contain anything (it can't be "full"), therefore, it doesn't make sense for a point to be "empty".

As a result, GeoMet has chosen to not attempt to address these problems, and simply raise an exception instead.

Example:

>>> import geomet
>>> import geomet.wkt as wkt
>>> import geomet.wkb as wkb
>>> pt = wkt.loads('POINT EMPTY')
>>> pt
{'type': 'Point', 'coordinates': []}
>>> wkb.dumps(pt)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/jdoe/geomet/geomet/wkb.py", line 216, in dumps
    return _dumps(obj, big_endian)
File "/home/jdoe/geomet/geomet/wkb.py", line 238, in _dumps
    raise ValueError(
ValueError: Empty geometries cannot be represented in WKB. Reason: The dimensionality of the WKB would be ambiguous.

See also

  • wellknown: A similar package for Node.js.
  • geo: A nearly-identical package for Elixir.

geomet's People

Contributors

ac avatar achapkowski avatar cool-rr avatar larsbutler avatar maralla avatar mwtoews avatar pbryan avatar sgillies avatar stianjensen avatar tirkarthi avatar tomplex avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

geomet's Issues

geomet.wkt.dumps fails with specific coordinates

I found a bug (using Hypothesis) with geomet.wkt.dumps when I try to round small values of coordinates with even smaller decimals. This is how you can reproduce this case:

import geomet

polygon = {'type': 'Polygon', 'coordinates': [[[0., 1.0000483598560095e-05]]]}
geomet.wkt.dumps(geometry, decimals=9)

which gives

Traceback (most recent call last):
File "", line 1, in
File "/home/victor/.cache/pypoetry/virtualenvs/watermill-K62Kegsu-py3.9/lib/python3.9/site-packages/geomet/wkt.py", line 79, in dumps
result = exporter(obj, decimals)
File "/home/victor/.cache/pypoetry/virtualenvs/watermill-K62Kegsu-py3.9/lib/python3.9/site-packages/geomet/wkt.py", line 266, in _dump_polygon
fmt = '(%s)' % ', '.join('(%s)' % r for r in rings)
File "/home/victor/.cache/pypoetry/virtualenvs/watermill-K62Kegsu-py3.9/lib/python3.9/site-packages/geomet/wkt.py", line 266, in
fmt = '(%s)' % ', '.join('(%s)' % r for r in rings)
File "/home/victor/.cache/pypoetry/virtualenvs/watermill-K62Kegsu-py3.9/lib/python3.9/site-packages/geomet/wkt.py", line 262, in
rings = (', '.join(' '.join(_round_and_pad(c, decimals)
File "/home/victor/.cache/pypoetry/virtualenvs/watermill-K62Kegsu-py3.9/lib/python3.9/site-packages/geomet/wkt.py", line 262, in
rings = (', '.join(' '.join(_round_and_pad(c, decimals)
File "/home/victor/.cache/pypoetry/virtualenvs/watermill-K62Kegsu-py3.9/lib/python3.9/site-packages/geomet/wkt.py", line 262, in
rings = (', '.join(' '.join(_round_and_pad(c, decimals)
File "/home/victor/.cache/pypoetry/virtualenvs/watermill-K62Kegsu-py3.9/lib/python3.9/site-packages/geomet/wkt.py", line 199, in _round_and_pad
rounded += '0' * (decimals - len(rounded.split('.')[1]))
IndexError: list index out of range

A curious thing I found is that greater values for decimals stops the error

Rework CI and release processes

  • Migrate to CircleCI (since it's more flexible than Travis-CI) --> Solved by #62.
  • Implement test automation for all supported Python versions --> Solved by #62.
  • Automate PyPI release process:
    • Define the exact workflow. For example, does the release happen on a release-* tag/branch, or does it happen once that branch is merged to master?
      • It should happen when a release-* branch is created
      • We could use some detail on what exactly needs to be included in a commit which triggers a new release. Perhaps we need to add a changelog and make that the release commit?
    • Add PyPI API credentials to CircleCI environment
    • Add some branch/tag protection (in GitHub, if possible; this needs to be investigated) to restrict who is allowed to release; this should be restricted to Owners only (@larsbutler).
    • Look at CircleCI "contexts" to define who is allowed to manually approve a PyPI release

Direct WKB <-> WKT conversion functions?

What do you think about adding a package with some small wrapper functions which convert directly from WKB <-> WKT (and perhaps GPKG)? They're easy enough for any user to implement, but it might be nice to include them as shortcuts in the package.

Add support for SRID encoding in WKB/EWKB

If the high byte of a geometry field in a WKB string is 0x02, this indicates that extra bytes follow with contain SRID--before the coordinate fields. See https://github.com/postgis/postgis/blob/d7b32e5df14ad7af677ee2def6084709e78ab2b7/liblwgeom/liblwgeom.h.in#L108.

This should be fairly straightforward to support.

Per #30, here are some examples:

  • 01010000003cdba337dcc351c06d37c1374d374840 is POINT(-71.060316 48.432044)
  • 0101000020e61000003cdba337dcc351c06d37c1374d374840 is the same Point with SRID=4326: SRID=4326;POINT(-71.060316 48.432044)

Use modern format strings

In any many places, we're still using % for string formatting. This still works, but it would be nice to clean it up and using the modern patterns, based on the format() function or string interpolation.

  • '{}'.format(foo)
  • f'{foo}'

conda-forge submission

Hi maintainers
I'm putting a library up on Conda-Forge, and it contains this library as a dependency. Are you alright with my using conda skeleton to submit this repository to conda-forge? I'm happy to list myself as the maintainer, and it includes all references back to this repository.
The library in question:
https://github.com/clcr/pyeo
The conda forge pull request:
conda-forge/staged-recipes#8972

geopackage WKB

In Geopackages there are extra header, is there any built in support for geopackage WKB?

WKT cannot load geocollections if one of the shapes is "EMPTY"

What is expected:

These shapes load up great:

geomet.wkt.loads("GEOMETRYCOLLECTION EMPTY")
geomet.wkt.loads("POINT EMPTY")

What happens:

It's when one of the shapes inside the geocollection contain an empty shape, I get a ValueError:

geomet.wkt.loads("GEOMETRYCOLLECTION (POLYGON((27 25,102 36,102 46,92 61,13 41,16 30,27 25)), LINESTRING EMPTY )")

Should it be able to create a json like so?

{'type': 'GeometryCollection', 'geometries': [
    {'type': 'Polygon', 'coordinates': [[[27.0, 25.0], [102.0, 36.0], [102.0, 46.0], [92.0, 61.0], [13.0, 41.0], [16.0, 30.0], [27.0, 25.0]]]},
    {'type': 'LineString', 'coordinates': []}
]}

Maybe of note:

When I try to load the above geocollection w/ the empty shape inside it to shapely, then dump it, I get this:

GEOMETRYCOLLECTION Z (POLYGON((27 25,102 36,102 46,92 61,13 41,16 30,27 25)), LINESTRING EMPTY )

with the "geometrycollection z". I'm new to wkt's so I'm not sure if the Z's expected as well, just thought I'd mention it.
Thanks!

Being able to programmatically insert POLYHEDRALSURFACE (Z) or TIN (Z) object from Python to PostGIS

Hello geomet's Team,

I was suggested by @tomplex to open a feature request here (but it seems the discussion tab is not yet available, therefore I'm creating an issue, even so it's not perfectly appropriated for discussing that).

Being able to programmatically insert POLYHEDRALSURFACE (Z) or TIN (Z) object from Python to PostGIS is not straightforward as explained for example on Shapely's discussions or even here at GDAL when I tried to start from a GeoJSON object, and for which issue I understand the closing, because they purely rely on the official GeoJSON definition.
But they actually do offer an integration of both these objects in OGR from WKT strings as very well explained here: https://gdal.org/development/rfc/rfc64_triangle_polyhedralsurface_tin.html and which you can also see in the mentioned issue, when I successfully build such object using geom = ogr.CreateGeometryFromWkt('POLYHEDRALSURFACE Z (((...).
But then I'm stuck for the writing to PostGIS.

Therefore I'm searching a tool to programmatically insert such "fancy" 3D data set (well, they are not so fancy because they do exist in PostGIS), a bit in the way Shapely does for simple points, lines or polygons (multi or not).

I'm naturally wondering if geomet could help in that direction, even so these geometric data types are not really well standardized yet, are they?

Well, such surfaces are referenced in those OGC documents:

So there hopefully seems to be a solid basis to start some work on.

Thanks a lot for taking the time to read!

Release version 1.0.0

Several significant changes have happened since the last 0.3.0 release. For instance:

A few things are needed before this release is done:

  • Update documentation on release procedures, and clearly define the process. With the addition of branch protection and a new automation workflow in CircleCI, there needs to be a clearly defined process to streamline future release.
  • Update repository, GitHub org, and README to reflect the addition of support for new geospatial formats (related: #59). For instance, instead of saying we just convert between GeoJSON/WKT/WKB, I think we should say something more general like "Conversion library for common geospatial data formats".
  • Update repository tags to include new supported formats (esri json, geopackage, etc.) to make the library easier to find.
  • Release 1.0.0 package to PyPI.
  • Create and push a 1.0.0 tag to GitHub.
  • Compile release notes and create a release for 1.0.0 on GitHub.

Add Python3.7 support

geomet already supports Python3.6 so it shouldn't be a stretch to support 3.7 as well. However, it would need to be explicitly added and tested.

Update documentation and project metadata to include support for GeoPackage

With the addition of #56, WKT, WKB, and GeoJSON aren't the only supported formats anymore.

Prerequisites:

  • Merge of #56
  • Resolution of #42
  • New minor version (0.3.0) release to PyPI

Once these prereqs are done, update:

  • The GitHub repo title
  • Update GitHub repo "topics" to include geopackage
  • Update any docs which are similar to "GeoMet - Convert GeoJSON to WKT/WKB, and vice versa" <-- include GeoPackage in the list of supported formats.

add support for FeatureCollection

here is a sample GEOJson i would like to convert:

{
 "type": "FeatureCollection", 
 "features": [
  {
   "geometry": {
    "type": "MultiPolygon", 
    "coordinates": [
     [
      [
       [
        2.8124, 
        51.18802228498421
       ], 
       [
        2.8124, 
        53.4376
       ], 
       [
        4.21885, 
        53.4376
       ], 
       [
        4.21885, 
        52.07174517270202
       ], 
       [
        3.964036, 
        51.967702
       ], 
       [
        4.0667323, 
        51.8438181
       ], 
       [
        3.6768609, 
        51.7058847
       ], 
       [
        3.8675851, 
        51.6732775
       ], 
       [
        4.21885, 
        51.43696797090517
       ], 
       [
        3.8389336, 
        51.605857
       ], 
       [
        3.4320307, 
        51.5284042
       ], 
       [
        3.8128001, 
        51.385549
       ], 
       [
        3.975224, 
        51.4623885
       ], 
       [
        4.2037, 
        51.3749
       ], 
       [
        4.2024, 
        51.3744
       ], 
       [
        4.194, 
        51.3765
       ], 
       [
        4.1871, 
        51.3757
       ], 
       [
        4.1771, 
        51.3676
       ], 
       [
        4.1523, 
        51.3687
       ], 
       [
        4.1401, 
        51.3612
       ], 
       [
        4.1241, 
        51.367
       ], 
       [
        4.1012394, 
        51.3566187
       ], 
       [
        3.9681846, 
        51.4080513
       ], 
       [
        3.8147588, 
        51.3299545
       ], 
       [
        3.5034764, 
        51.4078396
       ], 
       [
        3.186199, 
        51.3625791
       ], 
       [
        2.8124, 
        51.18802228498421
       ]
      ], 
      [
       [
        4.2025, 
        51.3746
       ], 
       [
        4.2027, 
        51.3746
       ], 
       [
        4.2031, 
        51.375
       ], 
       [
        4.2027, 
        51.375
       ], 
       [
        4.2025, 
        51.3746
       ]
      ], 
      [
       [
        3.6994, 
        51.3856
       ], 
       [
        3.7011, 
        51.3872
       ], 
       [
        3.6987, 
        51.3887
       ], 
       [
        3.7001, 
        51.3869
       ], 
       [
        3.6994, 
        51.3856
       ]
      ], 
      [
       [
        3.7278, 
        51.6257
       ], 
       [
        3.7277, 
        51.6267
       ], 
       [
        3.7263, 
        51.6259
       ], 
       [
        3.7273, 
        51.6254
       ], 
       [
        3.7278, 
        51.6257
       ]
      ], 
      [
       [
        4.009, 
        51.9171
       ], 
       [
        4.0097, 
        51.9171
       ], 
       [
        4.0118, 
        51.9178
       ], 
       [
        4.0087, 
        51.9177
       ], 
       [
        4.009, 
        51.9171
       ]
      ], 
      [
       [
        3.7174514, 
        51.6331783
       ], 
       [
        3.7152999, 
        51.6337002
       ], 
       [
        3.7185912, 
        51.632274
       ], 
       [
        3.7207608, 
        51.6342184
       ], 
       [
        3.7174514, 
        51.6331783
       ]
      ], 
      [
       [
        3.7231784, 
        51.6558376
       ], 
       [
        3.7165351, 
        51.6550415
       ], 
       [
        3.71524, 
        51.6535432
       ], 
       [
        3.7264159, 
        51.6502884
       ], 
       [
        3.7231784, 
        51.6558376
       ]
      ], 
      [
       [
        3.1051871, 
        51.311624
       ], 
       [
        3.1057755, 
        51.3117278
       ], 
       [
        3.107086, 
        51.3122626
       ], 
       [
        3.1063676, 
        51.3122107
       ], 
       [
        3.1051871, 
        51.311624
       ]
      ], 
      [
       [
        3.6854212, 
        51.6186038
       ], 
       [
        3.6977492, 
        51.6288383
       ], 
       [
        3.703791, 
        51.6218965
       ], 
       [
        3.7086049, 
        51.6311998
       ], 
       [
        3.7222999, 
        51.6249021
       ], 
       [
        3.7318951, 
        51.6328968
       ], 
       [
        3.7008244, 
        51.6436475
       ], 
       [
        3.6854212, 
        51.6186038
       ]
      ], 
      [
       [
        4.1870716, 
        51.3836068
       ], 
       [
        4.186773, 
        51.3833585
       ], 
       [
        4.1875717, 
        51.3834846
       ], 
       [
        4.1873637, 
        51.3836309
       ], 
       [
        4.1870716, 
        51.3836068
       ]
      ]
     ]
    ]
   }, 
   "type": "Feature", 
   "properties": {
    "FID": 19987.0
   }
  }
 ]
}

[Edited for formatting.]

srid option for geomet.esri.dumps does not do anything

The option for defining output srid for the esrijson geometry is actively ignored
I have installed version 1.0.0

import geomet.esri
from pprint import pprint

input_geojson =  {'coordinates': [494252.595744681, 7066045.31914894], 'type': 'Point'}

esri_json = geomet.esri.dumps(input_geojson, srid=32632)

pprint(esri_json)
# Expected output is 
# {'spatialReference': {'wkid': 32632},
# 'x': 494252.595744681,
# 'y': 7066045.31914894}
# Actual output is
# {'spatialReference': {'wkid': 4326},
# 'x': 494252.595744681,
# 'y': 7066045.31914894}

I did some investigating into the source code and found all the _dump_* functions in the esri.py file includes this line:

srid = _extract_geojson_srid(obj) or srid

And inside the _extract_geojson_srid function:

def _extract_geojson_srid(obj):
  ...
  return srid or 4326

Which means it will never return the srid value i explicitly define. Workaround is the add the crs field into the input_geojson

input_geojson =  {'coordinates': [494252.595744681, 7066045.31914894], 'type': 'Point', 'crs': {'type': 'name', 'properties': {'name': 'EPSG:32632'}}

The simple fix is to change all of the _extract_geojson_srid(obj) or srid to srid or _extract_geojson_srid(obj)

Error in wkt.dumps

There is bug in wkt._round_and_pad while rounding exponential representation of coordinates with values less than e-4. Float numbers with degree less, than 10^-4 have an exponential representation. Such representation does not includes dots. Therefore split('.') returns empty list.

I propose to fix follow strings:

  1. rounded = repr(round(value, decimals))
- rounded = repr(round(value, decimals))
+if repr(value).find('e-') == 1:
+    rounded = format(value, '.{}f'.format(decimals))
+else:
+    rounded = repr(round(value, decimals))
  1. Update test_cases in TestRoundAndPad.test by adding [(6e-6, 16), '0.0000060000000000'].

Floating point round errors for certain geometries when using string formatting

In Python 2 and 3, using string formatting to round / truncate precision does not properly preserve coordinates. Consider the following:

from shapely.geometry import shape
from geomet import wkt

geojson = {'type': 'Point', 'coordinates': [-83.2496395, 35.917330500000006]}

geom = shape(geojson)

print(wkt.dumps(geojson))
# POINT (-83.2496395000000007 35.9173305000000056)

Digging deeper, it appears this is an issue with Python's implementation of rounding when done through % or str.format:

fmt = '%%.%df' % 16
fmt % -83.2496395
# '-83.2496395000000007'
'{}'.format(-83.2496395)
# '-83.2496395'
'{:.16f}'.format(-83.2496395)
# '-83.2496395000000007'

However, when we use Python's round() function to achieve the desired precision, we see no mutation:

round(-83.2496395, 16)
# -83.2496395
'{}'.format(round(-83.2496395, 16))
# '-83.2496395'

I think that this instability is undesirable in a format conversion library - for some geometries, that small coordinate shift could be the different between validity and not.

It looks like a change could rather easily be made to the codebase to use round() instead of relying on string interpolation. Would you be receptive to a pull request to that effect?

Thanks for your time!

Add a 'strict/validate' mode to converters

In some situations, it would be inappropriate to allow mixed dimensions within, for example, a MULTI* geometry. For example, a MULTIPOINT containing a 2D point and a 3D point doesn't really make any sense (and probably also violates some spec, which I can't find at the moment).

So it might be useful to introduce checks to reject such inputs as invalid.

Accept hex as well as bytestrings in loads functions

It would be nice to be able to pass hex (like 0101000000F09EA0A7053B2340685ABD9383474340) into loads functions. We'd need to add a flag (like hex=True) to the wkb.loads function and then probably just use binascii.a2b_hex to convert the geometry from hex to bytestring, which could be passed along normally.

Round and format to specified decimal places

In the implementation so far, string formatting is used to control the number of digits after the decimal point. This is not a complete solution. Numbers should be rounded instead, using the built-in round function.

WKB does not support empty geometry

As far as I know, WKB does not support empty geometry, whereas WKT does. For example: POINT EMPTY. In a case like this, GeoMet should raise an exception or return a default value. I'm not sure yet which is more appropriate. Probably an exception is better.

I think GeometryCollections can be empty, if the WKB simply contains 0 geometries.

Test with multiple python versions

I know it works with 2.7. It probably works with python 2.4 and later, but I need to test.

I also want to know if it works with Python 3.x, and if it doesn't, make necessary modifications.

WKB.loads() fails

This one works: wkb.loads(bytes.fromhex('01010000003cdba337dcc351c06d37c1374d374840'))
But this one doesn't: wkb.loads(bytes.fromhex('0101000020e61000003cdba337dcc351c06d37c1374d374840'))

But should convert to: {'type': 'Point', 'coordinates': [-71.060316, 48.432044]}

WKT parsers don't handle negative numbers

Here's an example of what happens:

>>> from geomet import wkt
>>> wkt.loads('POINT (-10 10)')
Traceback (most recent call last):
  File "", line 1, in 
  File "geomet/wkt.py", line 64, in loads
    return importer(tokens, string)
  File "geomet/wkt.py", line 157, in __load_point
    coords.append(float(t))
ValueError: could not convert string to float: -

Add support for "EMPTY" WKT and WKB

WKB/WKT parsers/writers need to support empty geometries.

NOTE: WKB does not support POINT EMPTY. When converting an empty point to WKB, an exception should be raised.

Open issue: What's the best way to represent empty geometries in GeoJSON? There doesn't seem to be anything directly supported in the spec: http://geojson.org/geojson-spec.html

Error in wkt.dumps

Error in wkt.dumps while round coordinates with values less than e-4

I propose to fix follow strings:

  1. rounded = repr(round(value, decimals))
- rounded = repr(round(value, decimals))
+ format_string = '{:.%sf}' % decimals
+ rounded = format_string.format(round(value, decimals))
  1. Add test like something that
    def test_dumps_exponential_values(self):
        wkt.dumps(dict(type='Point', coordinates=[0.6, 6e-6]))

Support for EWKB?

First, thanks for this great library! I have been using Shapely to convert from/to WKB, but I can't use it for geometries with 4 dimensions. So, I was quite happy to find this project! 👍

But it seems that EWKB is not supported? Or am I missing an undocumented option?

Conversion from EWKB does not support 3d

Hello,

I noticed when adding a column of type 'POINTZ' with SRID in PostGIS, that geomet cannot convert the type to GeoJSON.

See for example:

SELECT ST_GeomFromEWKT('SRID=4326;POINT Z(5 50 15)');
                          st_geomfromewkt
--------------------------------------------------------------------
 01010000A0E6100000000000000000144000000000000049400000000000002E40
from geomet import wkb
g = wkb.loads(bytes.fromhex('01010000A0E6100000000000000000144000000000000049400000000000002E40'))
Traceback (most recent call last):
  File "/PyCharm.app/Contents/plugins/python/helpers/pydev/pydevconsole.py", line 364, in runcode
    coro = func()
  File "<input>", line 1, in <module>
  File "/.venv/lib/python3.9/site-packages/geomet/wkb.py", line 319, in loads
    _unsupported_geom_type(geom_type)
  File "/.venv/lib/python3.9/site-packages/geomet/wkb.py", line 335, in _unsupported_geom_type
    raise ValueError("Unsupported geometry type '%s'" % geom_type)
ValueError: Unsupported geometry type 'None'

010000A0 is not found as a type and therefore throws an exception.

This was address in #31, but implemented tests in #32 only checked for 2d conversion.

Add test cases for nested geometry collections

GeometryCollections can contain any type of geometry, including more GeometryCollections.

  • First, we need to make sure the parsers and serializers work with multiple nested cases.
  • Second, we need to verify that geometries of different dimensionality are not mixed (i.e., it is not allowed to put a 2d Point inside a 3d GeometryCollection, etc.)

Tests fail with Python 3.12

Attempting a build against Python 3.12 b3 fails some tests:

======================================================================
ERROR: test_malformed_wkt_no_ending_paren (geomet.tests.wkt_test.GeometryCollectionLoadsTestCase.test_malformed_wkt_no_ending_paren)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/geomet-1.0.0/geomet/tests/wkt_test.py", line 1219, in test_malformed_wkt_no_ending_paren
    wkt.loads(mp)
  File "/builddir/build/BUILD/geomet-1.0.0/geomet/wkt.py", line 147, in loads
    result = importer(tokens, string)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builddir/build/BUILD/geomet-1.0.0/geomet/wkt.py", line 638, in _load_geometrycollection
    geom = load_func(tokens, string)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable
======================================================================
FAIL: test_raises_unmatched_paren (geomet.tests.wkt_test.LineStringLoadsTestCase.test_raises_unmatched_paren)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/geomet-1.0.0/geomet/tests/wkt_test.py", line 324, in test_raises_unmatched_paren
    self.assertEqual('Invalid WKT: `LINESTRING (0.0 1.0`',
AssertionError: 'Invalid WKT: `LINESTRING (0.0 1.0`' != "could not convert string to float: ''"
- Invalid WKT: `LINESTRING (0.0 1.0`
+ could not convert string to float: ''
======================================================================
FAIL: test_malformed_wkt_misbalanced_parens (geomet.tests.wkt_test.MultiPointLoadsTestCase.test_malformed_wkt_misbalanced_parens)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/geomet-1.0.0/geomet/tests/wkt_test.py", line 521, in test_malformed_wkt_misbalanced_parens
    self.assertEqual(expected, str(ar.exception))
AssertionError: 'Invalid WKT: `MULTIPOINT ((0 0), (0 1)`' != "could not convert string to float: ''"
- Invalid WKT: `MULTIPOINT ((0 0), (0 1)`
+ could not convert string to float: ''
======================================================================
FAIL: test_raises_unmatched_paren (geomet.tests.wkt_test.PointLoadsTestCase.test_raises_unmatched_paren)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/geomet-1.0.0/geomet/tests/wkt_test.py", line 245, in test_raises_unmatched_paren
    self.assertEqual('Invalid WKT: `POINT (0.0 1.0`',
AssertionError: 'Invalid WKT: `POINT (0.0 1.0`' != "could not convert string to float: ''"
- Invalid WKT: `POINT (0.0 1.0`
+ could not convert string to float: ''
======================================================================
FAIL: test_raises_unmatched_paren (geomet.tests.wkt_test.PolygonLoadsTestCase.test_raises_unmatched_paren)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/geomet-1.0.0/geomet/tests/wkt_test.py", line 431, in test_raises_unmatched_paren
    self.assertEqual(
AssertionError: 'Invalid WKT: `POLYGON ((0.0 0.0, 1.0 4.0, 4.0 1.0, 0.0 0.0)`' != "could not convert string to float: ''"
- Invalid WKT: `POLYGON ((0.0 0.0, 1.0 4.0, 4.0 1.0, 0.0 0.0)`
+ could not convert string to float: ''
Could not read Geopackage geometry because of errors: Envelope indicator must be between 0-4
Could not read Geopackage geometry because of errors: Geopackage version must be 0

It appears that the convenient tokenizer has changed behaviour a bit.

Drop support for end-of-life Python versions.

GeoMet currently supports Python 2.7, 3.4, 3.5, 3.6, 3.7, 3.8, and 3.9. Versions 2.7, 3.4, 3.5, and 3.6 are end-of-life.

I originally pushed to maintain support for older versions just because of the how many older Python environments were installing GeoMet from PyPI (https://pypistats.org/packages/geomet). Those numbers are dropping. Although there are still a lot of downloads, the tooling we use (https://pypi.org/project/twine/) for releasing packages to PyPI has dropped support Python <3.7. I think it's finally time to follow suit.

If this breaks anything for users who are still running older Python versions, they can work around this by pinning geomet==0.3.0 in their requirements.txt file. Note, however, that doing so means that those users won't get any new features or bug fixes. In long term, it is recommend that users upgrade to using Python >= 3.7.

Geomet does not support explicit M, Z, or ZM WKT

>>> from geomet import wkt
>>> wkt.loads('POINT M (0 0 5)')
Traceback (most recent call last):
  File "<input>", line 1, in <module>
    wkt.loads('POINT M (0 0 5)')
  File "/private/tmp/v/lib/python3.6/site-packages/geomet/wkt.py", line 147, in loads
    result = importer(tokens, string)
  File "/private/tmp/v/lib/python3.6/site-packages/geomet/wkt.py", line 351, in _load_point
    raise ValueError(INVALID_WKT_FMT % string)
ValueError: Invalid WKT: `POINT M (0 0 5)`

We should allow parsing of WKT with extra dimensionality information.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.