wrobell / geotiler Goto Github PK

hiya - thanks for your library, it is very nice, and I use it in https://github.com/time4tea/gopro-dashboard-overlay

as the rendering code in my project gets called a lot - i was looking for places that I could make some easy gains in performance - so i ran the rendering code in viztracer.

I got the following trace:

as you can see there is quite some time spent in PIL imagedecode - this seems to be because the cache in the system caches the data files, but there is no image cache in the library. When drawing maps in my program we reuse map tiles a lot, as the map is moving pixel by pixel around the place.

i couldn't find a good place to put an image cache in, so i kind of bodged it, and ended up with a trace that looks like this:

you can see that the time to create the image is much reduced - from about 7.5ms to about 1ms - assuming a cached tile.

The code is smashed in right now, so no prizes for quality - but you are welcome to it if it is useful for you.

https://github.com/time4tea/gopro-dashboard-overlay/blob/geotiler-perf/gopro_overlay/geo_render.py

It is a "drop-in" replacement for the geotiler.render_map (at least how I use it)

Hope this is useful/interesting - anyhow thanks for making a nice library.

Cheers!

James

sphinx-build doc/ _build/ complains that these images were not found:

• /home/h/projects/geotiler/doc/cmd.rst:: WARNING: image file not readable: map-bluemarble.png
• /home/h/projects/geotiler/doc/cmd.rst:: WARNING: image file not readable: map-path.png
• /home/h/projects/geotiler/doc/usage.rst:: WARNING: image file not readable: map-osm.png
• /home/h/projects/geotiler/doc/usage.rst:: WARNING: image file not readable: map-stamen-toner.png

Indeed, they are not there...

I have created a project that utilizes geotiler attempted to bundle my project into an exe using auto-py-to-exe. However, when I run my program it crashes due to the following error:
File "geotiler\map.py", line 107, in init
File "geotiler\provider.py", line 112, in find_provider
File "geotiler\provider.py", line 164, in read_provider_data
FileNotFoundError: [Errno 2] No such file or directory: 'C:\Users\ellio\AppData\Local\Temp\_MEI313962\geotiler\source\osm.json'

Bundling files using auto-py-to-exe requires relative file paths due to it placing all files in a temporary folder upon compilation, but this does not appear to work with geotiler files.

I tried running the following:

fig = plt.figure()
ax = plt.subplot(111)

#
# download background map using OpenStreetMap
#
mm = geotiler.Map(center=(39.01480,-104.86509), zoom=16, size=(512,512))

img = geotiler.render_map(mm)
ax.imshow(img)

plt.show()
plt.close()

I get the following error:

/usr/local/lib/python3.5/dist-packages/geotiler/geo.py in rawProject(self, point)
    141     def rawProject(self, point):
    142         x, y = point
--> 143         return x, math.log(math.tan(0.25 * math.pi + 0.5 * y))
    144 
    145     def rawUnproject(self, point):

ValueError: math domain error

I didn't really change much from the examples that work in the docs except the coordinate and then tried some different zoom levels to see if that was an issue.
Am I just making a fundamental misunderstanding here?

When system's character encoding differs from utf8, osm.json fails to load. Fix is to open the file with encoding="utf8".

--- geotiler/provider.py~ 2016-10-03 13:20:22.589822292 +0200
+++ geotiler/provider.py  2016-10-04 09:42:04.890128325 +0200
@@ -109,7 +109,7 @@
     fn = os.path.join(base_dir(), id + '.json')
     if __debug__:
         logger.debug('loading map provider "{}" from {}'.format(id, fn))
-    with open(fn) as f:
+    with open(fn, encoding="utf8") as f:
         data = json.load(f)
         provider = MapProvider(data)
         return provider

See: openstreetmap/operations#737

Example:

Currently only Python 3.6 or later is supported :(

Please support 3.5, too ;)

With 3.5 i get:

  File ".../lib/python3.5/site-packages/geotiler/map.py", line 396
    tiles = (t async for t in tiles)
                   ^
SyntaxError: invalid syntax

As above, a correction in stamen.py is needed as fetches now fail.

Python 2.7 is still the defacto python version - consider maintaining (or at least creating of version for the current geotiler release, if you dont intend to support it) a 2.7 compliant version of geotiler. I think this would increase the popularity of a nice, easy to use library.

It would require eliminating use of the concurrent.futures package - this is fairly simple as a backport is available (https://pypi.python.org/pypi/futures)

I wrote a (long) tutorial on integrating GeoPandas and GeoTiler for mapping point and polygon data on a basemap. Also has code for high resolution mapping in the Jupyter Notebook (thanks to @wrobell). I hope someone here might find it useful.
https://gist.github.com/dfolch/ace80ca0307c571b6a1a719430c4052e

Creating rectangular map images (with fixed zoom level) results in messy/chaotic tile order on the map/image.
The behavior doesn't always occur, possibly it depends on the coordinates.

Environment:

macOS
new python3.7 venv
geotiler 0.1.4 via pip

Simple test script:

# coding: utf-8
import sys
import geotiler
print ('geotiler ver.:', geotiler.__version__)
print ('python ver.:', sys.version)

pts = (8.44759, 49.0259355)  # somewhere in Germany.

for size in [(400, 400), (400, 300)]:
    mm = geotiler.Map(center=pts, size=size, zoom=13, provider="osm")
    image = geotiler.render_map(mm)
    image.save('map%s_%s.png' % size)

Console output:

geotiler ver.: 0.14.0
python ver.: 3.7.3 (default, Mar 27 2019, 09:23:15)
[Clang 10.0.1 (clang-1001.0.46.3)]
configuration file /Users/f3k/.config/geotiler/geotiler.ini does not exist
configuration file /Users/f3k/.config/geotiler/geotiler.ini does not exist

Wrong image (rectangular: 400x300px):

Correct image (quadratic: 400x400px):

When running the example ex-shelvecache.py, geotiler 0.13, Python 3.6.4, iPython 6.2.1.

Traceback (most recent call last):

  File "<ipython-input-8-b158063d5933>", line 1, in <module>
    runfile('C:/Users/mw/Desktop/geotiler_test.py', wdir='C:/Users/mw/Desktop')

  File "C:\Users\mw\AppData\Local\Continuum\anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
    execfile(filename, namespace)

  File "C:\Users\mw\AppData\Local\Continuum\anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/Users/mw/Desktop/geotiler_test.py", line 116, in <module>
    img = render_map(mm)

  File "C:\Users\mw\AppData\Local\Continuum\anaconda3\lib\site-packages\geotiler\map.py", line 374, in render_map
    return loop.run_until_complete(task)

  File "C:\Users\mw\AppData\Local\Continuum\anaconda3\lib\asyncio\base_events.py", line 454, in run_until_complete
    self.run_forever()

  File "C:\Users\mw\AppData\Local\Continuum\anaconda3\lib\asyncio\base_events.py", line 408, in run_forever
    raise RuntimeError('This event loop is already running')

RuntimeError: This event loop is already running

If I create a map object like this:

m = geotiler.Map(extent=(50.78012548763749,6.047284217876666, 50.78286635348752,6.050775730265756),size=(100,100))

and then later call the map.extend parameter, I get (50.779366493225105, 6.046887877171803, 50.78365802764892, 6.051155516750894) instead. That is quite the difference. How does that come?

I have noticed an issue while using MapProxy as a local tile cache where the connection can be occasionally be dropped before the full request/response interaction has been completed.

This results in a series of errors like the ones below, and a set of error tiles being rendered onto the map.

Unable to download http://localhost:8080/tiles/layer/grid/11/1129/1228.png (error: Server disconnected)
Unable to download http://localhost:8080/tiles/layer/grid/11/1128/1227.png (error: Server disconnected)

Actually navigating to these tile endpoints works just fine, which shows that the tiles are generally being served correctly.

A quick hack shows that wrapping the body of fetch_tile in a simple retry loop is all that is needed to mitigate this and make the downloader a bit more robust.

I may take a look at implementing this myself if time allows, but I thought that I would bring it to your attention just in case it is something that you have already thought about.

Pillow 10 has removed the ImageDraw.textsize() method. The intended replacement, ImageDraw.textbbox(), is available beginning with Pillow 8.0.0.

python3.11 -m venv _e
. _e/bin/activate
pip install .[tests]
pip install numpy
python -m pytest

=================================== FAILURES ===================================
____________________________ test_render_error_tile ____________________________

    def test_render_error_tile():
        """
        Test rendering of error tile.
        """
        tile_img._error_image.cache_clear()
>       img = tile_img._error_image(10, 10)

geotiler/tests/test_tile_img.py:66:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

width = 10, height = 10

    @functools.lru_cache(maxsize=4)
    def _error_image(width, height):
        """
        Create error tile image.

        The error tile image is PIL image object showing message that a map
        tile could not be downloaded.

        :param width: Width of tile image.
        :param height: Height of tile image.
        """
        img = PIL.Image.new('RGBA', (width, height))
        draw = PIL.ImageDraw.Draw(img)
        msg = 'Error downloading map tile.'
>       tw, th = draw.textsize(msg)
E       AttributeError: 'ImageDraw' object has no attribute 'textsize'

geotiler/tile/img.py:87: AttributeError
______________________________ test_render_image _______________________________

    def test_render_image():
        """
        Test rendering map image.
        """
        tile = PIL.Image.new('RGBA', (10, 10))
        map = mock.MagicMock()
        map.size = [30, 20]
        map.provider.tile_width = 10
        map.provider.tile_height = 10

        with mock.patch('geotiler.tile.img._tile_image') as tf, \
                mock.patch.object(PIL.Image, 'new') as img_new:

            tf.return_value = tile
            data = (tile, tile, tile, tile)
            offsets = ((0, 0), (10, 0), (20, 0), (0, 10))
            tiles = _tile_generator(offsets, data)
>           image = _run_render_image(map, tiles)

geotiler/tests/test_tile_img.py:108:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
geotiler/tests/test_tile_img.py:51: in _run_render_image
    image = loop.run_until_complete(task)
/usr/lib64/python3.11/asyncio/base_events.py:653: in run_until_complete
    return future.result()
geotiler/tile/img.py:65: in render_image
    error = _error_image(provider.tile_width, provider.tile_height)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

width = 10, height = 10

    @functools.lru_cache(maxsize=4)
    def _error_image(width, height):
        """
        Create error tile image.

        The error tile image is PIL image object showing message that a map
        tile could not be downloaded.

        :param width: Width of tile image.
        :param height: Height of tile image.
        """
        img = PIL.Image.new('RGBA', (width, height))
        draw = PIL.ImageDraw.Draw(img)
        msg = 'Error downloading map tile.'
>       tw, th = draw.textsize(msg)
E       ValueError: not enough values to unpack (expected 2, got 0)

geotiler/tile/img.py:87: ValueError
___________________________ test_render_image_error ____________________________

    def test_render_image_error():
        """
        Test rendering map image with error tile.
        """
        tile = PIL.Image.new('RGBA', (10, 10))
        map = mock.MagicMock()
        map.size = 30, 20
        map.provider.tile_width = 10
        map.provider.tile_height = 10

        with mock.patch('geotiler.tile.img._tile_image') as tf:
            tf.return_value = tile
            data = (tile, tile, None, tile, None, tile)
            offsets = ((0, 0), (10, 0), (20, 0), (0, 10), (10, 10), (20, 10))
            tiles = _tile_generator(offsets, data)
>           image = _run_render_image(map, tiles)

geotiler/tests/test_tile_img.py:128:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
geotiler/tests/test_tile_img.py:51: in _run_render_image
    image = loop.run_until_complete(task)
/usr/lib64/python3.11/asyncio/base_events.py:653: in run_until_complete
    return future.result()
geotiler/tile/img.py:65: in render_image
    error = _error_image(provider.tile_width, provider.tile_height)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

width = 10, height = 10

    @functools.lru_cache(maxsize=4)
    def _error_image(width, height):
        """
        Create error tile image.

        The error tile image is PIL image object showing message that a map
        tile could not be downloaded.

        :param width: Width of tile image.
        :param height: Height of tile image.
        """
        img = PIL.Image.new('RGBA', (width, height))
        draw = PIL.ImageDraw.Draw(img)
        msg = 'Error downloading map tile.'
>       tw, th = draw.textsize(msg)
E       AttributeError: 'ImageDraw' object has no attribute 'textsize'

geotiler/tile/img.py:87: AttributeError

[...]

=========================== short test summary info ============================
FAILED geotiler/tests/test_tile_img.py::test_render_error_tile - AttributeError: 'ImageDraw' object has no attribute 'textsize'
FAILED geotiler/tests/test_tile_img.py::test_render_image - ValueError: not enough values to unpack (expected 2, got 0)
FAILED geotiler/tests/test_tile_img.py::test_render_image_error - AttributeError: 'ImageDraw' object has no attribute 'textsize'
========================= 3 failed, 40 passed in 0.49s ========================

I have some questions about this library. Maybe they would be useful to improve the documentation of the library

• in geotiler.Map():

  • What is the unit of the size parameter? image pixels?
  • some combinations of extend and zoom look very ugly. Is there a smart way of finding out ideal values of zoom or size?

• is there a way to extend the map extent? for example: i read the coordinates of a gpx file and get the boundaries of the track (min/max lat/long) and use that as map.extent. But i want a bigger area. I tried to increase by 10% the bbox, but img = geotiler.render_map(mm) crashes:

  DEBUG:asyncio:Using selector: EpollSelector
  DEBUG:geotiler.tile.img:combining tiles
  Killed
  

  maybe a Map.zomm_out(zoom_percentage) method? I tried to increase the bbox multiplying each entry by 1.1 or 0.9. It crashed. But when I summed 0.002, it worked. I chose the number by trial and error.

• maybe you should rename the internal map variables to my_map (or something else) to not confuse/overwrite with map(). Also in the examples.

• it would be nice to add a note to the docs that you need redis-server running for the Command Line Tools. And also a note in the readme about this online doc, it took me a while to find it.

Hi,
a fried used this lib and we stumbled across a small problem: The size of the tiles is expected to be 256x256. There are, however, providers with different resolutions, usually to provide "retina" or "high-res" tiles. When adding such a provider, the library messes up the images. It seems like its cutting only 256x256 from the 512x512 tiles. We changed the two "return 256" statements in provider.py to 512 and it worked.
So I guess you should either add an additional option in the json files, which defines the resolution, or even better automatically detect it once an image is downloaded.

I don't know if this is a problem that has also happened to others, but in case you need a solution, you should go to geotiler/map.py and add the following to the imports:

import nest_asyncio
nest_asyncio.apply()

I also suggest to the authors to add the two lines above, in case this becomes a recurring problem. Making a pull request didn't seem necessary for this small issue.

Hi, I've been using Geotiler in a map application. Recently I noticed that I couldn't load a map from any of the Stamen providers.

these providers all seem not to work now:
'stamen-terrain', 'stamen-terrain-background', 'stamen-terrain-lines', 'stamen-toner', 'stamen-toner-lite', 'stamen-watercolor'

I remember all of them working before, and 'osm' provider still works.

I am using Windows 10, Python 3.9.1. I first noticed the issue on Geotiler 0.14.7. I updated to 0.15.0 using pip and still had the issue.

I think it's due to Stamen changing their website or removing the map service. On stamen.org I used to see a demo of terrain, toner, and watercolor maps but don't see it now.

So this is probably out of your control, but I wanted to check if everyone sees the same issue.
Fortunately I can still use the OSM provider for now.

example Python code, which shows osm working vs stamen-terrain not working.

m = geotiler.Map(center=(0.0, 0.0), zoom=6, size=(500,500), provider="osm")
i = geotiler.render_map(m)
i.show() -> shows a map image of Atlantic Ocean.

m = geotiler.Map(center=(0.0, 0.0), zoom=6, size=(500,500), provider="stamen-terrain")
i = geotiler.render_map(m)
Cannot download a tile due to error: Unable to download http://b.tile.stamen.com/terrain/6/31/32.png (error: Cannot connect to host b.tile.stamen.com:80 ssl:default [getaddrinfo failed])
Cannot download a tile due to error: Unable to download http://c.tile.stamen.com/terrain/6/32/31.png (error: Cannot connect to host c.tile.stamen.com:80 ssl:default [getaddrinfo failed])
Cannot download a tile due to error: Unable to download http://a.tile.stamen.com/terrain/6/31/31.png (error: Cannot connect to host a.tile.stamen.com:80 ssl:default [getaddrinfo failed])
Cannot download a tile due to error: Unable to download http://d.tile.stamen.com/terrain/6/32/32.png (error: Cannot connect to host d.tile.stamen.com:80 ssl:default [getaddrinfo failed])
i.show() -> shows "error rendering map" image.

Some of my downloaded maps contain blank tiles with "Error downloading map tile" and terminal output shows the following error message:

Cannot download a tile due to error: <urlopen error [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond>

Normally I would expect the called script to raise an exception, but the library ignores it and generates map with a hole in it instead.

Hello, I ve installed geotiler, but when I m trying to run .py file there are some errors.
First
configuration file .config/geotiler/geotiler.ini does not exist
Second
could't find .netrc file

PS my code was from one of the example. I tried it in spyder (Python 3.6)
import geotiler map = geotiler.Map(center=(-6.069, 53.390), zoom=16, size=(512, 512)) map.extent image = geotiler.render_map(map)

What can be wrong? Thanks in advance for help!

I have been using geotiler for awhile now and really like it. If I save the map to the hard drive it looks great, but the image rendered in a Jupyter Notebook is low resolution. Does anyone know how to increase the resolution in the Notebook? Here is my workflow.

import geotiler
import matplotlib.pyplot as plt
%matplotlib inline

phx_tiler = geotiler.Map(center=(-112.00, 33.283), zoom = 12, size=(2000,2000))
phx_img = geotiler.render_map(phx_tiler)
plt.figure(figsize=(10,10))
plt.imshow(phx_img)

Attempting to run the following code in a Jupyter interactive window in Visual Studio Code results in an error:

import geotiler
map = geotiler.Map(center=(-6.069, 53.390), zoom=16, size=(512, 512))
image = geotiler.render_map(map)

Here is the error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-1-d6cca021a9a2> in <module>
      1 import geotiler
      2 map = geotiler.Map(center=(-6.069, 53.390), zoom=16, size=(512, 512))
----> 3 image = geotiler.render_map(map)

c:\Miniconda3\envs\replayenv\lib\site-packages\geotiler\map.py in render_map(map, tiles, downloader, **kw)
    374     task = render_map_async(map, downloader=downloader, **kw)
    375     loop = asyncio.get_event_loop()
--> 376     return loop.run_until_complete(task)
    377 
    378 async def render_map_async(map, tiles=None, downloader=None, **kw):

c:\Miniconda3\envs\replayenv\lib\asyncio\base_events.py in run_until_complete(self, future)
    590         """
    591         self._check_closed()
--> 592         self._check_running()
    593 
    594         new_task = not futures.isfuture(future)

c:\Miniconda3\envs\replayenv\lib\asyncio\base_events.py in _check_running(self)
    550     def _check_running(self):
    551         if self.is_running():
--> 552             raise RuntimeError('This event loop is already running')
    553         if events._get_running_loop() is not None:
    554             raise RuntimeError(

RuntimeError: This event loop is already running

Thank you very much for the library.

The OSM cycle based background is having this issue of printing API Key required in the output, how to add the API Key, thank you