GithubHelp home page GithubHelp logo

osm2pgsql-dev / osm2pgsql-themepark Goto Github PK

View Code? Open in Web Editor NEW
11.0 11.0 4.0 102 KB

A framework for creating osm2pgsql flex configurations

Home Page: https://osm2pgsql.org/themepark

License: Apache License 2.0

Lua 99.83% Shell 0.17%
openstreetmap postgis postgresql vector-tiles

osm2pgsql-themepark's People

Contributors

joto avatar pnorman avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar

Forkers

pnorman gbastkowski

osm2pgsql-themepark's Issues

Tilekiln config generation fails if directory not already created

When I uncommented the themepark:plugin('tilekiln'):write_config('tk') line from shortbread_v1_gen and ran LUA_PATH="./osm2pgsql-themepark/lua/?.lua;;" osm2pgsql -d flex -O flex -S ./osm2pgsql-themepark/config/shortbread_gen.lua osm-data.pbf I got an error:

2024-01-19 17:00:55 ERROR: Error loading lua config: ./osm2pgsql-themepark/lua/themepark/plugins/tilekiln.lua:73: Could open 'tk/ocean.10-14.sql.jinja2': tk/ocean.10-14.sql.jinja2: No such file or directory.

Upon examination I found that the issue was the tk directory did not exist, and it went away when I pre-created it.

Failed to execute osm2pgsql.process_way with shortbread configuration

Setup:

$ osm2pgsql --version
2023-12-13 22:46:26  osm2pgsql version 1.10.0 (1.10.0)
Build: RelWithDebInfo
Compiled using the following library versions:
Libosmium 2.20.0
Proj [API 6] 9.1.1
Lua 5.3.6

Themepark commit: a789b0e

Import settings

shortbread=# select * from osm2pgsql_properties;
          property           |                          value
-----------------------------+---------------------------------------------------------
 attributes                  | false
 db_format                   | 1
 flat_node_file              | /home/pnorman/nodes.bin
 import_timestamp            | 2023-12-04T01:00:01Z
 output                      | flex
 prefix                      | planet_osm
 style                       | /home/pnorman/osm2pgsql-themepark/config/shortbread.lua
 updatable                   | true
 version                     | 1.10.0
 current_timestamp           | 2023-12-05T12:26:08Z
 replication_base_url        | https://planet.openstreetmap.org/replication/minute
 replication_sequence_number | 5862920
 replication_timestamp       | 2023-12-05T12:26:08Z
(13 rows)

On my initial run of osm2pgsql-replication update -d shortbread I got an error in stage 2 processing. This happened on the second time it called osm2pgsql, not the first time.

ERROR: Failed to execute Lua function 'osm2pgsql.process_way': [string "/home/pnorman/osm2pgsql-themepark/lua/../them..."]:44: attempt to index a nil value (local 'info')
stack traceback:
        [string "/home/pnorman/osm2pgsql-themepark/lua/../them..."]:44: in local 'func'
        /home/pnorman/osm2pgsql-themepark/lua/themepark.lua:460: in function </home/pnorman/osm2pgsql-themepark/lua/themepark.lua:456>.
Full log ``` $ osm2pgsql-replication update -d shortbread 2023-12-13 21:15:50 [INFO]: Using replication service 'https://planet.openstreetmap.org/replication/minute'. 2023-12-13 21:25:30 osm2pgsql version 1.10.0 (1.10.0) 2023-12-13 21:25:30 Database version: 16.1 (Debian 16.1-1.pgdg120+1) 2023-12-13 21:25:30 PostGIS version: 3.4 2023-12-13 21:25:30 Loading properties from table '"public"."osm2pgsql_properties"'. 2023-12-13 21:25:30 Using flat node file '/home/pnorman/nodes.bin' (same as on import). 2023-12-13 21:25:30 Using output 'flex' (same as on import). 2023-12-13 21:25:30 Using style file '/home/pnorman/osm2pgsql-themepark/config/shortbread.lua' (same as on import). 2023-12-13 21:32:37 Reading input files done in 427s (7m 7s). 2023-12-13 21:32:37 Processed 3763108 nodes in 188s (3m 8s) - 20k/s 2023-12-13 21:32:37 Processed 586542 ways in 184s (3m 4s) - 3k/s 2023-12-13 21:32:37 Processed 15005 relations in 55s - 273/s 2023-12-13 21:32:39 Going over 94750 pending ways (using 4 threads) Left to process: 0....... 2023-12-13 21:33:42 Processing 94750 pending ways took 63s (1m 3s) at a rate of 1503.97/s 2023-12-13 21:33:42 Going over 42617 pending relations (using 4 threads) Left to process: 0....... 2023-12-13 21:36:29 Processing 42617 pending relations took 166s (2m 46s) at a rate of 256.73/s 2023-12-13 21:36:29 Entering stage 1c processing of 61576 ways... 2023-12-13 21:36:39 Going over 29805 pending relations (using 4 threads) Left to process: 0....... 2023-12-13 21:41:21 Processing 29805 pending relations took 282s (4m 42s) at a rate of 105.69/s 2023-12-13 21:41:21 Reprocess marked ways (stage 2)... 2023-12-13 21:41:21 There are 35933 ways to reprocess... 2023-12-13 21:41:25 Done postprocessing on table 'planet_osm_nodes' in 0s 2023-12-13 21:41:25 Done postprocessing on table 'planet_osm_ways' in 0s 2023-12-13 21:41:25 Done postprocessing on table 'planet_osm_rels' in 0s 2023-12-13 21:41:26 All postprocessing on table 'aerialways' done in 0s. 2023-12-13 21:41:26 All postprocessing on table 'boundaries' done in 1s. 2023-12-13 21:41:26 All postprocessing on table 'boundary_labels' done in 1s. 2023-12-13 21:41:26 All postprocessing on table 'bridges' done in 1s. 2023-12-13 21:41:28 All postprocessing on table 'buildings' done in 2s. 2023-12-13 21:41:26 All postprocessing on table 'bridges' done in 1s. [36/1130] 2023-12-13 21:41:28 All postprocessing on table 'buildings' done in 2s. 2023-12-13 21:41:28 All postprocessing on table 'dam_lines' done in 2s. 2023-12-13 21:41:28 All postprocessing on table 'dam_polygons' done in 2s. 2023-12-13 21:41:28 All postprocessing on table 'ferries' done in 2s. 2023-12-13 21:41:30 All postprocessing on table 'land' done in 1s. 2023-12-13 21:41:30 All postprocessing on table 'pier_lines' done in 1s. 2023-12-13 21:41:30 All postprocessing on table 'pier_polygons' done in 1s. 2023-12-13 21:41:30 All postprocessing on table 'place_labels' done in 1s. 2023-12-13 21:41:30 All postprocessing on table 'pois' done in 0s. 2023-12-13 21:41:30 All postprocessing on table 'public_transport' done in 0s. 2023-12-13 21:41:31 All postprocessing on table 'sites' done in 0s. 2023-12-13 21:41:33 All postprocessing on table 'streets' done in 2s. 2023-12-13 21:41:34 All postprocessing on table 'street_labels' done in 3s. 2023-12-13 21:41:36 All postprocessing on table 'streets_med' done in 5s. 2023-12-13 21:41:36 All postprocessing on table 'streets_low' done in 5s. 2023-12-13 21:41:36 All postprocessing on table 'street_polygons' done in 3s. 2023-12-13 21:41:36 All postprocessing on table 'streets_labels_points' done in 2s. 2023-12-13 21:41:37 All postprocessing on table 'water_polygons' done in 0s. 2023-12-13 21:41:37 All postprocessing on table 'water_polygons_labels' done in 0s. 2023-12-13 21:41:37 All postprocessing on table 'water_lines' done in 0s. 2023-12-13 21:41:38 All postprocessing on table 'water_lines_labels' done in 1s. 2023-12-13 21:41:39 All postprocessing on table 'addresses' done in 2s. 2023-12-13 21:41:39 osm2pgsql took 969s (16m 9s) overall. 2023-12-13 21:41:40 [INFO]: Data imported until 2023-12-05T12:26:08Z. Backlog remaining: 1 week(s) 1 day(s) 9 hour(s) 15 minute(s) 32 second(s) 2023-12-13 21:50:33 osm2pgsql version 1.10.0 (1.10.0) 2023-12-13 21:50:33 Database version: 16.1 (Debian 16.1-1.pgdg120+1) 2023-12-13 21:50:33 PostGIS version: 3.4 2023-12-13 21:50:33 Loading properties from table '"public"."osm2pgsql_properties"'. 2023-12-13 21:50:33 Using flat node file '/home/pnorman/nodes.bin' (same as on import). 2023-12-13 21:50:33 Using output 'flex' (same as on import). 2023-12-13 21:50:33 Using style file '/home/pnorman/osm2pgsql-themepark/config/shortbread.lua' (same as on import). 2023-12-13 21:57:55 Reading input files done in 441s (7m 21s). 2023-12-13 21:57:55 Processed 3866572 nodes in 174s (2m 54s) - 22k/s 2023-12-13 21:57:55 Processed 583471 ways in 186s (3m 6s) - 3k/s 2023-12-13 21:57:55 Processed 15513 relations in 81s (1m 21s) - 192/s 2023-12-13 21:57:58 Going over 93763 pending ways (using 4 threads) Left to process: 0....... 2023-12-13 21:57:58 Going over 93763 pending ways (using 4 threads) [0/1130] Left to process: 0....... 2023-12-13 21:59:05 Processing 93763 pending ways took 67s (1m 7s) at a rate of 1399.45/s 2023-12-13 21:59:05 Going over 41219 pending relations (using 4 threads) Left to process: 0....... 2023-12-13 22:01:21 Processing 41219 pending relations took 135s (2m 15s) at a rate of 305.33/s 2023-12-13 22:01:21 Entering stage 1c processing of 79306 ways... 2023-12-13 22:01:31 Going over 29307 pending relations (using 4 threads) Left to process: 0....... 2023-12-13 22:05:55 Processing 29307 pending relations took 264s (4m 24s) at a rate of 111.01/s 2023-12-13 22:05:55 Reprocess marked ways (stage 2)... 2023-12-13 22:05:55 There are 38348 ways to reprocess... 2023-12-13 22:05:56 ERROR: Failed to execute Lua function 'osm2pgsql.process_way': [string "/home/pnorman/osm2pgsql-themepark/lua/../them..."]:44: attempt to index a nil value (local 'info') stack traceback: [string "/home/pnorman/osm2pgsql-themepark/lua/../them..."]:44: in local 'func' /home/pnorman/osm2pgsql-themepark/lua/themepark.lua:460: in function . Traceback (most recent call last): File "/usr/local/bin/osm2pgsql-replication", line 724, in sys.exit(main()) ^^^^^^ File "/usr/local/bin/osm2pgsql-replication", line 716, in main return args.handler(props, args) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/bin/osm2pgsql-replication", line 552, in update subprocess.run(osm2pgsql, check=True) File "/usr/lib/python3.11/subprocess.py", line 571, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/local/bin/osm2pgsql', '--append', '--slim', '--prefix', 'planet_osm', '-d', 'shortbread', '/tmp/tmpp7o_wvaj/osm2pgsql_diff.osc.gz']' returned non-zero exit status 1. ```

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.