memory issue about docker

[jeremy-rutman]

ok it looks like you DON'T run pelias prepare polylines if you're using valhalla - and pelias prepare all includes that (the pelias commands are all defined here) - so you;d do pelias prepare placeholder and pelias prepare interpolation , skipping the pelias prepare polylines that comes first in the definition of prepare_all

[missinglink]

As the author of the code in question I can tell you that it's unlikely that the algorithm I used will work for the planet regardless of how much RAM you throw at it.

You should consider these errors as fatal and follow the instructions in the warning message I left.

[missinglink]

Please don't use the issue tracker for general support, this case was clearly documented in the error message you provided along with a solution.

[jeremy-rutman]

I thought that valhalla was already being used by the docker install , having seen the ~/pelias/images/valhalla dir and dockerfile.
Sorry , my mistake .
I had added virtual memory which avoided this error but iiuc , valhalla should be used in any case.

[jeremy-rutman]

I tried using a valhalla-generated file (the one conveniently linked to here) , putting

 "imports": { 
....
    "polyline": {
      "datapath": "/data/polylines",
      "files": [ "road_network.polylines" ]
    },
....

into ~/pelias/projects/planet/polylines/pelias.json after putting the unzipped/renamed polylines file at /tmp/pelias/planet/data/polylines/road_network.polylines (my DATA_DIR being /tmp/pelias/planet/data), ran pelias elastic stop , nuked the directory /tmp/pelias/planet/data/elasticsearch , and tried the install from start, wait, create, prepare but after prepare I still get the valhalla warning and see

converting /data/openstreetmap/planet-latest.osm.pbf to /data/polylines/extract.0sv

which was the previously-used polylines file . Is this expected ?

Trying again with pelias compose down does remove the elasticsearch index , preventing an 'index exists' message on pelias elastic start (I since realize that pelias elastic remove will take care of removing a partially-built index) but otherwise the same - the valhalla msg appears and an extract.0sv starts getting built. Those msgs are generated by polylines:docker_extract.sh, called by prepare_polylines() (in turn called by pelias prepare all) for any pbf over 1Gb so I hope I am still on the right track - conceivably, one should only run the prepare steps other than prepare_polylines() if a polylines file is being supplied?

[deanshelton913]

Suggestion: Why not roll valhalla into pelias as a dependency, and when a file is too large, pelias' prepare script could execute the valhalla scripts for you.

[missinglink]

FYI I just added this 3 days ago https://github.com/pelias/docker-valhalla-baseimage