GithubHelp home page GithubHelp logo

israelhikingmap / graphhopper-docker-image-push Goto Github PK

View Code? Open in Web Editor NEW
39.0 39.0 24.0 29 KB

Scripts to create and upload a docker image of graphhopper

License: MIT License

Dockerfile 13.52% Shell 86.48%

graphhopper-docker-image-push's People

Contributors

harelm avatar ilovesome1 avatar marisancans avatar rkube-rvk avatar zstadler avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar

Forkers

khoshbin 7040210 ilovesome1 rkube-rvk bigmichi1 umar18 bennyjonsson jtbaker helpbuttons asplin-solutions-ltd lermex chjuca markd2lp linhnv2801 readaptio truckspy mpa1977 kthouz rmmarquini bramws

graphhopper-docker-image-push's Issues

Out of memoy exception coming

I am trying with US map to deploy docker but it is giving me following error:

Untitled

Is it due to the memory ? I have 40GB of RAM installed.

cannot access web server from host

Thanks for maintaining this GH docker image :-)
I tried to use it, I could adapt the command line so as to feed GH with my own pbf data,
the docker run command makes the jetty web server up and running,
but I cannot access it from the host locally.

docker run -p 8989:8989 --env JAVA_OPTS="-Xmx1g -Xms1g -Ddw.server.application_connectors[0].bind_host=0.0.0.0 -Ddw.server.application_connectors[0].port=8989" --env TOOL_OPTS="-Ddw.graphhopper.datareader.file=/data/herault.osm.pbf -Ddw.graphhopper.graph.location=herault-gh" --entrypoint /bin/bash israelhikingmap/graphhopper -c "wget https://testpg.pagesperso-orange.fr/files/herault.pbf -O /data/herault.osm.pbf && java -Ddw.graphhopper.datareader.file=/data/herault.osm.pbf -Ddw.graphhopper.graph.location=herault-gh -jar *.jar server config-example.yml"

When I log into the container (with docker exec -it GHCONTAINER bash), I can check that all seems to work well (i.e. I can execute wget localhost:8989), but not from localhost.
I've found that I should comment out the bind_host lines in config-example.yml : https://discuss.graphhopper.com/t/how-to-connect-to-graphhopper/4470/6

But if I modify the config file on the running GH container, I don't how to restart the server. And I don't either how I could re-build the image with the modified config-example.yml : please could you help me to solve this ? Thanks

Add graphhopper.sh back to the image

The following PR and related issue causes some breaking changes:
graphhopper/graphhopper#2431
We need to decide if we would like to take ownership of this file and add it as part of this image or forfeit it.
The content of the file can be found below.
There are some assumptions in the script related to where to get the OSM file that can be simplified significantly and probably other less useful things that are probably not available when looking at an image and executable perspective.

#!/bin/bash
(set -o igncr) 2>/dev/null && set -o igncr; # this comment is required for handling Windows cr/lf
# See StackOverflow answer http://stackoverflow.com/a/14607651

GH_HOME=$(dirname "$0")
JAVA=$JAVA_HOME/bin/java
if [ "$JAVA_HOME" = "" ]; then
 JAVA=java
fi

vers=$($JAVA -version 2>&1 | grep "version" | awk '{print $3}' | tr -d \")
bit64=$($JAVA -version 2>&1 | grep "64-Bit")
if [ "$bit64" != "" ]; then
  vers="$vers (64bit)"
fi
echo "## using java $vers from $JAVA_HOME"

function printBashUsage {
  echo "Usage:"
  echo "-a | --action <action>    must be one the following actions:"
  echo "     --action import      creates the graph cache only, used for later faster starts"
  echo "     --action web         starts a local server for user access at localhost:8989 and API access at localhost:8989/route"
  echo "     --action build       creates the graphhopper web JAR"
  echo "     --action clean       removes all JARs, necessary if you need to use the latest source (e.g. after switching the branch etc)"
  echo "     --action measurement does performance analysis of the current source version via random routes (Measurement class)"
  echo "-c | --config <config>    specify the application configuration"
  echo "-d | --run-background     run the application in background (detach)"
  echo "-fd| --force-download     force the download of the OSM data file if needed"
  echo "-h | --help               display this message"
  echo "--host <host>             specify to which host the service should be bound"
  echo "-i | --input <file>       path to the input map file or name of the file to download. To download from geofabrik don't append 'latest.osm' and use _ instead of /, e.g. europe_ireland-and-northern-ireland.pbf"
  echo "--jar <file>              specify the jar file (useful if you want to reuse this script for custom builds)"
  echo "-o | --graph-cache <dir>  directory for graph cache output"
  echo "-p | --profiles <string>  comma separated list of vehicle profiles"
  echo "--port <port>             start web server at specific port"
  echo "-v | --version            print version"
}

VERSION=$(grep "<name>" -A 1 pom.xml | grep version | cut -d'>' -f2 | cut -d'<' -f1)

# one or two character parameters have one minus character'-' all longer parameters have two minus characters '--'
while [ ! -z $1 ]; do
  case $1 in
    -a|--action) ACTION=$2; shift 2;;
    -c|--config) CONFIG="$2"; shift 2;;
    -d|--run-background) RUN_BACKGROUND=true; shift 1;;
    -fd|--force-download) FORCE_DWN=1; shift 1;;
    -h|--help) printBashUsage
      exit 0;;
    --host) GH_WEB_OPTS="$GH_WEB_OPTS -Ddw.server.application_connectors[0].bind_host=$2"; shift 2;;
    -i|--input) FILE="$2"; shift 2;;
    --jar) JAR="$2"; shift 2;;
    -o|--graph-cache) GRAPH="$2"; shift 2;;
    -p|--profiles) GH_WEB_OPTS="$GH_WEB_OPTS -Ddw.graphhopper.graph.flag_encoders=$2"; shift 2;;
    --port) GH_WEB_OPTS="$GH_WEB_OPTS -Ddw.server.application_connectors[0].port=$2"; shift 2;;
    -v|--version) echo $VERSION
    	exit 2;;
    # forward VM options, here we assume no spaces ie. just one parameter!?
    -D*)
       GH_WEB_OPTS="$GH_WEB_OPTS $1"; shift 1;;
    # forward parameter via replacing first two characters of the key with -Ddw.graphhopper.
    *=*)
       echo "Old parameter assignment not allowed $1"; exit 2;;
    --*)
       GH_WEB_OPTS="$GH_WEB_OPTS -Ddw.graphhopper.${1:2}=$2"; shift 2;;
    -*) echo "Option unknown: $1"
        echo
        printBashUsage
	exit 2;;
    # backward compatibility
    *) REMAINING_ARGS+=($1); shift 1;;
  esac
done

if [ -z $ACTION ]; then
  ACTION=${REMAINING_ARGS[0]}
fi

if [ -z $FILE ]; then
  FILE=${REMAINING_ARGS[1]}
fi

if [ "$ACTION" = "" ]; then
 echo "## action $ACTION not found!"
 printBashUsage
fi

if [[ "$CONFIG" == *properties ]]; then
 echo "$CONFIG not allowed as configuration. Use yml"
 exit
fi

# default init, https://stackoverflow.com/a/28085062/194609
: "${CONFIG:=config.yml}"
if [[ -f $CONFIG && $CONFIG != config.yml ]]; then
  echo "copying non-default config file: $CONFIG"
  cp $CONFIG config.yml
fi
if [ ! -f "config.yml" ]; then
  echo "no config file was specified, using config-example.yml"
  cp config-example.yml $CONFIG
fi

function ensureOsm { 
  if [ "$OSM_FILE" = "" ]; then
    # skip
    return
  elif [ ! -s "$OSM_FILE" ]; then
    if [ -z $FORCE_DWN ]; then
      echo "File not found '$OSM_FILE'. Press ENTER to get it from: $LINK"
      echo "Press CTRL+C if you do not have enough disc space or you don't want to download several MB."
      read -e
    fi

    echo "## now downloading OSM file from $LINK and extracting to $OSM_FILE"

    if [ ${OSM_FILE: -4} == ".pbf" ]; then
       wget -S -nv -O "$OSM_FILE" "$LINK"
    elif [ ${OSM_FILE: -4} == ".ghz" ]; then
       wget -S -nv -O "$OSM_FILE" "$LINK"
       cd $DATADIR && unzip "$BASENAME" -d "$NAME-gh"
    else
       # make sure aborting download does not result in loading corrupt osm file
       TMP_OSM=temp.osm
       wget -S -nv -O - "$LINK" | bzip2 -d > $TMP_OSM
       mv $TMP_OSM "$OSM_FILE"
    fi

    if [[ ! -s "$OSM_FILE" ]]; then
      echo "ERROR couldn't download or extract OSM file $OSM_FILE ... exiting"
      exit
    fi
  else
    echo "## using existing osm file $OSM_FILE"
  fi
}

function ensureMaven {
  # maven home existent?
  if [ "$MAVEN_HOME" = "" ]; then
    # not existent but probably is maven in the path?
    MAVEN_HOME=$(mvn -v | grep "Maven home" | cut -d' ' -f3,4,5,6)
    if [ "$MAVEN_HOME" = "" ]; then
      # try to detect previous downloaded version
      MAVEN_HOME="$GH_HOME/maven"
      if [ ! -f "$MAVEN_HOME/bin/mvn" ]; then
        echo "No Maven found in the PATH. Now downloading+installing it to $MAVEN_HOME"
        cd "$GH_HOME"
        MVN_PACKAGE=apache-maven-3.6.3
        wget -O maven.zip http://archive.apache.org/dist/maven/maven-3/3.6.3/binaries/$MVN_PACKAGE-bin.zip
        unzip maven.zip
        mv $MVN_PACKAGE maven
        rm maven.zip
      fi
    fi
  fi
}

function execMvn {
  ensureMaven
  "$MAVEN_HOME/bin/mvn" "$@" > /tmp/graphhopper-compile.log
  returncode=$?
  if [[ $returncode != 0 ]] ; then
    echo "## compilation of parent failed"
    cat /tmp/graphhopper-compile.log
    exit $returncode
  fi
}

function packageJar {
  if [ ! -f "$JAR" ]; then
    echo "## building graphhopper jar: $JAR"
    echo "## using maven at $MAVEN_HOME"
    execMvn --projects web -am -DskipTests=true package
  else
    echo "## existing jar found $JAR"
  fi
}

## now handle actions which do not take an OSM file
if [ "$ACTION" = "clean" ]; then
 rm -rf ./android/app/target
 rm -rf ./*/target
 rm -rf ./target
 exit

elif [ "$ACTION" = "build" ]; then
 packageJar
 exit

fi

if [ "$FILE" = "" ]; then
  echo -e "no file specified?"
  printBashUsage
  exit
fi

# DATA_DIR = directories path to the file if any (if current directory, return .)
DATADIR=$(dirname "${FILE}")
# create the directories if needed
mkdir -p $DATADIR
# BASENAME = filename (file without the directories)
BASENAME=$(basename "${FILE}")
# NAME = file without extension if any
NAME="${BASENAME%.*}"

if [ "$FILE" == "-" ]; then
   OSM_FILE=
elif [ ${FILE: -4} == ".osm" ] || [ ${FILE: -4} == ".xml" ] || [ ${FILE: -4} == ".pbf" ]; then
   OSM_FILE="$FILE"
elif [ ${FILE: -7} == ".osm.gz" ]; then
   OSM_FILE="$FILE"
elif [ ${FILE: -3} == "-gh" ]; then
   OSM_FILE="$FILE"
   NAME=${FILE%%???}
elif [ ${FILE: -4} == ".ghz" ]; then
   OSM_FILE="$FILE"
   if [[ ! -d "$NAME-gh" ]]; then
      cd $DATADIR && unzip "$BASENAME" -d "$NAME-gh"
   fi
else
   # no known end -> no import
   OSM_FILE=
fi

LINK=$(echo $NAME | tr '_' '/')
if [ "$FILE" == "-" ]; then
   LINK=
elif [ ${FILE: -4} == ".osm" ]; then 
   LINK="http://download.geofabrik.de/$LINK-latest.osm.bz2"
elif [ ${FILE: -4} == ".ghz" ]; then
   LINK="https://graphhopper.com/public/maps/0.1/$FILE"
elif [ ${FILE: -4} == ".pbf" ]; then
   LINK="http://download.geofabrik.de/$LINK-latest.osm.pbf"
else
   # e.g. if directory ends on '-gh'
   LINK="http://download.geofabrik.de/$LINK-latest.osm.pbf"
fi

: "${JAVA_OPTS:=-Xmx1000m -Xms1000m}"
: "${JAR:=web/target/graphhopper-web-$VERSION.jar}"
: "${GRAPH:=$DATADIR/$NAME-gh}"

ensureOsm
packageJar

echo "## now $ACTION. JAVA_OPTS=$JAVA_OPTS"

if [[ "$ACTION" = "web" ]]; then
  export MAVEN_OPTS="$MAVEN_OPTS $JAVA_OPTS"
  if [[ "$RUN_BACKGROUND" == "true" ]]; then
    exec "$JAVA" $JAVA_OPTS -Ddw.graphhopper.datareader.file="$OSM_FILE" -Ddw.graphhopper.graph.location="$GRAPH" \
                 $GH_WEB_OPTS -jar "$JAR" server $CONFIG <&- &

    if [[ "$GH_PID_FILE" != "" ]]; then
       echo $! > $GH_PID_FILE
    fi
    exit $?
  else
    # TODO how to avoid duplicative command for foreground and background?
    exec "$JAVA" $JAVA_OPTS -Ddw.graphhopper.datareader.file="$OSM_FILE" -Ddw.graphhopper.graph.location="$GRAPH" \
                 $GH_WEB_OPTS -jar "$JAR" server $CONFIG
    # foreground => we never reach this here
  fi

elif [ "$ACTION" = "import" ]; then
  "$JAVA" $JAVA_OPTS -Ddw.graphhopper.datareader.file="$OSM_FILE" -Ddw.graphhopper.graph.location="$GRAPH" \
         $GH_IMPORT_OPTS -jar "$JAR" import $CONFIG

elif [ "$ACTION" = "measurement" ]; then
  ARGS="$GH_WEB_OPTS graph.location=$GRAPH datareader.file=$OSM_FILE \
       measurement.weighting=fastest measurement.ch.node=true measurement.ch.edge=false measurement.lm=true graph.flag_encoders=car|turn_costs=true \
       prepare.min_network_size=10000"

 function startMeasurement {
    execMvn --projects tools -am -DskipTests clean package
    COUNT=5000
    commit_info=$(git log -n 1 --pretty=oneline)
    JAR=tools/target/graphhopper-tools-$VERSION-jar-with-dependencies.jar
    echo -e "\nperform measurement via jar=> $JAR and ARGS=> $ARGS"
    "$JAVA" $JAVA_OPTS -cp "$JAR" com.graphhopper.tools.Measurement $ARGS measurement.count=$COUNT measurement.location="$M_FILE_NAME" \
            measurement.gitinfo="$commit_info"
 }


 # use all <last_commits> versions starting from HEAD
 last_commits=$3

 if [ "$last_commits" = "" ]; then
   startMeasurement
   exit
 fi

 current_commit=$(git log -n 1 --pretty=oneline | cut -d' ' -f1)
 commits=$(git rev-list HEAD -n $last_commits)
 for commit in $commits; do
   git checkout $commit -q
   M_FILE_NAME=$(git log -n 1 --pretty=oneline | grep -o "\ .*" |  tr " ,;" "_")
   M_FILE_NAME="measurement$M_FILE_NAME.properties"
   echo -e "\nusing commit $commit and $M_FILE_NAME"

   startMeasurement
   echo -e "\nmeasurement.commit=$commit\n" >> "$M_FILE_NAME"
 done
 # revert checkout
 git checkout $current_commit
fi

optimize=true doesn't work

Hey, im trying this docker image and can't get optimize=true to work.
on the graphhopper api it returns points_order an reorder the waypoints.
on this image it doesn't, do you have any hints?

Thanks

can't build, missing files to build?

on the Dockerfile line 17... its copying some files from somewhere.. how do I get this files?

$ docker build -t graphhopper .
Sending build context to Docker daemon  102.4kB
Step 1/15 : FROM maven:3.6.3-jdk-8 as build
 ---> d1b3f61d61f2
Step 2/15 : WORKDIR /graphhopper
 ---> Using cache
 ---> bcbf97944ea4
Step 3/15 : COPY graphhopper .
COPY failed: file not found in build context or excluded by .dockerignore: stat graphhopper: file does not exist

https://github.com/IsraelHikingMap/graphhopper-docker-image-push/blob/main/Dockerfile#L17

Image not published?

Still not sure, but...
I think the image build by the 'github action' is not getting uploaded to dockerhub because the upstream graphhopper version did not changed yet.

Consolidate build scripts

  • Modify build-and-publish.sh to use build.sh.
  • Modify build.sh to accept an optional --publish flag to further simplify its use by build-and-publish.sh.

To be implemented after #27 is resolved

There's no way to run the container on data that was generated

I would like to use the following command line to run the container with pre-processed data:

...
entrypoint: ./graphhopper.sh --host 0.0.0.0 -o /data/default-gh/ -c /data/gh-config.yml
...

This will currently fail with: No file or url were specified.
Basically the test needs to be better defined by checking file and graph settings, and maybe action? IDK...

There's currently a workaround of setting an non-existing input file.

Profiles don't seem to work

Hi there,

I try to run on the docker image, it runs fine. The problem is that only the car profile is available. The command I use is as follows
docker run -p 8989:8989 israelhikingmap/graphhopper --url https://download.geofabrik.de/europe/andorra-latest.osm.pbf --host 0.0.0.0 --profiles car,bike,foot

In UI I see only car option, when I send the request I get this response.

curl --location --request POST 'http://localhost:8989/route?key=' \ --header 'Content-Type: application/json' \ --data-raw '{ "points": [ [ 37.629913, 55.751231 ], [ 37.63914,55.742873 ] ], "profile": "foot", "elevation": true, "debug": false, "instructions": true, "locale": "de_DE", "optimize": "false", "points_encoded": true, "snap_preventions": [ "ferry" ], "details": [ "road_class", "road_environment", "max_speed", "average_speed" ], "alternative_route.max_paths": 3, "algorithm": "alternative_route" }'

Tried with another card, same result, can you give advice or is it a mistake?

Pull access denied ?

Hi,
I'm trying to run

docker run --entrypoint /bin/bash israelhikingmap/graphhhopper -c "wget https://download.geofabrik.de/europe/germany/berlin-latest.osm.pbf -O /data/berlin.osm.bpf && java -Ddw.graphhopper.datareader.file=/data/berlin.osm.pbf -Ddw.graphhopper.graph.location=berlin-gh -jar *.jar server config-example.yml"

and i receive:

Unable to find image 'israelhikingmap/graphhhopper:latest' locally

docker: Error response from daemon: pull access denied for israelhikingmap/graphhhopper, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.

Please help.

edit1:
I pull the image first manually, then when started, it saved the berlin file, but after start of server it can't find it:

2021-09-09 08:37:45.358 [main] INFO  o.e.jetty.setuid.SetUIDListener - Opened application@73ad7e90{HTTP/1.1, (http/1.1)}{localhost:8989}
2021-09-09 08:37:45.360 [main] INFO  o.e.jetty.setuid.SetUIDListener - Opened admin@4ba380c7{HTTP/1.1, (http/1.1)}{localhost:8990}
2021-09-09 08:37:45.363 [main] INFO  org.eclipse.jetty.server.Server - jetty-9.4.39.v20210325; built: 2021-03-25T14:42:11.471Z; git: 9fc7ca5a922f2a37b84ec9dbc26a5168cee7e667; jvm 11.0.12+7
2021-09-09 08:37:45.396 [main] INFO  com.graphhopper.GraphHopper - version 4.0|2021-09-09T01:56:19Z (5,19,4,4,5,7)
2021-09-09 08:37:45.403 [main] INFO  com.graphhopper.GraphHopper - graph CH|car|RAM_STORE|2D|no_turn_cost|,,,,, details:edges:0(0MB), nodes:0(0MB), name:(0MB), geo:0(0MB), bounds:1.7976931348623157E308,-1.7976931348623157E308,1.7976931348623157E308,-1.7976931348623157E308, CHGraph|car|NODE_BASED, shortcuts:0 (0MB), nodesCH:0 (0MB)
2021-09-09 08:37:45.665 [main] INFO  com.graphhopper.GraphHopper - No custom areas are used, custom_areas.directory not given
2021-09-09 08:37:45.685 [main] INFO  com.graphhopper.GraphHopper - start creating graph from /data/berlin.osm.pbf
2021-09-09 08:37:45.702 [main] INFO  com.graphhopper.GraphHopper - using CH|car|RAM_STORE|2D|no_turn_cost|,,,,, memory:totalMB:186, usedMB:18
2021-09-09 08:37:45.706 [main] ERROR io.dropwizard.cli.ServerCommand - Unable to start server, shutting down
java.lang.IllegalStateException: Your specified OSM file does not exist:/data/berlin.osm.pbf
        at com.graphhopper.reader.osm.OSMReader.readGraph(OSMReader.java:146)
        at com.graphhopper.GraphHopper.importOSM(GraphHopper.java:708)
        at com.graphhopper.GraphHopper.process(GraphHopper.java:672)
        at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:635)
        at com.graphhopper.http.GraphHopperManaged.start(GraphHopperManaged.java:103)
        at io.dropwizard.lifecycle.JettyManaged.doStart(JettyManaged.java:27)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
        at org.eclipse.jetty.server.Server.start(Server.java:423)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
        at org.eclipse.jetty.server.Server.doStart(Server.java:387)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at io.dropwizard.cli.ServerCommand.run(ServerCommand.java:53)
        at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:60)
        at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:98)
        at io.dropwizard.cli.Cli.run(Cli.java:78)
        at io.dropwizard.Application.run(Application.java:94)
        at com.graphhopper.http.GraphHopperApplication.main(GraphHopperApplication.java:35)
java.lang.IllegalStateException: Your specified OSM file does not exist:/data/berlin.osm.pbf
        at com.graphhopper.reader.osm.OSMReader.readGraph(OSMReader.java:146)
        at com.graphhopper.GraphHopper.importOSM(GraphHopper.java:708)
        at com.graphhopper.GraphHopper.process(GraphHopper.java:672)
        at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:635)
        at com.graphhopper.http.GraphHopperManaged.start(GraphHopperManaged.java:103)
        at io.dropwizard.lifecycle.JettyManaged.doStart(JettyManaged.java:27)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
        at org.eclipse.jetty.server.Server.start(Server.java:423)
        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
        at org.eclipse.jetty.server.Server.doStart(Server.java:387)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at io.dropwizard.cli.ServerCommand.run(ServerCommand.java:53)
        at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:60)
        at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:98)
        at io.dropwizard.cli.Cli.run(Cli.java:78)
        at io.dropwizard.Application.run(Application.java:94)
        at com.graphhopper.http.GraphHopperApplication.main(GraphHopperApplication.java:35)

Docker-compose up crashes

Starting today, the israelhikingmap/graphhopper image has updated so that the entrypoint ./graphhopper.sh is no longer found. Therefore, docker-compose doesn't start:

Starting graphhopper-docker_graphhopper_1 ... error

ERROR: for graphhopper-docker_graphhopper_1  Cannot start service graphhopper: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "./graphhopper.sh": stat ./graphhopper.sh: no such file or directory: unknown

ERROR: for graphhopper  Cannot start service graphhopper: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "./graphhopper.sh": stat ./graphhopper.sh: no such file or directory: unknown
ERROR: Encountered errors while bringing up the project.

Import custom OSM file

Do we have an option to import custom OSM file instead of URL ?

docker run -p 8989:8989 israelhikingmap/graphhopper --url --host 0.0.0.0

内存溢出

graphhopper:
restart:always
container_name: graphhopper
image: israelhikingmap/graphhopper:6.0
command: -i /maps/taiwan-latest.osm.pbf --host 0.0.0.0
volumes:
- ./data/SiteMapRouting/maps:/maps
ports:
- 8989:8989
我使用此方式部署docker由于osm.pbf文件过大,导致报错:内存溢出,我该怎么修改这段配置

How to use built in profiles

I have successfully deployed docker container on my Linux machine with default “car” profile. In the documentation: https://docs.graphhopper.com/#section/Map-Data-and-Routing-Profiles/OpenStreetMap, i can see multiple built in profile. Can i integrate multiple profiles ? How can i do that in docker run command ? I can see profile option with coma separated profiles but its not working. Here is the command:

docker run -p 8989:8989 israelhikingmap/graphhopper --url https://download.geofabrik.de/europe/andorra-latest.osm.pbf --host 0.0.0.0 -p|--profiles car,bike

ERROR io.dropwizard.cli.ServerCommand - Unable to start server, shutting down 

graphhopper_1  | 2022-12-29 18:14:20.276 [main] INFO  c.g.r.s.PrepareRoutingSubnetworks - car4wd - Marked 1122309 subnetworks (biggest: 345 edges) -> 6 components(s) remain (smallest: 458, biggest: 3619716 edges), total marked edges: 22334, took: 0.72206014s
graphhopper_1  | 2022-12-29 18:14:24.235 [main] INFO  c.g.r.s.PrepareRoutingSubnetworks - hike - Found 151280 subnetworks (139562 single edges and 11718 components with more than one edge, total nodes: 4916570), took: 3.9585626s
graphhopper_1  | 2022-12-29 18:14:24.533 [main] INFO  c.g.r.s.PrepareRoutingSubnetworks - hike - Marked 151272 subnetworks (biggest: 370 edges) -> 8 components(s) remain (smallest: 586, biggest: 4499766 edges), total marked edges: 39918, took: 0.29780337s
graphhopper_1  | 2022-12-29 18:14:28.543 [main] INFO  c.g.r.s.PrepareRoutingSubnetworks - mtb - Found 467104 subnetworks (454838 single edges and 12266 components with more than one edge, total nodes: 4916570), took: 4.0100274s
graphhopper_1  | 2022-12-29 18:14:28.618 [main] INFO  c.g.r.s.PrepareRoutingSubnetworks - mtb - Marked 467096 subnetworks (biggest: 370 edges) -> 8 components(s) remain (smallest: 446, biggest: 4199321 edges), total marked edges: 46872, took: 0.07496776s
graphhopper_1  | 2022-12-29 18:14:28.618 [main] INFO  c.g.r.s.PrepareRoutingSubnetworks - Finished finding and marking subnetworks for 3 jobs, took: 14.980878s, totalMB:1024, usedMB:580
graphhopper_1  | 2022-12-29 18:14:28.620 [main] INFO  com.graphhopper.GraphHopper - nodes: 1 878 423, edges: 2 458 285
graphhopper_1  | 2022-12-29 18:14:28.861 [main] ERROR io.dropwizard.cli.ServerCommand - Unable to start server, shutting down
graphhopper_1  | java.lang.IllegalStateException: location index was opened with incorrect graph: 3746312 vs. 3746618
graphhopper_1  |        at com.graphhopper.storage.index.LocationIndexTree.loadExisting(LocationIndexTree.java:136)
graphhopper_1  |        at com.graphhopper.GraphHopper.createLocationIndex(GraphHopper.java:1190)
graphhopper_1  |        at com.graphhopper.GraphHopper.initLocationIndex(GraphHopper.java:1205)
graphhopper_1  |        at com.graphhopper.GraphHopper.postProcessing(GraphHopper.java:1108)
graphhopper_1  |        at com.graphhopper.GraphHopper.process(GraphHopper.java:806)
graphhopper_1  |        at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:756)
graphhopper_1  |        at com.graphhopper.http.GraphHopperManaged.start(GraphHopperManaged.java:104)
graphhopper_1  |        at io.dropwizard.lifecycle.JettyManaged.doStart(JettyManaged.java:27)
graphhopper_1  |        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
graphhopper_1  |        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
graphhopper_1  |        at org.eclipse.jetty.server.Server.start(Server.java:423)
graphhopper_1  |        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
graphhopper_1  |        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
graphhopper_1  |        at org.eclipse.jetty.server.Server.doStart(Server.java:387)
graphhopper_1  |        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
graphhopper_1  |        at io.dropwizard.cli.ServerCommand.run(ServerCommand.java:53)
graphhopper_1  |        at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:60)
graphhopper_1  |        at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:98)
graphhopper_1  |        at io.dropwizard.cli.Cli.run(Cli.java:78)
graphhopper_1  |        at io.dropwizard.Application.run(Application.java:94)
graphhopper_1  |        at com.graphhopper.application.GraphHopperApplication.main(GraphHopperApplication.java:38)
graphhopper_1  | java.lang.IllegalStateException: location index was opened with incorrect graph: 3746312 vs. 3746618
graphhopper_1  |        at com.graphhopper.storage.index.LocationIndexTree.loadExisting(LocationIndexTree.java:136)
graphhopper_1  |        at com.graphhopper.GraphHopper.createLocationIndex(GraphHopper.java:1190)
graphhopper_1  |        at com.graphhopper.GraphHopper.initLocationIndex(GraphHopper.java:1205)
graphhopper_1  |        at com.graphhopper.GraphHopper.postProcessing(GraphHopper.java:1108)
graphhopper_1  |        at com.graphhopper.GraphHopper.process(GraphHopper.java:806)
graphhopper_1  |        at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:756)
graphhopper_1  |        at com.graphhopper.http.GraphHopperManaged.start(GraphHopperManaged.java:104)
graphhopper_1  |        at io.dropwizard.lifecycle.JettyManaged.doStart(JettyManaged.java:27)
graphhopper_1  |        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
graphhopper_1  |        at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
graphhopper_1  |        at org.eclipse.jetty.server.Server.start(Server.java:423)
graphhopper_1  |        at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
graphhopper_1  |        at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
graphhopper_1  |        at org.eclipse.jetty.server.Server.doStart(Server.java:387)
graphhopper_1  |        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
graphhopper_1  |        at io.dropwizard.cli.ServerCommand.run(ServerCommand.java:53)
graphhopper_1  |        at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:60)
graphhopper_1  |        at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:98)
graphhopper_1  |        at io.dropwizard.cli.Cli.run(Cli.java:78)
graphhopper_1  |        at io.dropwizard.Application.run(Application.java:94)
graphhopper_1  |        at com.graphhopper.application.GraphHopperApplication.main(GraphHopperApplication.java:38)
helpbuttonsorg_graphhopper_1 exited with code 1
` ``

docker-compose.yml
```
  graphhopper:
    image: israelhikingmap/graphhopper:6.2
    ports:
      - "8989:8989"
    volumes:
      - "./graphhopper:/gh-data:ro"
      - "./graphhopper-data:/data:rw"
    entrypoint: bash -c "chmod +x ./graphhopper.sh && ls /gh-data/ && ./graphhopper.sh --host 0.0.0.0 -i /gh-data/portugal-latest.osm.pbf -o /data/default-gh/ -c /gh-data/gh-config.yml"
    environment:
      - JVM_OPTS=-Xmx12g -Xms12g -XX:MaxPermSize=1024m

I expect the server to run, when i do
curl http://localhost:8989
but it crashes, memory issues maybe? I already put to 1024m im using the pbf of portugal got from:
https://download.geofabrik.de/europe/portugal-latest.osm.pbf

Error on container startup: executable file not found in $PATH

I pulled the version 4.0 of the GraphHopper Docker image and tried to run it as container, like this:

docker pull israelhikingmap/graphhopper:4.0
docker run -p 8989:8989 israelhikingmap/graphhopper:4.0

Unfortunately I received the following error on container startup:

docker: Error response from daemon: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "java $JAVA_OPTS $TOOL_OPTS -jar *.jar": executable file not found in $PATH: unknown.
ERRO[0038] error waiting for container: context canceled

Am I missing anything or is this a problem in the image?

Thanks in advance for any hint/help.

ARM64 images

Hi, thanks for your work on this! Would you be open to modifying the Actions pipeline to push arm64 versions of this image to the registry as well?

example under Linux

With the current example command:

sudo docker run --entrypoint /bin/bash israelhikingmap/graphhopper -c "wget https://download.geofabrik.de/europe/germany/berlin-latest.osm.pbf -O /data/berlin.osm.pbf && java -Ddw.graphhopper.datareader.file=/data/berlin.osm.pbf -Ddw.graphhopper.graph.location=berlin-gh -jar *.jar server config-example.yml"

I was able to start the import process and the server, but I wasn't able to access the service at https://localhost:8989/

I tried it via the --host 0.0.0.0 parameter or the port mapping -p 8989:8989 but without success. The only thing that worked was using the host network (--net="host"). (Maybe I should bind the GraphHopper service in the container to 0.0.0.0 via changing the config-example.yml, not sure.)

Also when I stop the service and restart it with the same command then it does the entire import again. I can avoid this via mounting a volume and downloading the pbf outside of the container like this:

wget https://download.geofabrik.de/europe/germany/berlin-latest.osm.pbf -O berlin.osm.pbf
docker run -v $(pwd)/:/data/ --entrypoint /bin/sh israelhikingmap/graphhopper -c "java -Ddw.graphhopper.datareader.file=/data/berlin.osm.pbf -Ddw.graphhopper.graph.location=/data/berlin-gh -jar *.jar server config-example.yml"

How to overwrite TOOL_OPT environment in docker-compose.yaml?

Please have a look at the following yaml configuration:

  graphhopper:
    image: israelhikingmap/graphhopper:latest
    ports:
      - 8989:8989
    environment:
      JAVA_OPTS: "-Xmx1g -Xms1g -Ddw.server.application_connectors[0].bind_host=0.0.0.0 -Ddw.server.application_connectors[0].port=8989"
      TOOL_OPTS: "-Ddw.graphhopper.datareader.file=/data.osm.pbf -Ddw.graphhopper.graph.location=default-gh"
    volumes:
      - ./demo.osm.pbf:/data.osm.pbf

As you can see I set TOOL_OPTS environment.

However, graphhopper is still trying to parse europe_germany_berlin.pbf:

## using java 11.0.11 (64bit) from /usr/local/openjdk-11
2021-12-20T09:28:27.948137095Z no config file was specified, using config-example.yml
2021-12-20T09:28:27.961433001Z ## using existing osm file /data/europe_germany_berlin.pbf
2021-12-20T09:28:27.961706434Z ## existing jar found web/target/graphhopper-web-4.0-SNAPSHOT.jar
2021-12-20T09:28:27.961735762Z ## now web. JAVA_OPTS=-Xmx1g -Xms1g -Ddw.server.application_connectors[0].bind_host=0.0.0.0 -Ddw.server.application_connectors[0].port=8989
2021-12-20T09:28:30.292116535Z 2021-12-20 09:28:30.288 [main] INFO  org.eclipse.jetty.util.log - Logging initialized @2318ms to org.eclipse.jetty.util.log.Slf4jLog
2021-12-20T09:28:30.384282193Z 2021-12-20 09:28:30.382 [main] INFO  i.d.server.DefaultServerFactory - Registering jersey handler with root path prefix: /
2021-12-20T09:28:30.385887977Z 2021-12-20 09:28:30.385 [main] INFO  i.d.server.DefaultServerFactory - Registering admin handler with root path prefix: /
2021-12-20T09:28:30.480115214Z 2021-12-20 09:28:30.479 [main] INFO  c.g.http.GraphHopperManaged - Loaded landmark splitting collection from 
2021-12-20T09:28:30.764499288Z 2021-12-20 09:28:30.763 [main] INFO  io.dropwizard.assets.AssetsBundle - Registering AssetBundle with name: assets for path /maps/*
2021-12-20T09:28:30.767785272Z 2021-12-20 09:28:30.767 [main] INFO  io.dropwizard.assets.AssetsBundle - Registering AssetBundle with name: webjars for path /webjars/*
2021-12-20T09:28:30.775239384Z 2021-12-20 09:28:30.774 [main] INFO  io.dropwizard.server.ServerFactory - Starting GraphHopperApplication
2021-12-20T09:28:30.775264878Z                         _     _
2021-12-20T09:28:30.775270897Z    __ _ _ __ __ _ _ __ | |__ | |__   ___  _ __  _ __   ___ _ __
2021-12-20T09:28:30.775274356Z   / _` | '__/ _` | '_ \| '_ \| '_ \ / _ \| '_ \| '_ \ / _ \ '__|
2021-12-20T09:28:30.775277770Z  | (_| | | | (_| | |_) | | | | | | | (_) | |_) | |_) |  __/ |
2021-12-20T09:28:30.775280913Z   \__, |_|  \__,_| .__/|_| |_|_| |_|\___/| .__/| .__/ \___|_|
2021-12-20T09:28:30.775284227Z   |___/          |_|                     |_|   |_|
2021-12-20T09:28:30.938663851Z 2021-12-20 09:28:30.938 [main] INFO  o.e.jetty.setuid.SetUIDListener - Opened application@70e889e9{HTTP/1.1, (http/1.1)}{0.0.0.0:8989}
2021-12-20T09:28:30.941283381Z 2021-12-20 09:28:30.940 [main] INFO  o.e.jetty.setuid.SetUIDListener - Opened admin@418c020b{HTTP/1.1, (http/1.1)}{localhost:8990}
2021-12-20T09:28:30.944250886Z 2021-12-20 09:28:30.943 [main] INFO  org.eclipse.jetty.server.Server - jetty-9.4.39.v20210325; built: 2021-03-25T14:42:11.471Z; git: 9fc7ca5a922f2a37b84ec9dbc26a5168cee7e667; jvm 11.0.11+9
2021-12-20T09:28:31.096064334Z 2021-12-20 09:28:31.095 [main] INFO  c.g.routing.ch.CHPreparationHandler - Creating CH preparations, totalMB:1024, usedMB:61
2021-12-20T09:28:31.103973810Z 2021-12-20 09:28:31.103 [main] INFO  com.graphhopper.GraphHopper - version 4.0|2021-06-15T07:21:51Z (5,17,4,4,5,7)
2021-12-20T09:28:31.108578548Z 2021-12-20 09:28:31.108 [main] INFO  com.graphhopper.GraphHopper - graph CH|car|RAM_STORE|2D|no_turn_cost|5,17,4,4,5, details:edges:173 195(6MB), nodes:148 134(2MB), name:(1MB), geo:192 164(1MB), bounds:13.073284284918405,13.763523223180844,52.333098515862936,52.67961645900346, CHGraph|car|NODE_BASED, shortcuts:104 553 (2MB), nodesCH:148 134 (2MB)
2021-12-20T09:28:31.109662831Z 2021-12-20 09:28:31.109 [main] INFO  c.g.http.GraphHopperManaged - loaded graph at:/data/europe_germany_berlin-gh, data_reader_file:/data/europe_germany_berlin.pbf, encoded values:car_subnetwork|version=288493657|bits=1|index=0|shift=0|store_both_directions=false,roundabout|version=283418129|bits=1|index=0|shift=1|store_both_directions=false,road_class|version=2008526613|bits=5|index=0|shift=2|store_both_directions=false,road_class_link|version=146107020|bits=1|index=0|shift=7|store_both_directions=false,road_environment|version=-444941817|bits=3|index=0|shift=8|store_both_directions=false,max_speed|version=650944097|bits=5|index=0|shift=11|store_both_directions=true,road_access|version=1917521501|bits=4|index=0|shift=21|store_both_directions=false, 1 ints for edge flags, edges:173 195(6MB), nodes:148 134(2MB), name:(1MB), geo:192 164(1MB), bounds:13.073284284918405,13.763523223180844,52.333098515862936,52.67961645900346, CHGraph|car|NODE_BASED, shortcuts:104 553 (2MB), nodesCH:148 134 (2MB)
2021-12-20T09:28:31.684713290Z 2021-12-20 09:28:31.683 [main] WARN  o.g.jersey.internal.inject.Providers - A provider com.codahale.metrics.health.HealthCheckRegistry registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider com.codahale.metrics.health.HealthCheckRegistry will be ignored. 
2021-12-20T09:28:31.973190196Z 2021-12-20 09:28:31.972 [main] INFO  i.d.jersey.DropwizardResourceConfig - The following paths were found for the configured resources:
2021-12-20T09:28:31.973218491Z 
2021-12-20T09:28:31.973226576Z     GET     / (com.graphhopper.http.resources.RootResource)
2021-12-20T09:28:31.973233133Z     GET     /health (com.graphhopper.resources.HealthcheckResource)
2021-12-20T09:28:31.973239342Z     GET     /i18n (com.graphhopper.resources.I18NResource)
2021-12-20T09:28:31.973245010Z     GET     /i18n/{locale} (com.graphhopper.resources.I18NResource)
2021-12-20T09:28:31.973251079Z     GET     /info (com.graphhopper.resources.InfoResource)
2021-12-20T09:28:31.973256865Z     GET     /isochrone (com.graphhopper.resources.IsochroneResource)
2021-12-20T09:28:31.973262520Z     POST    /match (com.graphhopper.resources.MapMatchingResource)
2021-12-20T09:28:31.973268435Z     GET     /mvt/{z}/{x}/{y}.mvt (com.graphhopper.resources.MVTResource)
2021-12-20T09:28:31.973274384Z     GET     /navigate/directions/v5/gh/{profile}/{coordinatesArray : .+} (com.graphhopper.navigation.NavigateResource)
2021-12-20T09:28:31.973280591Z     GET     /nearest (com.graphhopper.resources.NearestResource)
2021-12-20T09:28:31.973286499Z     GET     /route (com.graphhopper.resources.RouteResource)
2021-12-20T09:28:31.973291962Z     POST    /route (com.graphhopper.resources.RouteResource)
2021-12-20T09:28:31.973297691Z     GET     /spt (com.graphhopper.resources.SPTResource)
2021-12-20T09:28:31.973303238Z 
2021-12-20T09:28:31.975975068Z 2021-12-20 09:28:31.975 [main] INFO  o.e.j.server.handler.ContextHandler - Started i.d.j.MutableServletContextHandler@1ddc6db2{/,null,AVAILABLE}
2021-12-20T09:28:31.983337182Z 2021-12-20 09:28:31.982 [main] INFO  io.dropwizard.setup.AdminEnvironment - tasks = 
2021-12-20T09:28:31.983377196Z 
2021-12-20T09:28:31.983384392Z     POST    /tasks/log-level (io.dropwizard.servlets.tasks.LogConfigurationTask)
2021-12-20T09:28:31.983389925Z     POST    /tasks/gc (io.dropwizard.servlets.tasks.GarbageCollectionTask)
2021-12-20T09:28:31.983395539Z 
2021-12-20T09:28:31.984375890Z 2021-12-20 09:28:31.983 [main] INFO  o.e.j.server.handler.ContextHandler - Started i.d.j.MutableServletContextHandler@a6c54c3{/,null,AVAILABLE}
2021-12-20T09:28:31.996371903Z 2021-12-20 09:28:31.996 [main] INFO  o.e.jetty.server.AbstractConnector - Started application@70e889e9{HTTP/1.1, (http/1.1)}{0.0.0.0:8989}
2021-12-20T09:28:32.000617516Z 2021-12-20 09:28:31.999 [main] INFO  o.e.jetty.server.AbstractConnector - Started admin@418c020b{HTTP/1.1, (http/1.1)}{localhost:8990}
2021-12-20T09:28:32.000686170Z 2021-12-20 09:28:31.999 [main] INFO  org.eclipse.jetty.server.Server - Started @4030ms

Better url API in graphhopper.sh

The graphhopper.sh file has all kinds of quirks around bringing a osm pbf file.
After #11 I think we should change the implementation of getting the osm file such that:
if it starts with http or https the script will download the file and use it, otherwise assume it is in the file system in the given location.
This will avoid the need to force-download which is just counter intuitive command.
Mainly this part of the code:
https://github.com/IsraelHikingMap/graphhopper-docker-image-push/blob/main/graphhopper.sh#L227L239
But not only, there's a lot of code related to finding the file etc, which I believe can be simplified a lot.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.