mapbox / osm-mend

Mend referential integrity in OpenStreetMap data files

osm-mend

Fix referential integrity issues with any OpenStreetMap data file based on the current state of data accessible via the OpenStreetMap API.

Why?

Extracts of data from OpenStreetMap may not ensure referential integrity. There are several reasons that this might be the case, and generally applications that depend on OpenStreetMap extracts are tolerant of these situations. This tool exists for scenarios where you need to be certain about the referential integrity of a data file.

How does it work?

The script relies heavily on the osmium CLI tool to find missing references and apply changesets to a data file in order to fix those missing references. The steps in process are as follows:

• find missing references via osmium-tool check-refs
• gather the XML of parent elements that have missing references via osmium-tool getid
• look up the current state of missing references in the OpenStreetMap API
• build an .osc.xml changeset to apply to the original file. This changeset:
  • creates any element that was missing in the data file but found in the API
  • modifies parent elements that reference an element that has been deleted or never existed (API returned 404 or 410).
• applies the changeset to the original data file via osmium-tool apply-changes

How do I use it?

Install the tool, then use --help for more information.

$ npm install -g osm-mend
$ osm-mend --help
USAGE: osm-mend <input file> <output file>

  Note that both an input file and an output location are required.
  Existing files in the output location will be overwritten.

Please note that the script relies on pre-built osmium-tool binary that is not provided for Windows users.

osm-mend's Issues

Use osmium-tool via mason

v0.8.0 of mason contains the latest release of osmium-tool. Instead of vendoring (which too easily allows old versions to rot in place silently), let's start pulling versioned releases. We can follow the lead of directions-extracts: https://github.com/mapbox/directions-extracts/blob/ae86aa5dd2aa14c178ebc1d53a9fc6adb19891ff/Dockerfile#L26-L32

/cc @rclark