jamesbursa / geobuf

A compact binary encoding for geographic data.

Geobuf

Geobuf is a compact binary encoding for geographic data.

Geobuf provides lossless compression of GeoJSON data into protocol buffers. Advantages over using GeoJSON alone:

• Very compact: typically makes GeoJSON 6-8 times smaller.
• 2-2.5x smaller even when comparing gzipped sizes.
• Very fast encoding and decoding — even faster than native JSON parse/stringify.
• Can accommodate any GeoJSON data, including extensions with arbitrary properties.

The encoding format also potentially allows:

• Easy incremental parsing — get features out as you read them, without the need to build in-memory representation of the whole data.
• Partial reads — read only the parts you actually need, skipping the rest.

Think of this as an attempt to design a simple, modern Shapefile successor that works seamlessly with GeoJSON. Unlike Mapbox Vector Tiles, it aims for lossless compression of datasets — without tiling, projecting coordinates, flattening geometries or stripping properties.

Note that the encoding schema is not stable yet — it may still change as we get community feedback and discover new ways to improve it.

Sample compression sizes

                | normal    | gzipped

------------------- | --------- | -------- us-zips.json | 101.85 MB | 26.67 MB us-zips.pbf | 12.24 MB | 10.48 MB idaho.json | 10.92 MB | 2.57 MB idaho.pbf | 1.37 MB | 1.17 MB

API

encode

var buffer = geobuf.encode(geojson, new Pbf());

Given a GeoJSON object and a Pbf object to write to, returns a Geobuf as a Buffer object in Node or UInt8Array object in browsers.

decode

var geojson = geobuf.decode(new Pbf(data));

Given a Pbf object with Geobuf data, return a GeoJSON object.

Install

Node and Browserify:

npm install geobuf

Making a browser build:

npm install
npm run build-dev # dist/geobuf-dev.js (development build)
npm run build-min # dist/geobuf.js (minified production build)

Command Line

npm install -g geobuf

Installs these nifty binaries:

• geobuf2json: turn Geobuf from stdin or specified file to GeoJSON on stdout
• json2geobuf: turn GeoJSON from stdin or specified file to Geobuf on stdout
• shp2geobuf: given a Shapefile filename, send Geobuf on stdout

json2geobuf data.json > data.pbf
shp2geobuf myshapefile > data.pbf
geobuf2json data.pbf > data.json

Note that for big files, geobuf2json command can be pretty slow, but the bottleneck is not the decoding, but the native JSON.stringify on the decoded object to pipe it as a string to stdout. On some files, this step may take 40 times more time than actual decoding.

See Also

• geojsonp — the prototype that led to this project
• pygeobuf — Python implementation of Geobuf
• twkb — a geospatial binary encoding that doesn't support topology and doesn't encode any non-geographic properties besides id
• vector-tile-spec
• topojson — an extension of GeoJSON that supports topology
• WKT and WKB — popular in databases
• EWKB — a popular superset of WKB