GithubHelp home page GithubHelp logo

mapbox-cli-py's Introduction

mapbox-cli-py

Build Status Coverage Status

Command line interface to Mapbox Web Services based on https://github.com/mapbox/mapbox-sdk-py.

Installation

For users on OS X, we recommend installing with homebrew

$ brew install mapbox/cli/mapbox

For users familiar with Python and who already have pip installed on their system, you can create a virtual environment and install with

(venv)$ pip install mapboxcli

Installing locally without a virtual environment using

$ pip install --user mapboxcli

You'll then need to include ~/.local/bin in your $PATH.

Installing globally is not recommended but some users may want to do so under certain circumstances

$ sudo pip install mapboxcli

If you're interested in contributing, you'll want to install from master branch.

Setup

Use of the mapbox command line interface requires an access token. Your token is shown on the API access tokens page when you are logged in. The token can be provided on the command line

$ mapbox --access-token MY_TOKEN ...

or as an environment variable named MAPBOX_ACCESS_TOKEN

$ export MAPBOX_ACCESS_TOKEN=MY_TOKEN
$ mapbox ...

Usage

For any command that takes waypoints or features as an input you can either specify:

  • Coordinate pair(s) of the form "[0, 0]" or "0,0" or "0 0"
  • Sequence of GeoJSON features on stdin
  • GeoJSON FeatureCollection on stdin
  • Paths to GeoJSON file(s) containing either a single Feature or FeatureCollection.

Note that functions that accept points only, any non-point feature is filtered out.

directions

Usage: mapbox directions [OPTIONS] FEATURES...

  The Mapbox Directions API will show you how to get where you're going.

  mapbox directions "[0, 0]" "[1, 1]"

  An access token is required.  See "mapbox --help".

Options:
  --profile [mapbox/driving|mapbox/driving-traffic|mapbox/walking|mapbox/cycling]
                                  Routing profile
  --alternatives / --no-alternatives
                                  Whether to try to return alternative routes
  --geometries [geojson|polyline|polyline6]
                                  Format of returned geometry
  --overview [full|simplified|False]
                                  Type of returned overview geometry
  --steps / --no-steps            Whether to return steps and turn-by-turn
                                  instructions
  --continue-straight / --no-continue-straight
                                  Whether to see the allowed direction of
                                  travel when departing the original waypoint
  --waypoint-snapping TEXT        Controls waypoint snapping
  --annotations TEXT              Additional metadata along the route
  --language TEXT                 Language of returned turn-by-turn
                                  instructions
  -o, --output TEXT               Save output to a file
  --help                          Show this message and exit.

geocoding

Usage: mapbox geocoding [OPTIONS] [QUERY]

  This command returns places matching an address (forward mode) or places
  matching coordinates (reverse mode).

  In forward (the default) mode the query argument shall be an address such
  as '1600 pennsylvania ave nw'.

    $ mapbox geocoding '1600 pennsylvania ave nw'

  In reverse mode the query argument shall be a JSON encoded array of
  longitude and latitude (in that order) in decimal degrees.

    $ mapbox geocoding --reverse '[-77.4371, 37.5227]'

  An access token is required, see `mapbox --help`.

Options:
  --forward / --reverse           Perform a forward or reverse geocode.
                                  [default: forward]
  -i, --include                   Include HTTP headers in the output.
  --lat FLOAT                     Bias results toward this latitude (decimal
                                  degrees). --lon is also required.
  --lon FLOAT                     Bias results toward this longitude (decimal
                                  degrees). --lat is also required.
  -t, --place-type NAME           Restrict results to one or more place types.
  -o, --output TEXT               Save output to a file.
  -d, --dataset [mapbox.places|mapbox.places-permanent]
                                  Source dataset for geocoding, [default:
                                  mapbox.places]
  --country TEXT                  Restrict forward geocoding to specified
                                  country codes,comma-separated
  --bbox TEXT                     Restrict forward geocoding to specified
                                  bounding box,given in minX,minY,maxX,maxY
                                  coordinates.
  --features                      Return results as line-delimited GeoJSON
                                  Feature sequence, not a FeatureCollection
  --limit INTEGER                 Limit the number of returned features
  --help                          Show this message and exit.

mapmatching

Usage: mapbox mapmatching [OPTIONS] FEATURES...

  Mapbox Map Matching API lets you use snap your GPS traces to the
  OpenStreetMap road and path network.

        $ mapbox mapmatching trace.geojson

  An access token is required, see `mapbox --help`.

Options:
  --gps-precision INTEGER         Assumed precision of tracking device
                                  (default 4 meters)
  --profile [mapbox.driving|mapbox.cycling|mapbox.walking]
                                  Mapbox profile id
  --help                          Show this message and exit.

staticmap

Usage: mapbox staticmap [OPTIONS] MAPID OUTPUT

  Generate static map images from existing Mapbox map ids. Optionally
  overlay with geojson features.

    $ mapbox staticmap --features features.geojson mapbox.satellite out.png
    $ mapbox staticmap --lon -61.7 --lat 12.1 --zoom 12 mapbox.satellite
    out2.png

  An access token is required, see `mapbox --help`.

Options:
  --features TEXT              GeoJSON Features to render as overlay
  --lat FLOAT                  Latitude
  --lon FLOAT                  Longitude
  --zoom INTEGER               Zoom
  --size <INTEGER INTEGER>...  Image width and height in pixels
  --help                       Show this message and exit.

upload

Usage: mapbox upload [OPTIONS] TILESET [INFILE]

  Upload data to Mapbox accounts. All endpoints require authentication.
  Uploaded data lands at https://www.mapbox.com/studio/tilesets/ and can be used in new
  or existing projects.

  You can specify the tileset id and input file

    $ mapbox upload username.data mydata.geojson

  Or specify just the tileset id and take an input file on stdin

    $ cat mydata.geojson | mapbox upload username.data

  The --name option defines the title as it appears in Studio and defaults
  to the last part of the tileset id, e.g. "data"

  Note that the tileset must start with your username. An access token with
  upload scope is required, see `mapbox --help`.

Options:
  --name TEXT  Name for the data upload
  --help       Show this message and exit.

datasets

Usage: mapbox datasets [OPTIONS] COMMAND [ARGS]...

  Read and write GeoJSON from Mapbox-hosted datasets

  All endpoints require authentication. An access token with appropriate
  dataset scopes is required, see `mapbox --help`.

  Note that this API is currently a limited-access beta.

Options:
  --help  Show this message and exit.

Commands:
  create                 Create an empty dataset
  create-tileset         Generate a tileset from a dataset
  delete-dataset         Delete a dataset
  delete-feature         Delete a single feature from a dataset
  list                   List datasets
  list-features          List features in a dataset
  put-feature            Insert or update a single feature in a dataset
  read-dataset           Return information about a dataset
  read-feature           Read a single feature from a dataset
  update-dataset         Update information about a dataset

datasets list

Usage: mapbox datasets list [OPTIONS]

  List datasets.

  Prints a list of objects describing datasets.

      $ mapbox datasets list

  All endpoints require authentication. An access token with `datasets:read`
  scope is required, see `mapbox --help`.

Options:
  -o, --output TEXT  Save output to a file
  --help             Show this message and exit.

datasets create

Usage: mapbox datasets create [OPTIONS]

  Create a new dataset.

  Prints a JSON object containing the attributes of the new dataset.

      $ mapbox datasets create

  All endpoints require authentication. An access token with
  `datasets:write` scope is required, see `mapbox --help`.

Options:
  -n, --name TEXT         Name for the dataset
  -d, --description TEXT  Description for the dataset
  --help                  Show this message and exit.

datasets read-dataset

Usage: mapbox datasets read-dataset [OPTIONS] DATASET

  Read the attributes of a dataset.

  Prints a JSON object containing the attributes of a dataset. The
  attributes: owner (a Mapbox account), id (dataset id), created (Unix
  timestamp), modified (timestamp), name (string), and description (string).

      $ mapbox datasets read-dataset dataset-id

  All endpoints require authentication. An access token with `datasets:read`
  scope is required, see `mapbox --help`.

Options:
  -o, --output TEXT  Save output to a file
  --help             Show this message and exit.

datasets update-dataset

Usage: mapbox datasets update-dataset [OPTIONS] DATASET

  Update the name and description of a dataset.

  Prints a JSON object containing the updated dataset attributes.

      $ mapbox datasets update-dataset dataset-id

  All endpoints require authentication. An access token with
  `datasets:write` scope is required, see `mapbox --help`.

Options:
  -n, --name TEXT         Name for the dataset
  -d, --description TEXT  Description for the dataset
  --help                  Show this message and exit.

datasets delete-dataset

Usage: mapbox datasets delete-dataset [OPTIONS] DATASET

  Delete a dataset.

      $ mapbox datasets delete-dataset dataset-id

  All endpoints require authentication. An access token with
  `datasets:write` scope is required, see `mapbox --help`.

Options:
  --help  Show this message and exit.

datasets list-features

Usage: mapbox datasets list-features [OPTIONS] DATASET

  Get features of a dataset.

  Prints the features of the dataset as a GeoJSON feature collection.

      $ mapbox datasets list-features dataset-id

  All endpoints require authentication. An access token with `datasets:read`
  scope is required, see `mapbox --help`.

Options:
  -r, --reverse TEXT  Read features in reverse
  -s, --start TEXT    Feature id to begin reading from
  -l, --limit TEXT    Maximum number of features to return
  -o, --output TEXT   Save output to a file
  --help              Show this message and exit.

datasets put-feature

Usage: mapbox datasets put-feature [OPTIONS] DATASET FID [FEATURE]

  Create or update a dataset feature.

  The semantics of HTTP PUT apply: if the dataset has no feature with the
  given `fid` a new feature will be created. Returns a GeoJSON
  representation of the new or updated feature.

      $ mapbox datasets put-feature dataset-id feature-id 'geojson-feature'

  All endpoints require authentication. An access token with
  `datasets:write` scope is required, see `mapbox --help`.

Options:
  -i, --input TEXT  File containing a feature to put
  --help            Show this message and exit.

datasets read-feature

Usage: mapbox datasets read-feature [OPTIONS] DATASET FID

  Read a dataset feature.

  Prints a GeoJSON representation of the feature.

      $ mapbox datasets read-feature dataset-id feature-id

  All endpoints require authentication. An access token with `datasets:read`
  scope is required, see `mapbox --help`.

Options:
  -o, --output TEXT  Save output to a file
  --help             Show this message and exit.

datasets delete-feature

Usage: mapbox datasets delete-feature [OPTIONS] DATASET FID

  Delete a feature.

      $ mapbox datasets delete-feature dataset-id feature-id

  All endpoints require authentication. An access token with
  `datasets:write` scope is required, see `mapbox --help`.

Options:
  --help  Show this message and exit.

datasets create-tileset

Usage: mapbox datasets create-tileset [OPTIONS] DATASET TILESET

  Create a vector tileset from a dataset.

      $ mapbox datasets create-tileset dataset-id username.data

  Note that the tileset must start with your username and the dataset must
  be one that you own. To view processing status, visit
  https://www.mapbox.com/data/. You may not generate another tilesets from
  the same dataset until the first processing job has completed.

  All endpoints require authentication. An access token with `uploads:write`
  scope is required, see `mapbox --help`.

Options:
  -n, --name TEXT  Name for the tileset
  --help           Show this message and exit.

Alternative command syntax

When saving a fraction of a second matters you can call the mapboxcli module directly instead of using the installed program.

$ python -m mapboxcli --help

mapbox-cli-py's People

Contributors

andrewharvey avatar critical-path avatar darshanime avatar kant avatar lyzidiamond avatar mattficke avatar perrygeo avatar pratikyadav avatar rclark avatar sgillies avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mapbox-cli-py's Issues

No read method on valid file upload

Just got this somewhat-cryptic message on trying to upload this valid demo geojson file

| => mapbox upload --name landplanner.demopoints demo.geojson
Usage: mapbox upload [OPTIONS] TILESET [INFILE]

Error: Invalid value: Object `None` has no .read method, a file-like object is required

My key is properly configured (I gave it all 12 scopes just in case) and exported to $MAPBOX_ACCESS_TOKEN, and the file is valid, so I'm not sure what's going on here. Is my account missing some authorization globally?

MapboxAccessToken or MAPBOX_ACCESS_TOKEN?

-help says its MAPBOX_ACCESS_TOKEN but using this gave me the error

Before running this command you must set a Mapbox access token to an environment
variable named MapboxAccessToken. The access token must include the "uploads:write"
scope. Learn more about access tokens at https://www.mapbox.com/developers/api/#access-tokens.

So I tried export MapboxAccessToken= and this did the trick. I'm running v0.2.0.

cc @perrygeo

FutureWarning on re.split()

This is the only blocker for Python 3.5 support

With warnings.filterwarnings('error')

  File "/Users/mperry/work/mapbox-cli-py/mapboxcli/scripts/geocoding.py", line 114, in geocoding
    for lon, lat in map(coords_from_query, iter_query(query)):
  File "/Users/mperry/work/mapbox-cli-py/mapboxcli/scripts/geocoding.py", line 30, in coords_from_query
    vals = re.split(r"\,*\s*", query.strip())
  File "/Users/mperry/env/mapbox35/lib/python3.5/re.py", line 203, in split
    return _compile(pattern, flags).split(string, maxsplit)
FutureWarning: split() requires a non-empty pattern match.

Why is a future warning a blocker? Because for whatever reason (probably a bug in python 3.5 click), the future warning is captured in the output which messes up the CLI tests.

>       assert result.output.strip() == body
E       assert '/Users/mperr...71, 37.5227]}' == '{"query": [-7...71, 37.5227]}'
E         - /Users/mperry/env/mapbox35/lib/python3.5/re.py:203: FutureWarning: split() requires a non-empty pattern match.
E         -   return _compile(pattern, flags).split(string, maxsplit)
E           {"query": [-77.4371, 37.5227]}

tests/test_geocoding.py:70: AssertionError

Possible mapbox cli bug?

In closed issues I've found similar reports, but it seems that subsequent updates—I'm using v0.3.1—haven't dealt with the problem. I'm occasionally getting error messages (see screenshot below); in this case mapbox-cli-py is part of a shell script that runs within BitBar.

mapbox_error

Warning: could not load plugin

I installed with sudo pip install mapboxcli and I get the following when running mapbox. I haven't made much progress debugging myself. Any idea what's going on?

$ mapbox
Usage: mapbox [OPTIONS] COMMAND [ARGS]...

  This is the command line interface to Mapbox web services.

  Mapbox web services require an access token. Your token is shown on the
  https://www.mapbox.com/developers/api/ page when you are logged in. The
  token can be provided on the command line

    $ mapbox --access-token MY_TOKEN ...

  as an environment variable named MAPBOX_ACCESS_TOKEN (higher precedence)
  or MapboxAccessToken (lower precedence).

    $ export MAPBOX_ACCESS_TOKEN=MY_TOKEN
    $ mapbox ...

  or in a config file

    ; configuration file mapbox.ini
    [mapbox]
    access-token = MY_TOKEN

  The OS-dependent default config file path is something like

    ~/Library/Application Support/mapbox/mapbox.ini
    ~/.config/mapbox/mapbox.ini
    ~/.mapbox/mapbox.ini

Options:
  --version            Show the version and exit.
  -v, --verbose        Increase verbosity.
  -q, --quiet          Decrease verbosity.
  --access-token TEXT  Your Mapbox access token.
  -c, --config PATH    Config file
  --help               Show this message and exit.

Commands:
  config       † Warning: could not load plugin. See `mapbox config --help`.
  dataset      † Warning: could not load plugin. See `mapbox dataset --help`.
  directions   † Warning: could not load plugin. See `mapbox directions
               --help`.
  distance     † Warning: could not load plugin. See `mapbox distance --help`.
  geocoding    † Warning: could not load plugin. See `mapbox geocoding
               --help`.
  mapmatching  † Warning: could not load plugin. See `mapbox mapmatching
               --help`.
  staticmap    † Warning: could not load plugin. See `mapbox staticmap
               --help`.
  surface      † Warning: could not load plugin. See `mapbox surface --help`.
  upload       † Warning: could not load plugin. See `mapbox upload --help`.

mapbox returns driving instead of walking directions

I'm currently testing mapbox results for walking directions from one location to a public transportation stop/station in the Berlin area (Germany). However, mapbox returns only driving directions. The question is, if I'm doing something wrong, or if mapbox is simply incapable of handling such a request.

These are the commands:

MB_DIRECTIONS=$(mapbox directions "[$CURRENT_LONG, $CURRENT_LAT]" "[$STATION_LONG, $STATION_LAT]" --profile mapbox.walking --alternatives)
MB_INSTRUCTIONS=$(echo "$MB_DIRECTIONS" | jq '.routes[0].steps[] .maneuver.instruction' -M -r)
MB_DISTANCES=$(echo "$MB_DIRECTIONS" | jq '.routes[0].steps[] .distance' -M -r)

Variables for current lat/long and station lat/long are defined earlier.

This is the mapbox result (note the difference between linear and "walking" distance):

From: Herbert-von-Karajan-Straße 1, Berlin, Berlin 10785, Germany [52.510032,13.369693]
To: Potsdamer Straße, Berlin, Berlin 10785, Germany [52.507568,13.369186]

109     Head right
40      Turn right onto Tiergartenstraße (L 1137) at the end of the road
216     Turn right onto Ben-Gurion-Straße
153     Turn right onto Potsdamer Straße (B 1)
21      Turn left onto Potsdamer Straße
115     Turn right onto Potsdamer Straße (B 1) at the end of the road
0       You have arrived at your destination, on the right

Linear Distance: 275.9384 Meters
Walking Distance: 654 Meters
Approximate Walking Time: 8 Minutes

This is the (accurate) result in Apple Maps:
http://i.imgur.com/zLhpMQI.png

So Mapbox sends me the completely wrong way, north first, then east, then south, then onto the actual destination street, but not without a u-turn later on. These are clearly driving directions, and not even those are the fastest ones possible.

What's wrong here?

Upload: add ability to set data/tileset privacy

Currently, using upload still require user to manually go into Studio and set new tileset rights to public (default is private). Would be great to have a function added that sets privacy via CLI, especially in cases where upload replaces an existing tileset that is already exposed via styles.

maybe something like: mapbox upload <TILESET> <INFILE> (public,private)

Add subcommand for Directions API

First draft of proposed interface:

$ mbx directions --help
Usage: mbx directions [OPTIONS] WAYPOINTS...

  Calculate optimal route with turn-by-turn directions between up to 25
  waypoints.

    $ mbx directions "[-122.681032, 45.528334]" "[-122.71679, 45.525135]"

  An access token is required, see `mbx --help`.

Options:
  --profile [mapbox.driving|mapbox.walking|mapbox.cycling]
                                  Mapbox direction profile id
  --alternatives / --no-alternatives
                                  Generate alternative routes?
  --instructions [text|html]      Format for route instructions
  --geometry [geojson|polyline|false]
                                  Geometry encoding
  --steps / --no-steps            Include steps in the response
  --geojson / --no-geojson        Return geojson feature collection (default:
                                  full response json)
  -o, --output TEXT               Save output to a file.
  --help                          Show this message and exit.

Batch geocode

Continued from mapbox/mapbox-sdk-py#13 (comment)

Here's a proof-of-concept asyncio version of the batch_geocode CLI: https://gist.github.com/perrygeo/60633de1b2cd90b07b3d

Some issues with it

  • This only works on Python 3.4 (would need to port to Trollius to get python < 3.4 support).
  • no error handling
  • no match thresholds or resolving ambiguous results, it just takes the first feature
  • needs to respect rate limits
  • not clear if this would integrate with, replace or sit along side the current mapbox geocode

Handle uploads of existing s3 objects

It should be possible to accept an s3 uri as an upload object. With a file we transfer the data to s3 with stage and then calling create to start the processing. With an s3 uri we skip the transfer and call create directly

Questions

  • Does this work across accounts?
  • If not, there may be an intermediate option where we can use boto to transfer between s3 buckets (advantage: no need to pull data locally) then call create.

cc @rclark

world files for static maps api

We should add a --worldfile option the staticmaps command which would write a GDAL-recognizable world file alongside the image. This would help users to quickly grab a portion of their tilesets and load them into a local GIS viewer to compare to local data.

We'd need to determine the worldfile extension based on requested image.

Because the exact extent would only be know if bounds were provided, we could make it invalid to specify --worldfile with --auto. Or is the --auto behavior defined well enough to predict it?

Add a --first-feature option to the geocoder

Instead of getting a featurecollection with the first 5 results, many use cases at the command line will require selecting a single result, usually the first feature.

I've been using this pattern to get a GeoJSON Feature

mapbox geocoding "place" | jq -c .features[0]

Useful for geocoding multiple places and making a GeoJSON FeatureCollection

cat places.txt | parallel mapbox geocoding {} | jq -c .features[0] | fio collect

But jq isn't installed everywhere (it should be, it's awesome) and the arguments are not immediately clear.

Proposal

A --first-feature option to output the first, highest-ranked Feature (without line-breaks to facilitate streaming) rather than a FeatureCollection.

The parallel geocoding of multiple addresses becomes a little cleaner:

cat places.txt | parallel mapbox geocoding --first-feature {} | fio collect

The only reason I'd hesitate to implement this would be our design principles #10:

We'll rely on other programs like sed, awk, jq for finer parsing of responses from APIs at first, absorbing some of this when it makes sense.

So, does --first-feature make sense? Or do we just rely on jq?

Add mapbox-datasets-update-features with stdin or file input

I'd like to make this possible:

$ mapbox datasets create
$ fio cat a.shp b.shp c.shp | mapbox datasets update-features $id

reading a GeoJSON feature sequence from stdin, or reading from a GeoJSON feature collection file.

$ mapbox datasets create
$ fio cat a.shp b.shp c.shp | fio collect > lolwut.json
$ mapbox datasets update-features $id lolwut.json

Trouble setting access token

Hello,

I'm a junior developer and I'm playing around with the Mapbox CLI. I installed through Homebrew with no problems, but when I tried setting the token as per the instructions here on Github:

$ mapbox --access-token tokenfromstudiohomepagegoeshere12323342423442

I'm getting

Usage: mapbox [OPTIONS] COMMAND [ARGS]...
Error: Missing command.

Is there something basic I'm missing here?

Mapmatching for mutli-feature inputs

Currently the mapmatching API is limited to a single Feature with LineString geometry.

But that doesn't mean we have to limit the CLI to a single feature. What about taking multiple features and just calling the API multiple times (possibly asynchronously)?

Add subcommand for Upload API

First draft

$ mbx uploads --help
Usage: mbx uploads [OPTIONS] USERNAME INPUT TILESET

  Upload data to Mapbox accounts. All endpoints require authentication.
  Uploaded data lands at https://www.mapbox.com/data/ and can be used in new
  or existing projects.

    $ mbx uploads username data.geojson username.data

  An access token with upload scope is required, see `mbx --help`.

Options:
  --name TEXT  Name for the data upload
  --help       Show this message and exit.

CLI design principles and strategy

Replaces mapbox/mapbox-sdk-py#10.

We're developing a simple and easy to use CLI for the mapbox client module both as a service to customers and as a check on the usability of the module.

The CLI should be 99% about arg/opt definition and interoperation with other programs in the shell.

The CLI will have first class support for JSON and GeoJSON.

We'll rely on other programs like sed, awk, jq for finer parsing of responses from APIs at first, absorbing some of this when it makes sense.

Validation strategy

A general design question: What is our strategy for dealing with validation of input parameters?

The way it works now is

  • some known bad parameters are caught by the SDK and raise a python exception (the stacktrace is a bit intimidating and exposes the python underbelly to the CLI world)
  • other bad parameters are passed through to the API and the error message is conveyed (actually much cleaner errors, for example the static map size error in #19 returns Error: {"message":"Height must be between 1-1280."}) This puts a burden on the API when we could easily catch it before we hit the request.

I'm thinking the best approach is:

  1. do as much validation as possible in the SDK and raise a consistent exception type (ValueError seems too general)
  2. catch that exception at the CLI and re-raise using either a click.UsageError or a click.BadParameter

Alternatively, we could do all the validation at the CLI end and let the Python SDK play fast and loose, relying on the API response.

Doing the validation in both spots seems like a bad plan.

/cc @sgillies

Unpin mapbox-sdk-py requirement

We're developing this project outside of the mapbox-sdk-py project for product reasons, but want to keep it well synchronized. For 0.1, I set mapbox==0.5.0, which was a mistake because it prevents CLI users from getting mapbox-sdk-py bug fixes. I'll change it to mapbox>=0.5 and we'll continue like this, setting a new minimum version for every new mapbox-cli-py release.

Dataset API

The SDK PR is in the works. Meanwhile let's talk about the implications for the CLI...

As noted in the SDK planning ticket the Datasets API is a bit trickier as it involves two resources types: Datasets which are collections of Features. Then there are all of the standard "CRUD" operations on those resources.

How do we best represent these actions in the CLI? My initial thought is sub-sub-commands like:

mapbox dataset list-features
               read-features
               update-features
               delete-features
               list-datasets
               create-datasets
               read-datasets          

Add a mapbox-config command

Sort of like npm-config. It would tell the user

  • active access token
  • output verbosity

and whether these values are coming from environment variables or config file. It's also going to be handy for configuration testing.

Staticmap: size limit vs tile stitching for high resolution maps

There's an implicit 1280x1280 pixel limit on the size.

For example, creating wallpapers for my Thinkpad X1 Carbon fails with an API error:

mapbox staticmap --zoom 12 --size 2560 1440 --lat 52.500385 --lon 13.419779 mapbox.emerald out.png

It would be great, to either document that limit and catch the error already at the argparser level or provide transparent stitching, like @zugaldia does in his wallmapper (this may be out of scope, though).

1.0 release

  • Full API support
  • High quality documentation
  • Easy installation on Linux, OS X, Windows

Add subcommand for Distance API

First draft:

$ mbx distance --help
Usage: mbx distance [OPTIONS] WAYPOINTS...

  The Distance API returns all travel times between many points (also known
  as Distance Matrix). This is often used as input for solving routing
  optimization problems.

    $ mbx distance "[-122.681, 45.528]" "[-122.716, 45.525]" "[-122.691, 45.518]"

  The output is a json object with a "durations" key containing a 2D array
  of travel times between waypoints.

  An access token is required, see `mbx --help`.

Options:
  --profile [driving|walking|cycling]
                                  Mapbox direction profile id
  -o, --output TEXT               Save output to a file.
  --help                          Show this message and exit.

Packaging meta-ticket

A starting point for talking about how we're going to package and distribute this thing.

@tmcw mentioned that homebrew and apt-get should be the two primary targets.

I'd add that packaging a windows installer would really expand the potential audience. Whether it's worth the added maintenance burden (I don't even have access to a windows computer at the moment) is debatable.

  • identify target platforms
  • identify contributors with experience in those packaging systems
  • ...
  • everyone in the world has access to a one-line/one-click installer for command line access to Mapbox APIs

Input waypoints

The directions, distance and surface APIs all require input waypoints. Let's specify this in more detail...

At the python SDK level, all of the services take an iterable/list of GeoJSON-like feature mappings. At the command line, it's not clear to me exactly what the inputs will be.

The CLI will take x and normalize it to a list of features where x could potentially be

  • line-delimited GeoJSON features on the stdin stream (output of fio cat)
  • path to a GeoJSON file containing a featurecollection of points (probably the most common case?)
  • paths to GeoJSON files each containing point features (makes interop with geocoding easier since you're likely to get one feature per call)
  • stream or file of GeoJSON geometries
  • stream or multiple args with JSON arrays of coordinate pairs "[lat, lon]" (ala the input to reverse geocoder)
  • ... others? ...

So the question is, What types of inputs will we support for waypoints? On one hand I want to support the most common workflows even if that means accepting a bunch of different input formats. On the other hand, unix design principles - let other tools do the lifting to get data into our expected format.

And on a related note, if we're accepting GeoJSON and the service requires points, how should we handle other geometry types? Automatically explode multipoints and lines to points? Skip them? Be strict? Most of these validation decisions will likely be implemented at the SDK level but it might be useful to discuss them in this context.

cc @sgillies

Tests fail when MAPBOX_ACCESS_TOKEN is set

        result = runner.invoke(
            main_group,
            ['geocoding', '--reverse'],
            input='{0},{1}'.format(lon, lat),
            env={'MapboxAccessToken': 'bogus'})
>       assert result.exit_code == 0
E       assert -1 == 0
E        +  where -1 = <Result ConnectionError(u'Connection refused: GET https://api.mapbox.com/geoco...123xyz',)>.exit_code

tests/test_geocoding.py:93: AssertionError

where MAPBOX_ACCESS_TOKEN=123xyz is set in my environment. If I unset MAPBOX_ACCESS_TOKEN it works.

@sgillies is this just a matter of MAPBOX_ACCESS_TOKEN taking precedence over MapboxAccessToken?

geocoding fails with unicode input

continuation of mapbox/mapbox-sdk-py#110

The problem is in the CLI. Using the python SDK gives be similar results to the bites

>>> import mapbox
>>> result = mapbox.Geocoder().forward("Florianópolis, Brazil").json()
>>> for x in result['features']:
...     print x['text']
Florianópolis
Jardim Florianópolis
Rua Florianopólis
Rua Florianópolis
Rua Florianopolis

But using the CLI, I get an error

$ mapbox geocoding "Florianópolis, Brazil"
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py:1303: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  return ''.join(map(quoter, s))
Traceback (most recent call last):
  File "/Users/mperry/env/mapbox/bin/mapbox", line 9, in <module>
    load_entry_point('mapboxcli==0.2.0', 'console_scripts', 'mapbox')()
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/mperry/work/mapbox-cli-py/mapboxcli/scripts/geocoding.py", line 101, in geocoding
    q, types=place_type, lat=lat, lon=lon, country=country)
  File "/Users/mperry/work/mapbox-sdk-py/mapbox/services/geocoding.py", line 51, in forward
    dataset=self.name, query=address)
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/uritemplate/template.py", line 127, in expand
    return self._expand(var_dict, False)
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/uritemplate/template.py", line 89, in _expand
    expanded.update(v.expand(expansion))
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/uritemplate/variable.py", line 333, in expand
    expanded = expansion(name, value, opts['explode'], opts['prefix'])
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/uritemplate/variable.py", line 284, in _string_expansion
    return quote(value, self.safe)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 1303, in quote
    return ''.join(map(quoter, s))
KeyError: u'\xf3'

Failing test case pushed to the geounicode branch.

/cc @ingalls @sgillies

Add subcommand for Surface API

First draft

$ mbx surface --help
Usage: mbx surface [OPTIONS] MAPID LAYER FIELDS WAYPOINTS...

  Mapbox Surface API enables flexible querying of data stored in vector
  tiles on Mapbox, to create results like elevation profiles.

        $ mbx surface mapbox.mapbox-terrain-v1 contour ele \
        "[-122.681, 45.528]" "[-122.716, 45.525]"

  An access token is required, see `mbx --help`.

Options:
  -z, --zoom INTEGER              Zoom level to query (default: 14)
  --interpolate / --no-interpolate
                                  Weighted average interpolation (default:
                                  True)
  --geojson / --no-geojson        Return geojson feature collection (default:
                                  full response json)
  -o, --output TEXT               Save output to a file.
  --help                          Show this message and exit.

Upload arguments

For compatibility with mapbox-upload, the current upload CLI arguments follow suit. However, we might want to consider tweaking the design a bit

  1. mapbox upload DESTINATION-tileset and SOURCE-file - the current ordering is opposite what one would expect from unix tools like cp which generally follow the cmd source destination pattern. Switch 'em?
  2. we should consider making --name default to the basename of the file rather than None. This avoids having a bunch of "Untitled" tilesets in studio should you forget to supply the optional name.
  3. we should also consider making the username in the tileset id optional. We can infer the username from the access token and prepend it if there is no username supplied. This would help to write more reusable bash scripts intended to be run with different accounts.

Should we implement these changes? If so, do they belong here or could some be baked into SDK behavior?

Option to specify dataset for geocoding

We should include an option to use the mapbox.places-permanent dataset. The SDK supports it already, just a matter of specifying it in the service constructor.

0.3 release

Featuring new commands compatible with fio-cat and similar programs (#36).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.