azavea / lambnik

AWS Lambda mapnik renderer for PostGIS datasources

Lambnik - Lambda based mapnik rendering

Proof of concept mapnik renderer for PostGIS datasources using Python. Ie, don't run windshaft.

Instructions

1. Have recent versions of docker and docker-compose installed
2. MacOS only: sudo mkdir -p /opt/dbdata/lambnik && sudo chown "${USER}":staff /opt/dbdata/lambink
3. MacOS only: Open Docker preferences and add /opt/dbdata to the list in the File Sharing tab
4. Build the containers and populate the database: ./scripts/update.sh
5. Run the local Chalice server: ./scripts/server.sh

Test image now available at http://localhost:9000/tile.png

Deploying

Currently only a staging environment exists. To deploy it, first you'll need to setup an AWS CLI profile:

aws configure --profile=azavea-rd

Follow the prompts, entering your user credentials. If you don't have credentials for this environment, contact one of the project committers.

Once your profile is configured correctly, you can deploy with:

./scripts/publish.sh

The deployment will take a few minutes. When it's done, sign into the AWS Lambda console. Choose "functions" in the sidebar on the left, then the function "lambnik-tiler-staging".

Finally, expand the "Environment variables" section, and update the value of POSTGRES_PASSWORD. If you don't already have the password, it can be retrieved from the company password manager. Search for "lambnik".

You'll need to update the DB password after every deployment. Chalice does not yet support templating ENV variables in config.json.

lambnik's Issues

Configure map styles via Mapnik XML

Convert the mapnik rendering function to generically load map definitions and render tiles for multiple defined layers.

Manage RDS connections

There is an upper limit to connections the RDS service can accommodate. Lambnik originally established the PostGIS connection at the python-module level, which allowed a form of connection pooling across requests. However, the mapnik datasource object does not allow updating the table definition after instantiation, which prevents altering the query based on request-supplied parameters. The connections are now established during the execution of the Lambda function, meaning there is a 1:1 correspondence to invocation and connection.

Determine some mechanism to reduce the overall number of connections made while still allowing alterations to the query.

Long thread on the AWS forums where a number of folks are complaining about this.
While it's additional infrastructure, one of the few potentially useful suggestions in the thread was to put pgbouncer between RDS and Lambda.

Allow filtering features when rendering

Add the ability to perform request based attribute filtering.