miguelgrinberg / slam

Serverless deployment of Python APIs

slam

Slam is a simple command-line utility that makes it very easy to do serverless deployments of Python web applications to AWS, using the Lambda and API Gateway services. In particular, Slam supports the transparent deployment of WSGI compliant applications.

If you have 10 minutes to spare, you can watch a (now slightly outdated) screencast in which the main features of slam are demonstrated:

Or if you prefer something more formal, have a look at the documentation, which also includes a written tutorial.

slam's Issues

Getting an exit code 100: virtualenv error on build

I'm getting a few issues on build/deploy:

When running slam build or slam deploy, I get the following error:

Building lambda package...
b"Using base prefix '/usr/local/bin/../Cellar/python3/3.6.2/bin/../Frameworks/Python.framework/Versions/3.6'\nNew python executable in /Users/Work/Projects/test-slam-api/.slam/venv/bin/python3\nAlso creating executable in /Users/Work/Projects/test-slam-api/.slam/venv/bin/python\nERROR: The executable /Users/Work/Projects/test-slam-api/.slam/venv/bin/python3 is not functioning\nERROR: It thinks sys.prefix is '/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6' (should be '/Users/scuerda/Work/Projects/OTHER/birth-navigation/births-api/.slam/venv')\nERROR: virtualenv is not compatible with this system or executable\n"
Runtime error: Command failed with exit code 100.
See file slam_error.log for additional details.

2017-09-06 13:32:39,422 - slam - ERROR - [merry] Exception caught
Traceback (most recent call last):
  File "/Users/Work/Projects/test-slam-api/venv/lib/python3.6/site-packages/merry.py", line 26, in wrapper
    ret = f(*args, **kwargs)
  File "/Users/Work/Projects/test-slam-api/venv/lib/python3.6/site-packages/climax.py", line 194, in wrapper
    ctx = func(**filtered_args)
  File "/Users/Work/Projects/test-slam-api/venv/lib/python3.6/site-packages/slam/cli.py", line 321, in build
    package = _build(config, rebuild_deps=rebuild_deps)
  File "/Users/Work/Projects/test-slam-api/venv/lib/python3.6/site-packages/slam/cli.py", line 235, in _build
    _run_command('virtualenv .slam/venv')
  File "/Users/Work/Projects/test-slam-api/venv/lib/python3.6/site-packages/slam/cli.py", line 173, in _run_command
    proc.returncode)))
RuntimeError: Command failed with exit code 100.

I'm guessing that this is just an upstream issue with homebrew and how it's playing with the cli tools, but I might be missing something?

Add cli option to specify the name of the yaml config file to use

S3 Bucket naming conventions have changed

The function '_ensure_bucket_exists(s3, bucket, region)' in 'cli.py' fails to create an S3 bucket and raises an exception:

Unexpected error: An error occurred (InvalidBucketName) when calling the CreateBucket operation: The specified bucket is not valid. (ClientError)

I think this happens because one can no longer use capital letters in S3 bucket names, according to the most recent AWS-docs: https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html

Bucket names can consist only of lowercase letters, numbers, dots (.), and hyphens (-).

In the code (init(), line 119), it says:

random.choice(string.ascii_uppercase + string.digits)

Changing this line to:

random.choice(string.ascii_lowercase + string.digits)

makes the S3-bucket creation operation succeed.

Users Forum?

Miguel,
Is there a forum you would direct users to for discussing the implementation and development of “Slam”?
Thanks,
Tim

Issues with zip file missing dependencies

Having a similar issue to what @ThaWeatherman outlined in #9 where the output of slam build (zipfile) does not have the project dependencies.

I looked for similar issues within the lambda-uploader issue tracker and saw this rackerlabs/lambda-uploader#129. I tried to recreate my local project virtualenv, but only install slam instead of my full requirements.txt but that didn't help resolve the issue.

I'm using Python 3.6.2 via pyenv on OSX if that matters.

Here is my project repo: https://github.com/CT-Data-Collaborative/birth-data-api

Thanks in advance.

'async' (and 'invoke') are now Python keywords.

Running Slam on Python 3.8.1 fails with a Syntax Error because of a command line option and corresponding variable in main() called 'async', which is now a reserved Python keyword (from Python 3.7, I think).

To test, I changed the 'async' option to 'nowait', since I lack imagination.

This seems to work OK.

expose lambda version as an environment variable

What is between it and the Docker difference

thank you Mr. Miguel
i want ask .
Is this an alternative to the construction worker

thank you

flask module missing?

Hello
The tool seem quite powerful, I just have hard time getting the first run:
I run the slam advanced tutorial and seem to have the flask module missing, how can that be?
I run the files from the example and get:

slam logs --tail

Oct 01 21:42:39 START RequestId: af282c02-a6e0-11e7-a2f3-5b31c4c6f4b7 Version: $LATEST
Oct 01 21:42:39 (af271a63-a6e0-11e7-9075-cf62f417714a) Verifying Usage Plan for request: af271a63-a6e0-11e7-9075-cf62f417714a. API Key: API Stage: wgbbbxjdr0/dev
Oct 01 21:42:39 (af271a63-a6e0-11e7-9075-cf62f417714a) API Key authorized because method 'ANY /{proxy+}' does not require API Key. Request will not contribute to throttle or quota limits
Oct 01 21:42:39 (af271a63-a6e0-11e7-9075-cf62f417714a) Usage Plan check succeeded for API Key and API Stage wgbbbxjdr0/dev
Oct 01 21:42:39 (af271a63-a6e0-11e7-9075-cf62f417714a) Starting execution for request: af271a63-a6e0-11e7-9075-cf62f417714a
Oct 01 21:42:39 (af271a63-a6e0-11e7-9075-cf62f417714a) HTTP Method: GET, Resource Path: /tasks
Oct 01 21:42:40 (af271a63-a6e0-11e7-9075-cf62f417714a) Execution failed due to configuration error: Malformed Lambda proxy response
Oct 01 21:42:40 (af271a63-a6e0-11e7-9075-cf62f417714a) Method completed with status: 502
Oct 01 21:42:40 No module named 'flask': ModuleNotFoundError
Traceback (most recent call last):
File "/var/task/handler.py", line 39, in lambda_handler
from tasks_api import app as app # noqa
File "/var/task/tasks_api.py", line 6, in
from flask import Flask, jsonify, abort, request, make_response, url_for
ModuleNotFoundError: No module named 'flask'
Oct 01 21:42:40 END RequestId: af282c02-a6e0-11e7-a2f3-5b31c4c6f4b7
Oct 01 21:42:40 REPORT RequestId: af282c02-a6e0-11e7-a2f3-5b31c4c6f4b7 Duration: 217.80 ms Billed Duration: 300 ms Memory Size: 128 MB Max Memory Used: 32 MB

What can be corrected?

Thanks for your help!

edit: currently using python 3.5.2 could that be too old?

separate the deploy and publish options into separate commands

slam deploy currently is used to deploy to the dev stage, and also to publish a numbered version to another stage. This last action should be moved to a slam publish command, to make things more clear.

AWS Credentials

I have multiple AWS credentials in my computer. How can I specify the profile that I want slam to use?

Failing to deploy

(.venv)root@seandroplet:~/infant
$ slam deploy
Building lambda package...
Unexpected error: [Errno 2] No such file or directory: '/root/infant/.lambda_uploader_temp/lambda_package/.venv/lib/python3.5/types.py' (FileNotFoundError)
See file slam_error.log for additional details.

error log:

2017-06-18 04:12:50,278 - slam - ERROR - [merry] Exception caught
Traceback (most recent call last):
  File "/root/infant/.venv/lib/python3.5/site-packages/merry.py", line 26, in wrapper
    ret = f(*args, **kwargs)
  File "/root/infant/.venv/lib/python3.5/site-packages/climax.py", line 194, in wrapper
    ctx = func(**filtered_args)
  File "/root/infant/.venv/lib/python3.5/site-packages/slam/cli.py", line 348, in deploy
    lambda_package = _build(config, rebuild_deps=rebuild_deps)
  File "/root/infant/.venv/lib/python3.5/site-packages/slam/cli.py", line 236, in _build
    zipfile_name=package)
  File "/root/infant/.venv/lib/python3.5/site-packages/lambda_uploader/package.py", line 49, in build_package
    pkg.build(ignore)
  File "/root/infant/.venv/lib/python3.5/site-packages/lambda_uploader/package.py", line 77, in build
    self.package(ignore)
  File "/root/infant/.venv/lib/python3.5/site-packages/lambda_uploader/package.py", line 260, in package
    self._create_zip(package)
  File "/root/infant/.venv/lib/python3.5/site-packages/lambda_uploader/package.py", line 273, in _create_zip
   zf.write(absname, arcname)
  File "/usr/lib/python3.5/zipfile.py", line 1433, in write
    st = os.stat(filename)
FileNotFoundError: [Errno 2] No such file or directory: '/root/infant/.lambda_uploader_temp/lambda_package/.venv/lib/python3.5/types.py'

Seems to exist.

$ find . -name types.py
./.lambda_uploader_temp/lambda_package/.venv/lib/python3.5/types.py
./.lambda_uploader_temp/lambda_package/.venv/lib/python3.5/site-packages/boto3/dynamodb/types.py
./.lambda_uploader_temp/lambda_package/.venv/lib/python3.5/site-packages/click/types.py
./.lambda_uploader_temp/lambda_package/.venv/lib/python3.5/site-packages/pynamodb/types.py
./.venv/lib/python3.5/types.py
./.venv/lib/python3.5/site-packages/boto3/dynamodb/types.py
./.venv/lib/python3.5/site-packages/click/types.py
./.venv/lib/python3.5/site-packages/pynamodb/types.py
./.slam/venv/lib/python3.5/types.py
./.slam/venv/lib/python3.5/site-packages/boto3/dynamodb/types.py
./.slam/venv/lib/python3.5/site-packages/click/types.py
./.slam/venv/lib/python3.5/site-packages/pynamodb/types.py

DynamoDB table config yaml file (help/question)

i'm pretty new on using Flask and i encountered this error.
is it related to the lambda function generated by slam or what?

HTTP/1.1 502 Bad Gateway
Connection: keep-alive
Content-Length: 36
Content-Type: application/json
Date: Mon, 22 May 2017 08:12:29 GMT
Via: 1.1 ae7118021d10200685649b5e21d1bf9f.cloudfront.net (CloudFront)
X-Amz-Cf-Id: AbdRCe-NRfOJRACxZGUvLelG002kNugyfvgBhfBl5OFchVcOM1bSBQ==
X-Cache: Error from cloudfront
x-amzn-RequestId: 65d6c489-3ec6-11e7-97bc-e962a70966bf

{
    "message": "Internal server error"
}

Enhancements

• Enable API-Gateway logging on a per-stage basis

• Delete CloudWatch logs

• Refactor WSGI support as a plugin

• Create function type deployments (w/o API Gateway)

• Add custom resources/outputs to CFN template

• Add custom policies to Lambda function

• Unit test CFN generation

• Unit test DynamoDB plugin

• Add VPC configuration for Lambda

• Error handling in CLI

• Dump and/or tail logs

• Support Python 3

• Enable API key auth (too buggy for now)

• Lambda events

• Invoke CLI scripts over Lambda

• Static files plugin, hosted on S3

• Custom API-Gateway domains and certificates

the dynamodb permissions on the lambda are too wide

only the tables allocated for the service should be allowed, but not others

I'm getting InvalidBucketName when running under an aws account with restricted permissions

Hi @miguelgrinberg, thanks for creating slam!
I wanted to try this out so I followed the instructions of the first tutorial (fizzbuzz). I'm getting an error which is possibly related to the fact that I'm using a child aws account which does not have permissions to certain actions. I did not try with the root account yet, but I thought it would be nice to have a --verbose option to see what happened.
Here is the error log

2018-06-11 15:48:26,573 - slam - ERROR - [merry] Exception caught
Traceback (most recent call last):
  File "~/fizz/venv/lib/python3.6/site-packages/slam/cli.py", line 256, in _ensure_bucket_exists
    s3.head_bucket(Bucket=bucket)
  File "~/fizz/venv/lib/python3.6/site-packages/botocore/client.py", line 314, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "~/fizz/venv/lib/python3.6/site-packages/botocore/client.py", line 612, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (404) when calling the HeadBucket operation: Not Found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "~/fizz/venv/lib/python3.6/site-packages/merry.py", line 26, in wrapper
    ret = f(*args, **kwargs)
  File "~/fizz/venv/lib/python3.6/site-packages/climax.py", line 194, in wrapper
    ctx = func(**filtered_args)
  File "~/fizz/venv/lib/python3.6/site-packages/slam/cli.py", line 368, in deploy
    _ensure_bucket_exists(s3, bucket, region)
  File "~/fizz/venv/lib/python3.6/site-packages/slam/cli.py", line 262, in _ensure_bucket_exists
    s3.create_bucket(Bucket=bucket)
  File "~/fizz/venv/lib/python3.6/site-packages/botocore/client.py", line 314, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "~/fizz/venv/lib/python3.6/site-packages/botocore/client.py", line 612, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidBucketName) when calling the CreateBucket operation: The specified bucket is not valid.