duartemendes / aws-sam-api-proxy

Like running `sam local start-api` but keeping your containers warm and cozy

aws-sam-api-proxy 🚀

User and advocate of SAM? Improve development of API based Lambdas.

Execute your HTTP requests locally without hitting cold starts every time!

It's like running sam local start-api but keeping the containers (functions) around.

This tool will spin up a local server and proxy any incoming request to the expected lambda, here's how.

Features

• No cold starts on every request
  • in the first invocation a cold start is experienced, just like in the AWS environment
  • subsequent invocations are as fast as your code! 🏃‍♂️
• Context reutilization
• Always watching your distribution folder - rebuild your code and changes are propagated immediately
• Automatic discoverability of your Lambda functions
  • Global CodeUri, Runtime and Handlers are supported
  • Environment variables are passed through and can be overridden with --env-vars
  • All runtimes are supported
• Path parameters, querystring, body and headers are all passed through the request, with no changes
• Already supports Http Apis
• Supports local lambda function layers (ContentUri must be a relative path)
• Supports specifing your own docker socket path for added compatibility with Docker Desktop on Linux

Motivation

Developing AWS Lambdas and API Gateways locally is not usually the best experience.

I tend to develop with the help of unit tests but it just isn't enough. I always feel the need to make HTTP requests to properly test my template and code.

When I use sam local invoke or sam local start-api, even with the --skip-pull-image option, the cold starts on every request kills my mood and slows me down.

Another issue me and my team face, is that our APIs are behind a GraqhQL server. A query that should take less than a second to fulfil, takes about 6 to 7 seconds 🤨🔫

With this tool all this pain went away and I'm able to test my APIs much faster 😍

I hope somehow this tool is of any help to you. If you find a bug just open an issue or go ahead and fix it.

For more context you can read through aws-sam-cli/issues/239 and understand what led me to write this tool.

Requirements

The only dependency to use this CLI is docker. Make sure it's running.

CLI

Install

npm i aws-sam-api-proxy -g

Start API

Nice and easy:

cd ~/my-api
sam-proxy start my-api --port 3000

Or, with all the options available:

sam-proxy start my-api --port 3000 --base-path ~/my-api --template template.yaml --env-vars envVars.json --docker-network my_network --ref-overrides Env=dev,MyDynamoTable=http://localhost:8000 --log-level info --docker-socket-path /home/john-doe/.docker/desktop/docker.sock

Tearing down the house

For a specific API

sam-proxy teardown my-api

For all APIs

sam-proxy teardown-all

Environment Variables

Environment variables defined in your template.yaml either at the Function level or within Globals.Function are passed to the function containers. You can use --env-vars option to override variables pretty much as you would do using sam cli.

Overriding references

If the variable is a !Ref then the value will be resolved using the ref-overrides option passed to sam-proxy start.

Note that the references provided to sam-proxy start do not need to be defined inside the template's Parameters section. This allows you to fake resolution of other resources with !Ref. For example, passing --ref-overrides MyDynamoTable=http://localhost:8000 with a template as follows would allow your function to communicate with a local DynamoDB instance:

Resources:
  GetResources:
    Type: AWS::Serverless::Function
    Properties:
      Environment:
        Variables:
          DB_HOST: !Ref MyDynamoTable

  # Ignored by aws-sam-api-proxy:
  MyDynamoTable:
    Type: AWS::DynamoDB::Table
    # ...

More

sam-proxy --help

Running multiple APIs

If you want to run multiple APIs, the only detail to take in consideration is the port you choose to run your server on.

Why? All containers created by this tool will expose a port that is based on the server port.

Example:

Imagine your API has 4 endpoints and your server is running at port 3000.

4 containers are running, the following ports are being used: 3001, 3002, 3003 and 3004.

What you need to do, on the subsequent server, is to select an higher port. For this example, 3005 would be enough.

Template sample

My templates usually look something like this:

AWSTemplateFormatVersion: '2010-09-09'
Description: Resources API
Transform: AWS::Serverless-2016-10-31

Parameters:
  Env:
    Type: String
    AllowedValues:
      - dev
      - stg
      - prd

Globals:
  Function:
    Runtime: nodejs12.x
    MemorySize: 256
    Timeout: 10
    Environment:
      Variables:
        NODE_ENV: !Ref Env

Resources:
  GetResources:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: ./dist
      Handler: GetResourcesHandler.default
      Events:
        GetResourcesEvent:
          Type: Api
          Properties:
            Path: /resources
            Method: GET

Under the hood

Very succinctly, this tool does the following:

1. Parse your SAM template and your environment variables file if provided
2. Filter all functions triggered by Api or HttpApi events
3. Remove containers running for this API
4. Pull necessary docker images
5. Create and start containers for each Lambda function with the given environment variables
6. Spins up a server that acts as an API Gateway, proxying incoming requests to the expected containers

Keeping containers warm is possible due to the environment variable DOCKER_LAMBDA_STAY_OPEN that is given to the container on creation.

Logs

This tool will automatically log request details, duration and response status code.

If you want to check the logs of a function, just take a look at the container logs:

docker ps
docker logs get_resources_lambda

Credits

This tool wouldn't be possible without lambci/docker-lambda or dockerode. Thank you 🍻.

Roadmap

• Support JSON template
• Fill events missing data:
  • requestContext
  • multiValueHeaders
  • multiValueQueryStringParameters
  • cookies
  • stageVariables

aws-sam-api-proxy's Issues

Problem with path argument

Hi! First of all super great project!

Just doing some tentative but I have an issue with the path

sam-proxy start new-project-api --port 3000

TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received type undefined
    at validateString (internal/validators.js:112:11)
    at join (path.js:1040:7)
    at /usr/local/lib/node_modules/aws-sam-api-proxy/dist/serverlessFunctions/parseFromTemplate.js:81:32
    at Array.reduce (<anonymous>)
    at _default (/usr/local/lib/node_modules/aws-sam-api-proxy/dist/serverlessFunctions/parseFromTemplate.js:47:87)
    at _default (/usr/local/lib/node_modules/aws-sam-api-proxy/dist/index.js:71:73)
    at async Command.<anonymous> (/usr/local/lib/node_modules/aws-sam-api-proxy/dist/cli.js:37:3)

Adding separate start of docker container

Hi.
I have an issue when i try to run this in a docker-compose whit some other services. I have a sam-api-proxy image being built and run as a docker container.
The problem is the automated start of the lambda container. It would be nice to be able to start the lambda container that the proxy is forwarding to separately, so i can configure it and run it inside my docker-compose environment.
Now when I run sam-api-proxy inside a docker container, the lambda container that is started automatically doesn't have access to the filesystem inside the sam-api-proxy container.

It could also be that I'm missing something, I'm fairly new to using docker.
Apart from my issue, this works really great just running locally for testing. Nice work

0.0.7 release doesn't include code changes

Hi, I've just tried to switch from using my fork to using the main repo after you merged #6, but it looks like dist is still unchanged from previous versions.

Feature Request: Read env vars from template

It's kind of annoying to have to add environment variables to a separate envVars.json file as well as to template.yaml - especially the ones that can otherwise be defined just once across all functions in Globals.Function.Environment. This would probably require adding some kind of support for !Sub command, so that we can pluck env values out of the host environment and thereby use local development values without polluting the template.yaml file.

Problems with Response Header

Hi,

We are experimenting issue with response headers (goal is work with CORS)

This code in our lambda function but we are not able to see headers in the answer

awsProxyResponse.getHeaders().put(
	"Access-Control-Allow-Origin", "*"
);
awsProxyResponse.getHeaders().put(
	"Access-Control-Allow-Credentials", "true"
);
awsProxyResponse.getHeaders().put(
	"Access-Control-Allow-Headers",
	"Access-Control-Allow-Origin, Content-Type"
);
awsProxyResponse.getHeaders().put(
	"Access-Control-Allow-Methods", "GET,POST,PUT,DELETE,OPTIONS,HEAD"
);

We tried also adding this in yaml file

    Events:
       MyApp:
         Type: AWS::Serverless::Api
         Properties:
           Path: /myapp/{proxy+}
           Method: any
           Cors:
             AllowMethods: "'GET,POST,PUT,DELETE,OPTIONS,HEAD'"
             AllowHeaders: "'Access-Control-Allow-Headers, Access-Control-Allow-Origin, Content-Type'"
             AllowOrigin: "'*'"
             AllowCredentials: True

Thanks a lot for any help

Add support for layers

In response to this original issue and comment.

I'm not quite sure what you're doing behind the scenes, but I imagine if you're mounting a lambci container for each function in the SAM template, you could check for any local layers and mount a volume between the layer path and /opt on the lambci container.