numberemailer-webhook

Demo project for experimenting with SendGrid activity event webhooks using AWS SAM with Python and Flask REST API endpoint as webhook.

This project can be used on its own but, was originally conceived to work with the following peer projects:

numberemailer-sender is a SAM Python / Flask REST API for posting a random number along with an email address to which the SendGrid Email API is used to sending a random number email.
numberemailer is a static HTML website designed to be hosted on AWS S3 static website bucket and integrated with Segment.io for page and click tracking as well as generating random numbers and sending those number to a user specified email address via the numberemailer-sender project utilizing SendGrid

Webhook Endpoint

Method: POST
URI: /events/

accepts a POST request and saves the contents of the POST JSON body to AWS S3 bucket

Running Locally

An environment variables config file named .env.local in the root of the project. See .env.example file for details.

TODO: Add support for local environment variables to specify an IAM profile / role with permissions for writing to a non-prod S3 bucket for local dev/test work.

There is helper unix bash script for building and running the app in local api mode. This requires a .env.local to be present with values for the environment variables.

./start-local.sh

This will eventually output an endpoint you can hit on your local host such as http://localhost:3000

Test with HTTPie like so

SENT_AT=$(date '+%Y-%m-%d %H:%M:%S')
http -j POST http://localhost:3000/events/ [email protected] sent=$SENT_AT

Deploy to AWS

An environment variables file named .env is required to be present at the root of the project and will be used in conjunction with a unix helper script named deploy-wh.sh to deploy to AWS. See .env.example for the list of environment variables required.

Run the deploy-wh.sh script as shown below which will build the project then step you through an interactive guided deployment.

./deploy-wh.sh

After deploying the project it will output a key pieces of information:

SendGridWebHookApi URL
SendGridWebHookFunctionIamRole IAM Role ARN
SendGridWebHookFunction ARN

Make note of the SendGridWebHookFunctionIamRole value as that is used to create an S3 bucket and assign the appropriate IAM policy for this Lambda Function to write the SendGrid WebHook events to.

Update the .env file's variable LAMBDA_IAM_ROLE_ARN with the above mentioned Lambda IAM Role ARN value before executing the create-bucket.sh script.

./create-bucket.sh

You should now be able to POST the API Gateway URL and have the JSON body saved to a S3 object file with S3 key of bucket-name/sendgrid-events/year/month/day/hour/HHMMSS.json

About SAM

The Serverless Application Model Command Line Interface (SAM CLI) is an extension of the AWS CLI that adds functionality for building and testing Lambda applications. It uses Docker to run your functions in an Amazon Linux environment that matches Lambda. It can also emulate your application's build environment and API.

To use the SAM CLI, you need the following tools.

SAM CLI - Install the SAM CLI
Python 3 installed
Docker - Install Docker community edition

Test a single function by invoking it directly with a test event. An event is a JSON document that represents the input that the function receives from the event source. Test events are included in the events folder in this project.

Add a resource to your application

The application uses AWS Serverless Application Model (AWS SAM) to define application resources. AWS SAM is an extension of AWS CloudFormation with a simpler syntax for configuring common serverless application resources such as functions, triggers, and APIs. For resources not included in the SAM specification, you can use standard AWS CloudFormation resource types.

Fetch, tail, and filter Lambda function logs

To simplify troubleshooting, SAM CLI has a command called sam logs. sam logs lets you fetch logs generated by your deployed Lambda function from the command line. In addition to printing the logs on the terminal, this command has several nifty features to help you quickly find the bug.

NOTE: This command works for all AWS Lambda functions; not just the ones you deploy using SAM.

numberemailer-webhook$ sam logs -n SendGridWebHookFunction --stack-name numberemailer-webhook --tail

You can find more information and examples about filtering Lambda function logs in the SAM CLI Documentation.

Unit tests

Tests are defined in the tests folder in this project. Use PIP to install the pytest and run unit tests.

numberemailer-webhook$ pip install pytest pytest-mock --user
numberemailer-webhook$ python -m pytest tests/ -v

TODO: Add API unit tests

Cleanup

To delete the sample application that you created, use the AWS CLI. Assuming you used your project name for the stack name, you can run the following:

aws cloudformation delete-stack --stack-name stackname

Resources

See the AWS SAM developer guide for an introduction to SAM specification, the SAM CLI, and serverless application concepts.

Next, you can use AWS Serverless Application Repository to deploy ready to use Apps that go beyond hello world samples and learn how authors developed their applications: AWS Serverless Application Repository main page

amcquistan / numberemailer-webhook Goto Github PK