This project configures Amazon Web Services, in particular API Gateway, Simple Queue Service, and Lambda, to create an API endpoint which places incoming requests into a queue ("mailbox").
Requests can be popped off of this queue from behind a firewall and delivered to an internal service, without exposing that service to the Internet.
First, configure a virtual environment and install dependencies:
virtualenv --python=python3 .venv
.venv/bin/activate
pip install -r requirements.txt
command -v rehash && rehash
Configure the AWS command line with your credentials and default region. Other configuration methods, such as environment variables, are available.
Security considerations: Do not leave your AWS credentials lying around in ~/.aws
or your shell history file on the server. The configure
stage generates credentials with limited privileges that can be safely deployed instead.
The configure
subcommand will automatically provision the necessary AWS resources and return credentials and an HTTP endpoint URL:
$ python webhook-mailbox.py configure
Configured queue webhook-mailbox-388cfb with the following credentials:
AWS_ACCESS_KEY_ID=AKIATM5TRIWFWDM3I4GX
AWS_SECRET_ACCESS_KEY=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
URL: https://jofx96r5z4.execute-api.us-east-1.amazonaws.com/prod/
The watch
subcommand watches the queue. When a new message, it will issue a request to the given endpoint URL, copying the HTTP method, headers, URL parameters, and request body. The path is discarded.
AWS_ACCESS_KEY_ID= \
AWS_SECRET_ACCESS_KEY= \
python webhook-mailbox.py watch \
webhook-mailbox-e05a69 \
https://server.local/endpoint
The Generic Webhook Trigger plugin for Jenkins can be used to set up an HTTP endpoint to trigger a build.
Enable the trigger on your Jenkins project. It's a good idea to assign a token to it.
On the repository server, the hook URL should look like:
https://xyzzy.execute-api.us-east-1.amazonaws.com/prod/?token=project-token
For additional configuration, such as to set which branch to build, see these examples.
Then, run the mailbox watcher with the corresponding queue name and the Jenkins trigger invocation URL:
AWS_ACCESS_KEY_ID= \
AWS_SECRET_ACCESS_KEY= \
python webhook-mailbox.py watch \
webhook-mailbox-e05a69 \
https://jenkins.local/generic-webhook-trigger/invoke
A Dockerfile
is provided for convenience.
Note that currently -it
is required, because otherwise the program will hang and be unresponsive to keyboard input. This is a bug.
docker run \
-it \
-e AWS_ACCESS_KEY_ID= \
-e AWS_SECRET_ACCESS_KEY= \
-e AWS_DEFAULT_REGION=us-east-1 \
-e WEBHOOK_QUEUE_NAME=webhook-mailbox-e05a69 \
-e WEBHOOK_FORWARD_URL=https://jenkins.local/generic-webhook-trigger/invoke \
whoi/webhook-mailbox
It is possible the configuration could be done through CloudFormation or Terraform more robustly.