The Saga Pattern was published in 1987 by Hector Garcia-Molina and Kennet Salem from Princeton University. It's a design pattern that helps establish data consistency in distributed applications like Microservices. It coordinates transactions between multiple Microservices to ensure that the data will remain consistent depending on the success or failure of the Business Process transaction.
This repository contains source code that shows how to Orchestrate Microservices using Saga Pattern with AWS Step Functions. AWS Step Functions is a serverless function orchestrator that makes it easy to sequence AWS Lambda functions and multiple AWS services into business-critical applications.
Each request has a compensating request for rollback. The Saga goes like this:
Begin saga
Start book hotel request
End book hotel request
Start book flight request
End book flight request
Start book car rental request
End book car rental request
End saga
Before you start deploying, you can run unit tests and code coverage locally to test each function.
-
Install all dependencies from sub-folders
Run this command below to npm dependencies for each function. Note: if you're a Windows user, you can use Git Bash
find . -name node_modules -prune -o -name package.json -execdir npm install \;
-
Run Unit Test & Code Coverage
This project uses Jest for Unit Test implementation
npm run test
After running this command, you should get a result like an image below:
And to explore the Code Coverage, open the file generated at /coverage/lcov-report/index.html in your browser, and you should see a screen like the one bellow
Build lambda function, and prepare them for subsequent steps in the workflow
-
Build
sam build
-
Deploy guided
sam deploy --guided
-
Success Test
Sample JSON for Success test.
{ "tripId": "5c12d94a-ee6a-40d9-889b-1d49142248b7", "depart": "London", "departAt": "2021-07-10T06:00:00.000Z", "arrive": "Dublin", "arriveAt": "2021-07-12T08:00:00.000Z", "hotel": "holiday inn", "checkIn": "2021-07-10T12:00:00.000Z", "checkOut": "2021-07-12T14:00:00.000Z", "car": "Volvo", "carFrom": "2021-07-10T00:00:00.000Z", "carTo": "2021-07-12T00:00:00.000Z" }
To test this Payload, execute the follow CURL commands to send POST request to the State Machine via API Gateway.
Note: Make sure you have replaced the XXXXXX from https://XXXXXXX.execute-api.us-west-2.amazonaws.com/Prod/ with the unique id generated during your SAM deployment
DEMO_REGION=us-west-2 \ API_GW_URL=https://XXXXXXX.execute-api.$DEMO_REGION.amazonaws.com/Prod/ \ SUCCESS_JSON_PAYLOAD='{"tripId":"5c12d94a-ee6a-40d9-889b-1d49142248b7","depart":"London","departAt":"2021-07-10T06:00:00.000Z","arrive":"Dublin", "arriveAt":"2021-07-12T08:00:00.000Z","hotel":"holiday inn","checkIn":"2021-07-10T12:00:00.000Z","checkOut":"2021-07-12T14:00:00.000Z","car":"Volvo", "carFrom":"2021-07-10T00:00:00.000Z","carTo":"2021-07-12T00:00:00.000Z"}' \ executionArnPayload=$(curl -X POST -H 'Content-Type: application/json' $API_GW_URL -d "$SUCCESS_JSON_PAYLOAD" | jq '.executionArn' )
The commands above sends a POST with JSON payload to $API_GW_URL, then it queries the API output to filter a proprietary .executionArn and it sets this proprietary value locally to a variable executionArnPayload.
Now you can check the execution status
curl -X POST -H 'Content-Type: application/json' $API_GW_URL/Status -d "{ \"executionArn\": $executionArnPayload }" | jq .
The command above sends a POST with JSON payload that contains the executionArn from local variable $executionArnPayload and formats the output using jq. The output should contain the Status of the State Machine Execution, in this case it should show "SUCCESS"
-
Failure test.
Sample JSON for Failure test.
{ "tripId": "1ecb46a4-ce47-4e13-b19e-64c2d057bed1", "depart": "London", "departAt": "2021-07-10T06:00:00.000Z", "arrive": "Dublin", "arriveAt": "2021-07-12T08:00:00.000Z", "hotel": "holiday inn", "checkIn": "2021-07-10T12:00:00.000Z", "checkOut": "2021-07-12T14:00:00.000Z", "car": "Volvo", "carFrom": "2021-07-10T00:00:00.000Z", "carTo": "2021-07-12T00:00:00.000Z", "failBookFlight": true }
To test this Payload, execute the follow CURL commands to send a POST request to the State Machine via API Gateway.
FAILURE_JSON_PAYLOAD='{"tripId":"1ecb46a4-ce47-4e13-b19e-64c2d057bed1","depart":"London","departAt":"2021-07-10T06:00:00.000Z","arrive":"Dublin", "arriveAt":"2021-07-12T08:00:00.000Z","hotel":"holiday inn","checkIn":"2021-07-10T12:00:00.000Z","checkOut":"2021-07-12T14:00:00.000Z", "car":"Volvo", "carFrom":"2021-07-10T00:00:00.000Z","carTo":"2021-07-12T00:00:00.000Z","failBookFlight":true}' \ executionArnPayload=$(curl -X POST -H 'Content-Type: application/json' $API_GW_URL -d "$FAILURE_JSON_PAYLOAD" | jq '.executionArn' )
The commands above do the same thing as the step 1, the difference is this one sends a JSON payload that will fail, because it has "failBookFlight": true
curl -X POST -H 'Content-Type: application/json' $API_GW_URL/Status -d "{ \"executionArn\": $executionArnPayload }" | jq .
Click here --> https://us-west-2.console.aws.amazon.com/states/home?region=us-west-2#/statemachines to explore the State Machine execution from AWS Console, where you can find more details about your Stepfunction execution. Note: make sure you select the same region where you're working on, mine is us-west-2.
This project implements Observability using the combination of AWS CloudWatch and AWS X-Ray by enabling Tracing to all lambda functions in Globals Section and in the StateMachine definition from the template.yml. With this setting SAM will deploy the resources needed to allows X-Ray to generate an unique trace ID for each request and track its interaction with downstream calls, for more detailed instrumentations each function has implementation to instrument AWS SDK clients to wrap the aws-sdk library with the captureAWS method as the bellow code snippet, click here to learn more about Instrumenting Node.js code in AWS Lambda.
const AWSXRay = require("aws-xray-sdk-core");
const AWS = AWSXRay.captureAWS(require("aws-sdk"));
const db = new AWS.DynamoDB.DocumentClient();
As a result of this implementation, X-Ray will collect metadata for each request and record a trace with all downstream calls, and it provide dashboard like the images bellow, where it shows all services that your request interact with, a timeline that content a precise length of all calls and a list of CloudWatch logs produced by each functions invoked by this request, this gives you a deep insight about your application in runtime and can help debug, trace, troubleshoot or monitor your workload. Here’s the link https://us-west-2.console.aws.amazon.com/cloudwatch/home?region=us-west-2#servicelens:traces where you can find all your requests by Trace ID, open it and scroll down you should find a list that content all your Trace IDs, click in each one to access it dashboard.
To clean up the resources deployed by SAM project you can use AWS CloudFormation passing your SAM project name correspondent with stack-name from CloudFormation
aws cloudformation delete-stack --stack-name demo-saga-pattern-stepfunctions
See CONTRIBUTING for more information.
This library is licensed under the MIT-0 License. See the LICENSE file.