GithubHelp home page GithubHelp logo

lambda's Introduction

Codeship Status for smallwins/lambda


@smallwins/lambda 🌱🙌λ

  • Author your AWS Lambda functions as pure node style callbacks (aka errbacks)
  • Familiar middleware pattern for composition
  • Event sources like DynamoDB triggers and SNS topics too
  • Helpful npm scripts lambda-create, lambda-list, lambda-deploy and lambda-invoke (and more)

📡📡📡 λ returning json results 📫

Here is a vanilla AWS Lambda example for performing a sum. Given event.query.x = 1 it will return {count:2}.

exports.handler = function sum(event, context) {
  var errors = []
  if (typeof event.query === 'undefined') {
    errors.push(ReferenceError('missing event.query'))
  }
  if (event.query && typeof event.query != 'object') {
    errors.push(TypeError('event.query not an object'))
  }
  if (typeof event.query.x === 'undefined') {
    errors.push(ReferenceError('event.query not an object'))
  }
  if (event.query.x && typeof event.query.x != 'number') {
    errors.push(TypeError('event.query not an object'))
  }
  if (errors.length) {
    // otherwise Error would return [{}, {}, {}, {}]
    var err = errors.map(function(e) {return e.message})
    context.fail(err) 
  }
  else {
    context.succeed({count:event.query.x + 1})
  }
}

A huge amount of vanilla AWS Lambda code is working around quirky parameter validation. API Gateway gives you control over the parameters you can expect but this still means one or more of: headers, querystring, form body, or url parameters. Event source style lambdas are not much better because you can often still get differing payloads from different origin sources. In the example above we are validating one querystring parameter x. Imagine a big payload! 😮

Worse still, writing a good program we want to use JavaScript's builtin Error but it still needs manual serialization (and you still lose the stack trace). The latter part of this vanilla code uses the funky AWS context object.

We can do better:

var validate = require('@smallwins/validate')
var lambda = require('@smallwins/lambda')

function sum(event, callback) {
  var schema = {
    'query':   {required:true, type:Object},
    'query.x': {required:true, type:Number}
  }
  var errors = validate(event, schema)
  if (errors) {
    callback(errors)
  }
  else {
    var result = {count:event.query.x + 1}
    callback(null, result)
  }
}

exports.handler = lambda(sum)

@smallwins/validate cleans up parameter validation. The callback style above enjoys symmetry with the rest of Node and will automatically serialize Errors into JSON friendly objects including any stack trace. All you need to do is wrap a your node style function in lambda which returns your function with an AWS Lambda friendly signature.

➿➿➿ easily chain dependant actions ala middleware ➿➿➿

Building on this foundation we can compose multiple functions into a single Lambda. It is very common to want to run functions in series. Lets compose a Lambda that:

  • Validates parameters
  • Checks for an authorized account
  • And then returns data safely
  • Or if anything fails return JSON serialized Error array
var validate = require('@smallwins/validate')
var lambda = require('@smallwins/lambda')

function valid(event, callback) {
  var schema = {
    'body':          {required:true, type:Object},
    'body.username': {required:true, type:String},
    'body.password': {required:true, type:String}
  }
  validate(event, schema, callback)
}

function authorized(event, callback) {
  var loggedIn = event.body.username === 'sutro' && event.body.password === 'cat'
  if (!loggedIn) {
    // err first
    callback(Error('not found'))
  }
  else {
    // successful login
    event.account = {
      loggedIn: loggedIn,
      name: 'sutro furry pants'
    }
    callback(null, event)
  }
}

function safe(event, callback) {
  callback(null, {account:event.account})
}

exports.handler = lambda(valid, authorized, safe)

In the example above our functions are executed in series passing event through each invocation. valid will pass event to authorized which in turn passes it to save. Any Error returns immediately so if we make it the last function we just send back the resulting account data. Clean!

💾 save a record from a dynamodb trigger 💥🔫

AWS DynamoDB triggers invoke a Lambda function if anything happens to a table. The payload is usually a big array of records. @smallwins/lambda allows you to focus on processing a single record but executes the function in parallel on all the results in the Dynamo invocation.

var lambda = require('@smallwins/lambda')

function save(record, callback) {
  console.log('save a version ', record)
  callback(null, record)
}

exports.handler = lambda.triggers.dynamo.save(save)

🔖 respond to a message published on sns

Its very common to compose your application events using AWS SNS. @smallwins/lambda runs in parallel over the records in the trigger, similar to the Dynamo.

// somewhere in your codebase you'll want to trigger a lambda
var aws = require('aws-sdk')
var sns = new aws.SNS

sns.publish({
  Message: JSON.stringify({hello:'world'}),
  TopicArn: 'arn:aws:sns:us-east-1'
}, console.log)
// then, in your lambda
var lambda = require('@smallwins/lambda')

function msg(message, callback) {
  console.log('received msg ', message) // logs {hello:"world"}
  callback(null, message)
}

exports.handler = lambda.triggers.sns(msg)

💌 api 💭✨

  • lambda(...fns) create a Lambda that returns a serialized json result {ok:true|false}
  • lambda([fns], callback) create a Lambda and handle result with your own errback formatter
  • lambda.local(fn, fakeEvent, (err, result)=>) run a Lambda locally offline by faking the event obj
  • lambda.triggers.dynamo.insert(fn) run on INSERT only
  • lambda.triggers.dynamo.modify(fn) run on MODIFY only
  • lambda.triggers.dynamo.remove(fn) run on REMOVE only
  • lambda.triggers.dynamo.all(fn) run on INSERT, MODIFY and REMOVE
  • lambda.triggers.dynamo.save(fn) run on INSERT and MODIFY
  • lambda.triggers.dynamo.change(fn) run on INSERT and REMOVE
  • lambda.triggers.sns(fn) run for every sns trigger invocation; expects record.Sns.Message to be a serialized JSON payload

A handler looks something like this:

function handler(event, callback) {
  // process event, use to pass data
  var result = {ok:true, event:event}
  callback(null, result)
}

❗ regarding errors ❌⁉️

Good error handling makes programs easier to maintain. This is a great guide digging in more. When using @smallwins/lambda always use Error type as the first parameter to the callback:

function fails(event, callback) {
  callback(Error('something went wrong')
}

Or an Error array:

function fails(event, callback) {
  callback([
    Error('missing email'), 
    Error('missing password')
  ])
}

@smallwins/lambda serializes errors into Slack RPC style JSON. Easier to work with from API Gateway:

{
  ok: false, 
  errors: [
    {name:'Error', message:'missing email', stack'...'},
    {name:'Error', message:'missing password', stack'...'}
  ]
}

#! automatations 📝

@smallwins/lambda includes some helpful automation code perfect for npm scripts. If you have a project that looks like this:

project-of-lambdas/
 |-test/
 |-src/
 |  '-lambdas/
 |     |-signup/
 |     |  |-index.js
 |     |  |-test.js
 |     |  '-package.json <--- name property should equal the deployed lambda name
 |     |-login/
 |     '-logout/
 '-package.json

And a package.json like this:

{
  "name":"project-of-lambdas",
  "scripts": {
    "create":"AWS_PROFILE=smallwins lambda-create",
    "list":"AWS_PROFILE=smallwins lambda-list",
    "deploy":"AWS_PROFILE=smallwins lambda-deploy",
    "invoke":"AWS_PROFILE=smallwins lambda-invoke",
    "local":"AWS_PROFILE=smallwins lambda-local",
    "deps":"AWS_PROFILE=smallwins lambda-deps",
    "log":"AWS_PROFILE=smallwins lambda-log"
  }
}

You get:

⏩ npm run scripts 🏃💨

This is 🔑! Staying in the flow with your terminal by reducing hunts for information in the AWS Console. :shipit:📈

  • 👉 npm run create src/lambdas/forgot creates a new lambda named forgot at src/lambdas/forgot
  • 👉 npm run list lists all deployed lambdas and all their alias@versions
  • 👉 npm run deploy src/lambdas/signup brian deploys the lambda with the alias brian
  • 👉 npm run invoke src/lambdas/login brian '{"email":"[email protected]", "pswd":"..."}' to remote invoke a deployed lambda
  • 👉 npm run local src/lambdas/login brian '{"email":"[email protected]", "pswd":"..."}' to locally invoke a lambda
  • 👉 npm run deps src/lambdas/* for a report of all your lambda deps
  • 👉 npm run log src/lambdas/logout to view the cloudwatch invocation logs for that lambda (remote console.log statements show up here)

Note: these scripts assume each lambda has it's own nested package.json file with a name property that matches the lambda name.

testing ✅

You can invoke a Lambda locally with a mock payload using lambda.local. Say you have this lambda function:

// always-ok.js
var lambda = require('@smallwins/lambda')

function fakeFn(event, callback) {
  callback(null, Object.assign({hello:'world'}, event))
}

exports.handler = lambda(fakeFn)

You can imagine the test:

// always-test.js
var fn = require('./always-ok').handler

lambda.local(fn, {fake:'payload'}, console.log)
// logs {hello:'world', fake:'payload', ok:true}

./scripts/invoke.js is also a module and can be useful for testing. It will remotely invoke your lambda.

var invoke = require('@smallwins/lambda/scripts/invoke')

invoke('path/to/lambda', alias, payload, (err, response)=> {
  console.log(err, response)
})

lambda's People

Contributors

brianleroux avatar kristoferjoseph avatar oayandosu avatar remy avatar ryanblock avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

lambda's Issues

Remove "ok": true for API Gateway?

Hey! This project has been super helpful for me, going from knowing almost nothing about Lambda to getting something online and working, so thanks.

I did encounter a problem getting something to work with API Gateway though, which I admittedly don’t know very much about.

Works (without @smallwins/lambda):

function example (event, context, callback) {
  callback(null, {"statusCode": 200, "body": "results"})
}

exports.handler = example

// Returns "results"

Result from API Gateway:

Endpoint response body before transformations: {"statusCode":200,"body":"results"}
Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=0, Connection=keep-alive, Content-Length=35, Date=Tue, 18 Apr 2017 20:13:19 GMT, Content-Type=application/json}
Method response body after transformations: results
Method response headers: {X-Amzn-Trace-Id=Root=0}
Successfully completed execution
Method completed with status: 200

Doesn’t work (with @smallwins/lambda):

var lambda = require('@smallwins/lambda')

function example (event, callback) {
  callback(null, {"statusCode": 200, "body": "results"})
}

exports.handler = lambda(example)

// Returns {"message": "Internal server error"}

Result from API Gateway:

Endpoint response body before transformations: {"statusCode":200,"body":"results","ok":true}
Endpoint response headers: {x-amzn-Remapped-Content-Length=0, x-amzn-RequestId=0, Connection=keep-alive, Content-Length=45, Date=Tue, 18 Apr 2017 20:11:04 GMT, Content-Type=application/json}
Execution failed due to configuration error: Malformed Lambda proxy response
Method completed with status: 502

I think adding "ok": true to the object is causing problems. Based on some other issues I read, it didn’t seem like you could deviate from the object that the API Gateway was expecting ("statsCode", "body", and "headers" only).

When I modified @smallwins/lambda to stop adding "ok": true`, my broken example above worked as expected.

I had some trouble figuring out if that was the only change that needed to be made or if I was just messing up things in the API Gateway, but I tried with both Node v4 and 6 on Lambda and it seemed to resolve the issue.

Happy to open a PR if it makes sense that the change should be made. In the meantime, I’m still using the run scripts, but not this or the validation package.

feature: persist log streams for speed

I wrote something like scripts/log.js last night :-|

I already knew getting these logs would be slow, and it's real f'ing slow. I'd like to have the ability to persist the data, prolly as it's existing JSON form, locally. Basically the log command would update all the log streams to disk, then perform it's query over the persisted forms. You could imagine running some kind of "log updater" in the background, that sync's your logs every few minutes, so the log commands would have even less to do.

Interest?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.