GithubHelp home page GithubHelp logo

ca98am79 / connect-dynamodb Goto Github PK

View Code? Open in Web Editor NEW
144.0 4.0 66.0 309 KB

DynamoDB session store for Connect

Home Page: http://ca98am79.github.com/connect-dynamodb/

License: MIT License

Makefile 1.17% JavaScript 92.03% TypeScript 6.80%

connect-dynamodb's Introduction

Connect DynamoDB

connect-dynamodb is a DynamoDB session store backed by the 3.x @aws-sdk/client-dynamodb.

Installation

$ npm install connect-dynamodb

Options

Rational defaults are set but can be overridden in the options object. Credentials and configuration are automatically loaded using AWS defaults unless the client options is provided to override them.

  • client Optional AWS DynamoDBClient object from new DynamoDBClient({}) If supplied, this is used instead of standard node defaults.
  • table Optional DynamoDB server session table name (defaults to "sessions")
  • hashKey Optional hash key (defaults to "id")
  • prefix Optional key prefix (defaults to "sess")
  • reapInterval Legacy session expiration cleanup in milliseconds. ☣️ Legacy reap behaviors use DynamoDB scan functionality that can incur significant costs. Should instead enable DynamoDB TTL and select the expires field. TODO should we just remove it since we're already making a breaking change?

Usage

var options = {
  // Optional DynamoDB table name, defaults to 'sessions'
  table: "myapp-sessions",
  // Optional client for alternate endpoint, such as DynamoDB Local
  client: new DynamoDBClient({ endpoint: "http://localhost:8000" }),
  // Optional ProvisionedThroughput params, defaults to 5
  readCapacityUnits: 25,
  writeCapacityUnits: 25,
  // Optional special keys that will be inserted directly into your table (in addition to remaining in the session)
  specialKeys: [
    {
      name: "userId", // The session key
      type: "S", // The DyanamoDB attribute type
    },
  ],
  // Optional skip throw missing special keys in session, if set true
  skipThrowMissingSpecialKeys: true,
};

With connect

var connect = require("connect");
var DynamoDBStore = require("connect-dynamodb")(connect);
connect()
  .use(connect.cookieParser())
  .use(
    connect.session({
      store: new DynamoDBStore(options),
      secret: "keyboard cat",
    })
  );

With express 3

var DynamoDBStore = require("connect-dynamodb")(express);
var app = express(
  express.cookieParser(),
  express.session({ store: new DynamoDBStore(options), secret: "keyboard cat" })
);

With express 4

var app = express();
var session = require("express-session");
var DynamoDBStore = require("connect-dynamodb")({ session: session });
app.use(session({ store: new DynamoDBStore(options), secret: "keyboard cat" }));

OR

var app = express();
var session = require("express-session");
var DynamoDBStore = require("connect-dynamodb")(session);
app.use(session({ store: new DynamoDBStore(options), secret: "keyboard cat" }));

OR

import session from "express-session";
import connect from "connect-dynamodb";

interface SessionData {
  name: string;
  animal: "cow" | "pig";
}
const DynamoDBStore = connect<SessionData>(session);
app.use(session({ store: new DynamoDBStore(options), secret: "keyboard cat" }));

Testing

If you want to run the tests locally and have the AWS environment variables setup you can run the command:

npm test

You can also run it locally by running the following two scripts in separate terminals:

docker run -it --rm \
  --name=dynamodb-test \
  -p 127.0.0.1:8000:8000 \
  deangiberson/aws-dynamodb-local
export AWS_REGION=us-east-1
export AWS_ACCESS_KEY_ID=accesskey
export AWS_SECRET_ACCESS_KEY=secretaccesskey
export ENDPOINT=http://127.0.0.1:8000
npm test

IAM Permissions

Connect DynamoDB requires the following IAM permissions for DynamoDB:

  • CreateTable
  • PutItem
  • DeleteItem
  • GetItem
  • Scan
  • UpdateItem

Sample IAM policy (with least privilege):

(Replace <AWS ACCOUNT ID>, <TABLE NAME> and <SOURCE IP AND BITMASK>).

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "dynamodb:CreateTable",
        "dynamodb:DescribeTable",
        "dynamodb:PutItem",
        "dynamodb:DeleteItem",
        "dynamodb:GetItem",
        "dynamodb:Scan",
        "dynamodb:UpdateItem"
      ],
      "Resource": "arn:aws:dynamodb:*:<AWS ACCOUNT ID>:table/<TABLE NAME>"
    }
  ]
}

License

connect-dynamodb is licensed under the MIT license.

Donations

I made this in my spare time, so if you find it useful you can donate at my BTC address: 35dwLrmKHjCgGXyLzBSd6YaTgoXQA17Aoq. Thank you very much!

connect-dynamodb's People

Contributors

baraksi avatar brianmacarthur avatar bryce-larson avatar ca98am79 avatar daverickdunn avatar dianoga avatar ericabouaf avatar etiennea avatar intercept6 avatar jackmuskopf avatar jonathanfeller avatar kristian-ackar avatar lancegliser avatar lin826 avatar mblenton avatar mherger avatar mririgoyen avatar msroz avatar ollyfg avatar rafaelrpinto avatar roylines avatar rsenergy avatar shift-yuta-katayama avatar smaclell avatar therealsanmah avatar wayne-folkes avatar yutak23 avatar zeuzere avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

connect-dynamodb's Issues

NPM package need to be updated

Can you update the npm package with a new version so that we have the new package.json ( with optionalDepencencies)

Thx...
I did not do a pull since i don't know what version you want to put ...
Yves Laroche

Unclear why this npm package exists

Hi Team,

I have inherited a project that uses this package. Do you have any details of why this wrapper around the native aws-sdk for dynamo exists? What does this package offer over using the native?

Session Timeout Value?

I'm having trouble figuring out how you set the session timeout. I see the reap interval, but that looks like simple clean up of dead session records. How can you set the lifespan of the session?

Breaking Date changes in 1.0.12

The date math introduced in version 1.0.12, moving from millisecond unix epoch timestamps to second unix epoch timestamps, breaks session reaping when using sessions across multiple servers that serve sites to the same domain.

One of our servers, running connect-dynamodb v1.0.11 was reaping sessions and comparing against millisecond epoch timestamps, thus deleting every session made by our other server, running connect-dynamodb v1.0.12, since these timestamps are always shorter than their millisecond-length counterparts.

This results in sessions lasting no longer than 10 minutes (default reapInterval), and users being logged out automatically.

I'd consider this a breaking change, which definitely shouldn't happen when updating the patch version.

Deprecated express-session warnings: resave, saveUninitialized

express-session deprecated undefined resave option; provide resave option at src/config/session.ts:15:42
express-session deprecated undefined saveUninitialized option; provide saveUninitialized option at src/config/session.ts:15:42

Seeing the above in my API as I use it. Not sure if this is related to the #76 as I'm using that or a constant. Should confirm what it affects.

Configuration doesn't read region from environment variable AWS_REGION

The documentation for the AWS SDK mentions being able to set the AWS region via an environment variable here. connect-dynamodb doesn't yet respect this environment variable. If the developer is unaware of this it can lead to the session table being hosted in a different region than the rest of the database.

Does not fetch credentials from environment variables

It doe snot work without a credentials.json file

{ accessKeyId: AWS_ACCESS_KEY_ID, secretAccessKey: AWS_SECRET_KEY, region: 'us-east-1' } or smth like that would enable us not to have credential in the repo. I would improve it but I am sure someone understands these things much better than me.

express 4

I have gotten it to work on express 4 by doing

var connect = require('connect');
var DynamoDBStore = require('connect-dynamodb')(connect);

is the the right way of doing it?

TTL not set up on table creation

The library currently creates the DynamoDB table if it doesn't exists. In that process it doesn't seem to be properly setting the table TTL to the expires attribute of the sessions. Is there any reason for this?

Tests fail due to fs.exists TypeError [ERR_INVALID_CALLBACK]: Callback must be a function. Received undefined

On the master branch, using the commands README:

docker run -it --rm \
  --name=dynamodb-test \
  -p 127.0.0.1:8000:8000 \
  deangiberson/aws-dynamodb-local
export AWS_CONFIG_JSON='{"endpoint": "http://127.0.0.1:8000", "region": "us-east-1", "accessKeyId": "accesskey", "secretAccessKey": "secretaccesskey"}'
npm test

I run into this error:

TypeError [ERR_INVALID_CALLBACK]: Callback must be a function. Received undefined
    at maybeCallback (node:fs:177:3)
    at Object.exists (node:fs:260:3)
    at Object.<anonymous> (/Users/lancegliser/Projects/lancegliser/connect-dynamodb-v2-dependency/test/test.js:9:17)
    at Module._compile (node:internal/modules/cjs/loader:1101:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:190:29)
    at ModuleJob.run (node:internal/modules/esm/module_job:185:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:281:24)
    at async importModuleDynamicallyWrapper (node:internal/vm/module:437:15)
    at async formattedImport (/Users/lancegliser/Projects/lancegliser/connect-dynamodb-v2-dependency/node_modules/mocha/lib/nodejs/esm-utils.js:7:14)
    at async Object.exports.requireOrImport (/Users/lancegliser/Projects/lancegliser/connect-dynamodb-v2-dependency/node_modules/mocha/lib/nodejs/esm-utils.js:38:28)
    at async Object.exports.loadFilesAsync (/Users/lancegliser/Projects/lancegliser/connect-dynamodb-v2-dependency/node_modules/mocha/lib/nodejs/esm-utils.js:91:20)
    at async singleRun (/Users/lancegliser/Projects/lancegliser/connect-dynamodb-v2-dependency/node_modules/mocha/lib/cli/run-helpers.js:125:3)
    at async Object.exports.handler (/Users/lancegliser/Projects/lancegliser/connect-dynamodb-v2-dependency/node_modules/mocha/lib/cli/run.js:370:5)

test/test.js:9:17 is:

var fs = require('fs'),
// Other stuff
var config = fs.exists('./aws-config.json') && fs.readFileSync('./aws-config.json');

The definition of fs.exists reads:

deprecated export function exists( path: PathLike, callback: (exists: boolean) => void): void
Test whether or not the given path exists by checking with the file system. Then call the callback argument with either true or false:
import { exists } from 'fs';

exists('/etc/passwd', (e) => {
console.log(e ? 'it exists' : 'no passwd!');
});
The parameters for this callback are not consistent with other Node.js callbacks. Normally, the first parameter to a Node.js callback is an errparameter, optionally followed by other parameters. The fs.exists() callback has only one boolean parameter. This is one reason fs.access() is recommended instead of fs.exists().
Using fs.exists() to check for the existence of a file before callingfs.open(), fs.readFile() or fs.writeFile() is not recommended. > Doing so introduces a race condition, since other processes may change the file's state between the two calls. Instead, user code should open/read/write the file directly and handle the error raised if the file does not exist.

If I update the test config file read to the follow it works:

let config;
try {
    config = fs.readFileSync('./aws-config.json')
} catch {}
if (config) {
    client = new AWS.DynamoDB(JSON.parse(config));
}
else if (process.env.AWS_CONFIG_JSON) {
    var AWS = require('aws-sdk');
    config = JSON.parse(process.env.AWS_CONFIG_JSON);
    client = new AWS.DynamoDB(config);
}

Not sure I care to fix it, as I'm looking a different configuration in #76 but thought I'd document the issue and solution.

Disable checking if sessions table exists

Is there a way to disable always checking if the table exists?
The reason for this is that we plan to use express-session with dynamoDb and serverless (using aws lambda), so checking if the table exists each time a lambda function gets fired up seems redundant.

now >= result.Item.expires always returns false

In DynamoDBStore.prototype.get function,

now >= result.Item.expires always returns false, while debugging figure out that result.Item.expires is not a number so its unable to compare with 'now' and always returns false.

else block gets executed always :

else if (result.Item.expires && now >= result.Item.expires) {
fn(null, null);
} else {
var sess = result.Item.sess.S.toString();
sess = JSON.parse(sess);
fn(null, sess);
}

Did anyone face similar issue?

OnDemand Table

Is it possible to create an ondemand Table instead of a provisioned one.

I looked over your code and did not seem to think this was possible?
Is there a reason why?

Thanks so much for the package.

Hash Key must be id

I created a hash key called sessionID and got:

ValidationException: One or more parameter values were invalid: Missing the key sessionID in the item

Found out my table had to use the hash id to work properly with this module.

Maybe add the option to choose your own hash key? Or explicitly note it within the Readme/wiki?

Thanks!

make ProvisionedThroughput configurable

would you accept a PR to allow ProvisionedThroughput to be configurable?

e.g. 
var options = {
    // Optional DynamoDB table name, defaults to 'sessions'
    table: 'myapp-sessions',

    // Optional path to AWS credentials and configuration file
    // AWSConfigPath: './path/to/credentials.json',

    // Optional JSON object of AWS credentials and configuration
    AWSConfigJSON: {
        accessKeyId: <YOUR_ACCESS_KEY_ID>,
        secretAccessKey: <YOUR_SECRET_ACCESS_KEY>,
        region: 'us-east-1'
    },

    // Optional client for alternate endpoint, such as DynamoDB Local
    client: new AWS.DynamoDB({ endpoint: new AWS.Endpoint('http://localhost:8000')}),

    // Optional clean up interval, defaults to 600000 (10 minutes)
    reapInterval: 86400000,    // 1 day

   readCapacityUnits: 20,
   writeCapacityUnits: 20 
};

encryption

DynamoDB doesn't support encryption at rest, which means if you have sensitive data that you want to encrypt, this must be done at the application level.

Session data contains this kind of sensitive data, so one blocker for us using connect-dynamodb is the ability encrypt session data before saving it to dynamodb. One path we're pursuing is creating a connect-session-encrypted session store that wraps an existing session store and adds this kind of encryption. Another approach would be adding it to connect-dynamodb directly, since it might be a common-enough use case. I'm curious what you think.

Cannot read property length of undefined

Updated to 1.0.8 to get the fix for reaping but now encountering a different issue during testing. It appears to come from the line that references self.prefix.length in the reaping logic. Has anyone else seen/resolved this problem?

TypeError: Cannot read property 'length' of undefined 
  at destroyDataAt (/.../node_modules/connect-dynamodb/lib/connect-dynamodb.js:211:48)
  at Response.destroy (/.../node_modules/connect-dynamodb/lib/connect-dynamodb.js:219:9)
  at Response.onScan (/.../node_modules/connect-dynamodb/lib/connect-dynamodb.js:196:21)
  at Request.<anonymous> (/.../node_modules/aws-sdk/lib/request.js:355:18)
  at Request.callListeners (/.../node_modules/aws-sdk/lib/sequential_executor.js:105:20)
  at Request.emit /.../node_modules/aws-sdk/lib/sequential_executor.js:77:10)
  at Request.emit (/.../node_modules/aws-sdk/lib/request.js:615:14)
  at Request.transition (/.../node_modules/aws-sdk/lib/request.js:22:10)

(paths are abbreviated)

Table name doesn't default to 'sessions'

I omitted the table attribute to the config options and got this error.

Running "watch" task
Waiting...Express server listening on port 3000
{ [MissingRequiredParameter: Missing required key 'TableName' in params]
  code: 'MissingRequiredParameter',
  name: 'MissingRequiredParameter' } null
OK

It worked after adding the table attribute.

Session data incorrect

Hello,

I have tried to implement your solution and have managed to store sessions in dynamodb and retrieve them when my client make requests.
However, I have ran into an issue with some sessions that only contains an id, an expiration number but no other field (sess and type). When my client make a request with this id, I get a "Cannot read property 'S' of undefined" error in connect-dynamodb.js line 146
var sess = result.Item.sess.S.toString();

It makes sense, because there is no valid sess object.

I can try to detail my configuration more if needed, but do you please have any idea about what may cause this error ?
Thank you very much, best regards

aws-sdk in deployment package (aws lambda issue)

We use claudiajs to package our functions with:
claudia update --no-optional-dependencies ... (Idem: npm install --no-optional)
to reduce the size of our package.
As recommended by AWS to not include AWS-SDK in node_modules when deploying to lambda ...

and we notice that you install this in dependencies:
"dependencies": {
"aws-sdk": "",
"connect": "
"
},

it should be in optionalDependencies:
"optionalDependencies": {
"aws-sdk": "*"
},

It will not change anything, except that those that wish to not include this package will be able to do it...
by using npm install -no-optionnal
So the node_modules/connect-dynamodb directory will be a few Ko instead of ~30.4Mo

Small change for a huge difference, especially when sending to aws lamda

Thanks
The size of the package is the only buttleneck for us now...
Love your package (use it in Api-Gateway->Lambda->Dynamo)
Yves Laroche

PS.: I did a pull request...

  • "optionalDependencies": {
  • "aws-sdk": "*"
  • },

Is there a way to use this to connect to an existing table in an existing stack?

Hi, I'd rather not create a new session table every time I Deploy. I have a large cloud formation yaml file that spins up all the AWS resources I need in my stack. I was hoping to use this package to connect to the existing table. But the documentation looks like it only creates tables.

Is there a way to use this package to connect to my existing table?

Consistent reads

By Default Dynamodb issues non-consistent reads. It would be nice to have an option to use consistent reads.

Typescript support

Hi,

As the title, is there @types/connect-dynamodb or index.d.ts for Typescript? Thank you :)

Support for DynamoDB TTL

With the advent of DynamoDB TTL, it is now possible to automatically delete rows (sessions) from DynamoDB automatically based on a field (e.g. expires). Unfortunately, it only currently supports seconds since epoch (not milliseconds, which is what is stored in expires).

Is there a way to leverage this new DynamoDB feature?

Related: https://forums.aws.amazon.com/thread.jspa?threadID=251306, as it could also be a feature request on AWS's end.

Support passing express-session directly as a param in init

Allow the flexibility to pass the express-session directly as a parameter to the adapter constructor.

Today, then way connect-dynamodb seems to allow initialization is like this:

var session = require('express-session');
var DynamoDBStore = require('connect-dynamodb')({session: session});

Where as frameworks such as sails, initialize the store like this. Consequently, connect-dynamodb is not directly compatible with sails, which is gaining popularity quickly.

The issue is to request support initialisation by passing the express-session directly as a param. Like this:

var DynamoDBStore = require('connect-dynamodb')(require('express-session'));

This is today also supported by connect-mongo and will bring us feature-parity.

DynamoDB Key Value Store

Hi guys,

I've just tried the module and it works well. I found that the session items are stringified stored in DynamoDB, and I think there should be another way to store the session if it is an object: flattern the nested object and store the session as an item in db. Is it feasible/make sense?

ESM support

Hello,

Is it possible to use ES6 import like this (import DynamoDBStore from 'connect-dynamodb';) instead of require?

When I try the standard import I get errors relating to:
TypeError: Cannot read properties of undefined (reading 'Store')

Thanks, Alex

Issue with reaping

Not sure why, but reaping does not appear to be working for me, and the number of sessions stored in dynamo is growing seemingly without bound. I'm not using a custom reapInterval so it should be using the 10 min default, but there are sessions that are weeks old in my table. Any help would be appreciated!

add `.touch(sid, sess, fn)` method

express-session now supports a .touch method on session stores.

This method should just update the expiry. It allows you to set resave: false and avoid race conditions that cause session data trampling.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.