GithubHelp home page GithubHelp logo

comdb's Introduction

ComDB

CI Coverage Status Stability NPM Version JS Standard Style

A PouchDB plugin that transparently encrypts and decrypts its data so that only encrypted data is sent during replication, while encrypted data that you receive is automatically decrypted. Uses TweetNaCl for cryptography.

As an example, here's what happens when you replicate data to a CouchDB cluster:

const PouchDB = require('pouchdb')
PouchDB.plugin(require('comdb'))

const password = 'extremely secure value'

const db = new PouchDB(POUCH_PATH)
db.setPassword(password).then(async () => {
  await db.post({
    _id: 'gay-agenda',
    type: 'queerspiracy',
    agenda: ['be gay', 'do crimes']
  })
  // now replicate to a couchdb instance
  await db.replicate.to(`${COUCH_URL}/FALGSC`)
})

Now you can check the CouchDB for the encrypted information:

$ curl "$COUCH_URL/FALGSC/_all_docs?include_docs=true" | jq .
{
  "total_rows": 1,
  "offset": 0,
  "rows": [
    {
      "id": "...",
      "key": "...",
      "value": {
        "rev": "1-[...]"
      },
      "doc": {
        "_id": "...",
        "_rev": "1-[...]",
        "payload": "...",
      }
    }
}

ComDB can also restore encrypted data that it doesn't already have using your password.

// using a different and empty database
const db = new PouchDB(`${POUCH_PATH}-2`)
// but using the same password and encrypted copy
db.setPassword(password, { name: `${COUCH_URL}/FALGSC` })
// you can restore data from a remote source
return db.loadEncrypted().then(async () => {
  return db.allDocs({ include_docs: true })
}).then(({ rows }) => {
  const { doc } = rows[0].doc
  console.log(doc)
})
----
{ _id: 'gay-agenda',
  _rev: '1-[...]',
  type: 'queerspiracy',
  agenda: [ 'be gay', 'do crimes' ] }

This way, the server can't (easily) know anything about your data, but you can still maintain query indexes.

In the above example, we replicated data from a local encrypted copy of our data, but you can use a CouchDB instance as your encrypted copy. That way, your documents will be automatically backed up to the remote instance.

const db = new PouchDB(POUCH_PATH)
await db.setPassword(password, { name: COUCH_URL })

You can also set up encryption on another device by using db.exportComDB() and db.importComDB(). This is useful when you want to maintain a separate encrypted copy of your data, for example because you want that separate copy to live on another device, while retaining the ability to replicate with the original encrypted copy.

// on one machine, get the encryption key. it'll be a long string.
const key = await db.exportComDB()
// then on another machine, use the key with your password to set up encryption
const db = new PouchDB(POUCH_PATH)
await db.importComDB(password, key)
// now you can replicate over from the original encrypted backup
await db.replicate.from(COUCH_URL)

Now you can give your data to strangers with confidence!

For more examples, check out the /examples folder.

Install

You can get ComDB with npm:

$ npm i comdb

Now you can require() it in your node.js projects:

const PouchDB = require('pouchdb')
PouchDB.plugin(require('comdb'))

const db = new PouchDB(...)
await db.setPassword(...)

You can also use PouchDB and ComDB in the browser using browserify.

Usage

ComDB adds and extends several methods to PouchDB and any instances of it:

PouchDB.replicate(source, target, [opts], [callback])

ComDB wraps PouchDB's replicator to check if either the source or the target have an _encrypted attribute, reflecting that they are ComDB instances. If it finds the attribute, it changes the parameter to use the encrypted database rather than its decrypted one. If neither the source or target is a ComDB instance, the replicator behaves as normal.

You can also disable this functionality by passing comdb: false in the opts parameter:

PouchDB.replicate(db1, db2, { comdb: false })

The instance methods db.replicate.to and db.replicate.from automatically use PouchDB.replicate so that wrapping the static method causes the instance methods to exhibit the same behavior.

Original: PouchDB.replicate

async db.setPassword(password, [opts])

Mutates the instance with crypto tooling so that it can encrypt and decrypt documents.

await db.setPassword('hello world')
// db will now maintain an encrypted copy
  • password: A string used to encrypt and decrypt documents.
  • opts.name: A name or connection string for the encrypted database.
  • opts.opts: An options object passed to the encrypted database's constructor. Use this to pass any options accepted by PouchDB's constructor.

async db.destroy([opts], callback)

ComDB wraps PouchDB's database destruction method so that both the encrypted and decrypted databases are destroyed. ComDB adds two options to the method:

  • encrypted_only: Destroy only the encrypted database. This is useful when a backup has become compromised and you need to burn it.
  • unencrypted_only: Destroy only the unencrypted database. This is useful if you are using a remote encrypted backup and want to burn the local device so you can restore from backup on a fresh one.

Original: db.destroy()

async db.exportComDB()

Export the encryption key specific to your database's encrypted copy. This is necessary to creating new encrypted copies that can still replicate with the original, for example if you're creating an encrypted copy on a phone by replicating down from a server.

// on one machine
const db1 = new PouchDB('device-1')
await db1.setPassword(password)
const key = await db1.exportComDB()
// then, on another
const db2 = new PouchDB('device-2')
await db2.importComDB(password, key)
// now db2 can replicate with db1
await PouchDB.sync(db1, db2)

async db.importComDB(password, encryptionKey)

Set up ComDB, like db.setPassword(), but rather than generating a new encryption key for your encrypted copy, ComDB will use the given one. This allows you to replicate with other encrypted databases using the same password and encryption key.

// on one machine
const db1 = new PouchDB('device-1')
await db1.setPassword(password)
const key = await db1.exportComDB()
// then, on another
const db2 = new PouchDB('device-2')
await db2.importComDB(password, key)
// now db2 can replicate with db1
await PouchDB.sync(db1, db2)

async db.loadEncrypted(opts = {})

Load changes from the encrypted database into the decrypted one. Useful if you are restoring from backup:

// in-memory database is wiped on restart and so needs to be repopulated
const db = new PouchDB('local', { adapter: 'memory' })
// the encrypted DB lives on remote disk, so we can load docs from it
db.setPassword(PASSWORD, { name: REMOTE_URL }).then(async () => {
  await db.loadEncrypted()
  // all encrypted docs have been loaded into the decrypted database
})

Accepts the same options as PouchDB.replicate().

async db.loadDecrypted(opts = {})

Load changes from the decrypted database into the encrypted one. Useful if you are instrumenting encryption onto a database that already exists.

// db already exists, we are just adding encryption
const db = new PouchDB('local')
db.setPassword(PASSWORD).then(async () => {
  await db.loadDecrypted()
  // all decrypted docs have been loaded into the encrypted database
})

Accepts the same options as db.changes().

Recipe: End-to-End Encryption

ComDB can instrument end-to-end encryption of application data using pouchdb-adapter-memory, so that documents are only decrypted in memory while everything on disk remains encrypted.

Consider this setup:

// in-memory database is wiped on restart and so needs to be repopulated
const db = new PouchDB('local', { adapter: 'memory' })
// the encrypted copy lives on local disk, so we can load docs from it
db.setPassword(PASSWORD).then(async () => {
  // repopulate database from encrypted local copy
  await db.loadEncrypted()
  // decrypted database is up to date, app is ready to go
})

You can then replicate your encrypted database with a remote CouchDB installation to ensure you can restore your data even if your device is compromised:

// sync local encrypted with remote
const remoteDb = 'https://...' // CouchDB url
const sync = PouchDB.sync(db, remoteDb, { live: true, retry: true })

Now you'll have three copies of your data:

  • One in local memory, decrypted.
  • One on local disk, encrypted.
  • One on remote disk, encrypted.

The user syncs local disk with remote disk to have a remote encrypted backup, so the user can restore their info when switching devices. The local disk populates the in-memory database on startup, so that the only data that remains on disk remains encrypted. The user retains all their information locally, so they do not require network connectivity to use the app normally.

Development

To hack on ComDB, check out the issues page. To submit a patch, submit a pull request.

To run the test suite, use npm test in the source directory:

$ git clone garbados/comdb
$ cd comdb
$ npm i
$ npm test

A formal code of conduct is forthcoming. Pending it, contributions will be moderated at the maintainers' discretion.

License

Apache-2.0

comdb's People

Contributors

dependabot[bot] avatar garbados avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

comdb's Issues

Security questions

Say you lose your password. ComDB doesn't store your password and can only verify it, so we can't provide it if you lose it. Unless we do store it when asked to, in an encrypted way.

Consider a method such as .addSecurityQuestion(name, question, answer). Using a hash of the answer as a password, we then create a new Crypt instance to encrypt the user's password. This encrypted value and the Crypt instance's export string are then stored in the _local/comdb document like this:

{
  _id: '_local/comdb',
  _rev: '...',
  security_questions: {
    house: {
      question: 'In the house you grew up in, what is buried in the garden?',
      exportString: '...',
      payload: '...'
    }
  }
}

A matching db.removeSecurityQuestion(name) would remove the specified security question from _local/comdb.

Thus, a user can retrieve their password by answering exactly any one security question. The user must only ever opt-in to this type of password protection!

Can't get In E2E with memory to work

Hi,

tanks for this awesome lib!

I am trying to get the e2e encryption working in an angular 10 app. Just like you showed it here:
#4

But i have trouble with the pouchdb-adapter-memory.

When I import it as this:

import PouchDB from 'pouchdb'; // Works
import PouchAuth from 'pouchdb-authentication'; // Works
import comdb from 'comdb'; // Works
import PouchMemory from 'pouchdb-adapter-memory'; // Does not

PouchDB.plugin(PouchAuth);
PouchDB.plugin(comdb);
PouchDB.plugin(PouchMemory);

I get this at app start:

Error: ./node_modules/sublevel-pouchdb/node_modules/readable-stream/readable.js
Module not found: Error: Can't resolve 'stream' in 'C:\source\ngcp\node_modules\sublevel-pouchdb\node_modules\readable-stream
'``

Then I tried to import this: 

import PouchMemory from 'pouchdb/dist/pouchdb.memory';


and got

memory adapter plugin error: Cannot find global "PouchDB" object! Did you remember to include pouchdb.js?
core.js:6210 ERROR Error: Invalid plugin: got "[object Object]", expected an object or a function


I simply don't get it. Do you have any idea? I would be very thankful.
Btw. what I actually try to accomplish in the end is storing data encrypted on the client while having the decrypted version on the server. :-)

Recipe for Encrypting remote DB in place

Now that #21 is complete, I'm SO excited for this!

One issue is that I have many remote DBs with encrypted data on them. Because of the way replication works, I'm not sure how to encrypt the data in place. Do you have any suggestions?

Edit: (I MEANT to say DBs with unencrypted data... this confusion was resolved below)

Encryption of Database with Attachments

Hi,

I thought to have understood this nice project, but somehow...

I'm working local in my js app and I simply want to keep only the decrypted database in indexed db when the app isn't used.

So I do

const db = new PouchDB('test');
db.setPassword('my');

now working with the app -> when closing:

db.loadDecrypted();
db.destroy({ unencrypted_only: 'true' });`

When returning

const db = new PouchDB('test');
db.setPassword('my');
db.loadEncrypted();

This is working fine until there are attachments in the db.
Then it fails saying

"Failed to execute 'readAsArrayBuffer' on 'FileReader': parameter 1 is not of type 'Blob'."
Any help would be very appreciated!

Best Regards

Rotate keys

It should be possible for a user to reset their password and rotate their encryption keys. They might do this to lock out a device that has become compromised.

The process requires the old and new passwords. Given those, each document in the encrypted copy is re-encrypted using the new password and key. The executor of this rotation sets a rotating property on the _local/comdb document, and then unsets it once the rotation completes. The database will be in an intermediate state during this period; applications that encounter "Could not decrypt!" errors at this time should prompt the user for the new password and then wait for the _local/comdb) document to lose the rotating property.

Cannot decrypt using in-memory adapter

Given code like the following, running in a web page:

const PouchDB = require('pouchdb')
PouchDB.plugin(require('pouchdb-adapter-memory'))
PouchDB.plugin(require('comdb'))

const PASSWORD = 'blahzeblaht'
const db = new PouchDB('local', { adapter: 'memory' })
db.setPassword(PASSWORD).then(async () => {
  await db.loadEncrypted()
  await db.post({ name: 'bob' })
})

This will work the first time, but subsequent runs will encounter a "Could not decrypt!" error. This is because ComDB stores an exportString in a local document in the decrypted database, which it uses along with the given password to decrypt the encrypted database. Because the in-memory database is wiped each time the program ends, this exportString value is also wiped, essentially orphaning all your encrypted data. This means ComDB does not currently work with an in-memory adapter, pending architectural changes.

Is it possible End to End Encryption?

Hello @garbados , Diana, excellent work you have done here.

I wonder how difficult would be to have End to End Encryption, that way information is safe on both sides. WIth State Management Containers like Redux or Vuex you can have on Ram decrypted information.

I wonder if this makes sense to you.

Once again, thanks for the great job!!

Fully encrypt PARTIALLY encrypted database

This is related to the excellent work in #28.

As part of a software update, I have a use case where I want to administratively deploy new data to a remote database that is fully encrypted by comdb. The resulting database will have a mix of encrypted and unencrypted data. I would like the unencrypted data to be encrypted upon syncing with user client application (without double encrypted existing data).

transform-pouch bug breaks comdb when using couchdb

Transform-Pouch is broken against CouchDB, and as a result this occurs when running the CouchDB example in this repository:

$ node examples/couchdb.js 
...
TypeError: Cannot read property 'doc' of undefined
    at /home/garbados/code/comdb/examples/couchdb.js:40:12
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

There is a pending fix here. The dependency in this project will need to be updated as soon as that patch lands.

Loading encrypted documents created with `.post()` fails sometimes

Encrypted documents created with .post() fail to load using .loadEncrypted() if a version of that document already exists on the decrypted side. You can replicate this issue in a browser by running this code:

const db = new PouchDB('local')
db.setPassword(PASSWORD).then(async () => {
  await db.loadEncrypted()
  await db.post({ name: 'bob' })
})

This will work the first time. But if you run this in a browser and then reload the page, it will fail with a message like this:

Inquiry regarding `id` generation for encrypted record

When generating the id for the encrypted record it looks like you are hashing all the data in the record:

https://github.com/garbados/comdb/blob/master/index.js#L101-L103

I understand the reason for needing the ID to be deterministic but why use the hash of the entire record? It seems if you change any data on a record that would create a new record vs doing a revision on the previous record. I'm unsure what impact this has on syncing but at the very least it means compaction won't work as from PouchDB's perspective those are two different records rather than the current record and a previous record.

Would it be perhaps better to just hash the cleartext ID to create an obfuscated deterministic ID for the encrypted record? If you were worried about someone determining the cleartext ID (which might contain sensitive data since PouchDB encourages intelligent keys) via rainbow table or something you could always generate a per-record salt to be stored with the encrypted data.

Just wanted to inquire to make sure I'm not missing anything obvious on why that is a bad idea.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.