GithubHelp home page GithubHelp logo

pier4all / mongoose-versioned Goto Github PK

View Code? Open in Web Editor NEW
14.0 5.0 7.0 540 KB

Document versioning library for MongoDB using the mongoose package.

License: MIT License

JavaScript 100.00%
mongo mongodb mongoose versioning data-versioning

mongoose-versioned's Introduction

Versioning Module MongoDB

Module for versioning in MongoDB, inspired by Vermongo (https://www.npmjs.com/package/mongoose-vermongo, https://github.com/codela/mongoose-vermongo and https://github.com/thiloplanz/v7files/wiki/Vermongo).

It includes support for transactions to avoid inconsistency when performing an update or deletion since this operations involve the main and the shadow collection (see instructions below).

This module allows to keep the change history of every document and the deleted documents. The idea is to have a "main collection" storing the current document versions and a different collection called "shadow collection" to keep all the past versions and deleted documents.

See Basic Usage.

Use

In order to use the package just install it as a dependency

npm install mongoose-versioned

Basic usage

This package requires mongoose and it is added as a plugin to each individual model that will be versioned. Get familiar with mongoose before using this package (https://mongoosejs.com/docs/index.html).

// import versioning and mongoose related dependencies
const versioning = require('mongoose-versioned')
const constants = require('mongoose-versioned/source/constants')

const mongoose = require('mongoose')
mongoose.Promise = require('bluebird')
let Schema = mongoose.Schema

// connect to the database following mongoose instructions, for example:
let mongodb_uri = 'mongodb://localhost/test'

const versionItems = async(mongodb_uri) => {
  try {
      await mongoose.connect(mongodb_uri, { useUnifiedTopology: true, useNewUrlParser: true, useFindAndModify: false })
      console.log("Database.connect: DB connected ")
  } catch (err) {
      console.error(`Database.connect: MongoDB connection error. Please make sure MongoDB is running:` + err.message)
      throw new Error(err)
  }

  const db = mongoose.connection

  // create the model
  let itemSchema = new Schema({
    code: { type: Number, required: true, unique: true },
    name: { type: String, required: true, unique: false }
  })

  // add the versioning plugin to the schema and specify
  // the name of the shadow collection
  const name = 'item'
  itemSchema.plugin(versioning, {collection: name + "s.versioning", mongoose})

  // instantiate the model
  let Item = mongoose.model(name, itemSchema)
  // at this point a collection named 'tests'

  // add a new document
  const newItem = {
    code: 1,
    name: "first item"
  }

  let savedItem = await new Item(newItem).save()
  console.log(`saved item with id: ${savedItem._id}, version: ${savedItem._version}`)

  let id = savedItem._id

  // update document info
  savedItem.name = "modified item"

  // add edition information
  savedItem[constants.EDITOR] = "editing user"
  
  // perform the update
  let updatedItem = await savedItem.save()
  console.log(`updated item with name: ${updatedItem.name}, version: ${updatedItem._version}`)

  // find current version
  let foundCurrent = await Item.findVersion(id, 2, Item)
  console.log(`found current version ${foundCurrent._version}, name = ${foundCurrent.name}`)

  // find old version
  let foundOld = await Item.findVersion(id, 1, Item)
  console.log(`found current version ${foundOld._version}, name = ${foundOld.name}`)

  await db.close()
}

versionItems(mongodb_uri)

Using transactions

Transactions

Transactions can be used to ensure the database remains in a consistent state even if the operation fails. Update and delete operations involve changes in both main and shadow collections and therefore need to be wrapped in a transaction to ensure serialization.

The transaction should be stated before calling the update/delete operation and in addition the session should be stored in a reserved "_session" inside the document and passed as an option to save/delete method.

const versioning = require('mongoose-versioned')
const constants = require('mongoose-versioned/source/constants')

const mongoose = require('mongoose')
mongoose.Promise = require('bluebird')

[...]

try {
  // start transaction
  session = await mongoose.startSession()
  session.startTransaction()

  // store session in the document
  document[constants.SESSION] = session

  // save sending the session as option
  await document.save({session})

  // commit transaction
  await session.commitTransaction()
  session.endSession()

} catch(error) {
  if (session) session.endSession()
  const message = `Error updating document ${id} in the collection ${collection}.`
  processError(res, error, message)
}

mongoose-versioned's People

Contributors

espona avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

mongoose-versioned's Issues

Fill in ReadMe

The readme has no content. It would be good to have it filled in with basic usage instructions.

At the same time, I am a little confused about the status of the project, given the last code change was 4 years ago, but the last publish to NPM was 3 months ago (2022-12-04T23:04:19.221Z):

https://www.npmjs.com/package/mongoose-versioning

Operation `identities.versioning.createIndex()` buffering timed out after 10000ms

hey! thanks for the awesome library, been searching for one for a while.

I am trying to use this with Nestjs. It should be straightforward since it's just registering a plugin. but I am running into this issue.

This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason:
MongooseError: Operation `identities.versioning.createIndex()` buffering timed out after 10000ms
    at Timeout.<anonymous> (/home/nest/identity-api/node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:151:23)
    at listOnTimeout (node:internal/timers:569:17)
    at processTimers (node:internal/timers:512:7)

here is how I register the plugin. which is basically the Nestjs way of registering a plugin.

import * as versioning from 'mongoose-versioned';
import mongoose from 'mongoose';

  MongooseModule.forFeatureAsync([
      {
        name: Identity.name,
        useFactory: () => {
          const schema = IdentitySchema;

          schema.plugin(versioning, {
            collection: 'identities.versioning',
            mongoose,
          });
          return schema;
        },
      },]

Add Typescript support

It would be useful to convert this project to be Typescript based.

Is this something that would be of interest?

BTW I have taken an initial look at the changes involved and would be willing to contribute changes.

Versioning of old documents

Hey! I am trying to integrate versioning into my already-existing collection, yet I am facing issues with posts they were previously there before introducing the plug in

When updating one of them, I get: Error: modified and base versions do not match
This is because version is missing from the origianl document, so by I adding "_version" : 1 manually to the document, I bypass that and encounter TypeError: Cannot read property 'start' of undefined
adding

"_validity" : {
        "start" : ISODate("2022-08-05T16:37:07.442Z")
    },

fixed this issue.

is there a way where I can take care of those legacy records, and not to have to manually update them to include version 1 and. start date? Maybe that will be a good enhancement, to treat old documents are version 0 with a startDate of the object time stamp or something.

Thanks!

Wait until database is ready

I tried using mongoose-versioned today and ran into an issue because in our case the database is only connected to after the schemas are declared. Is there any way that mongoose-versioned can be lazy and only create the shadow database on connect or first update?

The error we get:

/Users/ajmas/projects/myproject/node_modules/mongoose-versioned/node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:175
          const err = new MongooseError(message);
                      ^
MongooseError: Operation `people_history.createIndex()` buffering timed out after 10000ms
    at Timeout.<anonymous> (/Users/ajmas/projects/myproject/node_modules/mongoose-versioned/node_modules/mongoose/lib/drivers/node-mongodb-native/collection.js:175:23)
    at listOnTimeout (node:internal/timers:557:17)
    at processTimers (node:internal/timers:500:7)

A snippet of our code:

// Mongoose creates 'person' as 'people', so just define the name we want
PersonSchema.plugin(Versioning, {collection: 'people_history' });

const PersonModel = mongoose.model<IPersonDB>('Person', PersonSchema) as SoftDeleteModel<IPersonDB>;

export default PersonModel;

_editor and _deleter custom values

Hey! I would like to use mongoose-versioned for my project, yet i can't really find any documentation for non-basic use cases
i would like to store custom values in _editor and _deleter fields

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.