scottwrobinson / camo Goto Github PK

I've been currently been running into problems with embedded documents when trying to execute the loadMany function on one of my models. I have my class that's an embedded document and another class based off the class from the embedded document

class myMainClass extends EmbeddedDocument { ... }
class subClass extends myMainClass { ... }

myMainClass.loadMany(query, fn)

This is how I currently have the code implemented, but myMainClass does not have the loadMany function. Am I doing something wrong or is there currently not a function in the EmbeddedDocument class?

It would be very nice if camo validator would accept field with declared type Data values as strings.
Currently we may use numbers or date instances but no date strings.

I think simple Date.parse would suffice to interpret strings.

Why:

I'm using camo (with my neat hack) in a browser (via my own REST client for camo) and after serializing/deserializing JSON objects, all Date fields become ISO date strings on the backend.

The workaround is to:
Date.prototype.toJSON = Date.prototype.getTime in the browser.

but I would like to leave Date.prototype.toJSON untouched and allow the camo validators to convert those date strings to Date objects.

I'm using meteorJS which does some odd things with the _id field for mongo records.
I want to use plain strings as _id s with Camo to avoid this type of mix/match when writing from a raw node app with camo:

I tried overriding the _id field in a document, but that didn't work. Is this something that can be done?

var Document = require('camo').Document;

class Lessons extends Document {
  constructor() {
    super('Lessons');
    this._id = 'id' + Math.random;
    this.cname = String;
    this.rive = String;
  }

  static collectionName() {
    return 'Lessons';
  }

}

info:
http://stackoverflow.com/questions/15318184/meteor-collection-objectid-vs-mongodb-objectid

I'm trying to mimic the Angular-Fullstack sync function over socket.io.

They do it by calling the 'post save' hook in mongoose. However, I'm not able to reproduce this with camo.

I'm aware of Camo having hooks, but they are exposed as Object based. I wonder if there is somehow to get this Class based.

I not sure if I'm making clear myself, I think I'm not.

Sorry for my bad english, and my bad explanation.

Thanks.

PS: I think maybe this can explain what I mean:
thing.socket.js

Because 'default' it's a reserved word.

When you create a new document, it gets return with all the reference as objects, but when it's loaded, only loades the ids.

Creating:

Room {
  _schema:
   { _id: { type: [Function: String] },
     title: { type: [Function: String], required: true },
     owner: { type: [Object], required: true },
     creationDate: { type: [Function: Date], default: 1455393132301 },
     clients: { type: [Object] },
     image: { type: [Function: Image] },
     classroom: { type: [Object] } },
  _id: 'WnvnnT8PEDNxVMVD',
  title: 'asdas',
  owner:
   User {
     _schema: { _id: [Object], user: [Object], role: [Object] },
     _id: 'ZLTyRwTJDx6S5gdn',
     user: 'asdasd',
     role: 'user' },
  creationDate: Sat Feb 13 2016 15:22:12 GMT-0430 (Hora estándar de Venezuela),
  clients:
   [ User {
       _schema: [Object],
       _id: 'ZLTyRwTJDx6S5gdn',
       user: 'asdasd',
       role: 'user' } ],
  image: undefined,
  classroom:
   Classroom {
     _schema:
      { _id: [Object],
        className: [Object],
        date: [Object],
        whiteboards: [Object] },
     _id: 'LJiplNc7Wz2u4lm5',
     className: 'asdas',
     date: Sat Feb 13 2016 15:22:12 GMT-0430 (Hora estándar de Venezuela),
     whiteboards: [ [Object] ] } }

Loading:

Room {
  _schema:
   { _id: { type: [Function: String] },
     title: { type: [Function: String], required: true },
     owner: { type: [Object], required: true },
     creationDate: { type: [Function: Date], default: 1455396264869 },
     clients: { type: [Object] },
     image: { type: [Function: Image] },
     classroom: { type: [Object] } },
  _id: 'nRF8AXEVaYSAggjW',
  title: 'sdf',
  owner:
   User {
     _schema: { _id: [Object], user: [Object], role: [Object] },
     _id: 'HtRw1URfSc23KEeA',
     user: 'asdas',
     role: 'user' },
  creationDate: Sat Feb 13 2016 16:14:08 GMT-0430 (Hora estándar de Venezuela),
  clients:
   [ User {
       _schema: [Object],
       _id: 'HtRw1URfSc23KEeA',
       user: 'asdas',
       role: 'user' } ],
  image:
   Image {
     _schema: { data: [Object], contentType: [Object] },
     data: undefined,
     contentType: undefined },
  classroom:
   Classroom {
     _schema:
      { _id: [Object],
        className: [Object],
        date: [Object],
        whiteboards: [Object] },
     _id: 'KumWJNLXNl6JEVEC',
     className: 'sdf',
     date: Sat Feb 13 2016 16:14:08 GMT-0430 (Hora estándar de Venezuela),
     whiteboards: [ 'DtCIuOKLRj9gdcz9' ] } }

Well, they both have the classroom, not just the reference, but the whiteboards (being a Array of Whiteboard documents) only load the id.

Is this the expected behavior? If it is, I'm sorry. I'm new with Document-oriented databases.

Thanks!

Doc1 extends Document {
...
}
Doc2 extends Document {
  constructor(){
    this.ref = {
      type: Doc1
    }
  }
}

Doc2.create({
   ref: "valid ObjectId string" 
})

In this case, It passes the validation, but reference id is saved as String not a ObjectId.

Example from your README will throw error: ReferenceError: schema is not defined

constructor () {
 super();
 schema({
     name: String,
     valuation: {
         type: Number,
         default: 10000000000,
         min: 0
     },
     employees: [String],
     dateFounded: {
         type: Date,
         default: Date.now
     }
 });
}

right code is:

constructor () {
 super();
 this.schema({
     name: String,
     valuation: {
         type: Number,
         default: 10000000000,
         min: 0
     },
     employees: [String],
     dateFounded: {
         type: Date,
         default: Date.now
     }
 });
}

Values for non-required null items should be left undefined when populating the database. This is a feature of document databases that keeps the column width from blowing out the data store for wide sparse documents. Any chance you can update this? I'm not quite sure where to look for that final set from Schema to key-value pair.

I am having an issue when trying to create (and save) an EmbeddedDocument. I have the following class:

class Reminder extends camo.EmbeddedDocument
{
  constructor()
  {
    super();
    this.enabled = { type: Boolean, required: true };
  }
}

Now, when I create a Reminder instance that does not adhere to the above, e.g. the enabled property is missing, like in this example:

Reminder.create({}).save();

I get the following error:

info: TypeError: that.collectionName is not a function

This is obviously an artifact of the new collectionName functionality in the validate function inside camo. Except, collectionName does not exist in EmbeddedDocument instances.

Probably quite an easy fix.

I am currently having an issue with the model/scheme validation in case of a loadOneAndUpdate call.

Specifically, consider the normal create function, as in camo's examples, which obviously validates the input. I've added a quite strict model as well:

class Dog extends camo.Document {
    constructor() {
        super('dogs');
        this.name = { type: String; required: true; };
        this.breed = { type: String; choices: [ 'Collie', 'Bulldog' ] };
    }
}

var lassie = Dog.create({
    name: 'Lassie',
    breed: 'Collie'
});

lassie.save().then(function(l) {
    console.log(l.id);
});

However, let's say I want to update the breed of a specified Dog entry. In this case, I would use the loadOneAndUpdate function since I want the other values to remain the same, and the create function obviously wouldn't work because of the missing name:

Dog.loadOneAndUpdate({ id: ... }, { breed: 'totallywrongvalue' }).then(function(l) { ... });

Note that the breed does not adhere to the model here. Yet, the document is actually updated with this invalid value.

To solve this issue, I guess loadOneAndUpdate would have to do some kind of validation. Perhaps additionally, an explicit validation function validate could be used and exposed which validates any given data against the model.

Quick bug post.

Whenever an Document is returned that contains an EmbeddedDocument, the object that represents the EmbeddedDocument always contains an element _id that is set to null.

It is obviously set to null because EmbeddedDocument's do not have id's, however the _id shouldn't be defined at all.

class Embedded extends camo.EmbeddedDocument
{
  constructor()
  {
    super();
    this.whatever = String;
  }
}
class ABC extends camo.Document
{
  constructor()
  {
    super();
    this.a =  { type: Embedded, required: false };
  }
}

Whenever ABC is returned, a will always be defined and contain a _id set to null.

Right now if you put some invalid validator in the schema, it throws a not found error.

that._schema[key].match.test is not a function

In the example, 'match' it's the name of the invalid validator.

I guess it should throw some information about where it is being used.

If you delete a document, the referenced id of the other document becomes null, and that's if you save the document with the referenced id.

This is a unexpected behavior. I think it should throw an error to show that's being used.

Hello.

If you set type to Array or [], the type is not being validated at all.

If you change https://github.com/scottwrobinson/camo/blob/master/test/document.test.js#L792 to anything other than array document will be saved and test will fail.

I don't know if i'm doing this properly, but I'm trying to hash the password in either 'PreSave' or 'PostValidate' (I supposse PostValidate get's calling just before PreSave).

PostValidate got called, I think PreSave is not. But in either case, I doesn't make permanent changes to the document. The changes that are made, are lost when they get save.

    postValidate () {
        if(this.password) {
            console.log('Salting');
            this.salt = new Buffer (crypto.randomBytes(16).toString('base64'), 'base64');
            console.log(this.salt);
            this.password = this.hashPassword(this.password);
        }
        this.updated = Date.now();
    }

    postSave () {
        console.log('Post save');
        console.log(this);
    }

The console outputs show they are being called. PostSave show changes in the document, but it the datastorage (I'm using NeDB) there is not salt in the document, and the password it's not hashed.

Any thoughs?

Thanks for camo. I don't think I ever got to post so much in a issue list, that is not mine own.

PS: I'm trying to using this with Passport, but all the examples are with 'mongoose'. I'm trying to write all the examples to camo, but I've fail, so far. Have you use Camo with Passport? How did you?

Hey,

Spent the last 6 hours trying to figure out how to deep populate a comments model. Nested models are not getting populated. With mongoose we can use a plugin called deepPopulate or populateDeep, and it works. But with Camo (I love camo btw) there really isnt a way to do it. I tried really hard and nothing works. Please let me know if you have any ideas..

Right now you can use populate to get just the fields you want, but, can you just exclude fields?

I mean something like this:

Person.loadOne({name: 'Billy'}, {populate: [-'address', -'spouse']});

or

Person.loadOne({name: 'Billy'}, {populate: ['-address', '-spouse']});

or maybe

Person.loadOne({name: 'Billy'}, {exclude: ['address', 'spouse']});

The expect behavior it's to exclude those fields in the loaded person. Of course right now you can just populate all the fields that you want, but if you add something else, you have to add it in populate too.

It's not a breaking feature, but it's a nice one to have I think.

Also, I see some issues about this. What if some one makes a request as:

Person.loadOne({name: 'Billy'}, {populate: ['address', '-spouse']});

Well, if you want to populate 'address' only, the '-spouse' will be exclude by default, that's the actual behavior.

You asked how to do this better here: https://github.com/scottwrobinson/camo/blob/master/lib/document.js#L41

Just use a static property static collectionName = 'people'; instead of using super('people').

Then your Document class can do this.constructor.collectionName to access it.

http://jsfiddle.net/resistdesign/wgsh2sex/

Are there production usable examples that one can reference? For example, catching validation errors, missing embedded documents e.t.c? Thanks

Hello there!

As I can see from the docs there is no currently direct way to specify that some field in document is required. Am I right?

If so I would like to request this functionality or even happy to make a pull request if you would accept one :)

The .create() method can be used to initialize a Document object with data, as well as nested EmbeddedDocuments, but it can't initialize a nested array of EmbeddedDocuments.

So this should work, but doesn't:

class Discount extends EmbeddedDocument {
    constructor() {
        super();
        this.authorized = Boolean;
        this.amount = Number;
    }
}

class Product extends Document {
    constructor() {
        super('products');
        this.name = String;
        this.discounts = [Discount];
    }
}

var product = Product.create({
    name: 'bike',
    discount: [{
        authorized: true,
        amount: 9.99
    },
    {
        authorized: false,
        amount: 187.44
    }]
});

Currently there is no easy way to make use of those atomic updates to underlying documents. Perhaps it would be easy to enhance Document.prototype.loadOneAndUpdate in such a way:

MyDoc.loadOneAndUpdate(query, {somePropToIncrement: 2}, {update: "inc"})
so the client implementation would be such like:

var update = options.update || "set";
db.findOneAndUpdate(query, {[`$${update}`]: values}, options);

I trying to use 'match', but as #41 it throws not a function.

Here is my schema:

class User extends Document {
    constructor(){
        super();
        //Just the part where using the match. 
        this.email = {
            type: String,
            match: '/.+\@.+\..+/'
        }
}

This issue happens when an object has an array of references and there are multiple references to the same object in the array.

In the code below, we give the user two references to the same Eye, but when we load the user there is only one reference there.

class Eye extends Document {
    constructor() {
        super('eyes');
        this.color = String;
    }
}

class User extends Document {
    constructor() {
        super('users');
        this.eyes = [Eye];
    }
}

var user = User.create();
var eye = Eye.create({color: 'blue'});

eye.save().then(function(e) {
    user.eyes.push(eye, eye);
    return user.save();
}).then(function(u) {
    return User.loadMany({});
}).then(function(users) {
    expect(users[0].eyes).to.have.length(2);    // FAILS
});

When instantiating a model with create() and initializing with data, the create() method doesn't use the virtual setters when assigning the data.

Example:

class User extends Document {
    constructor() {
        super('user');
        this.firstName = String;
        this.lastName = String;
    }

    set fullName(name) {
        var split = name.split(' ');
        this.firstName = split[0];
        this.lastName = split[1];
    }

    get fullName() {
        return this.firstName + ' ' + this.lastName;
    }
}

var user = User.create({
    fullName: 'Billy Bob'
});

The above code would not properly set firstName and lastName.

When using get and set methods in a Document, the Proxy correctly intercepts the call, but then any gets/sets done within the intercepted get/set is not intercepted by the Proxy.

Example:

class User extends Document {
    constructor() {
        super('user');
        this.firstName = String;
        this.lastName = String;
    }

    get fullName() {
        return this.firstName + ' ' + this.lastName;
    }
}

Calling user.fullName returns the string representation of the String object, which is the schema definition, and not the actual values of firstName and lastName.

Would there be an example of camos usage in the browser soon? Thanks

There are some inconsistencies with the _id to id field aliasing.

When I perform a load call (e.g. loadMany with {} empty query), camo returns objects with the _id field aliased to id:

{
  ...
  "id": "5669e44f5d531cbd00e7d106"
}

However, when specifying an actual input query to these load calls, I am unable to specify the id field, and instead have to specify MongoDB's original _id field, as using the id field yields no results at all:

MyModel.loadMany({
  "id": "5669e44f5d531cbd00e7d106"
});

In other words, for queries, the id field is not aliased back to _id, which is inconsistent with the load call result behavior, and requires the use of additional workarounds in the application using camo.

I have a specific need of supporting schema (camo's Document) definitions accross modules:

With a basic schema:

import {Document} from 'camo';

export default class User extends Document {
  constructor() {
    super();
    this.name = String;
    this.breed = String;
  }
}

Required from another module:

const uri = 'mongodb://127.0.0.1:27017/core';
log.info('Connecting to database uri="%s"...', uri);
connect(uri).then(db => {
  log.info('Connected to database uri="%s"', uri);
  return;
})
.then(() => {
  log.debug('creating...');
  const lassie = User.create({
    name: 'Lassie'
  });
  log.debug('saving...', lassie);
  return lassie.save().then((l) => {
    log.debug(l.id);
  });
})
.then(() => {
  return User.loadOne({name: 'Lassie'}).then((l) => {
    log.debug('Got Lassie!');
    log.debug('Her unique ID is', l.id);
  });
});

With latest version, this currently hangs on User.create.

More specifically, it looks like that it is this line that do hang:

super();

Which is compiled by babel 6 to something like:

var _this = _possibleConstructorReturn(this, Object.getPrototypeOf(User).call(this));

Any idea where this could come from or where should I look into?

I tried to test the count() function, given this example:

connect(uri).then(function(db) {
    database = db;
    console.log('db: ', db);

    var ibm = Company.create({
        name: 'ibm',
    });

    // ibm.save().then(function(l) {
    //     console.log('ibm: ', l.id);
    // });

    Company.loadMany({
        name: 'ibm'
    }).then(function(doc) {
        //console.log('docs:', doc);
    }).count(function(count) {
        console.log('count----> ', count);
    })

    Company.count({
        name: 'ibm'
    }).then(function(count) {
        console.log('count:', count);
    })
});

In both log cases, nothing happens, no errors or output. Am i doing something wrong? Thanks

Hi Scott!

some lodash methods are depricated, e.g. you used _.extends in base-document.js
see https://github.com/lodash/lodash/wiki/Roadmap

"Remove aliases all, any, backflow, collect, compose, contains, detect, extend, fold, foldr, head, include, inject, iteratee, methods, object, select, tail, & unique"

kind regards
gabriel

Just a comment for the documentation re. loadOneAndUpdate
For the people who have the same problem I had - you can't pass _id through in the values
It will fail silently (almost). Took some digging to find out why.

I've scrubbed the property from the object in my code, but could loadOneAndUpdate do that by default?

I like the camo API a lot, but wonder if you'd consider adding a way to use it more in a 'synchronous' style with the new await / async ES7 stuff?

Maybe add the timestamps fields by default?(created_at, updated_at).

Hello!

I want to set an unique index to an specific schema entry. Is this possible with camo?

Thanks in advance,
Rene Hollander

It would be nice if there is the 'required' option in schema options.

{
  type: String,
  required: true
}

It can be implemented by the 'validation' but It would be more convenient.

@ lib/document.js:31
// TODO: Is there a way to tell if a class is
// a subclass of something? Until I find out
// how, we'll be lazy use this.

If you need to get the subclass of an instance in a parent method - just use Object.getPrototypeOf(this).constructor

'use strict';

class A {
  getSubclass() {
    return Object.getPrototypeOf(this).constructor;
  }

  isSubclass() {
    return Object.getPrototypeOf(this).constructor !== A;
  }
}
class B extends A {}
class C extends B {}

// Static usage:
console.log(Object.getPrototypeOf(C).name); // B
// To get top parent you need to traverse the prototype chain:
console.log(Object.getPrototypeOf(Object.getPrototypeOf(C)).name); // A

// Instance usage:
let c = new C();
let a = new A();

console.log(c.getSubclass().name); // C
console.log(c.isSubclass()); // true
console.log(a.isSubclass()); // false

Maybe you can replace some lodash calls to ES6 calls, e.g. there are many new APIs (e.g. Array API) Instead of _.keys() you can call ["a", "b", "c"].keys() . Please have a look to https://github.com/lukehoban/es6features#readme

The only thing holding me from trying swapping Mongoose with Camo is the lack of a stream interface for reading, is it planned?

This issue happens when there are multiple objects in the database, each object has an array of references, and at least two of the object's arrays contain the same reference.

Consider the following example:

class Eye extends Document {
    constructor() {
        super('eyes');
        this.color = String;
    }
}

class User extends Document {
    constructor() {
        super('users');
        this.eyes = [Eye];
    }
}

var user1 = User.create();
var user2 = User.create();
var eye1 = Eye.create({color: 'blue'});
var eye2 = Eye.create({color: 'brown'});

eye1.save().then(function(e) {
    return eye2.save();
}).then(function(e) {
    user1.eyes.push(eye1, eye2);
    return user1.save();
}).then(function(u) {
    user2.eyes.push(eye1);
    return user2.save();
}).then(function(u) {
    return User.loadMany({});
}).then(function(users) {
    // Get user1
    var u1 = users[0].id === user1.id ? users[0] : users[1];

    // Ensure we have correct number of eyes
    expect(u1.eyes).to.have.length(2);    // FAILS
});

In this case, both user1 and user2 have a reference to eye1. But when we call .loadMany(), both user1 and user2 will have a duplicate reference to eye1, which is not correct.

Is there any easy way to use Camo with a non-ES6 codebase (Node.js standard CommonJS-style modules)? I would love to use it but I cannot switch our codebase over to ES6 modules as of yet (nor can I move to Node.js >= 2.x as of yet).

--removed--

Using babel-core@5 with babel-core/register and {stage: 0}.

This error: [TypeError: Class constructors cannot be invoked without 'new']

Is produced when running this code:

import { connect, Document } from 'camo';

const uri = 'mongodb://localhost:27017/camo';

let database;

class Contact extends Document {
  name = String;

  constructor () {
    super('contact');
  }
}

class Site extends Document {
  name = String;
  contacts = [Contact];

  constructor () {
    super('site');
  }
}

connect(uri)
  .then(db => {
    database = db;

    console.log('Connected!');

    const MA = new Site({
      name: 'Mass',
      contacts: [
        {
          name: 'One Guy'
        },
        {
          name: 'Some Dude'
        }
      ]
    });

    console.log('Initialized new Site.');

    MA.save()
      .then(data => {
        console.log('SAVED:', data);
      });

    console.log('Saving...');
  })
  .catch(err => {
    console.log('ERROR:', err);
    process.exit(1);
  });

How are these errors handled? I'm coming from Mongoose

I'm doing an API RESTful with camo using nedb as datastorage. The problem I'm facing is, when I update an array inside another array, I can't access that document anymore.

If I restart the server and load the datastorage again, I can access the data again, even the updated one, so it does saved it. How ever, I doesn't seem to be able to read it after.

Here's my code:

To update:

 Classroom.loadOne({_id: req.params.classId})
        .then(function(classroom){
            var shape = Shape.create({
                type: req.body.type
            });
            classroom.whiteboards[classroom.whiteboards.length-1].shapes.push(shape);
            classroom.save()
                .then(classroom => successHandler (res, shape))
                .catch(err => errorHandler (res, err));
        })
        .catch(err => errorHandler(res, err))

To read:

Classroom.loadOne({_id: req.params.classId})
        .then(classroom => successHandler (res, classroom))
        .catch(err => errorHandler (res, err));

successHandler and errorHandler, just send a response with 200 or 500 status, and jsonp of the document or the error.

Hope can help.

Thanks!

Looking at this project from the perspective of a TypeScript developer it is so close to what I would like to see from an ES6+ ODM. The big issue that keeps it from being perfect for TypeScript right now is that the type of a class property can jump from being "schema object" in the constructor to actual data type everywhere else. It would be great if you could place the schema information into separate symbols (and thus dummy/default values in the actual symbols) so that the typing information flows correctly to instances of that class. One way to do that and keep the class definitions easy to read might be to use ES2016 (ES7) Decorators.

This would be a useful convenience method to have. The big place I could use this is authentication code where the usernames are coming from an external source and I want to create a blank user record for a new to my app username.

Because of the importance of such a user lookup, it would be best if can attempt to be atomic in the database. Current version of my code is using the mixin mongoose-findorcreate for mongoose, and this convenience method would be another push over the hump of switching away from mongoose.

It's possible you might want to use an option on loadOne(...) instead of an entirely new method loadOneOrCreate(...), such as loadOne(..., { create: true }). For my use case I don't particularly need to know if this one was created for the first time or not, but that may be something that others are interested in and returning that information might be reason to use a separate method instead of sharing one. Your call.

Thanks.