arunoda / meteor-smart-collections

Meteor Collections Re-Imagined

This is a archived project

Smart Collection is now retired & Meteor's Collection implementation has fixes for most of the performance bottlenecks. It is also using the MongoDB oplog just like Smart Collections.

Meteor Smart Collections

This is a complete re-write of the MongoDB Collection implementation for Meteor. Designed with following 3 areas in mind

• Speed
• Efficiency (Memory & CPU)
• Scalability

This is not a toy project! But a complete Collection replacement with a well tested source code. Still we might have bugs :)

Click here for implementation details of Smart Collections

Install

Install Smart Collections from Atmosphere

mrt add smart-collections

Install From Git (If you are not using Meteorite)

mkdir -p packages
#make sure you created the packages folder
git submodule add https://github.com/arunoda/meteor-smart-collections.git packages/smart-collections

Usage

Replace Meteor.Collection with Meteor.SmartCollection. Just that!

eg:-

//old code
Posts = new Meteor.Collection('posts');

//with smart collections
Posts = new Meteor.SmartCollection('posts');

Compatibility

• Almost compatible with exiting Collection API
• But server side Cursor.observe() does not exists
• _id must be a String (will support ObjectID and numbers soon)

Scalability

• Can be easily scaled with mongodb oplog
• Follow this guide on how to scale meteor with Smart Collections.

meteor-smart-collections's Issues

horizontal scaling: current state and future?

Hi there!
In searching for scaling options I came across some update notes saying:

• smart-collection is a better way to scale meteor than meteor-cluster http://meteorhacks.com/meteor-cluster-introduction-and-how-it-works.html
• smart-collection's oplog feature is now part of the stock meteor http://meteorhacks.com/lets-scale-meteor.html

Now, this left me puzzled about the future of scaling to multiple meteor server instances.

I think support for multiple synced meteor servers that can work independently and redundantly (even in disconnected operation mode while backoffice servers are down) is almost there, with the new server-to-server DDP support. meteor/meteor@53de3f2

There only seem to be two points missing for Meteor.Collection():

• On the server if you specify a connection, also apply writes to local collection (like clients), before calling methods on the backoffice server.
• On the server if you specify a connection, support setting up local mongo collections (instead of using minimongo).

Then it could become really easy to dynamically scale up to multiple meteor instances (horizontally) that all connect to a central "backoffice" instance. Don't you think?

(PS: The idea and points originated at meteor/meteor#1723, but unfortunately these points don't seem to be part of the core roadmap.)

LocalCollection cursor.js ln47 (master)

I cant use mrt to test smart collections adding grounddb support so I forked master but gets:

Exception from sub undefined ReferenceError: LocalCollection is not defined

capped collections don't stay capped through SmartCollections

I created a capped collection with count limit of 100:

db.createCollection('last100', {capped : true, size: 1024*1024, max: 100 })

The MongoDB is correctly keeping the total count at 100 as items are added, but the client side is not doing so:

Using observeChanges, it appears that only "added" happens - not sure if that means the opLog doesn't record 'removes' for capped collections (and they're implicit) or if they're recorded in some different format than a normal remove or what.

This was the js I used (in a top-level .js, running on both client and server)

Last100 = new Meteor.SmartCollection("last100"); 
Query = Last100.find();
var count = 0;
Query.observeChanges({
    added: function (id, user) {
        count++;
        console.log("ADDED:", count, Query.count());
    },
    removed: function () {
        count--;
        console.log("REMOVED:", count, Query.count());
    }
});

Latency Compensation and Write Fence

See related discussion with David Glasser

latency compensation (the write fence): Meteor methods (such as the
auto-generated insert/update/remove collection methods) have two different
"done" messages. One is the "result" message which contains the method's
return value or error, and is delivered as soon as the method body returns
or throws. The other is the "updated" message, which specifies that any
writes done by the method have been reflected in data messages
(added/changed/removed/etc) sent from server to client. The latter method is
what links together the two components of DDP (methods and data). If any
collection write happens in a method body, then by use of an object called
the Write Fence, we ensure that the "updated" method does not get sent until
all possibly-affected cursors have been polled one time. The client uses
this message to prevent flicker (ie, latency compensation): essentially, any
documents that are modified by the client-side stub are "frozen" until the
method's "updated" message shows up, at which point we should have seen the
final value of the documents.

RegExp fix in filtering out oplogs at first

See: https://github.com/arunoda/meteor-smart-collections/blob/master/lib/mongo.js#L31

$geoWithin operator support

Hello,

Is $geoWithin or at least deprecated $within operator supported?

With SmartCollection I'm getting "Exception from sub PRhpASkNubaoSaWz5 Error: Unrecognized operator: $geoWithin". With Meteor Collections it works fine (both $within and $geoWithin)

.

Nothing, please delete :)

TokuMX version of MongoDB

Hello,

In my pursuit of looking for MongoDB transactions I stumbled upon TokuMX: http://www.tokutek.com/products/tokumx-for-mongodb/; a MongoDB fork that provides for some interesting implementation of ACID transactions as well as some performance improvements. Needless to say I was intrigued, however, there was one caveat they modified the MongoDB Oplog format to accommodate transactions so some modifications are required.

I would like to go about changing the Oplog feature through a hook? I know this may not be a suitable question for this repo

Regards,
David

Field filtering support

We need to filter fields in the changes too.

Collection methods return empty array, IE fetch(), forEach(), etc...

I'm declaring a collection like so:

Maps = new Meteor.SmartCollection('maps');

This file is location at:
/collections/ideas.js

Then I'm executing this function when the page loads:

Template.maps.rendered = function(){
    console.log(Maps.find({}).fetch());
}

The result is an empty array that gets logged to the console. If I execute the same function after the page has loaded, it returns a populated array.

I'm using the latest version v0.3.23

Standalone version for node community

With your knowledge of Oplog and Mongo have you ever thought about making standalone version of Smart Collection for the node community, I think this would be awesome and would have huge impact.

Think about it, I would be happy to help :-)

JB

Updating to latest version causes cpu to spike to 100% and stay there

Hi,

I updated a meteor app's smart-collections version from 1.1.16 to the latest version today, and after deploying the cpu usage is stuck at 100%. There is one fairly large collection so I expect cpu to be very high while that is initially loaded, but the app continued to climb in memory usage and cpu until it went unresponsive.

The only thing that had changed was updating smart-collections, so I am trying to think of info that might help identifying the issue:

• I am not using oplog
• Meteor v0.6.4.1
• There are no updates or inserts going on during this testing, just publishing and subscribing
• Autopublish is removed
• Downgrading back to 0.1.16 fixes the cpu issue entirely <---- !!!
• Network usage was also very high and dropped as soon as I downgraded, this makes me think the collections are being queried for changes or something at a high frequency?

Let me know if you need any more info, I will continue testing. Next step is to enable the oplog and see if that helps...

MongoDB oplog setup and settings for SmartCollections

SmartCollections uses oplog for finding changes. I have setup oplog by adding the right configuration to /etc/mongodb.conf and is called rs0. How do get SmartCollection to watch the oplog. Or what oplog settings to I need for SmartCollections to recognize it? What is the process of SmartCollections recognizing the oplog?

smartcollections

hi,I downloaded the package.Can u please guide me how to run a sample application using smart collections? using standard collection its working well

Typo in README docs

Effieciency - > Efficiency

Cursor with Limit

When we've cursor with limit, after the limit has reached, we should not add new documents to the collection.

With Sorting

If there is a sort we need to accept new documents and if those can be fit to the new limitedCursor, we need to add that document and remove the document at the end of the cursor.

When Removed

Need to get the next available document from the db. May need a query-rerun

Current Smart Collection Behaviour

Smart Collections does not detect this and keep sending accepting documents to the cursor.

Are we forgetting Meteor.Collection options or am I missing something?

http://docs.meteor.com/#meteor_collection

new Meteor.Collection(name, [options])

Mongo extensions (aggregation) support

I am migrating my application to use Smart Collections and I faced the lack of aggregation support which I had already overcome using a snippet based on meteor-mongo-extensions, so I decided to adapt this extension to work with Smart Collection.

My problem is that it was too easy to do I am afraid I am missing something else, so I created this topic to discuss this matter. Please, tell me if my thought is right.

• Mongo extensions seems to create a dummy collection to deal with the desired operations, and these operations seem to me that they do not need to use smart collections because the operations are volatile, are returned just in memory, except for mapReduce that saves its result in the db and should be opLog compliant.
• Considering the above, just linking Meteor.SmartCollection to the distinct and aggregate functions that use regular dummy collections should be enough.
• For the mapReduce we have 2 situations.
  • We can also bind the function as we did above, and then our "mapReduce created collections" won't be shared across mongodb instances, but will be accessible by the node process who called the function. [That would be particularly enough for me, because I delete the collection after some processing]
  • We can instantiate the dummy collection as a smart one instead of a regular one for this operation.

Here my is gist that just binds the smart collections to dummy regular collections.

Aggregate SmartCollection?

meteor/meteor#644 doesn't work with SmartCollections. =\

Do query re-run for geopartial queries

This is a temporary fix for #23

Update to Meteor 0.7.1.1

Hi, just updated to the latest version of meteor, apparently subscribing to a smart collection is causing this issue:

Awkwardly enough that doesn't happen when the collection is a Collection2 Smart collection. Is there an easy fix for this?

Thank you

Retire package from Atmosphere

meteor admin set-unmigrated arunoda:smart-collections

so it won't show at https://atmospherejs.com/arunoda/smart-collections

Consider field filtering at query level

Now fields are not consider when caching, but we need to consider that also.

Things to be implemented

• consider fields when doing initiateQuery on invalidator
• filter fields when adding it to the query - do field filtering inside _raw*
• remove field filtering from observer

Use of Meteor._wrapAsync

User callbacks needs to be wrapped with - Meteor.bindEnvironment

• Needs to add support in serverCollection if there is an callback argument passed
• In the cursor we need to cover forEach and map
• need to add support for _observeChanges in cursor

Some sample code - https://gist.github.com/arunoda/3c51d90dbbdc7e383e99

underscore dependency missing from package.js

ReferenceError: _ is not defined
at OpQueue._processQueue (packages/smart-collections/lib/op_queue.js:52)

find query is not working while using Meteor smart Collections

Hey there!

I am using the lates version of smart-collections and want to test it.

I read the "documentation" of this package - maybe i need a sample - but in my case it does not work.

First what i did:

app/collections/collections.js

MyCollection = new SmartCollection('Test');
app/server/publishs.js

Meteor.publish('test', function(){
return MyCollection.find();
});
app/client/env.js

Meteor.subscribe('test');
Error output (shorted):

Exception from sub STxdGoBM293NARNGe TypeError: Object function (name) { // 10
I20140603-00:24:41.299(2)? var self = this; // 11
I20140603-00:24:41.300(2)? self.name = name; // 12
I20140603-00:24:41.300(2)? // _id -> document (also containing id) // 13
I20140603-00:24:41.300(2)? self._docs = new LocalCollection._IdMap; // 14
I20140603-00:24:41.301(2)? // 15
I20140603-00:24:41.302(2)? self._observeQueue = new Meteor._SynchronousQueue(); // 16
When I remove the subscribtion no error is shown. When i change my SmartCollection to Collection the but do not remove the subscribtion the error is also not shown.

Error using Smartcollections

I am getting this error when I add smart-collections to my project and configured my app to use them

W20130829-22:19:39.970(-5)? (STDERR) events.js:71
W20130829-22:19:40.084(-5)? (STDERR) throw arguments[1]; // Unhandled 'error' event
W20130829-22:19:40.084(-5)? (STDERR) ^
W20130829-22:19:40.085(-5)? (STDERR) TypeError: Cannot call method 'added' of undefined
W20130829-22:19:40.087(-5)? (STDERR) at Query.snapshot (packages/smart-collections/lib/query.js:200)
W20130829-22:19:40.088(-5)? (STDERR) at Cursor.each (/home/shinank/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/c497bdc8fcec8bfe45b456c28822d80466f1b798/.build/npm/node_modules/mongodb/lib/mongodb/cursor.js:244:9)
W20130829-22:19:40.089(-5)? (STDERR) at Cursor.nextObject (/home/shinank/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/c497bdc8fcec8bfe45b456c28822d80466f1b798/.build/npm/node_modules/mongodb/lib/mongodb/cursor.js:678:5)
W20130829-22:19:40.089(-5)? (STDERR) at Cursor.nextObject.commandHandler (/home/shinank/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/c497bdc8fcec8bfe45b456c28822d80466f1b798/.build/npm/node_modules/mongodb/lib/mongodb/cursor.js:658:14)
W20130829-22:19:40.089(-5)? (STDERR) at Db._executeQueryCommand (/home/shinank/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/c497bdc8fcec8bfe45b456c28822d80466f1b798/.build/npm/node_modules/mongodb/lib/mongodb/db.js:1670:9)
W20130829-22:19:40.090(-5)? (STDERR) at Server.Base._callHandler (/home/shinank/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/c497bdc8fcec8bfe45b456c28822d80466f1b798/.build/npm/node_modules/mongodb/lib/mongodb/connection/base.js:382:41)
W20130829-22:19:40.091(-5)? (STDERR) at Server.connect.connectionPool.on.server._serverState (/home/shinank/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/c497bdc8fcec8bfe45b456c28822d80466f1b798/.build/npm/node_modules/mongodb/lib/mongodb/connection/server.js:472:18)
W20130829-22:19:40.091(-5)? (STDERR) at MongoReply.parseBody (/home/shinank/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/c497bdc8fcec8bfe45b456c28822d80466f1b798/.build/npm/node_modules/mongodb/lib/mongodb/responses/mongo_reply.js:68:5)
W20130829-22:19:40.096(-5)? (STDERR) at Server.connect.connectionPool.on.server._serverState (/home/shinank/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/c497bdc8fcec8bfe45b456c28822d80466f1b798/.build/npm/node_modules/mongodb/lib/mongodb/connection/server.js:430:20)
W20130829-22:19:40.099(-5)? (STDERR) at EventEmitter.emit (events.js:96:17)

In 0.3.12 updates to documents are not always being communicated back to the client correctly

I've noticed that, since updating to 0.3.12, there have been odd behaviours when making updates to documents. Changes are being made in the database, but these changes are not always being relayed back to the client. So there's a flicker while the latency compensation uses the changed doc on the client but then when the server info comes back, the change is reverted (as if the change was rejected by allow/deny functions). But this is not the case, as refreshing the browser (or just checking the db) shows that the update has been persisted. However, this behaviour doesn't happen consistently. Sometimes I refresh the browser and everything's working as normal, sometimes I refresh the browser, perform the identical actions, and notice this problem.

SmartCollection.allow not compatibile with core Meteor Collections

Hello,

On my server I have this

MyCollection.allow({
    remove: function(userId, doc) {
        console.log(userId);
        console.log(doc)
    }
})

And on client

MyCollection.remove(doc_id)

With SmartCollection server prints null for userId which is correct (no logged in users), but for doc I get undefined. With Meteor Collection it prints the object with doc_id

GeoPartial Queries

Since, minimongo query selector does not supports geo-partial queries this SmartCollections lacks it.

Simply db.find() in publish does not work

Hey there!

I am using the lates version of smart-collections and want to test it.

I read the "documentation" of this package - maybe i need a sample - but in my case it does not work.

First what i did:

app/collections/collections.js

MyCollection = new SmartCollection('Test');

app/server/publishs.js

Meteor.publish('test', function(){
     return MyCollection.find();
});

app/client/env.js

Meteor.subscribe('test');

Error output (shorted):

Exception from sub STxdGoBM293NARNGe TypeError: Object function (name) {                                                                   // 10
I20140603-00:24:41.299(2)?   var self = this;                                                                                    // 11
I20140603-00:24:41.300(2)?   self.name = name;                                                                                   // 12
I20140603-00:24:41.300(2)?   // _id -> document (also containing id)                                                             // 13
I20140603-00:24:41.300(2)?   self._docs = new LocalCollection._IdMap;                                                            // 14
I20140603-00:24:41.301(2)?                                                                                                       // 15
I20140603-00:24:41.302(2)?   self._observeQueue = new Meteor._SynchronousQueue();                                                // 16

When I remove the subscribtion no error is shown. When i change my SmartCollection to Collection the but do not remove the subscribtion the error is also not shown.

Collection _id Migration

How do I ensure that I have the correct _id on my collections when I migrate the data from the meteor collection to using smart collections?

update return error and affected count undefined

i try to update collection with Meteor.Collection ( even with async callback function ) , the return error and count is correct always.
but when i replace Meteor.Collection with Meteor.SmartCollection . all return value change to 'undefined' . what can i do ? or this is bug ?

WriteFence support tests

Writes tests to cover writefence support. Work on linker branch.

Is smart-collections compatible with Meteor 0.7.1.2?

Hello

Here the problem is:

W20140319-00:23:38.627(4)? (STDERR) } has no method 'compileSelector'
W20140319-00:23:38.627(4)? (STDERR) at new Query (packages/smart-collections/lib/query.js:8)
W20140319-00:23:38.628(4)? (STDERR) at Invalidator.initiateQuery (packages/smart-collections/lib/invalidator.js:55)
W20140319-00:23:38.628(4)? (STDERR) at SmartCollection.find (packages/smart-collections/lib/server_collection.js:83)
W20140319-00:23:38.629(4)? (STDERR) at Meteor.publish.Dictionary.find.lang (app/lib/common.js:69:22)
W20140319-00:23:38.629(4)? (STDERR) at /home/v/local/build/programs/server/boot.js:159:61
W20140319-00:23:38.630(4)? (STDERR) at Array.forEach (native)
W20140319-00:23:38.630(4)? (STDERR) at Function..each._.forEach (/home/v/.meteor/tools/f3947a4651/lib/node_modules/underscore/underscore.js:79:11)
W20140319-00:23:38.631(4)? (STDERR) at /home/v/local/build/programs/server/boot.js:159:5

ObjectID Support

Date and Subscriptions

Sometimes documents with Date does not sent to client

Try with Oplog Enabed and Disabled.

Client not receiving updates made through server call.

I create a record via Minimongo, it gets placed in the database:

Using mongo Directly:
db.calls.find({ended:false})
{ "participants" : [ { "_id" : "f58785da-7ed2-477f-bfa8-816f06987938", "role" : "call_owner", "state" : "on_call" }, { "_id" : "oCgsHB9jhptbEKucr", "role" : "care_recipient", "state" : "waiting" } ], "start_time" : ISODate("2013-08-13T19:53:55.709Z"), "ended" : false, "room_id" : "jT7dQZ6Rhu4KqX8Sj", "show_video" : true, "show_self_view" : false, "on_hold" : false, "_id" : "LmxeZJDm6N3KqgCfq" }

From Minimongo:
Calls.find().fetch()[0]
Object {_id: "LmxeZJDm6N3KqgCfq", participants: Array[2], start_time: Tue Aug 13 2013 13:53:55 GMT-0600 (MDT), ended: false, room_id: "jT7dQZ6Rhu4KqX8Sj"…}

I make a backend call where this record is updated:

Calls.update({'room_id': body.RoomId, 'participants._id': body.UserId, ended: false}, {$set: {'participants.$.state': 'hang_up', ended: true, end_time: new Date()}}, {multi: true});

The client data doesn't ever get invalidated/updated however.

As viewed from mongo:
db.calls.find({_id:'LmxeZJDm6N3KqgCfq'})
{ "_id" : "LmxeZJDm6N3KqgCfq", "answered" : true, "end_time" : ISODate("2013-08-13T19:54:36.322Z"), "ended" : true, "on_hold" : false, "participants" : [ { "_id" : "f58785da-7ed2-477f-bfa8-816f06987938", "role" : "call_owner", "state" : "hang_up" }, { "_id" : "oCgsHB9jhptbEKucr", "role" : "care_recipient", "state" : "on_call" } ], "room_id" : "jT7dQZ6Rhu4KqX8Sj", "show_self_view" : false, "show_video" : true, "start_time" : ISODate("2013-08-13T19:53:55.709Z") }

And from minimongo:
Calls.find().fetch()[0]
Object {_id: "LmxeZJDm6N3KqgCfq", participants: Array[2], start_time: Tue Aug 13 2013 13:53:55 GMT-0600 (MDT), ended: false, room_id: "jT7dQZ6Rhu4KqX8Sj"…}

As you can see Minimongo still registers the object as having ended as false, where as the database directly has it set to 'true'.

Add skip support in find()

multiple observeChanges() in a cursor

Currently there is only one observeChanges() can be called for a single cursor.

I've not checked with how Meteor.Collection works with this. If someone has use this? just tell me how it works.

• When you call second observeChanges does it reset the cursor again?
• Is new observeChanges hooks also get the initial docs list too, or only the changes

cc @glasser

.observe() support in SmartCollections

support end-to-end tailable smart/capped collections

tl;dr - tail -n 50 -f [collection]

The original SmartCollections already did the heavy lifting to do create/use a tailable cursor on a capped collection (the oplog). For scenarios where there are meteor apps surfacing logging information from a capped collection, it seems like it could be efficient/useful to support that end-to-end, such that the meteor server could publish this collection and clients could subscribe with a particular parameter for getting the last N items inserted into the collection along with the ongoing inserts that the server would publish.

The SmartCollections (and presumably the built-in Meteor.Collection in meteor 1.0) may be pretty close to that target efficiency already (being event-driven from the tail of the oplog) - the developer story I couldn't figure out is basically how to publish/subscribe the 'tail' of the collection as such.

The existing Meteor.Collection API seems to require (AFAICT) doing a sort by $natural descending, then a find with limit: N (say, 50), then fetching that, then reversing them in memory. Given the nature of the capped collection and tailable cursor, that seems much more hacky and less efficient than something that'd be able to do a tailable cursor for 'give me the last 50 documents inserted, then keep giving me new ones as they show up' (which AFAICT is viable with the tailable cursor?)

Related StackOverflow question:

http://stackoverflow.com/questions/19214889/how-best-to-tail-f-a-large-collection-in-mongo-through-meteor

Thanks!!

Dropping a table does not update the client

When dropping all entries in a table directly from the mongo console/terminal (from meteor mongo), the client's database is not cleared.

0.3.0

Make it compatible with Meteor.Collection.
Apply recommendations from David Glasser.

queue updates to a document until it gets inserted.

In 0.3.22 updates to documents are not always being communicated back to the client correctly

There are still occasional discrepancies between the version of a document in the db and the version in minimongo. Note: this only happens rarely, but it's confusing for the user when it does happen. The exact scenario: an update is made to the db document and this update is communicated to all other connected clients, except the client that actually made the update. (It takes a browser refresh to see the update on the client that made the update.)

Field Filtering support for nested object

Check how its performing field filtering support in the nested objects

Currently field filtering only works on the top level fields only. Need support for following

• nested objects
• $
• elemMatch
• slice

If anyone of the above unimplemened features found, throw an error

Updating fails

I cannot update an entry.

This is the code I am executing. It's embedded within a meteor method

    attendEvent: function(eventId){

        //make sure user is logged in
        if( !Meteor.user()){
            throw new Meteor.Error(401, "You need to login to create an event");
        }
        var email = Meteor.user().emails[0].address;
        var date = new Date().toString();

        Events.update({_id: eventId}, {$push: {attending: email}, $set: {lastUpdated:date}, $inc: {attendingCount: 1} });

        return true;
    },

The code above works when I enter it in the javascript console and it updates it properly. However, when I try to call it through Meteor.call, it fails with the following error

Your app is crashing. Here's the latest log.


/Users/ericyen/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/7603242bee01c0cc6ce63aae65bcc3f43b30bd8c/.npm/node_modules/mongodb/lib/mongodb/connection/server.js:594
        throw err;
              ^
TypeError: Cannot read property 'pendingVersions' of null
    at VersionManager.cleanVersion (app/packages/smart-collections/lib/version_manager.js:90:30)
    at processInvalidation (app/packages/smart-collections/lib/invalidator.js:228:33)
    at Cursor.each (/Users/ericyen/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/7603242bee01c0cc6ce63aae65bcc3f43b30bd8c/.npm/node_modules/mongodb/lib/mongodb/cursor.js:243:33)
    at Cursor.nextObject (/Users/ericyen/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/7603242bee01c0cc6ce63aae65bcc3f43b30bd8c/.npm/node_modules/mongodb/lib/mongodb/cursor.js:685:35)
    at Cursor.close (/Users/ericyen/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/7603242bee01c0cc6ce63aae65bcc3f43b30bd8c/.npm/node_modules/mongodb/lib/mongodb/cursor.js:899:5)
    at Cursor.nextObject (/Users/ericyen/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/7603242bee01c0cc6ce63aae65bcc3f43b30bd8c/.npm/node_modules/mongodb/lib/mongodb/cursor.js:685:17)
    at Cursor.nextObject.commandHandler (/Users/ericyen/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/7603242bee01c0cc6ce63aae65bcc3f43b30bd8c/.npm/node_modules/mongodb/lib/mongodb/cursor.js:658:14)
    at Db._executeQueryCommand (/Users/ericyen/.meteorite/packages/smart-collections/arunoda/meteor-smart-collections/7603242bee01c0cc6ce63aae65bcc3f43b30bd8c/.npm/node_modules/mongodb/lib/mongodb/db.js:1659:20)
    at g (events.js:192:14)
    at EventEmitter.emit (events.js:126:20)
=> Exited with code: 1
=> Your application is crashing. Waiting for file change.

When I switch my collection back to a normal one, it works as expected.

issue with v0.3.11 causing, infinite loop error

With this v0.3.11 released, there is an infinite error. Which gives this error message.

events.js:72
        throw er; // Unhandled 'error' event
              ^
RangeError: Maximum call stack size exceeded

Possible fix is on branch: fix-crash-issue

New Polling/Query-Rerun algorithm

Place all the query-rerun & polling into a single place.

• Group cursors based on query
• Each cursor have a query reference, which you see the states and allow to snapshot
• Polling is simply as snapshotting all the queries
• when grouping, sort and limit also need to take care of
• snapshot() accepts an callback which helps to manage states in the cursor related to sorting and limiting