darach / eep-js Goto Github PK

Hi,

In my application I'm using complex objects as data items to processed in windows and I have a custom function to accumulate and compensate. The sliding window uses a Buffer to store the values inside the window which make using objects impossible, since to compensate you need to have the evicting item.

I manually made a change to your code (lib/eep.window_sliding.js) to change the Buffer into an Array.

In this issue I'm just suggesting using the Array instead to make your implementation more generic. I know that this would somewhat increase the memory usage.

Here's what I changed:
var p = new Array(size);

Cheers,
Panickos

The experiment with embedded event processing has received its first pull request from brapse who should be added to a contributors file and a contributors section in package.json.

Given the following code using a fresh checkout of the repository on node v0.10.26:

var windows = eep.EventWorld.make().windows();
var win = windows.sliding(eep.Stats.min, 3);
win = windows.ordered(win);

win.on('emit', function(value) {
  console.log('min:' + value);
});

var values = [1,2,3,4,5,6];
for (var i in values) {
  win.enqueue(values[i]);
}

I get:

min:1
min:1
min:2
min:3

While I would have expected:

min:1
min:2
min:3
min:4

Replacing Stats.min by Stats.max I get the expected behavior, so I suspect there's an issue with Stats.min.

The most numerically stable way to take a rolling average is to use the strategy

new_average =  old_average + (new_value - old_average) / new_count

MeanFunction's accumulate should be

self.accumulate = function(v) { s+=(v-s)/++n; }
self.emit = function() { return s; }

Likewise, variance can use the form

Var(X) = E[X^2]-E[X]^2

where both are simple means, admitting a similar solution to above (e2 +=(v*v - e2)/n for second moment)

tyranically should be tyrannically

Compensating function is incorrect for kurtosis. Side effect of introducing compensating aggregate functions. Remove now, add back later.

I think to make this useful, a declarative language and parser is needed. (e.g. compare Esper). Anyway, this is the proper way to start to make such a framework generic. Of course the non-trivial thing is to make stream joins efficient.

Hi,

I've noticed that monotonic and periodic windows are identical right now.
It's not entirely true though, because monotonic, if I understand underlying idea correctly, should not receive clock in it's constructor. Instead, it should set monotonic clock by default, and periodic should use wall clock.

Is this behavior intended?

Hi,

It seems that it's very possible to avoid using Temporal and Temporal function (actually, it's done quite similarly in Erlang implementation).

I believe that what they do technically belongs to the Clock rather then these two.

Would you accept a patch that removes them & moves impl to Clock instead? Or do you think it's a bad idea and they serve their purpose somewhere I haven't really noticed.