Range queries for CountMinSketch about algebird

[avibryant]

BTW: the above could also be used with levels instances of Map[Long,V] where we have a Monoid[V], to allow arbitrary exact range queries with at most 2log_2(maxValue) plus() operations. We'd only want to use the exact version for sufficiently small values of maxValue, though if the Map is backed by disk storage, or sharded in memory across a cluster, this could still be reasonably large.

[johnynek]

This looks good to me. Some tests and I'll be convinced.

About the monoid approach, I like the idea. The problem is we are going to have some issues proving bounds. So, I wonder if we have a monoid on V and a metric on V, can we push the proof through so we can show that |sum(V) - approxSum(V)| < epsilon except with probability delta for some known epsilon, delta.

By the way, pushing this all through for any monoid with a metric is probably worth a paper, if it is possible. We may need to assume something about the structure of the metric.

[sritchie]

Nice! This blog post describes the dyadic range approach as well.

Apparently http://madlib.incubator.apache.org/ uses this algorithm to do range queries and estimate percentiles on big datasets.