russellluo / slidingwindow

Golang implementation of Sliding Window Algorithm for distributed rate limiting.

slidingwindow

Golang implementation of Sliding Window Algorithm for distributed rate limiting.

Installation

$ go get -u github.com/RussellLuo/slidingwindow

Design

slidingwindow is an implementation of the scalable rate limiting algorithm used by Kong.

Suppose we have a limiter that permits 100 events per minute, and now the time comes at the "75s" point, then the internal windows will be as below:

In this situation, the limiter has permitted 12 events during the current window, which started 15 seconds ago, and 86 events during the entire previous window. Then the count approximation during the sliding window can be calculated like this:

count = 86 * ((60-15)/60) + 12
      = 86 * 0.75 + 12
      = 76.5 events

Test Utility

For details, see testutil.

Documentation

For usage and examples see the Godoc.

License

MIT

slidingwindow's Issues

Limiter starts out allowing too many through?

Using the below code (clone-able from gist here), I expect to see output like:

[{allowed:1000 disallowed:12445195} {allowed:1001 disallowed:12568861} {allowed:999 disallowed:12469467}]

But instead I get this kind of thing:

[{allowed:1553 disallowed:12445195} {allowed:1001 disallowed:12568861} {allowed:999 disallowed:12469467}]

Note that the first 1-second bucket allowed 1553 rather than 1000. The exact number varies between runs, but it's always significantly more than 1000.

Why is that? (Is it something about the handling of the "previous" window when there is no previous?)

I'm concerned because I plan on aggressively removing limiters when they haven't been used for a few windows (because that would be a rate limit reset anyway), but I don't really want to frequently go over the limit by ~50%.

package main

import (
	"fmt"
	"time"

	sw "github.com/RussellLuo/slidingwindow"
)

func main() {
	const histogramBucketSize = time.Second
	const totalBuckets = 3
	const allowedPerBucket = 1000

	type histoEntry struct {
		allowed    int64
		disallowed int64
	}
	allowedHistogram := make([]histoEntry, totalBuckets)

	lim, _ := sw.NewLimiter(histogramBucketSize, allowedPerBucket, func() (sw.Window, sw.StopFunc) {
		return sw.NewLocalWindow()
	})

	start := time.Now()
	for {
		bucket := time.Now().Sub(start) / histogramBucketSize

		if bucket >= totalBuckets {
			break
		}

		if lim.Allow() {
			allowedHistogram[bucket].allowed++
		} else {
			allowedHistogram[bucket].disallowed++
		}
	}

	fmt.Printf("%+v\n", allowedHistogram)
}

Multiple windows per one synchronizer

Good day!

Is it possible to use one synchronizer with multiple windows and multiple limiters (ie: multiple clients)?

Pseudo code

synchronizer := NewNonBlockingSynchronizer()
for _, key := range keys {
  limiter, stop := slidingwindow.NewLimiter(interval, rate, func() (slidingwindow.Window, slidingwindow.StopFunc) {
    return slidingwindow.NewSyncWindow(key, synchronizer)
  })
}

SyncWindow code always starts Synchronizer

slidingwindow/window.go

Line 94 in 535bb99

w.syncer.Start()

however, in NonBlockingSynchronizer there are no checks for a double launch

slidingwindow/synchronizer.go

Line 131 in 535bb99

go s.syncLoop()

So I suspect that the number of goroutines will be roughly equal to the number of limiters, that sounds quite overkill.
Is it a bug, feature, or I missed something?