Comments (18)
@twelsh-aw
Yes, the issue is definitely there, I have a quick fix for it, but I also want to spend a bit more time to figure out why our tests don't catch that issue.
from ratelimit.
@rabbbit agree, I'll look at possible options with time mocking and will fix our tests in the following PR
from ratelimit.
Reverting - @storozhukBM we can re-do the whole PR again (possibly add some tests earlier?)
from ratelimit.
@rabbbit
I figured out that our tests continue to pass even if I break currently existing implementation.
For example if you change ratelimit_test.go:108
to t.clock.Sleep(interval-interval)
tests will still pass.
from ratelimit.
@rabbbit
Further findings. Tests do not able to detect issues because with mock clock, literally time, freezes between Take()
calls, to when we check now.Sub(oldState.last)
in atomicLimiter
or when we check now-timeOfNextPermissionIssue
in atomicInt64Limiter
it always returns 0
from ratelimit.
@rabbbit
If we add r.clock.Add(time.Microsecond)
between takes in (r *runnerImpl) startTaking
tests start to detect issues.
The problem with that is that we call startTaking
in different goroutines and "github.com/benbjohnson/clock" is not concurrently safe, so race detector fails when it finds some races inside mockClock state.
So we either should find concurrent mock clock implementation, or fix "github.com/benbjohnson/clock" or write our own.
from ratelimit.
I'm sorry, I won't be able to look at this in detail, for probably next ~10 days (limited laptop access).
For now, I can revert the clock implementation change in the meantime if you think it's better/stable - the tests were (mostly?) stable there. I'm opposed to implementing a clock as part of this package :)
Looks like we'll be stuck with the old implementation for a while - "code freeze" until we fix the tests.
from ratelimit.
@rabbbit I created a new PR as discussed #93
from ratelimit.
PR with fix for this issue #95
Also you can see on tests approach PR that it is able to detect this bug
from ratelimit.
Ack. This is pretty high on my priority list, will try to get to this soon. Again, sorry for the delay.
from ratelimit.
@rabbbit We already have a fix in place. What do you think about making atomicInt64Limiter a default implementation again and allowing people to have a try before cutting a new release tag? Or we can make this implementation public so that users can instantiate it. I'd like the new testing approach to be merged first, but if you're unsure about it, we can skip it for now.
from ratelimit.
The new atomic version seems to perform so much better than it feels we should enable it.
However, after that, we might just have to declare a code freeze until someone else has more cycles for reviews. Sadly this includes the new time-testing, would need to look at this in much more detail before merging in.
from ratelimit.
@rabbbit OK, so should I make a PR now?
from ratelimit.
SGTM; also, thanks for pushing this through :)
from ratelimit.
@rabbbit
Please take a look
#101
from ratelimit.
Closing as fixed - we reverted the bad change in #91, @storozhukBM fixed it in #95, but we were defaulting to the old implementation. #101 makes the new (faster) implementation the default.
from ratelimit.
@twelsh-aw this change is now merged, can you please try again and give us your feedback
from ratelimit.
thank you @rabbbit
from ratelimit.
Related Issues (20)
- X-Rate-Limit
- this line of doc is wrong
- When one of my operations has not called the atomicInt64Limiter.task() function for a long time, the speed limit may fail to be called again HOT 1
- change limit after start HOT 1
- Release all recent changes with a new v0.3.0 tag. HOT 1
- Unexpected results on CI with higher loop iterations
- This is rate shaping, not limiting HOT 1
- Take always return without block if it enters the case branch HOT 3
- [Docs] Need to illustrate the option `Slack` HOT 6
- The "slack" option doesn't make effect when use newAtomicInt64Based version rate limit
- Behavior of take in goroutine HOT 1
- Supply clock interface for testing? HOT 2
- context.Context support for Take? HOT 7
- If add some check in New function?
- Support for increasing/decreasing limits on the fly HOT 5
- What's the major diff v.s. https://pkg.go.dev/golang.org/x/time/rate HOT 1
- Adding support for an `Allow` method HOT 10
- Why does the example_test fail when I run it locally? HOT 2
- Can you support the take method of non blocking sleep
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ratelimit.