Comments (8)
Thanks :-) . It's good that you reference the issue from the commit.
- Actually I think the commit doesn't solve the problem completely since it still writes back the session data to the store. So the concurrency with the dashboard issue is still unsolved.
- Writing back is probably necessary since rate limiting is still handled the old way in the session storage. Should also be changed to the new method. Then the write back could be removed, improving performance because we save one database roundtrip.
- The transaction you added in "RedisStoreManager.decrement" doesn't have any effect since the DECR command is atomic anyways. I think you can remove the transaction.
- session_manager:125 The quota is reset to the maximum quota but it should be reset to max-1 since one call is already used.
from tyk.
I actually realised in the shower this morning that this commit doesn't fully solve the issue anyway, since we set the value of the Quota if the key is not found, strictly speaking, Scenario A still applies since a concurrent set of requests could set the same key to a maximum, now this isn't too much of an issue, but theoretically speaking you could bypass quotas with some clever timing.
I'm considering switching it around to be an INCR instead of a DECR, then there's no SET required (then just subtract Max from Remaining to update the session cache, however the race condition that is mentioned in the pattern applies where there is a risk of a floating key, though I feel it's unlikely to occur.
I think Scenario B is solved since the rate limiter uses the Redis store as the "truth" and the session store as a record, so if a password reset resets the quota counter, if the key is already present then it will just DECR (or INCR, as the case will be), the actual value in the session is irrelelvant except for in the reply and health-checks, which can't be guaranteed values.
Will send some more time on this today.
from tyk.
Tests passing, quota and rate limiter now redis based with finer-grained quota reset commands in the API. :-)
from tyk.
Awesome. Just a few things:
- The method session_manager.IsQuotaExceeded should be unused now, could be deleted
- In the current version of the quota algorithm the quota is handled like the rate limit. This means that QuotaRenews is ignored. The field could be removed. This also has the consequence, that a user could not set it manually any more (e.g. if he wants the renew to always occur at midnight).
- RedisStorageManager.Decrement could be removed now, no?
- RedisStorageManager.IncrememntWithExpire calls RedisStorageManager.Decrement if it has to reconnect (line 267). I assume this is a typo.
- Can't you just remove the reconnect attempts in RedisStorageManager? Does r.pool.Get() not deliver a connection reliably?
from tyk.
- Yes
- QuotaRenews is used to report back to health check headers, it is also used to set the initial rate limit. User can set it manually, updating a session via API will reset the quota in Redis (unless new param
?suppress_reset=1
is sent with request), this basically means quota renewals can be scripted to occur whenever if needed). This was actually how it all behaved, so we're preserving exiting functionality, but adding a workaround for session updates that shouldn't affect quota. - Yes it can, though not a priority
- Eek! Fixed in master.
- Leaving it as is for now
from tyk.
On 2.: QuotaRenews is not set any more in IsRedisQuotaExceeded. Thus it is not updated any more anywhere (since IsQuotaExceeded is not used any more). Or am I overlooking anything?
from tyk.
You're right, it isn't set, and really should be, it's used in reporting current session state back to the user through the rate-limit endpoint and reply headers:
https://github.com/lonelycode/tyk/blob/b28c02a59e9deec2bba398b1b3bdc2b4c50b31cc/api.go#L1204
Have just pushed an update to master branch.
from tyk.
Yeah, that solves it :-)
from tyk.
Related Issues (20)
- Optimize rate limit using Lua script
- [Q]: Adding more tests and increasing the Code Coverage HOT 4
- Double response from go plugin virtual endpoint HOT 1
- [TT-5070]Wrong HTTP status code when panic happens inside Go plugin HOT 1
- [TT-11223]ERROR: Tyk PUMP not able to connect to Redis Sentinels HOT 1
- Error with middleware in one gateway while other pods are working fine HOT 1
- KV Store config not read in tyk.conf (security.certificates.upstream) HOT 4
- User not authorized when using basic authentication HOT 1
- OAS API: transformRequestMethod.toMethod expects boolean instead of string HOT 1
- Automatic retries when there are specific errors from Upstream
- How to define proxy.listen_path as exact path? HOT 4
- HTTP/2 Continuation Frame Vulnerability
- graphql playground error for federation
- Add HTTP proxy support to MDCB
- Tyk cannot validate client certificates against a certificate authority HOT 2
- Profile Raw Editor cursor placement HOT 2
- Sd
- Sd HOT 1
- Support multiple JWKS sources / one source per OIDC provider/issuer
- [TT-12318] SSE Streaming is broken HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tyk.