Comments (2)
I'm also getting this error when I'm getting to 1800 messages on a channel I'm trying to dump. It would be nice if it could write whatever it had to a file instead of just exiting.
By the way I tried changing the tier values & recompiling, but even at 1/10/25/50 (tier1/tier2/tier3/tier4 values) it still throws the same error "slack rate limit exceeded, retry after 1s" after 1800 messages.
It's interesting that a 429 seems to get triggered after the same amount of messages, regardless of the timing.
I guess the "1s" message is coming from the slack-go library? The https://api.slack.com/docs/rate-limits#rate-limits__responding-to-rate-limiting-conditions implies it would normally be asking for a retry after 30 seconds.
from slackdump.
Hey @foucist thanks for confirming, yes, this message is from the slack library.
It's interesting that it hits the rate limit after 1800 messages square.
I suspect that it is because of there was no limiter on files downloads, so now file downloader and message dumper use the same limiter instance After refreshing the memory in the rate limit doc, and discovering that the rate limiting is per method, it seems that files are not an issue here, as the library just sends the Get request, instead of polling any of the API endpoints for that. I think what happens here is that by the time it reaches 1800 messages, our internal limiter drifts away from slack rate limiter, (that might be because the same limiter was incorrectly used for 2 different api methods) and therefore we get this "retry after 1s" error. I wrapped the calls with the retry function that will retry the API call should it see the RateLimitedError. Also, now each method has it's own limiter. (see #13).
Would you be able to check, if you compile the branch "rate-limit" would it be able to complete the download of that channel that currently fails for you? That would be great help in confirming this.
from slackdump.
Related Issues (20)
- Add support for exporting canvases and attachments HOT 3
- Variation in capitalization of "type" HOT 8
- When using export-type mattermost program downloads files anyway HOT 2
- Is possible to dump Emojis with information about who created? HOT 1
- How to export direct messages? HOT 1
- Attachments downloaded in standard export although no -download flag HOT 4
- Selective attachments download (i.e. only private/only public) HOT 1
- Callback error when exporting conversation HOT 11
- Add an homebrew installation method HOT 1
- Slack reports browser is not supported during login HOT 1
- Crashes on Linux and Windows HOT 2
- Incremental backup script is empty HOT 2
- failed to initialise the auth provider HOT 8
- Migrate off survey
- Leakless triggering false positive on Windows HOT 1
- Support Slack Email Challenge during EZ Login HOT 1
- Add support for exporting own messages only from timeframe HOT 4
- Chromium doesn't get launched on Guix HOT 10
- User export: Allow `-download` flag to also fetch avatars HOT 1
- Error when dumping channel with files, when there are files that was deleted. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
š Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ššš
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ā¤ļø Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from slackdump.