Comments (11)
Thanks for reporting this. We'll look at handling the throttling exception, and reporting it at a lower logging level.
from amazon-kinesis-client.
Thank you @kumarumesh so in this case, I'd appreciate it if you could level down the log message to INFO
or at least WARN
. I regard ERROR
as something that needs to be taken care of immediately, not benign,
from amazon-kinesis-client.
I also support lowering this to the warning level.
from amazon-kinesis-client.
Hi Rantav,
I got this error as well.
Notice that there is a limitation of 5 getRecord requests per second per shard and 1000 puts per second per shard.
You should take it into account when you determine the number of shards in your stream.
This is very important and can cause you a lot of problems!
for example if you read less than 1Mb/sec from each shard (the max is 2Mb/sec), but you do it in 7 getRecord requests/sec you will get ProvisionedThroughputExceededException and you'll probably don't know why. It may cause your applications to have delays on the stream
To solve this you need to do some fine tuning on your KinesisClientLibConfiguration - you can control the maxRecords in each read and the idleTimeBetweenReadsInMillis.
Lets say you have 7 applications working on a stream - if the idleTimeBetweenReadsInMillis is 1000 (1 sec) you'll probably get a lot of those exceptions. In this case you'll want to increase the idle time to 2 seconds. Also you want to read as many records as possible in each getRecords call, so I recommend setting maxRecords to 10000 (the max value allowed).
Hope this helps,
Ari.
from amazon-kinesis-client.
This has been fixed in the latest release. It will now be a warning unless it's throttled 6 times consecutively. Additionally the message is now reported from the the ThrottleReporter if you want to filter it out completely.
Feel free to reopen if you have any other questions or concerns.
from amazon-kinesis-client.
Hey, is it possible to migrate the implemented functionality to 2.X version also?
There is an open request about it: #539
from amazon-kinesis-client.
Hi Rantav,
Thanks for reporting this.
As you said, if you see this exception occasionally, and your application (that uses KCL to process Kinesis stream) doesn't fall behind in processing your Kinesis stream data, then it can be considered as a benign exception and can be ignored.
We recently updated our documentation to reflect this ( for details see section titled "Read Throttling" at http://docs.aws.amazon.com/kinesis/latest/dev/kinesis-record-processor-additional-considerations.html )
Let me know if you have further questions.
Thanks
Umesh
from amazon-kinesis-client.
+1
from amazon-kinesis-client.
I agree this should be lower down to warn level
from amazon-kinesis-client.
+1 For a Warn
from amazon-kinesis-client.
+1
from amazon-kinesis-client.
Related Issues (20)
- Please update your schema-registry-serde library in order to solve CVE issue HOT 1
- Need more test cases, samples, documentation for StreamConfigs in case of Multi Streams with KCL2.x.
- Support for both polling and fanout retrieval mode for multi-stream consumer configuration HOT 1
- amazon-kinesis-client-pom using old awssdk.version HOT 2
- i.n.c.ChannelInitializer Failed to initialize a channel. Closing: [id: 0x9a4e56d6] java.lang.VerifyError: Bad return type HOT 2
- Lease table is not updated when new shards are added causing stale workers HOT 1
- Support artifacts with all third-party dependencies relocated HOT 1
- Consolidate metrics to a common name space, like /aws/kinesis-client HOT 2
- Uneven distribution of shards over the consumer application workers HOT 1
- Retrieving shard consumer's current lease's hash range key HOT 2
- KCL2 -Multi stream consumer - Configured streams can be in same account and cross account HOT 4
- graceful shutdown of MultiLangDaemon worker that is assigned for completed shards is always timeout HOT 3
- STS Endpoint HOT 1
- Change to PollingConfig maxRecords breaks compat
- OutOfMemory due to huge number of 'org.apache.http.impl.conn.PoolingHttpClientConnectionManager' instances referenced by 'idle-connection-reaper' thread HOT 1
- Am I degrading my app if use multiple KCL at the same time? HOT 2
- Uncaught Netty exceptions on high volume data stream HOT 2
- Documentation on _CHECKPOINT_FREQ_SECONDS
- 2.6.0 release not available on Maven Central HOT 2
- ERROR: Unable to download MultiLangDaemon jar files from maven
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from amazon-kinesis-client.