awslabs / cloudwatch-logs-analyze-data Goto Github PK

I've been playing around with this model over the past few days, and am having troubles trying to analyze small data sets on a smallish Elasticsearch domain. I'm seeing tons of errors like the following from my Lambda function:

EsRejectedExecutionException[rejected execution (queue capacity 50) ...

Reading up on this it seems like Lambda is just trying to send too much data to the Elasticsearch cluster at a time. There doesn't seem to be a way to increase the queue capacity that I can find, so all I can think of is attempting to limit how fast Lambda is pushing in events from the S3 bucket.

Is there any way to limit how many Lambda functions get spun up when getting put events from the S3 bucket? Or some other way to slow things down?