When I write data to my Amazon Elasticsearch Service (Amazon ES) cluster, I get an error message similar to the following:

error":"elastic: Error 429 (Too Many Requests): rejected execution of org.elasticsearch.transport.TransportService$7@b25fff4 on 
EsThreadPoolExecutor[bulk, queue capacity = 200, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor@768d4a66[Running, 
pool size = 2, active threads = 2, queued tasks = 200, completed tasks = 820898]] [type=es_rejected_execution_exception]"

The "es_rejected_execution_exception" exception usually occurs when the Elasticsearch cluster has received a large number of requests and can't take any more requests. Each node has a thread pool queue that can hold between 50 and 200 requests, depending on which Elasticsearch version you are using. When the queue is full, new requests are rejected.

Note: It's not possible to increase the queue size in most Amazon ES versions. The queue exists for a reason—it limits requests to a manageable number. For more information, see Threadpool Section in the Elasticsearch documentation.

Use one of the following methods to resolve the "es_rejected_execution_exception" error:

  • Add more nodes: Each node has a bulk queue, so adding more nodes gives you more queuing capacity. To add nodes, see Configuring Amazon ES Domains (Console).
  • Switch to a larger instance type: The number of threads in the thread pool on each node for bulk requests is equal to the number of available processors. Switch to an instance with more virtual CPUs (vCPUs) to get more threads to process bulk requests. For more information, see Choosing Instance Types and Testing.
  • Optimize bulk request size: Instead of sending many small bulk requests, send a few large bulk requests.
  • Improve indexing performance: When documents are indexed faster, the bulk queue is less likely to reach capacity. For more information about performance tuning, see Indexing Performance Tips in the Elasticsearch documentation.

Did this page help you? Yes | No

Back to the AWS Support Knowledge Center

Need help? Visit the AWS Support Center

Published: 2018-10-30