How do I troubleshoot an ultraWarm storage migration issue in Amazon Elasticsearch Service?

Last updated: 2021-01-05

When you migrate an index to ultraWarm storage in Amazon ES, you might receive the following error message:

Before an UltraWarm storage migration, Amazon ES validates the amount of available hot disk space. Amazon ES automatically runs a force merge operation during the disk space validation. When low disk space is detected, the force merge operation request is blocked. The request is blocked because the force merge operation processes one shard at a time. Additionally, each shard requires twice the amount of hot disk space currently being occupied and an additional 20 GB of free storage space per node.

To resolve any low disk space issues before an index migration, perform the following:

1.    Check to see which nodes have shards that exist in the index:

2.    Check the available disk space per node:

The nodes must have enough free storage space to fulfill the disk space requirement for a migration (twice the shard size plus 20 GB). For more information about calculating the on-size disk of an index, see Calculating storage requirements.

3.    (Optional) If you don't have enough disk space, delete old or unused indices. This deletion frees up disk space on the node so that a force merge operation can succeed.

4.    (Optional) If you can't delete indices from the node, increase the EBS volume. For more information, see How do I add storage space to an Amazon Elasticsearch Service (Amazon ES) domain?

Additional tips:

• Delete old or unused documents from your index. Too many documents can increase shard sizes, which slow down a migration. Note: Deleting a document doesn't immediately free up disk space. For more information, see Why is the Deleted Documents metric so high in my Amazon Elasticsearch Service cluster?
• Monitor disk utilization metrics (such as FreeStorageSpace) in Amazon CloudWatch. For more information, see ClusterBlockException.