Posted On: Oct 5, 2021
Amazon OpenSearch Service now supports cross-cluster replication, enabling you to automate copying and synchronizing of indices from one domain to another at low latency in same or different AWS accounts or Regions. With cross-cluster replication, you can achieve high availability for your mission critical applications with sequential data consistency.
Amazon OpenSearch Service makes it easy for you to perform interactive log analytics, real-time application monitoring, website search, and more. To ensure redundancy and availability customers configure replicas and deploy their domains across multiple availability zones, protecting them against instance failures and availability zone outages. However, the domain itself can still be a single point of failure. To protect against such failures, customers previously had to create a second domain, fork their input data streams to the two clusters, and place a load balancer in front of the two domains to balance incoming search requests. This set-up adds complexity and cost as it requires you to use additional technologies like Apache Kafka or AWS Lambda to monitor and correct data inconsistencies between the domains.
With cross-cluster replication for Amazon OpenSearch Service, you can replicate indices at low latency from one domain to another in same or different AWS Regions without needing additional technologies. Cross-cluster replication provides sequential consistency while continuously copying data from the leader index to the follower index. Sequential consistency ensures the leader and the follower index return the same result set after operations are applied on the indices in the same order. Cross-cluster replication is designed to minimize delivery lag between the leader and the follower index. You can continuously monitor the replication status via APIs. Additionally, if you have indices that follow an index pattern, you can create automatic follow rules and they will be automatically replicated.
Cross-cluster replication is available on the service today for domains running Elasticsearch 7.10. Cross cluster replication is also available as an open-source feature in OpenSearch 1.1, which is planned to be available on the service soon.
Cross-cluster replication is available for Amazon OpenSearch Service across 25 regions globally. Please refer to the AWS Region Table for more information about Amazon OpenSearch Service availability. To learn more about cross-cluster replication, please see the documentation. To learn more about Amazon OpenSearch Service, please visit the product page.