Overview
Migration Assistant for Amazon OpenSearch Service assists you in migrating or upgrading your Elasticsearch and OpenSearch workloads to Amazon OpenSearch Service managed clusters or serverless collections. This AWS Solution automates manual tasks with a low-risk and prescriptive migration path for existing and live data. It also includes advanced features, such as a metadata migration tool and capture and replay comparison tooling to help you identify potential migration and upgrade issues earlier. The migration process is streamlined, performance and behavioral comparisons based on real customer workloads are enabled, and the pre-migration, migration, and validation phases are accelerated.
This solution prescribes a systematic migration workflow to upgrade, migrate, recover, and modify an OpenSearch Service cluster. The workflow includes a migration console command line interface (CLI) for management, a dedicated scaling group for existing data backfill, and a replayer to synchronize live traffic between source and target clusters. Users can pause or abort the migration without affecting production traffic, thereby reducing risk. Additionally, the backfill functionality minimizes further risk by retrieving data from a snapshot, leaving the source cluster unaffected, and supporting multi-hop migrations, which decreases the overall number of migrations required.
Benefits
Transfer data from an originating (source) cluster to a designated target (OpenSearch Service cluster or OpenSearch Serverless Collection).
Safely capture and replay traffic on source and target clusters to identify optimal performance while reducing migration risk through abort capabilities, comparison tools, source preservation, and multi-hop support.
Record requests and responses between the source and destination clusters for comparison, then forward the latency metrics and response codes to an analytics hub. You can analyze the data essential for transitioning your traffic from a legacy system to a new OpenSearch Service destination.
Technical details
You can automatically deploy this architecture using the implementation guide and the accompanying AWS CloudFormation template.
Step 1
Client traffic is directed to the existing cluster.
Step 2
An Application Load Balancer (ALB) is positioned in front of the traffic capture proxy to route traffic as needed. The ALB forwards traffic to the capture proxy, which then relays it to the source while simultaneously replicating the traffic to Amazon Managed Streaming for Apache Kafka (Amazon MSK).
Step 3
With continuous traffic capture in place, a Reindex-from-Snapshot (RFS) is initiated by the user through the Migration Management Console.
Step 4
After a backfill has been completed, the captured traffic is replayed by the user with a Traffic Replayer.
Step 5
The performance and behavior of traffic routed to the source and target clusters are analyzed by reviewing relevant logs and metrics.
Step 6
After confirming the target cluster’s functionality meets expectations, the user redirects clients to the new target. Additionally, the user can retire and discard the old cluster’s infrastructure.
Step 1
Client traffic is directed to the existing cluster.
Step 2
An Application Load Balancer (ALB) is positioned in front of the traffic capture proxy to route traffic as needed. The ALB forwards traffic to the capture proxy, which then relays it to the source while simultaneously replicating the traffic to Amazon Managed Streaming for Apache Kafka (Amazon MSK).
Step 3
With continuous traffic capture in place, a Reindex-from-Snapshot (RFS) is initiated by the user through the Migration Management Console.
Step 4
After a backfill has been completed, the captured traffic is replayed by the user with a Traffic Replayer.
Step 5
The performance and behavior of traffic routed to the source and target clusters are analyzed by reviewing relevant logs and metrics.
Step 6
After confirming the target cluster’s functionality meets expectations, the user redirects clients to the new target. Additionally, the user can retire and discard the old cluster’s infrastructure.
Related content
In manufacturing enterprises, digital transformation and Industry 4.0 are likely at the top of your mind. Learn how to digitally transform your factory with Machine Downtime Monitor on AWS.
- Publish Date