Overview
Migration Assistant for Amazon OpenSearch Service assists you in migrating your self-managed Elasticsearch and OpenSearch clusters to Amazon OpenSearch Service managed clusters or serverless collections. This AWS Solution automates manual migration tasks by facilitating existing and live data migration. It also includes advanced features, such as traffic capture and replay, to help identify potential migration and upgrade issues earlier. This solution not only streamlines the migration process but also enables performance and behavioral comparisons based on real customer workloads.
By deploying this solution, you will get a migration suite that allows you to follow a systematic workflow to run an upgrade, migration, and modification to an OpenSearch cluster. The suite includes a migration console command line interface (CLI) for management, a dedicated container for existing data backfill, and a replayer for processing data streams.
Benefits
Follow the process provided by this solution to transfer data from an originating (source) cluster to a designated target (OpenSearch cluster or OpenSearch Serverless Collection).
Safely capture traffic on the original cluster for simultaneous or deferred replay on a destination cluster. Opt to replay the recorded traffic at the same or varied speeds across multiple target clusters to identify the optimal cluster size and performance tailored to your real workloads.
Record requests and responses between the source and destination clusters for comparison, then forward the latency metrics and response codes to an analytics hub. You can analyze the data essential for transitioning your traffic from a legacy system to a new OpenSearch Service destination.
Make contributions and modify or extend this open source solution to meet your needs.
Technical details
You can automatically deploy this architecture using the implementation guide and the accompanying AWS CloudFormation template.
Step 1
Traffic is directed to the existing cluster, reaching each coordinator node.
Step 2
A Capture Proxy is added before each coordinator node in the cluster, allowing for traffic capture and storage in Amazon Managed Streaming for Apache Kafka (MSK).
Step 3
After continuous traffic capture is in place, the user initiates a historical backfill.
Step 4
Following the backfill, the user replays captured traffic captured using a Traffic Replayer.
Step 5
The user evaluates the outcomes from routing traffic to both the original and the new cluster.
Step 6
After confirming the new cluster’s functionality meets expectations, the user dismantles all related stacks, retaining only the new cluster’s setup. Additionally, the user can retire and discard the old cluster’s legacy infrastructure and all of the solution’s stacks, keeping only the new cluster.
Step 1
Traffic is directed to the existing cluster, reaching each coordinator node.
Step 2
A Capture Proxy is added before each coordinator node in the cluster, allowing for traffic capture and storage in Amazon Managed Streaming for Apache Kafka (MSK).
Step 3
After continuous traffic capture is in place, the user initiates a historical backfill.
Step 4
Following the backfill, the user replays captured traffic captured using a Traffic Replayer.
Step 5
The user evaluates the outcomes from routing traffic to both the original and the new cluster.
Step 6
After confirming the new cluster’s functionality meets expectations, the user dismantles all related stacks, retaining only the new cluster’s setup. Additionally, the user can retire and discard the old cluster’s legacy infrastructure and all of the solution’s stacks, keeping only the new cluster.
Related content
In manufacturing enterprises, digital transformation and Industry 4.0 are likely at the top of your mind. Learn how to digitally transform your factory with Machine Downtime Monitor on AWS.
- Publish Date