AllCloud accelerates OpenSearch migration for financial services platform

Legacy search infrastructure becomes a performance bottleneck as businesses grow. Companies can have billions of searchable documents across dozens of connected applications, making it difficult to scale efficiently without disrupting services. On top of that, licensing fees can add considerably to operating expenses.

To help its clients increase efficiency, AllCloud uses Amazon OpenSearch Service to simplify AI-powered search, observability, and vector database operations with a secure, cost-effective managed service. Because it’s a managed service, it reduces operational overhead while providing enterprise-grade security, making it possible for businesses to quickly deploy real-time search, analytics, and generative AI applications.

To simplify the migration process, AllCloud uses the Migration Assistant for Amazon OpenSearch Service (Migration Assistant) for a flexible, multi-version workload migration with rollback capacity and virtually no downtime. This solution is available through the AWS Solutions Library, which offers vetted solutions and guidance for business and technical use cases.

The Migration Assistant helps customers safely migrate or upgrade their Elasticsearch and OpenSearch workloads to Amazon OpenSearch Service managed clusters by providing a systematic workflow with built-in validation tools and risk-mitigation features. It streamlines the migration process through a command line interface–based console, using automated data backfill and live traffic synchronization. Users can pause or stop migrations without impacting production systems, ultimately reducing risk and accelerating the entire migration lifecycle.

AllCloud’s migrations include a phased cutover with rigorous testing and tailored automation to minimize business risk. To durably store live traffic, Migration Assistant uses Amazon Managed Streaming for Apache Kafka (Amazon MSK)—a streaming data service that manages Apache Kafka infrastructure and operations—which is crucial for maintaining service availability. Amazon Simple Storage Service (Amazon S3), an object storage service, provides the durable and scalable object storage that’s needed for managing snapshot repositories throughout the migration process.

AllCloud also implements customizations to meet individual customers’ specific business requirements. For example, the company can extend the source code of the Migration Assistant to handle complex index naming conventions, and it can implement JavaScript transformers for sophisticated metadata and data management. Using the AWS Cloud Development Kit (AWS CDK) to define cloud application resources, AllCloud automates the deployment and scaling of the AWS resources that are required for migrations. This adaptability, combined with Amazon MSK for message streaming and custom automation scripts, makes it possible for AllCloud to achieve virtually zero-downtime cutovers for clients while maintaining data security and compliance within the AWS environment.

By using Migration Assistant for Amazon OpenSearch Service, AllCloud’s clients can modernize their environments to support vector-based workloads, semantic search, and real-time AI applications. This helps position customers for success in implementing machine learning and large language model–powered capabilities going forward.

In one use case, a migration successfully transferred 8 TB of production data and 1.76 billion documents across quality assurance, staging, and production environments while maintaining service availability, with 546 Mbps live traffic handled in production. The customer removed 100,000 dollars in annual licensing costs—a 36 percent reduction—while gaining scalability with GPU-boosted OpenSearch. “Our work using the Migration Assistant for Amazon OpenSearch Service reflects AllCloud’s commitment to customer success,” says Peter Nebel, chief security officer at AllCloud. “This engagement highlights our ability to deliver tailored, high-impact, AWS-based solutions that align with our customers’ evolving needs.”