Amazon OpenSearch Ingestion adds support for ingesting data from self-managed sources
Amazon OpenSearch Ingestion now allows you to ingest data from self-managed OpenSearch, Elasticsearch and Apache Kafka clusters, eliminating the need to run and manage 3rd party tools like Logstash to migrate your data from self-managed sources into Amazon OpenSearch Service. Now you can seamlessly migrate or continuously replicate your data from all OpenSearch versions and Elasticsearch 7.x versions either on Amazon EC2 or on-premises environments into Amazon OpenSearch Service managed clusters or Serverless collections.
You can now migrate data from all indices, or just specific indices, from one or more self-managed OpenSearch/Elasticsearch clusters to one or more Amazon OpenSearch Service managed clusters or Serverless collections. Amazon OpenSearch Ingestion will continually detect new indices in the self-managed source cluster that need to be processed and can even be scheduled to reprocess indices at a configurable interval to pick up on new documents. Similarly, Amazon OpenSearch Ingestion pipelines can consume data from one or more topics in your self-managed Kafka cluster and transform the data before writing it to Amazon OpenSearch Service or Amazon S3. You can check out the complete list of features in this blog post.
This feature is available in all the 15 AWS regions that Amazon OpenSearch Ingestion is currently available in: US East (Ohio), US East (N. Virginia), US West (Oregon), US West (N. California), Europe (Ireland), Europe (London), Europe (Frankfurt), Asia Pacific (Tokyo), Asia Pacific (Sydney), Asia Pacific (Singapore), Asia Pacific (Mumbai), Asia Pacific (Seoul), Canada (Central), South America (São Paulo), and Europe (Stockholm).
To learn more, see the Amazon OpenSearch Ingestion webpage and the Amazon OpenSearch Service Developer Guide.