Amazon OpenSearch Ingestion adds supports to ingest streaming data from Confluent Cloud
Amazon OpenSearch Ingestion now allows you to seamlessly ingest streaming data from Confluent Cloud Kafka clusters into your Amazon OpenSearch Service managed clusters or Serverless collections without the need for any third-party data connectors. With this integration, you can now use Amazon OpenSearch Ingestion to perform near-real-time aggregations, sampling and anomaly detection on data ingested from Confluent Cloud, helping you to build efficient data pipelines to power your complex observability use cases.
Amazon OpenSearch Ingestion pipelines can consume data from one or more topics in a Confluent Kafka cluster and transform the data before writing it to Amazon OpenSearch Service or Amazon S3. While reading data from Confluent Kafka clusters via Amazon OpenSearch Ingestion, you can configure the number of consumers per topic and tune different fetch parameters for high and low priority data. You can also optionally use Confluent Schema Registry to specify your data schema to dynamically read data at ingest time. You can also check out this blog post by Confluent to learn more about this feature.
This feature is available in the following 14 regions : US East (Ohio), US East (N. Virginia), US West (Oregon), Europe (Ireland), Europe (London), Europe (Frankfurt), Asia Pacific (Tokyo), Asia Pacific (Sydney), Asia Pacific (Singapore), Asia Pacific (Mumbai), Asia Pacific (Seoul), Canada (Central), South America (Sao Paulo), and Europe (Stockholm).
To learn more, see the Amazon OpenSearch Ingestion webpage and the Amazon OpenSearch Ingestion Developer Guide.