Posted On: Apr 27, 2023
Today Amazon OpenSearch Service announces Amazon OpenSearch Ingestion, a new fully managed data ingestion tier that allows you to ingest and process petabyte-scale data before indexing it in OpenSearch-managed clusters or serverless collections. OpenSearch Ingestion provides a no-code capability to filter, transform, redact, and route data prior to indexing it in OpenSearch. OpenSearch Ingestion automatically provisions and scales the underlying resources for even the most demanding and unpredictable workloads.
OpenSearch Ingestion is powered by OpenSearch Data Prepper, an open-source, data pipeline for preprocessing telemetry data (logs, distributed traces, and metrics) to prepare it for analysis and visualization in OpenSearch. OpenSearch Ingestion can be used as an alternate to self-managing Logstash or other streaming data pipelines. Because OpenSearch Ingestion is a fully managed serverless service, it removes the complexities of managing a multi-node cluster for data ingestion like choosing the right instance types, applying security patches, and adding or removing nodes to optimize for data volume fluctuations, enabling your team to focus on more critical work.
To get started with OpenSearch Ingestion, you can create a pipeline in the AWS Management Console and define your source, processors, and the destination cluster or collection. You can also start from a blueprint for the most common ingestion use cases.
OpenSearch Ingestion is now available in the following AWS Regions: US East (Ohio), US East (N. Virginia), US West (Oregon), US West (N. California), Europe (Ireland), Europe (London), Europe (Frankfurt), Asia Pacific (Tokyo), Asia Pacific (Sydney), and Asia Pacific (Singapore).
To learn more, see the Amazon OpenSearch Ingestion webpage and the Amazon OpenSearch Service Developer Guide.