Amazon OpenSearch Service Documentation

With Amazon OpenSearch Service, choose from a selection of open source engine options. You can deploy and run the latest versions of OpenSearch, as well as 19 versions of ALv2 Elasticsearch (7.10 and earlier). The service also includes visualization capabilities with OpenSearch Dashboards and Kibana (7.10 and earlier).

You can set up and configure your Amazon OpenSearch Service cluster using the AWS Management Console or a single API call through the AWS Command Line Interface (CLI). You can specify the number of instances, instance types, storage options, and modify or delete existing clusters at any time.

Amazon OpenSearch Service allows you to upgrade your OpenSearch and Elasticsearch clusters (up to version 7.10) to newer versions, using in-place version upgrades. In-place upgrades can reduce the hassle of taking a manual snapshot, restoring it to a cluster running the newer version, and updating all your endpoint references.

Amazon OpenSearch Service is designed to provide built-in event monitoring and alerting, helping you monitor the data stored in your cluster and automatically send notifications based on pre-configured thresholds. Built using the OpenSearch alerting plugin, this feature helps you configure and manage alerts using your Kibana (Elasticsearch version 7.10 and previous) or OpenSearch Dashboards interface and the REST API. You can receive notifications via custom webhooks, Slack, Amazon Simple Notification Service (SNS), and Amazon Chime. You can also view cluster health metrics including number of instances, cluster health, searchable documents, CPU, and memory, as well as disk utilization for data and master nodes through Amazon CloudWatch.

With Amazon OpenSearch Service, you may no longer have a need for OpenSearch query domain-specific language (DSL) proficiency. Write SQL queries with OpenSearch SQL or use the OpenSearch Piped Processing Language (PPL), a query language that lets you use pipe (|) syntax, to explore, discover, and query your data. OpenSearch Dashboards also includes a SQL and PPL workbench.

Amazon OpenSearch Service offers built-in OpenSearch Dashboards and Kibana (Elasticsearch version 7.10 and previous) and integrates with Logstash, so you can ingest and visualize your data using the open source tools you prefer. Perform trace analytics with Amazon OpenSearch Service’s support for the open source OpenTelemetry standard and continue to use your existing code with direct access to Elasticsearch APIs (up to version 7.10) and plugins such as Kuromoji, Phonetic Analysis, Ingest Processor Attachment, Ingest User Agent Processor, and Mapper Murmur3.

With Amazon OpenSearch Service, you can connect your applications to your managed Elasticsearch (version 7.10 and previous) or OpenSearch environment from your Amazon Virtual Private Cloud (VPC) or via the public Internet, configuring network access using VPC security groups or IP-based access policies. You can also authenticate users and control access using Amazon Cognito, AWS Identity and Access Management (IAM), or basic authentication with a username and password. Amazon OpenSearch Service leverages the OpenSearch security plugin, enabling you to define granular permissions for indices, documents, or fields. You can also extend Kibana (Elasticsearch version 7.10 and previous) with read-only views and secure multi-tenant support. Amazon OpenSearch Service also supports built-in encryption for data at-rest and in-transit to help you protect your data when it is stored in your domain or in automated snapshots and transferring between nodes in your domain. 

Amazon OpenSearch Serverless is designed to automatically provision and continually adjust to get fast data ingestion rates and millisecond response times during changing usage patterns and demand.

Hot storage is designed to allow for fast retrieval of frequently accessed data. UltraWarm is a warm storage tier that is designed to complement Amazon OpenSearch Service’s hot storage tier by providing lower cost storage for older and less-frequently accessed data while still providing an interactive querying experience. UltraWarm stores data in Amazon S3 and uses custom, optimized nodes, purpose-built on the AWS Nitro System, to cache, pre-fetch, and query that data quickly.

With UltraWarm, you can retain up to 3 PB of data in a single Amazon OpenSearch Service cluster. You can also easily query and visualize the data in your Kibana (version 7.10 and previous) or OpenSearch Dashboards interface. Analyze both your recent (weeks) and historical (months or years) log data without spending hours or days restoring archived logs.

Cold storage is designed to be a low-cost storage option for Amazon OpenSearch Service, which allows you to retain infrequently accessed data in Amazon S3 and only pay for compute when you need it. Cold storage builds on UltraWarm, which provides specialized nodes that store data in Amazon S3 and uses a sophisticated caching solution to provide an interactive experience. By decoupling compute resources from storage, cold storage lets you retain virtually any amount of data in your Amazon OpenSearch Service domain while helping reduce cost. Detach historical or infrequently accessed warm data while not in use and free up compute. Discover and selectively attach your cold data to your domain’s UltraWarm nodes in seconds with your choice of a Kibana (version 7.10 and previous) or OpenSearch Dashboards interface and easy-to-use APIs. With cold storage, you can query the attached cold data with a similar interactive experience and performance as your warm data.

OpenSearch Service is designed to provide real-time document search capabilities that go beyond database search. This fully managed service uses the OpenSearch engine for search. Core search capabilities are designed to:

• Acquire data from a database or content management system, a web or intranet crawler, or a streaming service

• Provide search APIs to build a frontend on top of the search services

• Power searches across many attributes

• Find new documents that match a set of saved queries with prospective search (percolation)

• Assess usage patterns and performs capacity planning and cost prediction with OpenSearch Service monitoring capabilities

• Use built-in machine learning (ML) algorithms for k-nearest neighbors (k-NN) search to accomplish vector search, similarity search, semantic search, and more

• Use built-in ML algorithms for Learning to Rank to calculate relevance scores.

• Use multiple query languages, including SQL 

For additional information about service controls, security features and functionalities, including, as applicable, information about storing, retrieving, modifying, restricting, and deleting data, please see https://docs.aws.amazon.com/index.html. This additional information does not form part of the Documentation for purposes of the AWS Customer Agreement available at http://aws.amazon.com/agreement, or other agreement between you and AWS governing your use of AWS’s services.