Amazon Elasticsearch Service is a fully managed service that enables you to search, analyze, and visualize your log data cost-effectively, at petabyte-scale.
Elasticsearch clusters are challenging to setup, scale, and manage. As a fully managed service, Amazon Elasticsearch Service manages the setup, deployment, configuration, patching, and monitoring of your Elasticsearch clusters for you, so you can spend less time managing your clusters and more time building your applications. With a few clicks in the AWS console, you can create highly scalable, secure, and available Elasticsearch clusters. Amazon Elasticsearch Service offers open source Elasticsearch APIs, managed Kibana, integration with Logstash and other AWS services, and SQL querying, so you can continue to use your existing tools and code.
Easy to deploy and manage
Setup and configuration: Getting started with Amazon Elasticsearch Service is easy. You can setup and configure your Amazon Elasticsearch Service cluster using the AWS Management Console or a single API call through the AWS Command Line Interface (CLI). You can specify the number of instances, instance types, storage options, and modify or delete existing clusters at any time.
In-place upgrades: Amazon Elasticsearch Service enables you to easily upgrade your Elasticsearch clusters to newer versions without any downtime, using in-place version upgrades. With in-place upgrades, you no longer need to go through the hassle of taking a manual snapshot, restoring it to a new cluster running the newer version of Elasticsearch, and updating all of your endpoint references. Learn more »
Event monitoring and alerting: Amazon Elasticsearch Service provides built-in event monitoring and alerting, enabling you to monitor the data stored in your cluster and automatically send notifications based on pre-configured thresholds. Built using the Open Distro for Elasticsearch alerting plugin, this feature allows you to configure and manage alerts using your Kibana interface and the REST API and receive notifications via custom webhooks, Slack, Amazon Simple Notification Service (SNS), and Amazon Chime. You can also view cluster health metrics including number of instances, cluster health, searchable documents, CPU, memory, and disk utilization for data and master nodes through Amazon CloudWatch, at no additional charge.
SQL querying: Amazon Elasticsearch Service supports querying of your Elasticsearch cluster using the SQL syntax. Built using the Open Distro for Elasticsearch SQL plugin, this feature provides more than 40 SQL functions, data types, and commands, including direct export to CSV and query translation from SQL to Elasticsearch JSON. You can also connect to your existing SQL-based business intelligence and ETL tools via a JDBC driver.
Integration with open source tools: Amazon Elasticsearch Service offers built-in Kibana and integration with Logstash, so you can ingest and visualize the data using the open source tools you like. It also includes integration for analyzing trace data that supports the open source OpenTelemetry standard. You can continue to use your existing code with direct access to Elasticsearch APIs and plugins such as Kuromoji, Phonetic Analysis, Ingest Processor Attachment, Ingest User Agent Processor, and Mapper Murmur3.
Highly scalable and available
Scalability: Amazon Elastisearch Service lets you store up to 3 PB of data in a single Elasticsearch cluster and scale up or down easily as your needs change. You can monitor the state of your cluster through Amazon CloudWatch metrics and add or remove instances via a simple API call or a few clicks in the AWS console. You can also modify SSD-powered Amazon Elastic Block Store (EBS) volumes to accommodate your workload requirements.
Availability: Amazon Elasticsearch Service supports three Availability Zones (AZ) deployments, enabling you to deploy your instances across multiple AZs for better availability and failure tolerance. You can enable three AZ deployments for both existing and new clusters at no extra cost using the AWS console, CLI, or SDKs. If you enable replicas for your indexes, the primary and replica shards will automatically be distributed across nodes providing cross-zone replication. Learn more »
Durability: You can build data durability for your Amazon Elasticsearch cluster through automated and manual snapshots. You can use snapshots to recover your cluster or to create a new cluster with preloaded data. By default, the Amazon Elasticsearch Service will automatically create hourly snapshots of each domain and retain them for 14 days at no extra charge. These snapshots are stored in Amazon S3, which is designed for 99.999999999% (11 9’s) durability.
With Amazon Elasticsearch Service you can securely connect your applications to your managed Elasticsearch environment from your VPC or via the public Internet, configuring network access using VPC security groups or IP-based access policies. You can also securely authenticate your users and control access using Amazon Cognito, AWS Identity and Access Management (IAM), or basic authentication using username and password. Amazon Elasticsearch Service leverages the Open Distro for Elasticsearch security plugin to enable you to define granular permissions for indices, documents, or fields and to extend Kibana with read-only views and secure multi-tenant support. Amazon Elasticsearch Service supports built-in encryption for data at-rest and in-transit so you can protect your data both when it is stored in your domain or in automated snapshots, and when it is transferred between nodes in your domain. Amazon Elasticsearch Service is HIPAA eligible and compliant with PCI DSS, SOC, ISO, and FedRamp standards, making it easy for you to build applications that meet compliance requirements.
Pay only for what you use: With Amazon Elasticsearch Service, you pay only for what you use. There is no upfront fee or usage requirement. You can reserve instances for for a one- or three-year term to get significant cost savings on usage as compared to on-demand instances. For more details, visit the pricing page.
UltraWarm is a warm storage tier that complements the existing Elasticsearch hot storage tier by providing less expensive storage for older and less-frequently accessed data while still providing an interactive experience. UltraWarm stores data in Amazon S3 while using custom, highly-optimized nodes, purpose-built on the AWS Nitro System, to cache, pre-fetch, and query that data. This allows you to:
- Retain up to 3 PB of data in a single Amazon Elasticsearch Service cluster while reducing cost per GB by nearly 90% compared to existing Elasticsearch storage tiers.
- Run fast, interactive analytics on both your recent (weeks) and historical (months or years) log data without needing to spend hours or days restoring it from the archives.
- Easily query and visualize across both your recent and historical log data via your Kibana interface, enabling you to quickly identify and troubleshoot performance issues.
When searching and analyzing data, you don’t need to worry about which tier of storage that data is currently in as that is handled automatically. To start using UltraWarm, sign in to the AWS console, create an Amazon Elasticsearch Service cluster, and when selecting your nodes, enable UltraWarm. You can select UltraWarm1.medium.elasticsearch or UltraWarm1.large.elasticsearch instances.
Cold storage is the lowest cost storage tier that allows you to retain infrequently accessed data in Amazon S3 and only pay for compute when you need it. Based on your data retention, query latency, and budgeting requirements, you now have a selection of hot, UltraWarm, or cold storage options to choose from. Cold storage builds on UltraWarm, which provides specialized nodes that store data in Amazon S3 and uses a sophisticated caching solution to provide an interactive experience. By decoupling compute resources from storage, cold storage allows you to:
- Retain any amount of data in your Amazon Elasticsearch Service domain while reducing cost per GB to near Amazon S3 storage prices.
- Detach historical or infrequently accessed warm data while not in use and free up compute to help lower costs.
- Easy-to-use APIs and a Kibana interface enable you to discover and selectively attach your cold data to your Amazon Elasticsearch Service domain’s UltraWarm nodes in seconds.
- Query the attached cold data with a similar interactive experience and performance as your warm data.
Additionally, Index State Management (ISM) policies can be used to automate the lifecycle of your data across all storage tiers. To start using cold storage, sign in to the AWS console and enable cold storage on a new or existing UltraWarm-enabled Amazon Elasticsearch Service domain.