OpenSearch is a distributed, open-source search and analytics suite used for a broad set of use cases like real-time application monitoring, log analytics, and website search. OpenSearch provides a highly scalable system for providing fast access and response to large volumes of data with an integrated visualization tool, OpenSearch Dashboards, that makes it easy for users to explore their data. Like Elasticsearch and Apache Solr, OpenSearch is powered by the Apache Lucene search library. OpenSearch and OpenSearch Dashboards were originally derived from Elasticsearch 7.10.2 and Kibana 7.10.2.
Q: Why was OpenSearch created?
Developers embrace open source software for many reasons, one of the most important being the freedom to use that software where and how they wish. On January 21, 2021, Elastic NV announced that they would change their software licensing strategy and not release new versions of Elasticsearch and Kibana under the permissive ALv2 license. Instead, Elastic is releasing Elasticsearch and Kibana under the Elastic license, with source code available under the Elastic License or Server Side Public License (SSPL). These licenses are not open source and do not offer users the same freedoms. Because of this, we made the decision to create and maintain a fork from the last ALv2 version of Elasticsearch and Kibana. The fork is called OpenSearch and is available under ALv2.
Q: What license is OpenSearch released under?
All of the software in the OpenSearch project is released under the Apache License, Version 2.0 (ALv2). ALv2 grants well-understood and permissive usage rights that match the freedoms people expect with open source software: freedoms such as being able use, modify, extend, monetize, and resell the open source software where and how they want. For OpenSearch, we believe this license will enable broad adoption and contributions benefiting all members of the community. We have also published permissive usage guidelines for the OpenSearch trademark, so you can use the name to promote your offerings.
Q: Are Elasticsearch and Kibana open source?
Elastic announced they will no longer publish new versions of Elasticsearch and Kibana under the ALv2 and instead publish new versions under the proprietary Elastic License with source code available under the Elastic License of SSPL. This change means that Elasticsearch and Kibana versions after 7.10.2 are not open source software.
Q: Why should I use OpenSearch?
OpenSearch enables you to easily ingest, secure, search, aggregate, view, and analyze data. These capabilities are popular for use cases such as application search, log analytics, and more. With OpenSearch, you benefit from having an open source product you can use, modify, extend, monetize, and resell how you want. At the same time, we will continue to provide a secure, high-quality search and analytics suite with a rich roadmap of new and innovative functionality.
Q: Is OpenSearch available in Amazon Elasticsearch Service?
Not yet. The source code and pre-release software are available at opensearch.org for customers interested in experimenting with and testing the open source software.
Q: What does the release of OpenSearch mean for Amazon Elasticsearch Service customers?
You can rest assured that neither Elastic’s license change or our decision to create OpenSearch will have any negative impact on the Amazon Elasticsearch Service you currently enjoy. We will continue to offer the 19 versions of Elasticsearch and Kibana supported on the service today. We will add support for OpenSearch when it is ready for production use. At that time, we will rename the service to Amazon OpenSearch Service. There will be no price changes, no deprecations, or forced migration off any version of Elasticsearch running on the service. All the APIs will be backward compatible, so there will no impact to customer workloads and development processes. Just as we do today, we will provide you with a seamless upgrade path to new versions of the software.
Q: How do the 19 versions of Elasticsearch offered on our service compare with what is offered on Elastic’s Elastic Cloud?
Elastic’s managed service only allows customers to deploy three versions of Elasticsearch at any time (the latest version, the previous minor version, and the latest from the previous major version). This approach contrasts with ours for Amazon Elasticsearch Service. Our customers have the freedom to continue with the version of Elasticsearch that they have deployed and certified for their workloads. They only need to upgrade versions when it suits their needs, not the vendors’. In fact, unlike Elastic Cloud, we backport bug and security patches to all affected versions running on the service and do not require you to upgrade to get a bug or security fix.
Q: How does the OpenSearch announcement affect customers and partners that offer SaaS products using Amazon Elasticsearch Service?
Our current and future customers and partners that build a SaaS product on Amazon Elasticsearch Service will see no change and can continue to take advantage of our existing and new innovations to build a compelling offering for their customers. OpenSearch will strive to maintain a modular architecture to make it easy for our customers and partners to innovate and extend.
Q: Is AWS equipped to maintain and advance a project like OpenSearch?
When AWS decides to offer a service based on an open source project, we ensure that we are equipped and prepared to maintain it ourselves, if necessary. We bring years of experience working with Elasticsearch and Kibana codebases and have made upstream code contributions to both Elasticsearch and Apache Lucene (the core search library that Elasticsearch is built on). We have added several features in open source like security, alerting, anomaly detection, index state management, and trace analytics that are widely used and deployed in production by our community and customers. We are well equipped to maintain and advance the project ourselves. Also, the community-backed codebase will help accelerate new innovations and will allow everyone to move faster in improving stability, scalability, resiliency, and performance. Already many organizations including SAP, CapitalOne, RedHat, Logz.io, Aiven.io, Bonsai, Logit.io, Search Guard, and BAInsight have publicly backed OpenSearch.
Q: What are some features that OpenSearch already provides that were not previously available in open source Elasticsearch?
|Advanced Security||Offers encryption, authentication, authorization, and auditing features. They include integrations with Active Directory, LDAP, SAML, Kerberos, JSON web tokens, and more. OpenSearch also provides fine-grained role-based access control to indices, documents and fields.|
|SQL Query Syntax||Provides the familiar SQL query syntax. Use aggregations, group by, and where clauses to investigate your data. Read data as JSON documents or CSV tables so you have the flexibility to use the format that works best for you.|
|Reporting||Schedule, export, and share reports from dashboards, saved searches, alerts and visualizations.|
|Anomaly Detection||Leverage Machine Learning anomaly detection based on the Random Cut Forest (RCF) algorithm to automatically detect anomalies as your data is ingested. Combine with Alerting to monitor data in near real time and automatically send alert notifications.|
|Index Management||Define custom policies to automate routine index management tasks, such as rollover and delete, and apply them to indices and index patterns.|
|Performance Analyzer and RCA Framework||Query numerous cluster performance metrics and aggregations. Use PerfTop, the command line interface (CLI) to quickly display and analyze those metrics. Use the root cause analysis (RCA) framework to investigate performance and reliability issues in clusters.|
|Asynchronous Search||Run complex queries without worrying about the query timing out and with Asynchronous Search queries run in the background. Track query progress and retrieve partial results as they become available.|
|Trace Analytics||Ingest and visualize OpenTelemetry data for distributed applications. Visualize the flow of events between these applications to identify performance problems.|
|Alerting||Automatically monitor data and send alert notifications automatically to stakeholders. With an intuitive interface and a powerful API, easily set up, manage, and monitor alerts. Craft highly specific alert conditions using OpenSearch’s full query language and scripting capabilities.|
|k-NN search||Using Machine Learning, run the nearest neighbor search algorithm on billions of documents across thousands of dimensions with the same ease as running any regular OpenSearch query. Use aggregations and filter clauses to further refine similarity search operations. k-NN similarity search powers use cases such as product recommendations, fraud detection, image and video search, related document search, and more.|
|Piped Processing Language||The Piped Processing Language provides a familiar query syntax with a comprehensive set of commands delimited by pipes (|) to query data.|
|Dashboard Notebooks||Combine dashboards, visualizations, text, and more to provide context and detailed explanations when analyzing data.|