Launching Open Distro for Elasticsearch security features on Amazon Elasticsearch Service

We are excited to announce that we are making new Open Distro for Elasticsearch security features available on Amazon Elasticsearch Service. Amazon Elasticsearch Service is frequently used for sensitive enterprise workloads, and today’s launch adds multiple capabilities to give you even tighter control over your data. New features include the ability to use roles to define granular permissions for indices, documents, and fields, and to extend Kibana with read-only views and tenant-level dashboards and visualizations.

This release enables multiple users and teams to securely share an Amazon Elasticsearch Service cluster without having the ability to view or manipulate data they are not authorized to access. Today’s release supports authentication via AWS Identity and Access Management (IAM) with Amazon Cognito providing user-level authentication in Kibana, as well as a new built-in user database, which makes it easy to configure users with support for simple authentication from Kibana. But this is only the beginning, in the coming months we plan to launch additional Open Distro for Elasticsearch security features on Amazon Elasticsearch Service including single sign-on (SSO) with SAML and Open ID Connect, and audit logging allowing you to monitor user activity for compliance and security related events.

Over ten months ago, we announced Open Distro for Elasticsearch, a completely open source version of Elasticsearch, the popular Apache Lucene-based search engine. We launched this initiative because we and others recognized the need to protect open source innovation for Elasticsearch, ensuring the long-term viability of the technology and community. Since then, Open Distro for Elasticsearch has seen millions of downloads, dozens of external contributors on GitHub, and seven major new releases. We have also seen significant interest and use by enterprises and independent software vendors (ISVs) who care about the long-term flexibility and assurances open-source software provides. While we have invested significantly in Open Distro for Elasticsearch, we also continue to contribute to the upstream Elasticsearch project making contributions such as addressing inefficiencies in snapshotting, automatic removal of write-blocks, and addressing scaling issues in shard routing. We have also continued to grow our contributions to Apache Lucene, the foundational technology that enables Elasticsearch and Apache Solr.

The security features included in Open Distro for Elasticsearch have been extremely popular, as open source Elasticsearch lacks even basic security features. To provide security for Open Distro for Elasticsearch, we worked with floragunn GmbH, a security specialist software company that develops Search Guard, a popular alternative to Elastic’s own commercial security features. Unfortunately, Elastic is targeting floragunn GmbH, filing a copyright infringement lawsuit against them. We want to make the community aware that AWS performed our own due diligence prior to partnering with floragunn and found no evidence that Search Guard misappropriated any copyrighted material. Additionally, we evaluated all of the code specified by Elastic in their litigation, as well as engaged third party experts, and we are confident that there is no basis to Elastic’s claims.

We find it unfortunate that Elastic has resorted to litigation against floragunn, a valuable member of the Elasticsearch community. This kind of behavior is misaligned with the spirit of open source and is detrimental to the vibrancy of the community.

Now more than ever, we believe it is important to have and protect a truly open source version of Elasticsearch. We remain excited to work with the community to deliver innovative new features in Open Distro for Elasticsearch community and to make those capabilities available to Amazon Elasticsearch Service customers. New features that have either been recently released or are currently under development include streaming anomaly detection, k-nearest neighbor search, index lifecycle automation, job scheduler, and a root cause analysis module that leverages Open Distro for Elasticsearch’s performance analyzer. We encourage everyone interested in advancing Open Distro for Elasticsearch to reach out to us on the community site and for Amazon Elasticsearch Service customers to give the new security features a try – available now on domains running Elasticsearch 6.7 or greater in all 21 AWS regions where Amazon Elasticsearch Service is available. Learn more here.