Stepping up for a truly open source Elasticsearch

Last week, Elastic announced they will change their software licensing strategy, and will not release new versions of Elasticsearch and Kibana under the Apache License, Version 2.0 (ALv2). Instead, new versions of the software will be offered under the Elastic License (which limits how it can be used) or the Server Side Public License (which has requirements that make it unacceptable to many in the open source community). This means that Elasticsearch and Kibana will no longer be open source software. In order to ensure open source versions of both packages remain available and well supported, including in our own offerings, we are announcing today that AWS will step up to create and maintain a ALv2-licensed fork of open source Elasticsearch and Kibana.

What this means for the Open Distro for Elasticsearch community

We launched Open Distro for Elasticsearch in 2019 to provide customers and developers with a fully featured Elasticsearch distribution that provides all of the freedoms of ALv2-licensed software. Open Distro for Elasticsearch is a 100% open source distribution that delivers functionality practically every Elasticsearch user or developer needs, including support for network encryption and access controls. In building Open Distro, we followed the recommended open source development practice of “upstream first.” All changes to Elasticsearch were sent as upstream pull requests (#42066, #42658, #43284, #43839, #53643, #57271, #59563, #61400, #64513), and we then included the “oss” builds offered by Elastic in our distribution. This ensured that we were collaborating with the upstream developers and maintainers, and not creating a “fork” of the software.

Choosing to fork a project is not a decision to be taken lightly, but it can be the right path forward when the needs of a community diverge—as they have here. An important benefit of open source software is that when something like this happens, developers already have all the rights they need to pick up the work themselves, if they are sufficiently motivated. There are many success stories here, like Grafana emerging from a fork of Kibana 3.

When AWS decides to offer a service based on an open source project, we ensure that we are equipped and prepared to maintain it ourselves if necessary. AWS brings years of experience working with these codebases, as well as making upstream code contributions to both Elasticsearch and Apache Lucene, the core search library that Elasticsearch is built on—with more than 230 Lucene contributions in 2020 alone.

Our forks of Elasticsearch and Kibana will be based on the latest ALv2-licensed codebases, version 7.10. We will publish new GitHub repositories in the next few weeks. In time, both will be included in the existing Open Distro distributions, replacing the ALv2 builds provided by Elastic. We’re in this for the long haul, and will work in a way that fosters healthy and sustainable open source practices—including implementing shared project governance with a community of contributors.

What this means for Amazon Elasticsearch Service customers

You can rest assured that neither Elastic’s license change, nor our decision to fork, will have any negative impact on the Amazon Elasticsearch Service (Amazon ES) you currently enjoy. Today, we offer 18 versions of Elasticsearch on Amazon ES, and none of these are affected by the license change.

In the future, Amazon ES will be powered by the new fork of Elasticsearch and Kibana. We will continue to deliver new features, fixes, and enhancements. We are committed to providing compatibility to eliminate any need to update your client or application code. Just as we do today, we will provide you with a seamless upgrade path to new versions of the software.

This change will not slow the velocity of enhancements we offer to our customers. If anything, a community-owned Elasticsearch codebase presents new opportunities for us to move faster in improving stability, scalability, resiliency, and performance.

What this means for the open source community

Developers embrace open source software for many reasons, perhaps the most important being the freedom to use that software where and how they wish.

The term “open source” has had a specific meaning since it was coined in 1998. Elastic’s assertions that the SSPL is “free and open” are misleading and wrong. They’re trying to claim the benefits of open source, while chipping away at the very definition of open source itself. Their choice of SSPL belies this. SSPL is a non-open source license designed to look like an open source license, blurring the lines between the two. As the Fedora community states, “[to] consider the SSPL to be ‘Free’ or ‘Open Source’ causes [a] shadow to be cast across all other licenses in the FOSS ecosystem.”

In April 2018, when Elastic co-mingled their proprietary licensed software with the ALv2 code, they promised in “We Opened X-Pack”: “We did not change the license of any of the Apache 2.0 code of Elasticsearch, Kibana, Beats, and Logstash — and we never will.” Last week, after reneging on this promise, Elastic updated that same page with a footnote that says “circumstances have changed.”

Elastic knows what they’re doing is fishy. The community has told them this (e.g., see Brasseur, Quinn, DeVault, and Jacob). It’s also why they felt the need to write an additional blustery blog (on top of their initial license change blog) to try to explain their actions as “AWS made us do it.” Most folks aren’t fooled. We didn’t make them do anything. They believe that restricting their license will lock others out of offering managed Elasticsearch services, which will let Elastic build a bigger business. Elastic has a right to change their license, but they should also step up and own their own decision.

In the meantime, we’re excited about the long-term journey we’ve embarked on with Open Distro for Elasticsearch. We look forward to providing a truly open source option for Elasticsearch and Kibana using the ALv2 license, and building and supporting this future with the community.

To learn more, read the What is OpenSearch? FAQ.

An earlier version of this post incorrectly indicated that the Jenkins CI tool was a fork. We thank @abayer for the correction.