AWS Open Source Blog

Category: Analytics

Open Distro for Elasticsearch logo.

Open Distro for Elasticsearch 1.1.0 released

We are happy to announce that Open Distro for Elasticsearch 1.1.0 is now available for download! Version 1.1.0 includes the upstream open source versions of Elasticsearch 7.1.1, Kibana 7.1.1, and the latest updates for alerting, SQL, security, performance analyzer, and Kibana plugins, as well as the SQL JDBC driver. You can find details on enhancements, […]

Read More
An Open Distro for Elasticsearch cluster with balanced resource usage

Use Elasticsearch’s _rollover API For efficient storage distribution

Many Open Distro for Elasticsearch users manage data life cycle in their clusters by creating an index based on a standard time period, usually one index per day. This pattern has many advantages: ingest tools like Logstash support index rollover out of the box; defining a retention window is straightforward; and deleting old data is […]

Read More

Add Single Sign-On to Open Distro for Elasticsearch Kibana Using SAML and ADFS

Open Distro for Elasticsearch Security (Open Distro Security) comes with authentication and access control out of the box. Prior posts have discussed LDAP integration with Open Distro for Elasticsearch and JSON Web Token authentication with Open Distro for Elasticsearch. Security Assertion Markup Language 2.0 (SAML) is an open standard for exchanging identity and security information […]

Read More
Diagram showing where PartiQL fits with other data sources.

Announcing PartiQL: One query language for all your data

Data is being gathered and created at rates unprecedented in history. Much of this data is intended to drive business outcomes but, according to the Harvard Business Review, “…on average, less than half of an organization’s structured data is actively used in making decisions…” The root of the problem is that data is typically spread […]

Read More
Setup of the centralized logging demo app

Centralized Container Logging with Fluent Bit

by Wesley Pettit and Michael Hausenblas AWS is built for builders. Builders are always looking for ways to optimize, and this applies to application logging. Not all logs are of equal importance. Some require real-time analytics, others simply need to be stored long-term so that they can be analyzed if needed. It’s therefore critical to […]

Read More
Open Distro for Elasticsearch logo

Open Distro for Elasticsearch version 1.0.0 is now available

Open Distro for Elasticsearch 1.0.0 is now available for you to download and run! The 1.0.0 release includes Elasticsearch 7.0.1 and Kibana 7.0.1 from upstream and the latest versions of the Open Distro for Elasticsearch plugins for alerting, performance analyzer, SQL, and security. The Kibana UI components for security and alerting are also part of […]

Read More
A portion of the YML for specifying a monitor in Open Distro for Elasticsearch

Manage Your Open Distro for Elasticsearch Alerting Monitors With odfe-monitor-cli

When you use Open Distro for Elasticsearch Alerting, you create monitors in Kibana. Setting up monitors with a UI is fast and convenient, making it easy to get started. If monitoring is a major workload for your cluster, though, you may have hundreds or even thousands of monitors to create, update, and tune over time. […]

Read More
Open Distro for Elasticsearch Security plugin selecting the tenant for Kibana visualizations and dashboards

Set up Multi-Tenant Kibana Access in Open Distro for Elasticsearch

中文版 – Elasticsearch has become a default choice for storing and analyzing log data to deliver insights on your application’s performance, your security stance, and your users’ interactions with your application. It’s so useful that many teams adopt Elasticsearch early in their development cycle to support DevOps. This grass-roots adoption often mushrooms into a confusing […]

Read More
Open Distro for Elasticsearch logo

New! Open Distro for Elasticsearch’s Job Scheduler Plugin

中文版 – Open Distro for Elasticsearch’s JobScheduler plugin provides a framework for developers to accomplish common, scheduled tasks on their cluster. You can implement Job Scheduler’s Service Provider Interface (SPI) to take snapshots, manage your data’s lifecycle, run periodic jobs, and much more. When you use Job Scheduler, you build a plugin that implements interfaces provided […]

Read More
A Kibana dashboard with metrics gathered by Open Distro for Elasticsearch's Performance Analyzer plugin

Store Open Distro for Elasticsearch’s Performance Analyzer Output in Elasticsearch

中文版 – Open Distro for Elasticsearch‘s Performance Analyzer plugin exposes a REST API that returns metrics from your Elasticsearch cluster. To get the most out of these metrics, you can store them in Elasticsearch and use Kibana to visualize them. While you can use Open Distro for Elasticsearch’s PerfTop to build visualizations, PerfTop doesn’t retain data […]

Read More