What is the ELK stack?

The ELK stack is an acronym used to describe a stack that comprises of three popular open-source projects: Elasticsearch, Logstash, and Kibana. Often referred to as Elasticsearch, the ELK stack gives you the ability to aggregate logs from all your systems and applications, analyze these logs, and create visualizations for application and infrastructure monitoring, faster troubleshooting, security analytics, and more.

E = Elasticsearch
Elasticsearch is an open-source, RESTful, distributed search and analytics engine built on Apache Lucene. Support for various languages, high performance, and schema-free JSON documents makes Elasticsearch an ideal choice for various log analytics and search use cases. Learn more »

L = Logstash
Logstash is an open-source data ingestion tool that allows you to collect data from a variety of sources, transform it, and send it to your desired destination. With pre-built filters and support for over 200 plugins, Logstash allows users to easily ingest data regardless of the data source or type. Learn more »

K = Kibana
Kibana is an open-source data visualization and exploration tool for reviewing logs and events. Kibana offers easy-to-use, interactive charts, pre-built aggregations and filters, and geospatial support and making it the preferred choice for visualizing data stored in Elasticsearch. Learn more » 

Why is the ELK stack so popular?

The ELK Stack is popular because it fulfills a need in the log analytics space. As more and more of your IT infrastructure move to public clouds, you need a log management and analytics solution to monitor this infrastructure as well as process any server logs, application logs, and clickstreams. The ELK stack provides a simple yet robust log analysis solution for your developers and DevOps engineers to gain valuable insights on failure diagnosis, application performance, and infrastructure monitoring – at a fraction of the price.

The ELK stack – Choosing the right option

You can choose to deploy and manage the ELK stack yourself. But, would you prefer that your developers or DevOps engineers spend time on building innovative applications or on managing operational tasks such as deployment, upgrades, software installation and patching, backups, and monitoring? Also, scaling up and down to meet your business requirements or achieving security and compliance is a challenge with the self-managed option.

Or you can choose an easier, scalable, and more secure option.

Introducing Amazon Elasticsearch Service

Amazon Elasticsearch Service is a fully managed service that makes it easy for you to deploy, secure, and operate Elasticsearch at scale. The service offers support for Elasticsearch APIs, built-in Kibana, and integration with Logstash, so you can continue to use your existing tools and code – without worrying about operational overhead.

Amazon Elasticsearch Service also integrates with other AWS services such as Amazon Kinesis Data Firehose, Amazon CloudWatch Logs, and AWS IoT giving you the flexibility to select the data ingestion tool that meets your use case requirements.

Amazon ES ELK diagram

How do I get started with Amazon Elasticsearch Service?

You're in luck — we created this step-by-step guide to help you get started with Amazon Elasticsearch Service. With AWS Free Tier, you can spin up your first domain at no risk or cost to you. Alternatively, you can follow this lab to get hands-on experience on Amazon Elasticsearch Service.