Logstash is a light-weight, open-source, server-side data processing pipeline that allows you to collect data from a variety of sources, transform it on the fly, and send it to your desired destination. It is most often used as a data pipeline for Elasticsearch, an open-source analytics and search engine. Because of its tight integration with Elasticsearch, powerful log processing capabilities, and over 200 pre-built open-source plugins that can help you easily index your data, Logstash is a popular choice for loading data into Elasticsearch.

Logstash benefits

Easily load unstructured data

Logstash allows you to easily ingest unstructured data from a variety of data sources including system logs, website logs, and application server logs. 

Pre-built filters

Logstash offers pre-built filters, so you can readily transform common data types, index them in Elasticsearch, and start querying without having to build custom data transformation pipelines.

Flexible plugin architecture

With over 200 plugins already available on Github, it is likely that someone has already built the plugin you need to customize your data pipeline. But if none is available that suits your requirements, you can easily create one yourself.

Getting started with Logstash on AWS

To make it easy for customers to run Elasticsearch and ingest data into it, AWS offers Amazon Elasticsearch Service, a fully managed service that delivers Elasticsearch with easy integration with Logstash. To get started, simply launch your Amazon Elasticsearch Service domain and start loading data from your Logstash server. You can try Logstash and Amazon Elasticsearch Service for free using the AWS Free Tier.

 

Alternative data ingestion solutions

Amazon Elasticsearch Service offers built-in integrations with Amazon Kinesis Firehose, Amazon CloudWatch Logs, and AWS IoT to help you more easily ingest data into Elasticsearch. Alternatively, you can also build your own data pipeline using open-source solutions such as Apache Kafka and Fluentd. To find out which ingestion tool is right for your use case, visit Amazon Elasticsearch Service Data Ingestion page.

Learn more about Amazon Elasticsearch Service pricing

Visit the pricing page
Ready to build?
Get started with Amazon Elasticsearch Service
Have more questions?
Contact us