Logstash is a light-weight, open-source, server-side data processing pipeline that allows you to collect data from a wide variety of sources, transform it on the fly, and send it to your desired destination. It is most often used as a data pipeline for Elasticsearch, a popular analytics and search engine. Logstash is a popular choice for loading data into Elasticsearch because of its tight integration, powerful log processing capabilities, and over 200 pre-built open-source plugins that can help you get your data indexed the way you want it.

Get Started with AWS for Free

Create a Free Account
Or Sign In to the Console

AWS Free Tier includes 750 hrs per month of a t2.micro.elasticsearch or t2.small.elasticsearch instance and 10GB per month of optional Amazon EBS storage (Magnetic or General Purpose)


View AWS Free Tier Details »

Benefit_ElastiCache_Performance

Logstash allows you to easily ingest unstructured data from a variety of data sources including system logs, website logs, and application server logs.

Benefit_ElastiCache_Performance

Logstash offers pre-built filters, so you can readily transform common data types, index them in Elasticsearch, and start querying without having to build custom data transformation pipelines.

Benefit_ElastiCache_Performance

With over 200 plugins already available on Github, it is likely that someone has already built the plugin you need to customize your data pipeline. But if none is available that suits your requirements, you can easily write your own plugin.

Ingest un-structured and semi-structured logs generated by servers, applications, mobile devices, and more for a wide variety of applications such as digital marketing, application monitoring, fraud detection, ad tech, gaming, and IoT. Logstash provides plugins to quickly load data from a variety of data sources.

Capture server logs and push them into your Elasticsearch cluster using Logstash. Elasticsearch indexes the data and makes it available for analysis in near real-time (less than one second). You can then use Kibana to visualize the data and perform operational analyses like identifying network issues and disk I/O problems. Your on-call teams can perform statistical aggregations to identify root cause and fix issues.

It’s easy to get started with Logstash on AWS. Amazon Elasticsearch Service supports integration with Logstash. Simply sign into the AWS Management Console, launch your first Amazon Elasticsearch Service domain and start loading your data from your Logstash server.

Depending on your specific use case, there is a number of alternative solutions that can help you more easily ingest data into Elasticsearch. Amazon Elasticsearch Service offers built-in integrations with Amazon Kinesis Firehose, Amazon CloudWatch Logs, and AWS IoT for this purpose. You can also build your own data pipeline using open-source solutions including Apache Kafka and Apache Fluentd. For more information, see Amazon Elasticsearch Service Data Ingestion page.