Amazon Kinesis Data Firehose makes it easy to stream machine-generated data to Splunk for operational intelligence. Kinesis Data Firehose can stream data to your Splunk cluster in real-time at any scale. This integration supports Splunk versions with HTTP Event Collector (HEC), including Splunk Enterprise and Splunk Cloud.
To get started, simply sign into the Kinesis management console and create a Kinesis delivery stream. Then specify your Splunk cluster as a destination for the delivery stream. You can now start sending data to your delivery stream, and Kinesis Data Firehose will automatically load the data into your splunk cluster in real-time. For more information, see the Kinesis Data Firehose developer guide.
Blog posts
Ready, Set, Stream with the Kinesis Data Firehose and Splunk Integration
Featured re:Invent 2017 Sessions
Cox Automotive Empowered to Scale with Splunk Cloud & AWS (ABD208)
In this session learn how Cox Automotive is using Splunk Cloud for real time visibility into its AWS and hybrid environments to achieve near instantaneous MTTI reduce auction incidents by 90% and proactively predict outages. We also introduce a highly anticipated capability that allows you to ingest transform and analyze data in real time using Splunk and Amazon Kinesis Firehose to gain valuable insights from your cloud resources. It's now quicker and easier than ever to gain access to analytics-driven infrastructure monitoring using Splunk Enterprise and Splunk Cloud.
Real-Time Streaming Applications on AWS: Use Cases and Patterns (ABD203)
To win in the marketplace and provide differentiated customer experiences, businesses need to be able to use live data in real time to facilitate fast decision making. In this session, you learn common streaming data processing use cases and architectures. First, we give an overview of streaming data and AWS streaming data capabilities. Next, we look at a few customer examples and their real-time streaming applications. Finally, we walk through common architectures and design patterns of top streaming data use cases.
Recorded Webinars
How TrueCar Gains Actionable Insights with Splunk Cloud
Moving your entire data center to the cloud is no easy feat! TrueCar’s technology platform team was tasked with just that—and in search of a more scalable monitoring and troubleshooting solution that could increase infrastructure and application performance, enhance its security posture, and drive product improvements. The company landed on Splunk Cloud running on AWS and deployed it in one day! In this webinar, you’ll learn how TrueCar leverages both AWS and Splunk capabilities to gain insights from its data in real time.
Watch the webinar to learn how TrueCar's experience running Splunk Cloud on AWS with Amazon Kinesis Data Firehose can help you:
- Gain historical insights with additional data retention
- Provide better visibility into AWS billing
- Obtain security insights and threat detection
Real-Time Log Analytics using Amazon Kinesis Data Firehose (Jun 2017)
Log analytics is a common big data use case that allows you to analyze log data from websites, mobile devices, servers, sensors, and more for a wide variety of applications such as digital marketing, application monitoring, fraud detection, ad tech, gaming, and IoT. Moving your log analytics to real time can speed up your time to information allowing you to get insights in seconds or minutes instead of hours or days. In this session, you will learn how to ingest and deliver logs with no infrastructure using Amazon Kinesis Data Firehose. We will show how Amazon Managed Service for Apache Flink can be used to process log data in real time to build responsive analytics. Finally, we will show how to use Amazon Elasticsearch Service to interactively query and visualize your log data.
Learning Objectives:
- Understand how to easily build an end to end, real time log analytics solution.
- Get an overview of collecting and processing data in real-time using Amazon Kinesis.
- Learn how to Interactively query and visualize your log data using Amazon Elasticsearch Service.
Streaming ETL for Data Lakes using Amazon Kinesis Data Firehose (May 2017)
Data lakes enable your employees across the organization to access and analyze massive amounts of unstructured and structured data from disparate data sources, many of which generate data continuously and rapidly. Making this data available in a timely fashion for analysis requires a streaming solution that can durably and cost-effectively ingest this data into your data lake. Amazon Kinesis Data Firehose is a fully managed service that makes it easy to prepare and load streaming data into AWS. In this tech talk, we will provide an overview of Kinesis Data Firehose and dive deep into how you can use the service to collect, transform, batch, compress, and load real-time streaming data into your Amazon S3 data lakes.
Learning Objectives:
- Understand key requirements for collecting, preparing, and loading streaming data into data lakes.
- Get an overview of transmitting data using Kinesis Data Firehose.
- Learn how to perform data transformations with Kinesis Data Firehose.
Get started with Amazon Kinesis Data Firehose