Amazon Data Firehose resources

Developer guide

API reference

Conceptual introduction

 Hand-on tutorial

Amazon Kinesis makes it easy to collect process and analyze real-time streaming data so you can get timely insights and react quickly to new information. In this session we present an end-to-end streaming data solution using Kinesis Streams for data ingestion Kinesis Analytics for real-time processing and Amazon Data Firehose for persistence. We review in detail how to write SQL queries using streaming data and discuss best practices to optimize and monitor your Kinesis Analytics applications. Lastly we discuss how to estimate the cost of the entire system.

Watch session recording | Download presentation 

Want to ramp up your knowledge of AWS big data web services and launch your first big data application on the cloud? We walk you through simplifying big data processing as a data bus comprising ingest, store, process, and visualize. You build a big data application using AWS managed services, including Amazon Athena, Amazon Kinesis, Amazon DynamoDB, and Amazon S3. Along the way, we review architecture design patterns for big data applications and give you access to a take-home lab so that you can rebuild and customize the application yourself. You should bring your own laptop and have some familiarity with AWS services to get the most from this session.

Download presentation 

In recent years, there has been an explosive growth in the number of connected devices and real-time data sources. Because of this, data is being produced continuously and its production rate is accelerating. Businesses can no longer wait for hours or days to use this data. To gain the most valuable insights, they must use this data immediately so they can react quickly to new information. In this workshop, you learn how to take advantage of streaming data sources to analyze and react in near real-time. You are presented with several requirements for a real-world streaming data scenario and you're tasked with creating a solution that successfully satisfies the requirements using services such as Amazon Kinesis, AWS Lambda and Amazon SNS.

Download presentation

Reducing the time to get actionable insights from data is important to all businesses and customers who employ batch data analytics tools are exploring the benefits of streaming analytics. Learn best practices to extend your architecture from data warehouses and databases to real-time solutions. Learn how to use Amazon Kinesis to get real-time data insights and integrate them with Amazon Aurora Amazon RDS Amazon Redshift and Amazon S3. The Amazon Flex team describes how they used streaming analytics in their Amazon Flex mobile app used by Amazon delivery drivers to deliver millions of packages each month on time. They discuss the architecture that enabled the move from a batch processing system to a real-time system overcoming the challenges of migrating existing batch data to streaming data and how to benefit from real-time analytics.

Watch session recording | Download presentation

To win in the marketplace and provide differentiated customer experiences, businesses need to be able to use live data in real time to facilitate fast decision making. In this session, you learn common streaming data processing use cases and architectures. First, we give an overview of streaming data and AWS streaming data capabilities. Next, we look at a few customer examples and their real-time streaming applications. Finally, we walk through common architectures and design patterns of top streaming data use cases.

Watch session recording | Download presentation

In this session learn how Cox Automotive is using Splunk Cloud for real time visibility into its AWS and hybrid environments to achieve near instantaneous MTTI reduce auction incidents by 90% and proactively predict outages. We also introduce a highly anticipated capability that allows you to ingest transform and analyze data in real time using Splunk and Amazon Data Firehose to gain valuable insights from your cloud resources. It's now quicker and easier than ever to gain access to analytics-driven infrastructure monitoring using Splunk Enterprise and Splunk Cloud.

Watch session recording | Download presentation

Log analytics is a common big data use case that allows you to analyze log data from websites, mobile devices, servers, sensors, and more for a wide variety of applications such as digital marketing, application monitoring, fraud detection, ad tech, gaming, and IoT. Moving your log analytics to real time can speed up your time to information allowing you to get insights in seconds or minutes instead of hours or days. In this session, you will learn how to ingest and deliver logs with no infrastructure using Amazon Data Firehose. We will show how Amazon Managed Service for Apache Flink can be used to process log data in real time to build responsive analytics. Finally, we will show how to use Amazon Elasticsearch Service to interactively query and visualize your log data.

Learning Objectives:

• Understand how to easily build an end to end, real time log analytics solution.
  
• Get an overview of collecting and processing data in real-time using Amazon Kinesis.
• Learn how to Interactively query and visualize your log data using Amazon Elasticsearch Service.
  

Watch session recording | Download presentation

Data lakes enable your employees across the organization to access and analyze massive amounts of unstructured and structured data from disparate data sources, many of which generate data continuously and rapidly. Making this data available in a timely fashion for analysis requires a streaming solution that can durably and cost-effectively ingest this data into your data lake. Amazon Data Firehose is a fully managed service that makes it easy to prepare and load streaming data into AWS. In this tech talk, we will provide an overview of Firehose and dive deep into how you can use the service to collect, transform, batch, compress, and load real-time streaming data into your Amazon S3 data lakes.

Learning Objectives:

• Understand key requirements for collecting, preparing, and loading streaming data into data lakes.
• Get an overview of transmitting data using Firehose.
• Learn how to perform data transformations with Firehose.
  

Watch recording | Download presentation

Moving your entire data center to the cloud is no easy feat! TrueCar’s technology platform team was tasked with just that—and in search of a more scalable monitoring and troubleshooting solution that could increase infrastructure and application performance, enhance its security posture, and drive product improvements. The company landed on Splunk Cloud running on AWS and deployed it in one day! In this webinar, you’ll learn how TrueCar leverages both AWS and Splunk capabilities to gain insights from its data in real time.

Watch the webinar to learn how TrueCar's experience running Splunk Cloud on AWS with Amazon Data Firehose can help you:

• Gain historical insights with additional data retention

• Provide better visibility into AWS billing

• Obtain security insights and threat detection
  

Watch recording

by Jeremy Ber and Michael Greenshtein, 09/02/2021

by Jeff Barr, 03/31/2021

by Sakti Mishra, 08/18/2020

by Bryan Liston, 02/13/2027

by Viral Shah, 09/08/2020

by Rajeev Chakrabarti , 04/22/2019

by Imtiaz Sayed and Masudur Rahaman Sayem , 06/29/2020

by Re Alvarez-Parmar , 09/05/2017

by Aravind Kodandaramaiah , 05/04/2017

by Ian Robinson , Chaitanya Shah , and Ben Snively , 03/09/2017