Amazon Redshift Streaming Ingestion

Ingest streaming data into your data warehouse for real-time analytics

Natively integrating with Amazon streaming engines, Amazon Redshift Streaming Ingestion ingests hundreds of megabytes of data per second so you can query data in near real time. With Amazon Redshift Streaming Ingestion, you can connect to multiple Amazon Kinesis Data Streams or Amazon Managed Streaming for Apache Kafka (MSK) data streams and pull data directly to Amazon Redshift without staging data in Amazon Simple Storage Service (S3). Define a scheme or choose to ingest semi-structured data with SUPER data type; set up and manage extract, load, and transform (ELT) pipelines with SQL.

Benefits

High throughput, low latency

Process large volumes of streaming data from multiple sources with low latency and high throughput to derive insights in seconds.

Simplified ingestion process

Directly ingest streaming data into your data warehouse from Kinesis Data Streams and MSK without the need to stage in Amazon S3.

Easy to get started managing downstream processing

Perform rich analytics on streaming data within Amazon Redshift using familiar SQL. Define and build materialized views on top of streams directly. Create and manage downstream ELT pipelines by creating MV on MVs, using user-defined functions and stored procedures in Amazon Redshift.

Use cases

Improve gaming experience

Increase in-game conversion, player retention, and optimize gaming experience by analyzing real-time data from gamers.

Analyze IoT data in real-time

Analyze data from thousands of IoT devices and use machine learning (ML) within Amazon Redshift to improve operations, predict customer churn, and grow your business. 

Analyze clickstream user data

The average customer visits dozens of websites in a single session, yet marketers typically analyze only their own websites. Analyze authorized clickstream data ingested into the warehouse to assess your customers’ footprint and behaviors.

Conduct real-time troubleshooting

By accessing and analyzing streaming data from application log files and network logs, developers and engineers can conduct real-time troubleshooting, deliver better products, and alert systems for preventative measures.

Real-time retail analytics on streaming POS data

Access and visualize in near real time all POS retail sales transaction data for real-time analytics, reporting, and visualization. 

Customer Quotes

”LiveMe is a live broadcast app that attracts more than 1 million anchors from over 220 countries. Our app powers more than 100k hours of live broadcasts every day. We use Amazon Redshift's streaming ingestion and other Amazon services for risk control over users' financial activity such as recharge, refund, and rewards. With Amazon Redshift, we are able to view risk control reports and data in near real time, instead of on an hourly basis. This significantly improved our business efficiency.”

PengBo Yang, CTO, Joyme (parent company of LiveMe)

Resources

AWS online tech talk

Accelerate application development with real-time data streams.

Blog

Learn how to ingest data streams across accounts.

Get started with Amazon Redshift Streaming Ingestion

Get started with Amazon Redshift Streaming Ingestion
Start today »
Check out more details
Check out the Amazon Redshift Streaming Ingestion demo
Watch the demo »
Review the step-by-step guide
Check out the guide »