AWS Database Blog

Category: Amazon Kinesis

Creating Amazon Timestream interpolated views using Amazon Kinesis Data Analytics for Apache Flink

Many organizations have accelerated their adoption of stream data processing technologies in an effort to more quickly derive actionable insights from their data. Frequently, it is required that data from streams be computed into metrics or aggregations and stored in near real-time for analysis. These computed values should be generated and stored as quickly as […]

Read More

Performing analytics on Amazon Managed Blockchain

Managed Blockchain follows an event-driven architecture. We can open up a wide range of analytic approaches by streaming events to Amazon Kinesis. For instance, we could analyze events in near-real time with Kinesis Data Analytics, perform petabyte scale data warehousing with Amazon RedShift, or use the Hadoop ecosystem with Amazon EMR. This allows us to use the right approach for every blockchain analytics use case.
In this post, we show you one approach that uses Amazon Kinesis Data Firehose to capture, monitor, and aggregate events into a dataset, and analyze it with Amazon Athena using standard SQL.

Read More

Accelerating Nylas’s feature development with AWS Data Lab

This is a guest post by David Ting, VP of Engineering at Nylas. In their own words, Nylas is a pioneer and leading provider of universal communications APIs that allow developers to quickly connect their applications to every email, calendar, or contacts provider in the world. Over 26,000 developers around the globe use the Nylas […]

Read More

The most-viewed AWS Database Blog posts in 2018

The following 10 posts were the most viewed AWS Database Blog posts that we published during 2018. You can use this list as a guide to catch up on your Database Blog reading, or read a post again that you found particularly useful. Query your AWS database from your serverless application How the AWS Schema […]

Read More

Analyze user behavior using Amazon Elasticsearch Service, Amazon Kinesis Data Firehose and Kibana

Let’s assume that you work for an ecommerce company and you want to provide the best user experience to your customers. A customer could land on a product page by coming from a recommendation on another page in your application or from a search engine. Whatever the route, you want to ensure that your customers […]

Read More

Use the AWS Database Migration Service to Stream Change Data to Amazon Kinesis Data Streams

In this post, we discuss how you can use AWS Database Migration Service (AWS DMS) to stream change data into Amazon Kinesis Data Streams. An earlier post, Load CDC Data, discussed real-time data processing architecture. As part of that, it covered how to capture changes in an Amazon RDS for Microsoft SQL Server database using AWS […]

Read More

Load CDC data from relational databases to Amazon Kinesis using Database Migration Service

UPDATE, March 2019: Since this blog post was published, DMS has launched a new capability to set Kinesis as a target. For the most updated guidance, please visit Use the AWS Database Migration Service to Stream Change Data to Amazon Kinesis Data Streams. ———————————— Many large enterprises are moving their data processing from batch to […]

Read More

Stream changes from Amazon RDS for PostgreSQL using Amazon Kinesis Data Streams and AWS Lambda

In this post, I discuss how to integrate a central Amazon Relational Database Service (Amazon RDS) for PostgreSQL database with other systems by streaming its modifications into Amazon Kinesis Data Streams. An earlier post, Streaming Changes in a Database with Amazon Kinesis, described how to integrate a central RDS for MySQL database with other systems […]

Read More

Stream data into an Aurora PostgreSQL Database using AWS DMS and Amazon Kinesis Data Firehose

Using AWS Database Migration Service (AWS DMS), you can migrate data from various sources to most widely used commercial and open-source databases. The service supports homogenous migrations such as Oracle to Oracle. It also supports heterogeneous migrations between different database platforms, such as Oracle to Amazon Aurora or Microsoft SQL Server to MySQL. In addition, […]

Read More

Send Apache Web Logs to Amazon Elasticsearch Service with Kinesis Firehose

We have many customers who own and operate Elasticsearch, Logstash, and Kibana (ELK) stacks to load and visualize Apache web logs, among other log types. Amazon Elasticsearch Service provides Elasticsearch and Kibana in the AWS Cloud in a way that’s easy to set up and operate. Amazon Kinesis Firehose provides reliable, serverless delivery of Apache […]

Read More