AWS Database Blog

Category: Amazon Kinesis

Use the AWS Database Migration Service to Stream Change Data to Amazon Kinesis Data Streams

In this post, we discuss how you can use AWS Database Migration Service (AWS DMS) to stream change data into Amazon Kinesis Data Streams. An earlier post, Load CDC Data, discussed real-time data processing architecture. As part of that, it covered how to capture changes in an Amazon RDS for Microsoft SQL Server database using AWS […]

Read More

Load CDC data from relational databases to Amazon Kinesis using Database Migration Service

UPDATE, March 2019: Since this blog post was published, DMS has launched a new capability to set Kinesis as a target. For the most updated guidance, please visit Use the AWS Database Migration Service to Stream Change Data to Amazon Kinesis Data Streams. ———————————— Many large enterprises are moving their data processing from batch to […]

Read More

Stream changes from Amazon RDS for PostgreSQL using Amazon Kinesis Data Streams and AWS Lambda

In this post, I discuss how to integrate a central Amazon Relational Database Service (Amazon RDS) for PostgreSQL database with other systems by streaming its modifications into Amazon Kinesis Data Streams. An earlier post, Streaming Changes in a Database with Amazon Kinesis, described how to integrate a central RDS for MySQL database with other systems […]

Read More

Stream data into an Aurora PostgreSQL Database using AWS DMS and Amazon Kinesis Data Firehose

Using AWS Database Migration Service (AWS DMS), you can migrate data from various sources to most widely used commercial and open-source databases. The service supports homogenous migrations such as Oracle to Oracle. It also supports heterogeneous migrations between different database platforms, such as Oracle to Amazon Aurora or Microsoft SQL Server to MySQL. In addition, […]

Read More

Send Apache Web Logs to Amazon Elasticsearch Service with Kinesis Firehose

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details. We have many customers who own and operate Elasticsearch, Logstash, and Kibana (ELK) stacks to load and visualize Apache web logs, among other log types. Amazon Elasticsearch Service provides Elasticsearch and Kibana in the AWS Cloud in a way that’s […]

Read More

Streaming Changes in a Database with Amazon Kinesis

Emmanuel Espina is a software development engineer at Amazon Web Services. In this blog post, I will discuss how to integrate a central relational database with other systems by streaming its modifications through Amazon Kinesis. The following diagram shows a common architectural design in distributed systems. It includes a central storage referred to as a […]

Read More