AWS Database Blog

Tag: Kinesis

Analyze user behavior using Amazon Elasticsearch Service, Amazon Kinesis Data Firehose and Kibana

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details. Let’s assume that you work for an ecommerce company and you want to provide the best user experience to your customers. A customer could land on a product page by coming from a recommendation on another page in your application […]

Read More

Use the AWS Database Migration Service to Stream Change Data to Amazon Kinesis Data Streams

In this post, we discuss how you can use AWS Database Migration Service (AWS DMS) to stream change data into Amazon Kinesis Data Streams. An earlier post, Load CDC Data, discussed real-time data processing architecture. As part of that, it covered how to capture changes in an Amazon RDS for Microsoft SQL Server database using AWS […]

Read More

Stream data into an Aurora PostgreSQL Database using AWS DMS and Amazon Kinesis Data Firehose

Using AWS Database Migration Service (AWS DMS), you can migrate data from various sources to most widely used commercial and open-source databases. The service supports homogenous migrations such as Oracle to Oracle. It also supports heterogeneous migrations between different database platforms, such as Oracle to Amazon Aurora or Microsoft SQL Server to MySQL. In addition, […]

Read More

Automatically Archive Items to S3 Using DynamoDB Time to Live (TTL) with AWS Lambda and Amazon Kinesis Firehose

Adam Wagner is a solutions architect at Amazon Web Services. Earlier this year, Amazon DynamoDB released Time to Live (TTL) functionality, which automatically deletes expired items from your tables, at no additional cost. TTL eliminates the complexity and cost of scanning tables and deleting items that you don’t want to retain, saving you money on […]

Read More

Serverless Scaling for Ingesting, Aggregating, and Visualizing Apache Logs with Amazon Kinesis Firehose, AWS Lambda, and Amazon Elasticsearch Service

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details. Pubali Sen and Shankar Ramachandran are solutions architects at Amazon Web Services. In 2016, AWS introduced the EKK stack (Amazon Elasticsearch Service, Amazon Kinesis, and Kibana, an open source plugin from Elastic) as an alternative to ELK (Amazon Elasticsearch Service, the […]

Read More

How to Stream Data from Amazon DynamoDB to Amazon Aurora using AWS Lambda and Amazon Kinesis Firehose

Aravind Kodandaramaiah is a partner solutions architect with the AWS Partner Program Introduction We find that customers running AWS workloads often use both Amazon DynamoDB and Amazon Aurora. Amazon DynamoDB is a fast and flexible NoSQL database service for all applications that need consistent, single-digit millisecond latency at any scale. Its flexible data model and reliable performance […]

Read More

Send Apache Web Logs to Amazon Elasticsearch Service with Kinesis Firehose

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details. We have many customers who own and operate Elasticsearch, Logstash, and Kibana (ELK) stacks to load and visualize Apache web logs, among other log types. Amazon Elasticsearch Service provides Elasticsearch and Kibana in the AWS Cloud in a way that’s […]

Read More

Streaming Changes in a Database with Amazon Kinesis

Emmanuel Espina is a software development engineer at Amazon Web Services. In this blog post, I will discuss how to integrate a central relational database with other systems by streaming its modifications through Amazon Kinesis. The following diagram shows a common architectural design in distributed systems. It includes a central storage referred to as a […]

Read More