AWS Database Blog
Category: Analytics
Archive data from Amazon DynamoDB to Amazon S3 using TTL and Amazon Kinesis integration
In this post, we share how you can use Amazon Kinesis integration and the Amazon DynamoDB Time to Live (TTL) feature to design data archiving. Archiving old data helps reduce costs and meet regulatory requirements governing data retention or deletion policies. Amazon Kinesis Data Streams for DynamoDB captures item-level modifications in a DynamoDB table and […]
Read MoreCombine Amazon Neptune and Amazon OpenSearch Service for geospatial queries
Many AWS customers are looking to solve their business problems by storing and integrating data across a combination of purpose-built databases. The reason for that is purpose-built databases provide innovative ways to build data access patterns that would be challenging or inefficient to solve otherwise. For example, we can model highly connected geospatial data as […]
Read MoreStore and stream sports data feeds using Amazon DynamoDB and Amazon Kinesis Data Streams
Online bookmakers are innovating to offer their clients continuously updated sports data feeds that allow betting throughout the duration of matches. In this post, we walk through a solution to ingest, store, and stream sports data feeds in near real-time using Amazon API Gateway, Amazon DynamoDB, and Amazon Kinesis Data Streams. In betting, odds represent […]
Read MoreBuild interactive graph data analytics and visualizations using Amazon Neptune, Amazon Athena Federated Query, and Amazon QuickSight
Customers have asked for a way to interact with graph datasets in Amazon Neptune using business intelligence (BI) tools such as Amazon QuickSight. Although some BI tools offer generic HTTP connectors that allow you to define a set of REST API calls to extract data from REST endpoints, you have to predefine either Gremlin or […]
Read MoreBuild a fault-tolerant, serverless data aggregation pipeline with exactly-once processing
The business problem of real-time data aggregation is faced by customers in various industries like manufacturing, retail, gaming, utilities, and financial services. In a previous post, we discussed an example from the banking industry: real-time trade risk aggregation. Typically, financial institutions associate every trade that is performed on the trading floor with a risk value […]
Read MoreBuild a near real-time data aggregation pipeline using a serverless, event-driven architecture
The collection, aggregation, and reporting of large volumes of data in near real time is a challenge faced by customers from many different industries, like manufacturing, retail, gaming, utilities, and financial services. In this post, we present a serverless aggregation pipeline in AWS. We start by defining the business problem, introduce a serverless architecture for […]
Read MoreFilter Amazon Aurora database activity stream data for segregation and monitoring
Most organizations need to monitor activity on databases containing sensitive information to ensure security auditing and compliance. Although some security operations teams might be interested in monitoring all activities like read, write, and logons, others might want to restrict monitoring to activities that lead to changes in data and data structures only. In this post, […]
Read MoreBuilding a data discovery solution with Amundsen and Amazon Neptune
September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details. In this post, we discuss the need for a metadata and data lineage tool and the problems it solves, how to rapidly deploy it in the language you prefer using the AWS Cloud Development Kit (AWS CDK), as well as […]
Read MoreAnalyze database performance with Amazon CloudWatch metric streams
With the announcement of Amazon CloudWatch Metric Streams, you can now stream near-real-time metrics data to a destination such as Amazon Simple Storage Service (Amazon S3). Metric Streams supports two primary use cases: Third-party providers – You can stream metrics to partners to power dashboards, alarms, and other tools that rely on accurate and timely […]
Read MoreNear real-time processing with Amazon Kinesis, Amazon Timestream, and Grafana
As organizations adopt and deploy home-connected smart devices, they face challenges utilizing device telemetry data in narrow and broad contexts. Examples of such home-connected devices are smart meters and home sensors that emit telemetry and measurements as time series data. In a narrow context, operational teams use data to understand if devices are operating within […]
Read More