AWS Big Data Blog

Encrypt and Decrypt Amazon Kinesis Records Using AWS KMS

Customers with strict compliance or data security requirements often require data to be encrypted at all times, including at rest or in transit within the AWS cloud. This post shows you how to build a real-time streaming application using Kinesis in which your records are encrypted while at rest or in transit. Amazon Kinesis overview […]

Read More

Big Data Resources on the AWS Knowledge Center

The AWS Knowledge Center answers the questions we receive most frequently from AWS customers. It is a resource for you that is distinct from AWS Documentation, the AWS Discussion Forums, and the AWS Support Center. It covers questions from across every AWS service. Specific big data services covered on the Knowledge Center include Amazon EMR, Amazon Athena, […]

Read More

Running R on Amazon Athena

This blog post has been translated into Japanese. Data scientists are often concerned about managing the infrastructure behind big data platforms while running SQL on R. Amazon Athena is an interactive query service that works directly with data stored in S3 and makes it easy to analyze data using standard SQL without the need to […]

Read More

Amazon Redshift Monitoring Now Supports End User Queries and Canaries

Ian Meyers is a Solutions Architecture Senior Manager with AWS The serverless Amazon Redshift Monitoring utility lets you gather important performance metrics from your Redshift cluster’s system tables and persists the results in Amazon CloudWatch. This serverless solution leverages AWS Lambda to schedule custom SQL queries and process the results. With this utility, you can use […]

Read More

Month in Review: February 2017

Another month of big data solutions on the Big Data Blog! Take a look at our summaries below and learn, comment, and share. Thank you for reading! NEW POSTS Implement Serverless Log Analytics Using Amazon Kinesis Analytics In this post, learn how how to implement a solution that analyzes streaming Apache access log data from an […]

Read More

Analyzing VPC Flow Logs with Amazon Kinesis Firehose, Amazon Athena, and Amazon QuickSight

Many business and operational processes require you to analyze large volumes of frequently updated data. Log analysis, for example, involves querying and visualizing large volumes of log data to identify behavioral patterns, understand application processing flows, and investigate and diagnose issues. VPC flow logs capture information about the IP traffic going to and from network […]

Read More

Analyze Security, Compliance, and Operational Activity Using AWS CloudTrail and Amazon Athena

As organizations move their workloads to the cloud, audit logs provide a wealth of information on the operations, governance, and security of assets and resources. As the complexity of the workloads increases, so does the volume of audit logs being generated. It becomes increasingly difficult for organizations to analyze and understand what is happening in […]

Read More