AWS Big Data Blog

Month in Review: February 2017

by Derek Young | on | Permalink | Comments |  Share

Another month of big data solutions on the Big Data Blog! Take a look at our summaries below and learn, comment, and share. Thank you for reading! NEW POSTS Implement Serverless Log Analytics Using Amazon Kinesis Analytics In this post, learn how how to implement a solution that analyzes streaming Apache access log data from an […]

Read More

Join us next week at Strata + Hadoop World in San Jose, CA

by Jorge A. Lopez | on | Permalink | Comments |  Share

We’re back in San Jose for the Strata conference, March 13-16, 2017, to talk all things big data at AWS and show you some of our latest innovations. Come meet the AWS Big Data team at booth #928, where big data experts will be happy to answer your questions, hear about your requirements, and help […]

Read More

Analyzing VPC Flow Logs with Amazon Kinesis Firehose, Amazon Athena, and Amazon QuickSight

Many business and operational processes require you to analyze large volumes of frequently updated data. Log analysis, for example, involves querying and visualizing large volumes of log data to identify behavioral patterns, understand application processing flows, and investigate and diagnose issues. VPC flow logs capture information about the IP traffic going to and from network […]

Read More

Big Updates to the Big Data on AWS Training Course!

by Sara Snedeker | on | Permalink | Comments |  Share

AWS offers a range of training resources to help you advance your knowledge with practical skills so you can get more out of the cloud. We’ve updated Big Data on AWS, a three-day, instructor-led training course to keep pace with the latest AWS big data innovations. This course allows you to hear big data best […]

Read More

Analyze Security, Compliance, and Operational Activity Using AWS CloudTrail and Amazon Athena

As organizations move their workloads to the cloud, audit logs provide a wealth of information on the operations, governance, and security of assets and resources. As the complexity of the workloads increases, so does the volume of audit logs being generated. It becomes increasingly difficult for organizations to analyze and understand what is happening in […]

Read More

Harmonize, Search, and Analyze Loosely Coupled Datasets on AWS

You have come up with an exciting hypothesis, and now you are keen to find and analyze as much data as possible to prove (or refute) it. There are many datasets that might be applicable, but they have been created at different times by different people and don’t conform to any common standard. They use […]

Read More

Scheduled Refresh for SPICE Data Sets on Amazon QuickSight

Jose Kunnackal is a Senior Product Manager for Amazon Quicksight This blog post has been translated into Japanese. In November 2016, we launched Amazon QuickSight, a cloud-powered, business analytics service that lets you quickly and easily visualize your data. QuickSight uses SPICE (Super-fast, Parallel, In-Memory Calculation Engine), a fully managed data store that enables blazing […]

Read More

Create Tables in Amazon Athena from Nested JSON and Mappings Using JSONSerDe

Most systems use Java Script Object Notation (JSON) to log event information. Although it’s efficient and flexible, deriving information from JSON is difficult. In this post, you will use the tightly coupled integration of Amazon Kinesis Firehose for log delivery, Amazon S3 for log storage, and Amazon Athena with JSONSerDe to run SQL queries against these logs without […]

Read More

AWS Big Data is Coming to HIMSS!

by Christopher Crosbie | on | Permalink | Comments |  Share

The AWS Big Data team is coming to HIMSS, the industry-leading conference for professionals in the field of healthcare technology. The conference brings together more than 40,000 health IT professionals, clinicians, administrators, and vendors to talk about the latest innovations in health technology. Because transitioning healthcare to the cloud is at the forefront of this […]

Read More

Migrate External Table Definitions from a Hive Metastore to Amazon Athena

For customers who use Hive external tables on Amazon EMR, or any flavor of Hadoop, a key challenge is how to effectively migrate an existing Hive metastore to Amazon Athena, an interactive query service that directly analyzes data stored in Amazon S3. With Athena, there are no clusters to manage and tune, and no infrastructure to […]

Read More