AWS Big Data Blog

Analyze a Time Series in Real Time with AWS Lambda, Amazon Kinesis and Amazon DynamoDB Streams

This is a guest post by Richard Freeman, Ph.D., a solutions architect and data scientist at JustGiving. JustGiving in their own words: “We are one of the world’s largest social platforms for giving that’s helped 26.1 million registered users in 196 countries raise $3.8 billion for over 27,000 good causes.” Introduction As more devices, sensors […]

Read More

AWS Partner Post Spotlight: Attunity

Partners are a vital part of the AWS ecosystem, and AWS Partners have made important contributions to the AWS Big Data Blog. This month’s Partner Post Spotlight is on Attunity, who co-authored the post “Using Attunity CloudBeam at UMUC to Replicate Data to Amazon RDS and Amazon Redshift.” Their post explains how UMUC used Attunity […]

Read More

Big Data Website Gets a Big Makeover at AWS

Jorge A. Lopez is responsible for Big Data Solutions Marketing at AWS The big data ecosystem is evolving at a tremendous pace, giving rise to a plethora of tools, use cases, and applications. The new AWS Big Data website is now the ideal starting point to learn about new and existing capabilities, and the services […]

Read More

Analyze Your Data on Amazon DynamoDB with Apache Spark

Manjeet Chayel is a Solutions Architect with AWS Every day, tons of customer data is generated, such as website logs, gaming data, advertising data, and streaming videos. Many companies capture this information as it’s generated and process it in real time to understand their customers. Amazon DynamoDB is a fast and flexible NoSQL database service […]

Read More

Month in Review: February 2016

Lots for big data enthusiasts in February on the AWS Big Data Blog. Take a look! Submitting User Applications with spark-submit Learn how to set spark-submit flags to control the memory and compute resources available to your application submitted to Spark running on EMR. Learn when to use the maximizeResourceAllocation configuration option and dynamic allocation […]

Read More

Optimize Spark-Streaming to Efficiently Process Amazon Kinesis Streams

Rahul Bhartia is a Solutions Architect with AWS Martin Schade, a Solutions Architect with AWS, also contributed to this post. Do you use real-time analytics on AWS to quickly extract value from large volumes of data streams? For example, have you built a recommendation engine on clickstream data to personalize content suggestions in real time […]

Read More

Introducing On-Demand Pipeline Execution in AWS Data Pipeline

Marc Beitchman is a Software Development Engineer in the AWS Database Services team Now it is possible to trigger activation of pipelines in AWS Data Pipeline using the new on-demand schedule type. You can access this functionality through the existing AWS Data Pipeline activation API. On-demand schedules make it easy to integrate pipelines in AWS […]

Read More

Join us at the AWS Big Data Meetup on February 24th in Palo Alto

Join and RSVP! Guest Speaker: Cory Dolphin from Twitter Learn about how Answers, Fabric’s realtime analytics product, which processes billions of events in realtime, using Twitter’s new stream processing engine, Heron. Cory will explain some of the challenges the team faced while scaling Storm, and how Heron has helped them fly faster. Specifically, Cory will describe how Heron’s […]

Read More

Process Amazon Kinesis Aggregated Data with AWS Lambda

Ian Meyers is a Solutions Architecture Senior Manager with AWS Last year, we introduced the Amazon Kinesis Producer Library (KPL) to simplify the development of applications that need to send data to Amazon Kinesis Streams. Many customers use aggregation, which allows you to send multiple records to a single Amazon Kinesis Streams record.  Although the […]

Read More

Big Data Analytics Options on AWS: Updated White Paper

Erik Swensson is an Enterprise Solutions Architect Manager for AWS The big data ecosystem is growing quickly. Many AWS services have recently been added, such as AWS Lambda, Amazon Elasticsearch Service, Amazon Kinesis Firehose, and Amazon Machine Learning. We’ve also made significant enhancements to existing analytics offerings, such as supporting JSON documents in Amazon DynamoDB […]

Read More