Scientists, developers, and many other technologists from many different industries are taking advantage of Amazon Web Services to perform big data analytics and meeting the challenges of the increasing volume, variety, and velocity of digital information.  Amazon Web Services offers an comprehensive, end-to-end portfolio of cloud computing services to help you manage big data by reducing costs, scaling to meet demand, and increasing the speed of innovation.

See the AWS Big Data solutions for every stage of the big data lifecycle:

Collect > Stream > Store > RDBMS | Data Warehouse | NoSQL > Analytics > Archive

Getting up to speed with big data in the cloud has never been easier. Explore the Getting Started section, with tutorials and resources, to help start your first project.




The AWS Big Data Blog is intended for solutions architects, data scientists and developers to learn big data best practices, discover which managed AWS dig data services are the best fit for their use case, and help get started and to go deep on AWS big data services. The goal of the blog is to make this the hub for anyone to discover new ways to collect, store, process, analyze, and visualize data at any scale. Readers will find short tutorials with code samples, case studies that demonstrate the unique benefits of working with big data on AWS, new feature announcements, partner and customer generated demos and tutorials, with tips and best practices for using AWS big data services.

View the AWS Big Data Blog »

blog

Visit the new AWS Big Data Blog, for example, read how to use Dynamo DB to get 100,000 gamers up and running for less than a cup of coffee.

trainingCourse

The new Big Data Technology Fundamentals course is perfect for getting started in learning how to run big data applications in the AWS Cloud.  This is a free, online training course and is intended for individuals who are new to big data concepts, including solutions architects, data scientists, and data analysts.

The course covers the development of big data solutions using the Hadoop ecosystem, including MapReduce, HDFS, and the Pig and Hive programming frameworks.

Learn more about the Big Data Technology Fundamentals course »

Learn why AWS customer, Metamarkets, uses AWS for big data and real time analytics for digital marketing workloads.

View other big data use cases »

From running highly successful marketing campaigns that generate 100x the normal traffic to performing multiple rounds of real-time bidding of ads in under 200 milliseconds, digital marketing solutions require big data technologies to be competitive in today's online markeplace.


Learn more about Digital Marketing on AWS »

bigdata-metamarkets-thumbnail

It feels like everything generates data today, from your customers on social networks to the instances running your web applications.  AWS makes it easy to provision the storage, computation, and database services you need to turn that data into information for your business.  AWS also has data transfer services which can move big data into and out of the cloud quickly such as AWS Direct Connect and our Import/Export service. Furthermore, all inbound data traffic into AWS is free.

Learn how to send hard drives to the cloud with AWS Import/Export »

Learn how to get your private fiber line to the cloud with AWS Direct Connect »

integ-cloud-backup

RDS is available for MySQL, PostgreSQL, Oracle, and SQL Server.

kinesis-thumb2
2:08
Watch the Introduction to Amazon Kinesis video

Amazon Kinesis is a managed service for real-time processing of streaming big data. Amazon Kinesis supports data throughput from megabytes to gigabytes of data per second and can scale seamlessly to handle streams from hundreds of thousands different sources. Designed to provide for high availability and durability in cost-effective manner, you can now focus on making sense of your data which will enable you to make better decisions faster and at lower costs.

Learn more about Amazon Kinesis »

Whether you’re storing pharmaceutical data for analysis, financial data for computation and pricing, or multimedia files such as photos and videos, Amazon Simple Storage Service (S3) is the ideal big data cloud storage solution to store original content durably. Designed for eleven 9's of durability, with no single point of failure, Amazon S3 is your fundamental big data object store.

Learn more about Amazon S3 »

Amazon Elastic Block Store (EBS) provides hard drives for as persistent storage for virtual machines.  Amazon EBS volumes offer the consistent and low-latency performance needed to run big data workloads such as your own relational or NoSQL databases, enterprise applications, and high performance distributed network file systems.

Learn more about Amazon EBS »

diff_aws_report_new
3:01
AWS Report with Deepak Singh, Amazon EC2

NoSQL data stores benefit greatly from the speed of solid state drives (SSDs). Amazon DynamoDB uses them by default, but if you are using alternatives from the AWS Marketplace, such as Cassandra or MongoDB, you can accelerate your access with on-demand access to terabytes of solid state storage, with the High I/O instance class.

Learn more about the options with EC2 instance types »

When you need a NoSQL database without the operational burden to run it, look no further than Amazon DynamoDB.  It is a fast, fully-managed NoSQL database service that makes it simple and cost-effective to store and retrieve any amount of data, and serve any level of request traffic. 

Amazon DynamoDB has provisioned guaranteed throughput and single-digit millisecond latency make it a great fit for gaming, ad tech, mobile and many other big data applications.

Learn more about Amazon DynamoDB »

dropcam-video-thumbnail
5:22
Watch the Dropcam re:invent Keynote


test-database

Find out more details on Amazon RDS

Big data innovation goes beyond NoSQL, it is more about bringing the appropriate technology to use on your data depending on your business needs. Relational databases deliver fast, predictable, and consistent performance; and it is optimized for transactional workloads such as point of sales or financial history.  Relational databases play a complementary role to NoSQL databases in many comprehensive big data architectures.

Amazon RDS allows you to easily to set up, operate, and scale a relational database in the cloud. It provides cost-efficient and resizable capacity while managing time-consuming database administration tasks, freeing you up to focus on your applications and business.

Learn more about Amazon RDS »

Amazon Redshift provides a fast, fully-managed, petabyte-scale data warehouse for less than $1000 per terabyte per year. Amazon Redshift delivers fast query and I/O performance for virtually any size dataset by using columnar storage technology and parallelizing and distributing queries across multiple nodes. In just a few minutes, you can easily provision a fully managed data warehouse with automated backups and built-in encryption. Plug in easily with your existing business intelligence tools.

Learn more about Amazon Redshift »

big_data_redshift
3:22
Introduction to Amazon Redshift

amazon-emr-thumb
Introduction to Amazon Elastic MapReduce

Amazon Elastic MapReduce (EMR) provides the powerful Apache Hadoop framework on Amazon EC2 as a easy-to-use managed service. With Amazon EMR, you can focus on your map/reduce queries and take advantage of the broad ecosystem of Hadoop tools, while deploying to a high-scale, secure infrastructure platform.  Run big data analytics jobs in the cloud with ease; let Amazon EMR do the work of managing your Hadoop clusters.

Learn more about Amazon EMR »

How fast could your project go with another 1000 virtual machines? How about 10,000? The Amazon Spot Market, integrated into Amazon Elastic MapReduce, lets you choose your own price for the computing resources you need to do analytics with cloud computing. That means you can choose your own balance of cost and performance, overclocking your analytics when you need to, or reducing costs significantly.

Get started with Spot instances »

request_instances
4:13
Getting Started with Spot Instances

Thumb-GettingStartedWithGlacier
3:17
Watch the Getting Started with Glacier video

Amazon Glacier allows you to offload the administrative burdens of operating and scaling archival storage to AWS, and makes retaining data for long periods, whether measured in years or decades, especially simple.  Amazon Glacier is an extremely low-cost cold storage service starting at $0.01 per GB per month. There are no upfront capital commitments, and all ongoing operational expenses are included in the price.

Learn more about Amazon Glacier »