Category: Amazon EC2


EC2’s M3 Instances Go Global; Reduced EC2 Charges and Lower Bandwidth Prices

We continue to work to make AWS more powerful and less expensive, and to pass the savings on to you. To that end, I have three important announcements:

  • EC2’s M3 instance family is now available in all AWS Regions including AWS GovCloud (US).
  • On-Demand prices for EC2 instances in the M1, M2, M3, and C1 families have been lowered.
  • Prices for data transfer between AWS Regions have also been lowered.

M3 Global Rollout
We launched the M3 family of EC2 instances last fall, with initial availability in the US East (Northern Virginia) Region. Also known as the Second Generation Standard Instances, members of the M3 family (Extra Large and Double Extra Large) feature up to 50% higher absolute CPU performance than their predecessors. All instances are 64-bit and are optimized for applications such as media encoding, batch processing, caching, and web serving.

I’m pleased to announce that you can now launch M3 instances in the US East (Northern Virginia), US West (Northern California), US West (Oregon), AWS GovCloud (US), Europe (Ireland), Asia Pacific (Singapore), Asia Pacific (Tokyo), and Asia Pacific (Sydney) Regions. We plan to make the M3 instances available in the South America (Brazil) Region in the coming weeks.

On-Demand Price Reduction
We are reducing the price of On-Demand Amazon EC2 instances running Linux world-wide, effective February 1, 2013.

This reduction applies to all members of the M1 (First Generation Standard), M2 (High Memory), M3 (Second Generation Standard), and C1 (High-CPU) families. The size of the reductions vary, but generally average 10-20%. Here are the reductions, by family and by Region:

  Savings (%)
Region M1 M2 M3 C1 Medium C1 Extra Large
 US East (Northern Virginia) 7.7% 8.9% 13.8% 12.1% 12.1%
 US West (Northern California) 27.7% 9.1% 11.3% 11.3%
 US West (Oregon) 7.7% 8.9% 12.1% 12.1%
 AWS GovCloud (US) 22.3% 9.3% 9.8% 9.8%
 Europe (Ireland) 23.5% 9.1% 11.3% 11.3%
 Asia Pacific (Singapore) 5.9% 2.2% 1.6% 1.9%
 Asia Pacific (Tokyo) 4.3% 2.5% 2.6% 2.6%
 Asia Pacific (Sydney) 5.9% 2.2% 1.6% 1.9%
 South America (So Paulo) 30.4% 20.6% 13.0% 13.0%

Pricing for the M3 instances in the Regions where they are newly available already reflect the economies of scale that allowed us to make the reductions that we are announcing today.

As usual, your AWS bill will automatically reflect the lower prices.

Regional Data Transfer Price Reduction
With nine AWS Regions in operation (and more in the works), you can already build global applications that have a presence in two or more Regions.

Previously, we have charged normal internet bandwidth prices for data transfer between Regions. In order to make this increasingly common scenario even more cost-effective, we are significantly lowering the cost of transferring data between AWS Regions (by 26% to 83%), effective February 1, 2013. You can already transfer data into a Region at no charge ($0.00 / Gigabyte); this price reduction applies to data leaving one Region for another Region.

Here are the details:

Region Old Price / GB New Price / GB Savings
US East (Northern Virginia) $0.120 $0.020 83%
US West (Northern California) $0.120 $0.020 83%
US West (Oregon) $0.120 $0.020 83%
AWS GovCloud (US) $0.155 $0.030 81%
Europe (Ireland) $0.120 $0.020 83%
Asia Pacific (Singapore) $0.190 $0.090 53%
Asia Pacific (Tokyo) $0.200 $0.090 55%
Asia Pacific (Sydney) $0.190 $0.140 26%
South America (So Paulo) $0.250 $0.160 36%

This pricing applies to data transferred out of Amazon EC2 instances, Amazon S3 buckets, and Amazon Glacier vaults.

The new pricing also applies to CloudFront origin fetches. In other words, the cost to use CloudFront in conjunction with static data stored in S3 or dynamic data coming from EC2 will decline as a result of this announcement. This is an important aspect of AWS — the services become an even better value when used together.

Let’s work through an example to see what this means in practice. Suppose you are delivering 100 TB of content per month to your users, with a 10% cache miss rate (90% of the requests are delivered from a cached copy in a CloudFront edge location), and that this content comes from the Standard or Europe (Ireland) Amazon S3 Region. The cost of your origin fetches (from CloudFront to S3) will drop from $1,228.68 to $204.80, an 83% reduction.

Again, your next AWS bill will reflect the lower prices. You need not do anything to benefit.

Jeff;

EC2 for In-Memory Computing – The High Memory Cluster Eight Extra Large Instance

Our new High Memory Cluster Eight Extra Large (cr1.8xlarge) instance type is designed to host applications that have a voracious need for compute power, memory, and network bandwidth such as in-memory databases, graph databases, and memory intensive HPC.

Here are the specs:

  • Two Intel E5-2670 processors running at 2.6 GHz with Intel Turbo Boost and NUMA support.
  • 244 GiB of RAM.
  • Two 120 GB SSD for instance storage.
  • 10 Gigabit networking with support for Cluster Placement Groups.
  • HVM virtualization only.
  • Support for EBS-backed AMIs only.

This is a real workhorse instance, with a total of 88 ECU (EC2 Compute Units). You can use it to run applications that are hungry for lots of memory and that can take advantage of 32 Hyperthreaded cores (16 per processor). We expect this instance type to be a great fit for in-memory analytics systems like SAP HANA and memory-hungry scientific problems such as genome assembly.

The Turbo Boost feature is very interesting. When the operating system requests the maximum possible processing power, the CPU increases the clock frequency while monitoring the number of active cores, the total power consumption and the processor temperature. The processor runs as fast as possible while staying within its documented temperature envelope.

NUMA (Non-Uniform Memory Access) speeds access to main memory by optimizing for workloads where the majority of requests for a particular block of memory come from one of the two processors. By enabling processor affinity (asking the scheduler to tie a particular thread to one of the processors) and taking care to manage memory allocation according to prescribed rules, substantial performance gains are possible. See this Intel article for more information on the use of NUMA.

Pricing starts at $3.50 per hour for Linux instances and $3.831 for Windows instances, both in US East (Northern Virginia). One year and three year Reserved Instances and Spot Instances are also available.

These instances are available in the US East (Northern Virginia) Region. We plan to make them available in other AWS Regions in the future.

— Jeff;

Webinar – Scalable Database Architectures

Ok, there you are, about to build an application that has to store lots and lots of data in cost-effective yet scalable fashion. Should you use a relational model or should you go NoSQL? Should you use DynamoDB, Redshift, RDS, ElastiCache, or more than one?

We’ve set up webinar for later this week to help you learn more about building highly scalable database applications on AWS. This is an updated version of one of the most popular sessions from re:Invent.

Attend our Scalable Database Architecture webinar at 10 AM PT on Thursday, January 17th to learn from the people and companies behind a presidential election, an online learning platform, and a tool to detect brand abuse:

You will also hear from Sundar Raghavan, General Manager for the Amazon Relational Database Service.

The webinar is free but you have to register!

— Jeff;

 

Amazon CloudWatch – Alarm Actions

As you probably know, Amazon CloudWatch provides monitoring services for your cloud resources and your applications. You can track cloud, system, and application metrics, see them visually, and arrange to be notified (via a CloudWatch alarm) if they go beyond a value that you specify. For example, you can track the CPU load of your EC2 instances and receive a notification (via email and/or Amazon SNS) if it exceeds 90% for a period of 5 minutes.

Today we are giving you the ability to stop or terminate your EC2 instances when a CloudWatch alarm is triggered. You can use this as a failsafe (detect an abnormal condition and then act) or as part of your application’s processing logic (await an expected condition and then act).

Before we dig in, I should remind you of one thing. If you are using EBS-backed EC2 instances, you can stop them at any point, with the option to restart them later, while retaining the same instance ID and root volume (this is, of course, distinct from the associated termination option).

Failsafe Ideas
If you (or your developers) are forgetful, you can detect unused EC2 instances and shut them down. You could do this by detecting a very low load average for an extended period of time. This type of failsafe could be used to reduce your AWS bill by making sure that you are not paying for resources you’re not actually using.

You could also implement a failsafe that would detect runaway instances (for example, CPU pegged at 100% for an extended period of time). Perhaps your application gets stuck in a loop from time to time (only when you are not looking, of course). You could also use our CloudWatch monitoring scripts to detect and act on other situations, such as excessive memory utilization).

Processing Logic
Many AWS applications will pull work from an Amazon SQS queue, do the work, and then pass the work along to the next stage of a processing pipeline. You can detect and terminate worker instances that have been idle for a certain period of time.

You can use a similar strategy to get rid of instances that are tasked with handling compute-intensive batch processes. Once the CPU goes idle and the work is done, terminate the instance and save some money!

Application Integration
You can also create CloudWatch alarms based on Custom Metrics that you observe on an instance-by-instance basis. You could, for example, measure calls to your own web service APIs, page requests, or message postings per minute, and respond as desired.

Setting Up Alarm Actions
You can set up alarm actions from the EC2 or CloudWatch tabs of the AWS Management Console. Let’s say you want to start from the EC2 tab. Right-click on the instance of interest and choose Add/Edit Alarms:

Choose your metrics, set up your notification (SNS topic and optional email) and check Take the action, and choose either Stop or Terminate this instance:

The console will confirm the creation of the alarm, and you’re all set (if you asked for an email notification, you need to confirm the subscription within three days):

Your Turn
I can speak for the entire CloudWatch team when I say that we are interested in hearing more about how you will put this feature to use. Feel free to leave a comment and I’ll pass it along to them ASAP.

— Jeff;

 

 

 

AWS Management Console Improvements – Tablet and Mobile Support

Managing your AWS resources has become easier and more direct over the years! Let’s do a quick recap before we dig in:

  • We launched Amazon SQS (2004) and Amazon S3 (2006) as pure APIs, with no tool support whatsoever. Developers were quick to build all sorts of interesting tools around the web service APIs.
  • Later in 2006, we introduced Amazon EC2, this time with a set of command-line tools.
  • Sometime in 2007 we entered the visual, browser-based era with the release of ElasticFox.
  • In early 2009 we released the AWS Management Console and have focused our development efforts there ever since that launch.

Over the years we’ve made many incremental improvements to the AWS Management Console. We’ve also improved the overall look and feel a couple of times. The goal remains unchanged – to provide you with a clean and efficient way to see and manage your AWS resources.

Today we are ready to take another step forward. We’re making some big improvements to the existing Console, and we’re also introducing a brand-new Console App for Android devices. In this post I’ll give you a visual tour of both applications..

AWS Management Console Improvements
We heard your feedback that the growing number of services in the Management Console (21 and counting), increased service functionality, and new form factors such as tablets, required an update to our designs. Our focus is to make AWS easier to use by increasing customizability and improving information display on your screen of choice.

We started with a focus on customization to make the Console work better for you. We moved Region selection into the Consoles navigation and made it work seamlessly across all of the services. You can also customize the Console navigation with shortcuts to the AWS services that you use the most often:

We learned that many Console users spent a lot of time alternately selecting one of a pair of AWS resources in order to compare and contrast certain settings. This wasn’t a good use of your time, so we added inline resource summaries to give you quick access to key resource attributes.

We then reviewed the Console’s information management and display features to improve readability and to put your information front and center. We learned that monitoring resource statistics is one of the most frequent actions and users wanted more space to view graphs. The new Monitoring View makes it easier for you to see statistics for your resources. You can easily filter your resources and hit the new Select All button to see stacked graphs for your resources. You can even change this view to see all the graphs or individual large graphs on one screen.

We also learned that many users wanted as much space as possible for table information. To accommodate this, we added a collapse option to the side navigation pane and moved the table details to let the table fill the screen.

Expanded
Collapsed

Finally, we know that many of you use (or would like to use) the Console from your tablet device, so we now support endless scrolling within the current page. Your resources are just a swipe away! We also optimized the use of horizontal and vertical space and made the buttons and selectors large enough to ensure easy access.

These improvements will be rolled out across the AWS services on slightly different schedules. We look forward to your feedback on these new designs!

AWS Management Console App
This new app provides mobile-relevant tasks that are a good companion to the full web experience including the ability to quickly and easily view and manage your existing EC2 instances and CloudWatch alarms from your Android phone. You can view your total AWS service charges and switch between AWS accounts and regions from within the app. As with the web-based management console, sign-in is as simple as entering your AWS or IAM credentials.

You can:

  • View a summary of your EC2 instances, CloudWatch alarms, total service charges, and the AWS Service Health status, with optional filtering on the instances and alarms.
  • Look at EC2 instance metrics and status checks to check the state of your environment.
  • Stop or reboot your EC2 instances.
  • List CloudWatch alarms by state or time.
  • View CloudWatch graphs.
  • Examine the automated actions configured for each CloudWatch alarm.
  • View detailed AWS service health status, including recent AWS service events and notifications.

Here’s a tour:

We plan to add support for additional services very quickly, so stay tuned (and use the app’s feedback function to tell us what you think). We are also planning to support mobile devices running other operating systems.

Download the AWS Management Console for Android and get started today.

By the way, the Management Console team is hiring! If youre interested in building web or mobile user interfaces, check out our open positions:

— Jeff;

 

Even More Features for AWS GovCloud (US)

What’s GovCloud?

AWS GovCloud (US) is a gated community cloud designed to support the compliance needs of customer workloads with direct or indirect ties to U.S. Government functions, services, or regulations. The AWS GovCloud (US) framework adheres to U.S. International Traffic in Arms Regulations (ITAR) requirements.

Hot on the heels of my last GovCloud post, I’ve got even more good news today. Two more AWS services are now available for use in AWS GovCloud (US).

If you are part of the US Federal Government, a US state or local government, if you provide services to them, or if you need to follow Federal regulations on Controlled Unclassified Information (CUI) data protection, you can now make use of the following services:

Amazon Relational Database Service (RDS) – Launch database instances running MySQL, Oracle Database, or Microsoft SQL Server and let RDS take care of all of the common database administration and scaling tasks for you.

Amazon DynamoDB – Use this fully managed NoSQL database to build highly scalable dynamic applications while taking advantage of fast, SSD-based storage and automatic replication across three Availability Zones.

As I mentioned before, we recently added support for High Performance Computing, Elastic Load Balancing, Auto Scaling, CloudWatch Alarms, the Simple Notification Service, the Simple Queue Service, and the ElasticWolf management application.

Putting it all together, you now have access to a very broad and powerful set of services in AWS GovCloud (US). Here are some of the ways that our other AWS customers have been using GovCloud and AWS:

BioSense 2.0
The US Centers for Disease Control and Prevention hosts the BioSense 2.0 application on AWS. Watch this video to learn how they use Amazon S3, Amazon EC2, Elastic MapReduce, and the Amazon Simple Email Service (for push notifications) to build a system that meets their needs for privacy, security, and availability while saving money:

The BioSense application gives local, state, and federal officials the power to monitor and assess a set of symptoms that, taken together, characterize a disease or other threat to public health.

NIH Microbiome Cloud
http://www.ncats.nih.gov was established to speed up the basic scientific discovery for patient therapies and treatments.  A new website was created on AWS GovCloud (US) to help catalyze the generation of innovative methods and technologies that will enhance the development, testing and implementation of diagnostics and therapeutics across a wide range of human diseases and conditions.

For more information about the AWS GovCloud (US), visit the GovCloud site or contact us.

— Jeff;

The New EC2 High Storage Instance Family

In our continuing quest to ensure that Amazon EC2 is applicable to an incredibly broad array of use cases, we are launching a new instance family today.

The High Storage Eight Extra Large (hs1.8xlarge) instances are a great fit for applications that require high storage depth and high sequential I/O performance. Each instance includes 117 GiB of RAM, 16 virtual cores (providing 35 ECU of compute performance), and 48 TB of instance storage across 24 hard disk drives capable of delivering up to 2.4 GB per second of I/O performance.

This instance family is designed for data-intensive applications that require high storage density and high sequential I/O — data warehousing, log processing, and seismic analysis (to name a few). We know that these applications can generate or consume tremendous amounts of data and that you want to be able to run them on EC2.

The storage on this instance family is local, and has a lifetime equal to that of the instance. You should think of these instances as building blocks that you can use to build a complete storage system. You should build a degree of redundancy into your storage architecture (e.g. RAID 1, 5, or 6) and you should use a fault-tolerant file system like HDFS or Gluster. Of course, you should also back up your data to Amazon S3 for increased durability.

Here’s some nmon output with all of the disks visible:

You can launch multiple High Storage Eight Extra Large instances in a placement group for high bandwidth low latency networking between the instances.

High Storage instances are available now in the US East (Northern Virginia) Region and will be made available in other AWS Regions in the coming months. On-Demand pricing is $4.60 per hour in US East (Northern Virginia). You can also purchase one and three year Reserved Instances (Light, Medium, or Heavy). See the EC2 pricing page for more information.

— Jeff;

The AWS Report – Jeff Lawson of Twilio

In the latest episode of The AWS Report, I spoke with Jeff Lawson of Twilio to learn more about what they do and how they use AWS:

As Jeff noted, Twilio has a presence in six AWS Regions and their service is available in 40 countries. He told me that none of their developers or admins are allowed to log in to their production systems since this is a source of human error.

— Jeff;

PS – I wrote about Twilio for the first time way back in 2008. It was wonderful to hear how they’d been able to grow over the last 4 years, aided by AWS.

 

EBS Snapshot Copy (Between Regions)

To make it even easier for you to build AWS applications that span regions, we’re introducing a new EBS Snapshot Copy feature today. You can now copy EBS snapshots between EC2 Regions.

Why Copy?

So, why would you want to copy an EBS Snapshot from one AWS Region to another? Here are some of the more common use cases:

  • Geographic Expansion – You want to be able to launch your application in a new Region.
  • Migration – You want to be able to migrate your application from one Region to another. 
  • Disaster Recovery – You want to back up your data and your log files across different geographical locations at regular intervals to minimize data loss and recovery time.

EBS Snapshot Copy simplifies each of these use cases by simplifying the copy process.

Ok, How do I Copy Snapshots?
You can now copy EBS Snapshots from one AWS Region to another. You can copy any accessible Snapshots that are in the “completed” status. This includes Snapshots that you created, Snapshots that were shared with you, and also Snapshots from the AWS Marketplace, VM Import/Export, and Storage Gateway. If you copy a Marketplace product to a new Region, you need to make sure that the product is supported in the destination Region.

You can initiate copies from the AWS Management Console or from the command line. You can also use the new CopySnapshot function from your own code. Here’s how you initiate a copy operation from the Console:

After you choose the Copy Snapshot operation, the Console will ask you where you would like to copy the snapshot:

While the copy is underway, you can watch the progress by switching to the destination Region:

What Else Do I Need to Know?
Here are a few important facts about this new feature:

  • The AWS Management Console shows the progress of a Snapshot copy in progress, you can check the percentage complete there.
  • You can initiate multiple Snapshot Copy commands simultaneously either by selecting and copying multiple Snapshots to the same region, or by copying a snapshot to multiple regions in parallel.  The in-progress copies do not affect the performance of the associated EBS Volumes.
  • The console-based interface is push-based; you log in to the source Region and tell the console where you’d like the Snapshot to end up. The API and the command line are, by contrast, pull-based and you must run them within the destination Region.

Pricing
You will be charged the regular data transfer prices when you copy snapshots between Regions. Once you have created a new Snapshot, you’ll pay the regular price to store it in the destination Region.

Coming Soon
As you can see, EBS Snapshot Copy, will let you simplify your geographic expansion, data center migration, and disaster recovery. As a next step, we will implement copying of AMIs (Amazon Machine Images) between Regions. You’ll be able to copy public and custom-created AMIs to the Region of your choice.

— Jeff;

AWS Expansion in Brazil – Elastic Beanstalk, Provisioned IOPS for EBS and RDS

We launched an AWS Region in Brazil almost a year ago, along with Portuguese and Spanish versions of the AWS Blog.

Today we are adding the following new AWS functionality in the Region:

AWS Elastic Beanstalk – You can now deploy and manage .NET, PHP, Python, Ruby, and Java applications in the AWS Cloud using AWS Elastic Beanstalk. With this new addition to our lineup in Brazil, you can get your application online more quickly and allow Elastic Beanstalk to handle system updates, scaling, monitoring, and lots more.

EBS Provisioned IOPS – You can now create EBS Provisioned IOPS volumes in Brazil, with up to 2000 IOPS per EBS volume. This will give you more control and the ability to create fast, responsive applications on AWS.

RDS Provisioned IOPS – You can now create RDS Database Instances with up to 10,000 Provisioned IOPS (for MySQL and Oracle) or 7,000 (SQL Server). This gives you the power to create fast, responsive database-driven applications on AWS. As part of today’s launch, this is also available in the Asia Pacific (Singapore) Region.

— Jeff;

PS[0] – Check out the AWS Products and Services by Region to find out which services are available in each AWS Region.

PS[1] – This post is also available in Portuguese and Spanish.