Category: Amazon EC2


AWS Marketplace – Support for Windows Applications (ISVs Wanted)

Today we are opening up the AWS Markeplace to applications running on Microsoft Windows. We’re launching with an initial set of applications, with many more in the pipeline for release in the near future. If you are a Windows ISV (Independent Software Vendor), you should consider making your products available in the Marketplace. I’ll tell you why in just a minute.

What’s the Marketplace?
The AWS Marketplace makes it easy for you to find, compare, and immediately start using a wide variety of applications and technical services in the cloud. The applications you run are hosted on Amazon EC2 instances running Linux or Microsoft Windows, with 1-click deployment and billing integrated into your existing AWS account. You pay only for what you use, allowing you to eliminate on-going hardware and software costs when your needs change or your projects come to an end.

Now For Windows
We launched the AWS Marketplace earlier this year with support for applications and services running on Linux. Effective immediately, we now support applications that run on Microsoft Windows Server. We support a number of versions of Windows Server including Windows Server 2008, Windows Server 2008 R2, and Windows Server 2012.

You can choose to run Windows on the full spectrum of EC2 instance types, from micro instances up to our largest cluster-computing instances, and in any of our nine regions worldwide.  Windows is also available in the AWS Free Usage Tier, giving you the ability to test the environment for up to 750 hours per month without incurring charges, and then easily migrate to full-fledged environments when you are ready. 

Windows instances work seamlessly with existing AWS features like Amazon Elastic Block Store (EBS), Amazon CloudWatch, AWS CloudFormation, Amazon Virtual Private Cloud, AWS Elastic Beanstalk, Elastic Load Balancing, and Amazon Relational Database Service.  EC2 Windows instances are pre-configured for security and high-performance on AWS, and are set up for automatic patching and deployment.  All Windows EC2 instances come with basic support, and business- and enterprise-level support is available.

Windows Applications in the Marketplace
The following Windows applications are now available in the AWS Marketplace:

  • Parallels Plesk – A control panel for hosting and managing multiple web sites on a single server. 
  • Quest Toad – A database productivity tool for DBAs, developers, and analysts.
  • MicroStrategy – A comprehensive BI (Business Intelligence) platform.
  • SAP Afaria – Mobile device management.

ISVs Wanted
If you are a Windows ISV, we want you to list your software in the AWS Marketplace so that you can sell it to hundreds of thousands of active AWS customers located all over the world. Because the AWS Marketplace shortens the time between discovery and deployment to a matter of minutes, you’ll find that the sales cycle is dramatically shortened as well — decisions that once took days or weeks now take just minutes. Once your application is in the Marketplace, we’ll take care of all billing, collections, and disbursements for you so that you can focus on building your products and your business. You can now sell your traditional on-premises software in the cloud using a SaaS (Software-as-a-Service) model. Learn more about Selling on Amazon Marketplace.

— Jeff;

AWS Marketplace – Additional EC2 Operating System Support (FreeBSD, Debian, CentOS)

We’re working hard to make the AWS Marketplace even more flexible and to make sure that it contains the operating systems, tools, and other resources that you need. Today we are adding support for three new open source operating systems: FreeBSD, CentOS, and Debian. We are also making it easier for you to find software that runs on the operating system of your choice.

Expanded Operating System Support
You can now launch three new operating systems from within the AWS Marketplace:

FreeBSD is an advanced operating system for modern server, desktop and embedded computer platforms. FreeBSD provides advanced networking, impressive security features, and world class performance, and is used by some of the world’s busiest websites.

Debian is a popular and influential Linux distribution. The current stable release includes support for over 29,000 packages.

CentOS is a free, Enterprise-class Linux distribution derived from publicly available sources. CentOS conforms fully with the upstream vendor’s redistribution policy and aims to be 100% binary compatible with their offering.

These operating systems come directly from the Open Source community and are available at no charge other than the usual cost for the EC2 instances. You can find them in the Operating Systems section of the Marketplace.

Improved Searching
You can now search for software that’s running on the operating system of your choice:

I hope that you enjoy these new additions to the Marketplace.

— Jeff;

EBS Volume Status Checks

We’ve received some great feedback on the EC2 instance status checks that were released earlier this year. Our customers appreciate the fact that we can detect and report on how their EC2 instances are performing.

Today we are adding a new status check for EBS Provisioned IOPS volumes. As you know, you can now provision up to 2000 IOPS when you create a new EBS volume:

We are now making a new I/O Performance status check available for each Provisioned IOPS volume. The status check will tell you how well your volume is doing with respect to the number of IOPS that have been provisioned.  In the background, we measure the volume’s I/O performance each minute and determine if it is Normal, Degraded, Severely Degraded or Stalled. 

A volume’s performance may fall below normal if you are accessing data on it for the first time, making frequent snapshots at peak usage times, or accessing the volume from an EC2 instance that isn’t EBS-optimized.

You can view the status of your Provisioned IOPS volumes in the AWS Management Console:

You can also retrieve the status by calling the DescribeVolumeStatus function. To learn more about this feature, visit the Monitoring the Status of Your Volumes section of the EC2 documentation.

We are also publishing two new CloudWatch metrics for each volume at one minute intervals:

VolumeThroughputPercentage is the percentage of IOPS delivered out of the IOPS provisioned for an EBS volume.

VolumeConsumedReadWriteOps is the total amount of read and write operations consumed in the period. Provisioned IOPS volumes process your applications reads and writes in I/O block sizes of 16KB or less. Every increase in I/O size above 16KB will linearly increase the resources you need to achieve the same IOPS rate.

You can view these metrics in the AWS Management Console and you can access them through the CloudWatch APIs.

— Jeff;

 

Windows Server 2012 Now Available on AWS

Today we have an awesome guest post from a General Manager on the Amazon EC2 team. Enjoy!

— Jeff;


Before talking about Windows Server 2012, let me first introduce myself. My name is Tom Rizzo and Im the General Manager for the Windows team in AWS. I previously worked at Microsoft in a variety of business groups including Exchange Server, SQL Server and SharePoint Server. Im excited to be at Amazon helping AWS be the best place for customers to run Windows and Windows workloads.

Were trying something new on the AWS blog by introducing a series we call Windows Wednesdays to help you learn more about the work were doing to support Windows on AWS. While well try to post at least twice a month about some new feature, tidbit or something you may not know about Windows on AWS, I wont promise that we wont miss a Wednesday or two but well do our best!

With that said, we couldn’t wait for Wednesday for this announcement! AWS is excited to add Windows Server 2012 as an option for customers in addition to our existing Windows Server 2003 R2, 2008 and 2008 R2 offerings.  Windows Server 2012 provides a number of exciting enhancements including a Server Manager for multi-server setup, Powershell Improvements, Internet Information Services 8.0 and the .NET Framework 4.5.

What’s New in AWS for Windows Server?
Beyond whats new from Microsoft in terms of Windows Server 2012 functionality, here are some highlights specific to Windows Server 2012 on AWS.

Free Tier
First, you can use Windows Server, including Windows Server 2012, as part of the AWS Free Usage Tier. By using it, you get 750 hours of a Micro Instance per month for a full year!  Its a great way to start evaluating Windows Server 2012 and when youre ready to use it in production, just stop your instance and restart it on a bigger instance.  No migration and no messy moving around on different server hardware.  AWS makes it easy for you to grow the power of your instances with a few clicks of the mouse.

New Languages
With the global expansion of AWS, we want to make sure we meet the needs of our worldwide customers.  To that end, today were publishing 31 Windows Server 2012 AMIs which includes AMIs for 19 different languages and for Windows with SQL Server 2008 and 2008 R2 (Express, Web and Standard). The AMIs are available in every Region and can be used with any instance type.

AWS Elastic Beanstalk Support for Windows Server 2012
AWS Elastic Beanstalk allows you to focus on building your application, without having to worry about the provisioning and maintenance aspects of your applications. Elastic Beanstalk already supports Java, PHP, Python, Ruby, and Windows Server 2008 R2 based .NET applications. Starting today, it will also support Windows Server 2012 based .NET applications. You can conveniently deploy your applications from Visual Studio or the AWS Management Console.

.NET 4.5 and Visual Studio 2012 Support
To support building applications on Windows with .NET, AWS supports the new .NET framework shipped with Windows Server 2012 – .NET 4.5. With integrated Visual Studio support, including the new Visual Studio 2012, getting started with building AWS applications is as easy as creating an AWS project in Visual Studio as shown below..

Included with the Visual Studio tools is the AWS Explorer which allows you to see all of your AWS resources without leaving the Visual Studio environment.  In addition, you can deploy to AWS with just a few clicks and can decide whether you want to deploy to EC2 instances or use Elastic Beanstalk as the target for your applications.

Getting Started
To help you get started with Windows Server 2012, we put together a quick introduction video that steps you through the process of creating a Windows Server 2012 instance, expanding the size of the root volume, adding an EBS volume, and connecting to the instance.

If you are attending AWS re:Invent next week, please feel free to drop by the EC2 booth to learn more. We’ll also be presenting several sessions on Windows.

— Tom

 

AWS in Action – Behind the Scenes of a Presidential Campaign

Now that the 2012 US Presidential Election is over, I’d like to take you behind the scenes and tell you a bit about the technology that powered President Obama’s successful reelection campaign!

To set the stage, imagine setting up the technology infrastructure needed to power a dynamic, billion-dollar organization under strict time limits using volunteer labor, with traffic peaking for just one day, and then shutting everything down shortly thereafter.

The words “mission critical” definitely apply here. With the opportunity to lead the United States as the prize, the stakes were high.

Cloud Computing for the Win
The campaign’s technology team built, deployed, ran, and scaled up their applications on AWS. After the election, they backed it all up to Amazon S3 and scaled way, way down.

The campaign used AWS to avoid an IT investment that would have run into the tens of millions of dollars. Along the way they built and ran more than 200 applications on AWS, scaled to support millions of users. One of these apps, the campaign call tool, supported 7,000 concurrent users and placed over two million calls on the last four days of the campaign. Here’s a graph which depicts the growth in call volume in the days leading up to the election:

Data is Key
A database running on Amazon RDS, served as the primary registry of voter file information. This database integrated data from a number of sources including www.barackobama.com and donor information from the finance team) in order to give the campaign managers a dynamic, fully-integrated view of what was going on. Alongside this database, an analytics system running on EC2 Cluster Compute instances (cc2.8xl).  Another database cluster ran Leveldb on a trio of High-Memory Quadruple Extra Large instances.

This array of databases allowed campaign workers to target and segment prospective voters, shift marketing resources based on near real-time feedback on the effectiveness of certain ads, drive a donation system that collected over one billion dollars (the 30th largest ecommerce site in the world).

Complex Applications
The applications run by the Obama campaign are comparable in scope and complexity to those seen in the largest enterprises and data-rich startups. For example:

  • Massive data modeling using Vertica and Elastic MapReduce.
  • Multi-channel media management via TV, print, web, mobile, radio and email using dynamic production, targeting, retargeting, and multi-variant testing like you’d find in a sophisticated digital media agency.
  • Social coordination and collaboration of volunteers, donors, and supporters.
  • Massive transaction processing.
  • Voter abuse prevention and protection, including capture of incoming incidents and dispatch of volunteers.
  • A rich information delivery system for campaign news, polls, information on the issues, voter registration, and more.

The applications made use of virtually every AWS service including EC2, Route 53, SQS, DynamoDB, SES, RDS, VPC, EBS Provisioned IOPS, SNS, ElastiCache, Elastic Load Balancing, Auto Scaling, and CloudFront. They also took advantage of our team of Solution Architects and AWS Premium Support.

Learn More at re:Invent
We’ve assembled an impressive “Big Data and the US Presidential Campaign” panel for AWS re:Invent.  You’ll get to hear from the people who built and ran the applications described above:


Picture courtesy of @Trammel.

I am looking forward to learning even more about their use of technology to help win the election.

— Jeff;

Spotlight on Cost: TellApart

Part of our modus operandi at AWS is to help customers lower the costs of operating their infrastructure. When we are able to achieve better efficiencies in our own datacentres, we pass those savings back on to our customers. That can come in the form of price reductions, or in a variety of payment models.

With Amazon EC2, for example, you can choose to request compute resources as and when you need them, and pay a fixed on-demand price per hour. For more predictable workflows, you can pay a small up-front price for reserved capacity. This allows us to plan our own operations more efficiently, and so we’re able to pass those savings back to you in the form of a reduced hourly price. However, even with the combined reserved capacity and on-demand requests, there can still be unused capacity available in the AWS datacentres.

Rather than let this capacity go to waste, we offer it to customers at a variable price, on the EC2 Spot Market. You can name your own price for this under utilized capacity by placing bids, and achieve significant savings compared to on-demand or reserved capacity. Spot instances allow you to run time insensitive tasks at a lower cost, or to overclock your infrastructure, but adding more bang per unit cost.

To help illustrate how customers are using spot instances, I wanted to introduce you to some customers who have seen some steep cost reductions by integrated spot into their workflows. Today’s willing volunteer is ecommerce data legend, TellApart.

TellApart reduced costs by 75%

“At TellApart, our goal is to build a big data platform that enables retailers to unlock the power of their customer data”, said Julie Black, Director of Engineering at TellApart. “Every day were analyzing massive amounts of shopping data from hundreds of millions of users and building complex machine learning algorithms that empower retailers to identify their highest quality customers and deliver them perfectly targeted marketing.”

So how do they use spot instances?

“At the core of our system is our data processing pipeline. It serves as the backbone of our machine learning system and produces thousands of business metrics consumed daily by our clients”, continues Julie, “To make this all work, we use Amazon Elastic MapReduce to bring up a Hadoop cluster that can batch process log data generated by our distributed fleet of machines.”

“We use Spot Instances for all Hadoop Task Nodes for the flexibility and cost effectiveness we require. Through simple bidding optimizations on the Spot market, we have been able to cut our data processing costs more than 75%”.

Using spot instances with Hadoop and Amazon Elastic MapReduce is very straight forward: you can specify your bid price right in the AWS console when spinning up new clusters in just a few clicks. Easy money.

The Amazon EC2 Spotathon, 2012

If you’d like to shine a spotlight on what you do (or are planning to do) with the spot market, you’ll be interested in the very first Spotathon. There’s a grand prize of $2500 of AWS credit available to the winner who shows the most innovative, low cost use of spot instances in their application architecture.

That’s around 43,000 compute hours of the new m3.xlarge instances at the current spot price.

You can register for the Spotathon today. Looking forward to seeing how you’re using spot.

~ Matt

New Asia Pacific (Sydney) Region in Australia – EC2, DynamoDB, S3, and Much More

It is time to expand the AWS footprint once again, with a new Region in Sydney, Australia. AWS customers in Australia can now enjoy fast, low-latency access to the suite of AWS infrastructure services.

New Region
The new Sydney Region supports the following AWS services:

We also have an edge location for Route 53 and CloudFront in Sydney.

This is our ninth Region; see the AWS Global Infrastructure Map for more information. You can see the full list in the Region menu of the AWS Management Console:

Customers
Over 10,000 organizations in Australia and New Zealand are already making use of AWS. Here’s a very small sample:

  • The Commonwealth Bank of Australia runs customer-facing web applications on AWS as part of a cloud strategy that has been underway for the past five years. The seamless scaling enabled by AWS has allowed their IT department to focus on innovation.
  • Brandscreen, a fast-growing Australian start-up,has developed a real-time advertising trading platform for the media industry.They use Elastic MapReduce to process vast amounts of data to test out machine learning algorithms. They store well over 1 PB of data in Amazon S3 and add another 10 TB every day.
  • MYOB uses AWS to host the MYOB Atlas, a simple website builder that enables businesses to be online within 15 minutes.  They currently have more than 40,000 small and medium-sized businesses using Atlas on the AWS cloud.
  • Halfbrick Studios hosts the highly acclaimed Fruit Ninja game on AWS. They use DynamoDB and multiple Availability Zones to host tens of millions of regular players.

AWS Partner Network
A number members of the AWS Partner Network have been preparing for the launch of the new Region. Here’s a sampling (send me email with launch day updates):

  • Canonical is working to bring the official Ubuntu AMIs to our new Region. The latest supported images for Ubuntu Server 10.04 LTS, 11.10, 12.04 LTS and 12.10 have been migrated over. Daily images have been turned on for the new region. The Amazon Quickstart list is also populated with the proper image ID’s.
  • The enStratus cloud management platform is available in the new Region.
  • RightScale‘s cloud management platform (see my interview with RightScale CEO Michael Crandell to learn more) is available in the new Region.
  • Acquia provides hosted Drupal (again, see my interview with Acquia’s Tom Erickson to learn more) to over 2,400 customers. They are working to ensure that their service will be available to customers in the new Region.
  • ESRI is the leading provider of Geographic Information Systems, with over one million users in more than 350,000 organizations. They are making their ArcGIS platform available in the new Region.
  • The CloudBerry Labs S3 Explorer supports the new region.
  • MetaCDN provides global cloud-based content delivery, video encoding and streaming services. They are working to ensure that their video encoding, persistent storage and delivery services will be available to customers in the new Region.
  • Bulletproof launched Bulletproof Managed AWS in October, removing a significant barrier to AWS entry for enterprise and government customers that require enterprise management. Bulletproof’s Managed AWS includes 24/7 proactive incident response, AWS Enterprise Support and 24/7 application and database management.

We already have a vibrant partner ecosystem in the region. Local Systems Integrators include ASG, Bulletproof Networks, Fronde, Industrie IT, The Frame Group, Melbourne IT, SMS IT and Sourced Group.

On the Ground
In order to serve enterprises, government agencies, academic institutions, small-to-mid size companies, startups, and developers, we now have offices in Sydney, Melbourne, and Perth. We will be adding a local technical support operation in 2013 as part of our global network of support centers, all accessible through AWS Support.

Listen to Andy
AWS Senior Vice President Andy Jassy will be speaking at our Customer Appreciation Day (November 13, 2012). You can register for and attend the live event if you are in Sydney, or you can watch the live stream from anywhere in the world.

— Jeff;

The Amazon EC2 Spotathon


Stephen Elliott Talks About EC2 Spot Instances

If you have used EC2 Spot Instances in a unique way, you need to enter the Amazon EC2 Spotathon. This is an open-ended coding challenge designed to allow you to show off what you have done (and are doing) with Spot Instances.

We’re looking for entrants from a wide variety of industries including (but most definitely not limited to) life sciences, financial services, media, consumer apps, business analytics, gaming, and web analytics.

We want to hear about what you application does, how it incorporates Spot Instances, how much money you have saved, the performance benefits you’ve measured, and the computational scale that you’ve been able to achieve.

The Grand Prize is $2,500 in AWS Credit (which you can use to run even more Spot Instances) and the Runner-Up Prize is another $1,000 in AWS Credit.

I’m really looking forward to seeing what you come up with. The submission deadline is November 16, 2012 so enter today!

— Jeff;

Raising the Roof – More Provisioned IOPS for EBS

When we launched Provisioned IOPS for EBS earlier this year (blog post, EBS home page) we promised even more IOPS (Input/Output Operations Per second) over time.

Time moves fast on the AWS team, and I’m happy to announce that you can now provision up to 2,000 IOPS per EBS volume (double the previous limit of 1000). This is an order of magnitude more IOPS than you can expect from a high-end 15,000 RPM disk drive. If you’ve been RAIDing multiple volumes together for even higher performance, you may be able to achieve the desired level of performance with far fewer volumes.

To recap, Provisioned IOPS volumes allows you to set the level of throughput that you need.  EBS will consistently deliver the desired performance over the lifetime of the volume.  For even higher performance, you can stripe multiple Provisioned IOPS volumes together, giving you the ability to deliver thousands of IOPS per logical volume to your EC2-powered application. These volumes deliver consistent performance and are well-suited to database storage, transaction processing, and other heavy random I/O loads.  For maximum performance and to fully utilize the IOPS provisioned on an EBS volume, you can request the launch of EBS Optimized EC2 instances. An EBS-Optimized instance is provisioned with dedicated throughput to EBS. The m1.large, m1.xlarge, and m2.4xlarge instance types are currently available as EBS-Optimized instances. m1.large instances can transfer data to and from EBS at a rate of 500 Mbit/second; m1.xlarge and m2.4xlarge instances can transfer data at a rate of 1000 Mbit/second. This is additional throughput, and doesn’t affect other general purpose network throughput already available on the instance.

Our friends over at SAP have been using EBS Provisioned IOPS to host SAP HANA (a realtime data platform) on AWS. I was recently chatting with some of our colleagues there and they told me that they have used Provisioned IOPS to achieve over 20,000 IOPS in some of their test scenarios. They also enjoy the predictable performance that comes with the use of Provisioned IOPS.

The team at Parse moved their MongoDB database over to Provisioned IOPS EBS volumes last month. As you can see from their blog post, end-to-end latency was halved and they no longer see periodic latency spikes due to MongoDB write locks. Kevin Lacker, CTO of Parse, told us:

Amazon’s Provisioned IOPS helped us improve our MongoDB query speed by over 60% and drastically reduced latency spikes to our clusters.  We are thrilled to welcome the new 2000 P-IOPS volumes.

Weve heard from 10gen, the developer of MongoDB, about their usage of EBS Provisioned IOPS. 10gen has many customers running MongoDB workloads of up to 20,000 IOPS on a single instance.  For these workloads, 10gen has told us that the high performance and consistency of EBS Provisioned IOPS are a great fit.

Update: Charity Majors of Parse wrote a really nice post, MongoDB with Amazons EBS and Provisioned IOPS. She discusses their move from RAID 10 to a new striped configuration using Provisioned IOPS volumes, which simplified their architecture while also improving performance. The change reduced database latency and greatly improved their cold-start time.

If you are still learning about Provisioned IOPS, be sure to watch this quick (and yet highly informative) video (but replace Arun’s “1000” with “2000”):

We also have a recorded webinar that will help you to learn more about running high performance databases on EC2 using EBS:

Finally, be sure to check out our EBS Provisioned IOPS Benchmarking Guide to learn more about the best tools and techniques for measuring and evaluating the performance of your EBS Provisioned IOPS volumes.

— Jeff;