Category: Amazon VPC

VPC So Far
The Amazon Virtual Private Cloud (VPC) lets you provision a logically isolated section of the AWS Cloud where you can launch AWS resources in a virtual network that you define. Earlier this year, in my post Virtual Private Clouds for Everyone, I outlined some important changes to Amazon EC2 which combine its ease of use with the advanced networking features of Amazon VPC.

As part of that launch, we introduced the concept of the default subnet for default VPCs. EC2 instances created in the default subnet automatically received public IP addresses; instances in the other subnets did not. This was a very sensible default behavior because it made the new VPC feature more or less transparent.

More Control
Today we are launching a new feature that gives you additional control of public IP addresses in VPC at launch time. Here’s the scoop:

• When you launch an instance into a default subnet, you now have the ability to decide if the instance is given a public IP address. Until now, launches into a default subnet were always assigned a public IP address and there was no way to remove it.
• When you launch an instance into a nondefault subnet, you can now choose to assign a public IP address as part of the launch. In the past you had to launch the instance and then allocate and attach an Elastic IP address after the instance became available.

The AWS Management Console includes complete support for this new feature. If you are launching into a default VPC and you don’t select a subnet, or if you are making a Spot Instance request, you can check or uncheck a single check box in the launch wizard to select the desired outcome:

If you choose a specific subnet for your launch, you will see a similar checkbox in the ENI (Elastic Network Interface) section of the launch wizard:

The initial setting of the checkbox (checked or unchecked) reflects the behavior introduced when we launched VPCs for everyone. In other words, if you want the existing behavior, leave the checkbox the way you found it.

Some Notes
I should note that this feature is available only at launch time. If you do not assign a public IP address to an instance during the launch, you can associate an Elastic IP address after the instance has been launched.

Also, if, for some reason, you decide to attach an Elastic IP address to an instance that already has a public IP address, the Elastic IP address will replace the public IP address. When you detach the Elastic IP address from the instance, a new public IP address will be assigned to the instance.

We expect to add CloudFormation and Auto Scaling support before too long.

To learn more about this new feature, read the Using Instance Addressing topic in the EC2 documentation.

— Jeff;

AWS Elastic Beanstalk for .NET now supports the Amazon Virtual Private Cloud (VPC), seamlessly integrates with the Amazon Relational Database Service (RDS), and can be customized using configuration files.

Elastic Beanstalk allows you to easily deploy and manage .NET applications on AWS. Because Elastic Beanstalk leverages Windows Server 2008 R2 and Windows Server 2012, you can run .NET applications with minimal changes.

VPC Integration
With Amazon VPC, you can set up your own virtual network and you can configure Elastic Beanstalk to run your .NET applications inside of this logically isolated section of the AWS cloud. For example, you can create a private subnet where you host your Elastic Beanstalk backend services and then expose the public-facing web application in a public subnet. Visit the AWS Elastic Beanstalk Developer Guide to learn more about Using AWS Elastic Beanstalk with Amazon VPC.

RDS Integration
If your application relies on a relational database, you can easily configure an Amazon RDS DB Instance for your Elastic Beanstalk .NET application. Using the AWS Toolkit for Visual Studio or the AWS Management Console, you can add an RDS DB Instance with just a few clicks. The connection information is automatically exposed to your application through a connection string. Visit Using Amazon RDS in the AWS Elastic Beanstalk Developer Guide.

Configuration Files
Elastic Beanstalk configuration files are YAML text files that allow you to customize your environment in two ways:

1. You can customize the software running inside your environment by downloading files, running commands, installing agents and packages, and setting environment variables.
2. You can provision and configure additional resources such as DynamoDB tables, SQS queues, and CloudWatch alarms.

For example, if your application requires write permissions to app_data, you can grant it these permissions using the following configuration file:

Visit Customizing and Configuring AWS Elastic Beanstalk Environments in the AWS Elastic Beanstalk Developer Guide to learn more about configuration files and for additional examples.

— Jeff + Saad;

If you use or plan to use Amazon EC2, you need to read this post!

History Lesson
Way back in the beginning (OK, 2006) we launched Amazon EC2. When you launched an instance we gave it a public IP address and a DNS hostname. You had the ability to create a Security Group for ingress filtering and attach it to the instance at launch time. Each instance could have just one public IP address, which could be an Elastic IP if desired. Later, we added private IP addresses and an internal DNS hostname to each instance. Let’s call this platform “EC2-Classic” (that will be on the quiz, so remember it).

In 2009 we introduced the Amazon Virtual Private Cloud, better known as the VPC. The VPC lets you create a virtual network of logically isolated EC2 instances and an optional VPN connection to your own datacenter.

In 2011 we announced a big upgrade to EC2’s networking features, with enhanced Security Groups (ingress and egress filtering and the ability to change membership on running instances), direct Internet connectivity, routing tables, and network ACLs to control the flow of traffic between subnets.

We also added lots of other features to VPC in the past couple of years including multiple IP addresses, multiple network interfaces, dedicated instances, and statically routed VPN connections.

How to Make it Easier to Get the Power of VPC and the Simplicity of EC2
We want every EC2 user to be able to benefit from the advanced networking and other features of Amazon VPC that I outlined above. To enable this, starting soon, instances for new AWS customers (and existing customers launching in new Regions) will be launched into the “EC2-VPC” platform. We are currently in the process of enabling this feature, one Region at a time, starting with the Asia Pacific (Sydney) and South America (So Paulo) Regions. We expect these roll-outs to occur over the next several weeks. We will update this post in the EC2 forum each time we enable a Region.

You dont need to create a VPC beforehand – simply launch EC2 instances or provision Elastic Load Balancers, RDS databases, or ElastiCache clusters like you would in EC2-Classic and well create a VPC for you at no extra charge.  Well launch your resources into that VPC and assign each EC2 instance a public IP address. You can then start taking advantage of the features I mentioned earlier: assigning multiple IP addresses to an instance, changing security group membership on the fly, and adding egress filters to your security groups. These VPC features will be ready for you to use, but you need not do anything new and different until you decide to do so.

We refer to the automatically provisioned VPCs as default VPCs.  They are designed to be compatible with your existing shell scripts, CloudFormation templates, AWS Elastic Beanstalk applications, and Auto Scaling configurations. You shouldnt need to modify your code because youre launching into a default VPC.

Default VPCs for (Almost) Everyone
The default VPC features are available to new AWS customers and existing customers launching instances in a Region for the first time. If youve previously launched an EC2 instance in a Region or provisioned ELB, RDS, or ElastiCache in a Region, we wont create a default VPC for you in that Region.

If you are an existing AWS customer and you want to start gaining experience with this new behavior, you have two options. You can create a new AWS account or you can pick a Region that you haven’t used (as defined above). You can see the set of available platforms in the AWS Management Console (this information is also available through the EC2 APIs and from the command line). Be sure to check the Supported Platforms and Default VPC values for your account to see how your account is configured in a specific Region.

You can determine if your account is configured for default VPC within a particular Region by glancing at the top right corner of the EC2 Dashboard in the AWS Management Console. Look for the Supported Platforms item.  EC2-VPC means your instances will be launched into Amazon VPC.

Here is what you will see if your AWS account is configured for EC2 Classic and EC2-VPC (without a default VPC):

You can also see the supported platforms and the default VPC values using the EC2 API and the Command Line tools.

All Set Up
As I noted earlier in this post, we’ll create a default VPC for you when you perform certain actions in a Region. It will have the following components:

• One default subnet per Availability Zone.
• A default route table, preconfigured to send traffic from the default subnets to the Internet.
• An Internet Gateway to allow traffic to flow to and from the Internet.

Each VPC will have its own private IP range (172.31.0.0/16 to be precise); each subnet will be a “/20” (4096 IP addresses, minus a few that are reserved for the VPC).

EC2 instances created in the default VPC will also receive a public IP address (this turns out to be a very sensible default given the preconfigured default route table and Internet Gateway). This is a change from the existing VPC behavior, and is specified by a new PublicIP attribute on each subnet. We made this change so that we can support the EC2-Classic behavior in EC2-VPC. The PublicIP attribute can’t currently be set for existing subnets but we’ll consider allowing this in the future (let us know if you think that you would find this to be of use).

You can modify your default VPC as you see fit (e.g., creating or deleting subnets, creating or modifying route tables, adding VPN connections, etc.)  You can also create additional, nondefault VPCs just like the VPCs you can create today.

Once you are up and running in a VPC within an AWS Region, you’ll have access to all of the AWS services and instance types that are available in that Region (see the List of AWS Offerings by Region page for more information). This includes new and important services such as Amazon Redshift, AWS OpsWorks, and AWS Elastic Beanstalk.

New VPC Features
We’re also adding new features to VPC. These are available to all VPC users:

DNS Hostnames – All instances launched in a default VPC will have private and public DNS hostnames. DNS hostnames are disabled for existing VPCs, but can be enabled as needed.  If youre resolving a public hostname for another instance in the same VPC, it will resolve to the private IP of the target instance.  If youre resolving a public hostname for an instance outside of your VPC, it will resolve to the public IP address of that instance.

DNS Name Resolution – DNS resolution is enabled in all VPCs but weve added the ability for you to disable use of the Amazon provided DNS service in your VPC as needed.

ElastiCache – You can now create ElastiCache cache nodes within VPC (both default and nondefault VPCs).

RDS IP Addresses – RDS database instances in VPC can now be provisioned as Internet-facing or private-facing instances. Internet-facing RDS database instances have public IP addresses so that they can be accessed from EC2 Classic instances or on-premises systems. For more information on this feature, read the documentation on Amazon RDS and the Amazon Virtual Private Cloud Service.

Learning About VPC
To learn more about Amazon VPC, please consult the following resources:

• VPC Home Page.
• VPC Documentation.
• Detecting Your Supported Platforms.
• Differences Between Amazon EC2-Classic and EC2-VPC.

Were happy to give you the advanced features of Amazon VPC coupled with the simplicity of Amazon EC2. If you have any questions, the AWS Support Team is ready, willing, and able to help.

— Jeff;

It is time to expand the AWS footprint once again, with a new Region in Sydney, Australia. AWS customers in Australia can now enjoy fast, low-latency access to the suite of AWS infrastructure services.

New Region
The new Sydney Region supports the following AWS services:

• 
  A Tranquil Beach – Shoal Bay, Australia

  Amazon Elastic Compute Cloud (EC2) and related services including Elastic Block Storage, Virtual Private Cloud, VM Import, VM Export, Auto Scaling, and Elastic Load Balancing.

• Amazon Elastic Map Reduce (EMR).
• Amazon DynamoDB.
• Amazon SimpleDB.
• Amazon Relational Database Service (RDS).
• AWS CloudFormation.
• Amazon Simple Queue Service (SQS).
• Amazon Simple Notification Service (SNS).
• Amazon CloudWatch.
• Amazon Simple Storage Service (S3).
• AWS Storage Gateway.
• AWS Elastic Beanstalk.
• AWS CloudFront.
• Amazon Route 53.
• AWS Direct Connect.

We also have an edge location for Route 53 and CloudFront in Sydney.

This is our ninth Region; see the AWS Global Infrastructure Map for more information. You can see the full list in the Region menu of the AWS Management Console:

Customers
Over 10,000 organizations in Australia and New Zealand are already making use of AWS. Here’s a very small sample:

• The Commonwealth Bank of Australia runs customer-facing web applications on AWS as part of a cloud strategy that has been underway for the past five years. The seamless scaling enabled by AWS has allowed their IT department to focus on innovation.
• Brandscreen, a fast-growing Australian start-up,has developed a real-time advertising trading platform for the media industry.They use Elastic MapReduce to process vast amounts of data to test out machine learning algorithms. They store well over 1 PB of data in Amazon S3 and add another 10 TB every day.
• MYOB uses AWS to host the MYOB Atlas, a simple website builder that enables businesses to be online within 15 minutes.  They currently have more than 40,000 small and medium-sized businesses using Atlas on the AWS cloud.
• Halfbrick Studios hosts the highly acclaimed Fruit Ninja game on AWS. They use DynamoDB and multiple Availability Zones to host tens of millions of regular players.

AWS Partner Network
A number members of the AWS Partner Network have been preparing for the launch of the new Region. Here’s a sampling (send me email with launch day updates):

• Canonical is working to bring the official Ubuntu AMIs to our new Region. The latest supported images for Ubuntu Server 10.04 LTS, 11.10, 12.04 LTS and 12.10 have been migrated over. Daily images have been turned on for the new region. The Amazon Quickstart list is also populated with the proper image ID’s.
• The enStratus cloud management platform is available in the new Region.
• RightScale‘s cloud management platform (see my interview with RightScale CEO Michael Crandell to learn more) is available in the new Region.
• Acquia provides hosted Drupal (again, see my interview with Acquia’s Tom Erickson to learn more) to over 2,400 customers. They are working to ensure that their service will be available to customers in the new Region.
• ESRI is the leading provider of Geographic Information Systems, with over one million users in more than 350,000 organizations. They are making their ArcGIS platform available in the new Region.
• The CloudBerry Labs S3 Explorer supports the new region.
• MetaCDN provides global cloud-based content delivery, video encoding and streaming services. They are working to ensure that their video encoding, persistent storage and delivery services will be available to customers in the new Region.
• Bulletproof launched Bulletproof Managed AWS in October, removing a significant barrier to AWS entry for enterprise and government customers that require enterprise management. Bulletproof’s Managed AWS includes 24/7 proactive incident response, AWS Enterprise Support and 24/7 application and database management.

We already have a vibrant partner ecosystem in the region. Local Systems Integrators include ASG, Bulletproof Networks, Fronde, Industrie IT, The Frame Group, Melbourne IT, SMS IT and Sourced Group.

On the Ground
In order to serve enterprises, government agencies, academic institutions, small-to-mid size companies, startups, and developers, we now have offices in Sydney, Melbourne, and Perth. We will be adding a local technical support operation in 2013 as part of our global network of support centers, all accessible through AWS Support.

Listen to Andy
AWS Senior Vice President Andy Jassy will be speaking at our Customer Appreciation Day (November 13, 2012). You can register for and attend the live event if you are in Sydney, or you can watch the live stream from anywhere in the world.

— Jeff;

Judging from the number of requests that I have had for this particular combination of EC2 features, I’m expecting this to be a very popular post.

You can now launch EC2 micro (t1.micro) instances within a Virtual Private Cloud (VPC). The AWS Free Usage Tier now extends to t1.micro instances running inside of a VPC.

The micro instances provide a small amount of consistent CPU power, along with the ability to increase it in short burst when additional cycles are available. They are a good match for lower throughput applications and web sites that require additional compute cycles from time to time.

With this release, you now have everything that you need to create and experiment with your very own Virtual Private Cloud at no cost. This is pretty cool and I’m sure you’ll make good use of it.

— Jeff;

The Amazon Virtual Private Cloud (VPC) gives you the power to create a private, isolated section of the AWS Cloud. You have full control of network addressing. Each of your VPCs can include subnets (with access control lists), route tables, and gateways to your existing network and to the Internet.

You can connect your VPC to the Internet via an Internet Gateway and enjoy all the flexibility of Amazon EC2 with the added benefits of Amazon VPC.  You can also setup an IPsec VPN connection to your VPC, extending your corporate data center into the AWS Cloud.  Today we are adding two options to give you additional VPN connection flexibility:

1. You can now create Hardware VPN connections to your VPC using static routing. This means that you can establish connectivity using VPN devices that do not support BGP such as Cisco ASA and Microsoft Windows Server 2008 R2. You can also use  Linux to establish a Hardware VPN connection to your VPC. In fact, any IPSec VPN implementation should work.
2. You can now configure automatic propagation of routes from your VPN and Direct Connect links (gateways) to your VPC’s routing tables. This will make your life easier as you wont need to create static route entries in your VPC route table for your VPN connections.  For instance, if youre using dynamically routed (BGP) VPN connections, your BGP route advertisements from your home network can be automatically propagated into your VPC routing table.

If your VPN hardware is capable of supporting BGP, this is still the preferred way to go as BGP performs a robust liveness check on the IPSec tunnel. Each VPN connection uses two tunnels for redundancy; BGP simplifies the failover procedure that is invoked when one VPN tunnel goes down.

Static Routing
We added the static routing option for a number of reasons. First, BGP can be difficult to set up and to manage, and we don’t want to ask you to go to all of that trouble if all you want to do is set up a VPN connection to a VPC. Second, some firewalls and entry-level routers support IPSec but not BGP. These devices are very popular in corporate branch offices. As I mentioned above, this change dramatically increases the number of VPN devices that can be used to connect to a VPC. We have tested the static routing “No BGP” option with devices from Cisco, Juniper, Yamaha, Netgear, and Microsoft. We’ve assembled a list of VPN devices that weve tested for dynamic and statically routed VPN connections

You can select this option when you create the VPN connection between a VPN and one of your customer gateways:

If you choose this option you must also enter one or more routes (CIDR addresses) to indicate which traffic is to be routed back to your customer gateways (your home network).

For client-side redundancy, you can use two customer gateway devices (two VPN connections).  That way, if your gateway device goes down, or needs maintenance, the other one can continue to carry your traffic into the VPC. On the AWS side, we have multiple redundant VPN concentrators to handle failover in case of device failure.

Route Propagation
You can automatically propagate your VPN Connection routes (whether statically entered or advertised via BGP) to your VPC route table:

In order to enable this option for a particular routing table, you must establish an association between the table and a gateway like this:

You can also arrange to update multiple routing tables from the same virtual private gateway.

As you can see, you can access these new VPN features from the AWS Management Console. They are also accessible through the VPC APIs and the command line tools.

— Jeff;

As you can tell from some of my recent posts, we’re adding new features to the Virtual Private Cloud and to the Relational Database Service at a very rapid clip. Today’s release involves both of these services!

You can now use launch RDS database instances running Microsoft SQL Server inside of a Virtual Private Cloud (VPC). You can now get all of the benefits of the Relational Database Service including backup management, automatic failure detection and recovery, software patching (both OS and database), and easy scaling, all within a private, isolated section of the AWS Cloud.

— Jeff;

The Amazon RDS team is rolling out new features at a very rapid clip. Here’s the latest and greatest:

Oracle Data Pump
Customers have asked us to make it easier to import their existing databases into Amazon RDS. We are making it easy for you to move data on and off of the DB Instances by using Oracle Data Pump. A number of scenarios are supported including:

• Transfer between an on-premises Oracle database and an RDS DB Instance.
• Transfer between an Oracle database running on an EC2 instance and an RDS DB Instance.
• Transfer between two RDS DB Instances.

These transfers can be run in either direction. We currently support the network mode of Data Pump where the job source is an Oracle database. Transfers using Data Pump should be considerably faster than those using the original Import and Export utilities. Oracle Data Pump is available on all new DB Instances running Oracle Database 11.2.0.2.v5. To use Data Pump with your existing v3 and v4 instances, please upgrade to v5 by following the directions in the RDS User Guide. To learn more about importing and exporting data from your Oracle databases, check out our new import/export guide.

 — Jeff;

Amazon EC2 instances within a Virtual Private Cloud (VPC) can now have multiple IP addresses. This oft-requested feature builds upon several other parts of AWS including Elastic IP Addresses and Elastic Network Interfaces.

Use Cases
Here are some of the things that you can do with multiple IP addresses:

• Host multiple SSL websites on a single instance. You can install multiple SSL certificates on a single instance, each associated with a distinct IP address.
• Build network appliances. Network appliances such as firewalls and load balancers generally work best when they have access to multiple IP addresses on a network interface.
• Move private IP addresses between interfaces or instances. Applications that are bound to specific IP addresses can be moved between instances.

The Details
When we launched the Elastic Network Interface (ENI) feature last December, you were limited to a maximum of two ENI’s per EC2 instance, each with a single IP address. With today’s release we are raising these limits, allowing you to have up to 30 IP addresses per interface and 8 interfaces per instance on the m2.4xl and cc2.8xlarge instances, with proportionally smaller limits for the less powerful instance types. Inspect the limits with care if you plan to use lots of interfaces or IP addresses and expect to switch between different instance sizes from time to time.

When you launch an instance or create an interface, a private IP address is created at the same time. We now refer to this as the “primary private IP address.” Amazingly enough, the other addresses are called “secondary private IP addresses.” Because the IP addresses are assigned to an interface (which is, in turn attached to an EC2 instance), attaching the interface to a new instance will also bring all of the IP addresses (primary and secondary) along for the ride.

You can also allocate Elastic IP addresses and associate them with the primary or secondary IP addresses of an interface. Logically enough, the Elastic IP’s also come along for the ride when the interface is attached to a new instance. You will, of course, need to create an Internet Gateway in order to allow Internet traffic into your VPC.

In addition to moving interfaces to other instances, you can also move secondary private IP addresses between interfaces or instances. The Elastic IP associated to the secondary private IP will move with the private IP to its new home.

As I mentioned when we launched the ENI feature, each ENI has its own MAC Address, Security Groups, and a number of other attributes. With today’s release, these attributes apply to all of the IP addresses associated with the ENI.

In order to make use of multiple interfaces and IP addresses, you will need to configure your operating system accordingly. We are planning to publish additional documentation and some scripts to show you how to do this. Code and scripts running on your instance can consult the EC2 instance metadata to ascertain the current ENI and IP address configuration.

Console Support
The VPC tab of the AWS Management Console includes full support for this feature. You can manage the IP addresses associated with each interface of a running instance:

You can associate IP addresses with network interfaces:

You can set up interfaces and IP addresses when you launch a new instance:

The number of supported IP addresses varies by instance type. Consult the Elastic Network Interfaces documentation to see how many addresses are supported by the instance types that you use. Depending on your use case, you may also need to configure your EC2 network interfaces to avoid asymetric routing.

Pricing
You can use one Elastic IP Address per instance at no charge (as long as it is mapped to an EC2 instance), as has always been the case. We are reducing the price for Elastic IP Addresses not mapped to running instances to $0.005 (half of a penny) per hour in both EC2 and VPC.

Each additional Elastic IP Address on an instance will also cost you $0.005 per hour. We have also changed the billing model for Elastic IP Addresses to prorate usage, so that you’ll be charged for partial hours as appropriate.

There is no charge for private IP addresses.

I hope that you have some ideas for this important new feature, and that you are able to make good use of it.

— Jeff;

Today’s guest post comes to you courtesy of Spencer Dillard, Product Manager for AWS Elastic Load Balancing.

— Jeff;

One of the challenges weve heard about many times from customers is the challenge of load balancing between tiers of an application. While Elastic Load Balancing addresses many of the complexities of building a highly available application, it doesnt help when you need to balance the load between multiple back-end instances. Until now. As of today, you can create an internal load balancer in your VPC and place your non-internet-facing instances behind the internal load balancer. Heres a simple overview:

The internet-facing load balancer has public IP addresses and the usual Elastic Load Balancer DNS name. Your web servers can use private IP addresses and restrict traffic to the requests coming from the internet-facing load balancer. The web servers in turn will make requests to the internal load balancer, using private IP addresses that are resolved from the internal load balancers DNS name, which begins with internal-. The internal load balancer will route requests to the application servers, which are also using private IP addresses and only accept requests from the internal load balancer.

With this change, all of your infrastructure can use private IP addresses and security groups so the only part of your architecture that has public IP addresses is the internet-facing load balancer. Because the DNS record is publicly resolvable, you could also use a VPN connection and address the internal load balancer from your on-premise environment through the VPN tunnel.

Getting started is easy. Using the AWS Console, simply select the checkbox to make your new load balancer an internal load balancer. Everything else stays the same.

As part of this change, weve also relaxed the constraints on the size of the subnet you need to attach the load balancer to. You can now attach a load balancer to your subnets that have a /27 or larger size.

Im looking forward to hearing about the new scenarios this enables for you. Let us know what you think!

— Spencer