Category: Amazon RDS

The Amazon RDS team is rolling out new features at a very rapid clip. Here’s the latest and greatest:

Oracle Data Pump
Customers have asked us to make it easier to import their existing databases into Amazon RDS. We are making it easy for you to move data on and off of the DB Instances by using Oracle Data Pump. A number of scenarios are supported including:

• Transfer between an on-premises Oracle database and an RDS DB Instance.
• Transfer between an Oracle database running on an EC2 instance and an RDS DB Instance.
• Transfer between two RDS DB Instances.

These transfers can be run in either direction. We currently support the network mode of Data Pump where the job source is an Oracle database. Transfers using Data Pump should be considerably faster than those using the original Import and Export utilities. Oracle Data Pump is available on all new DB Instances running Oracle Database 11.2.0.2.v5. To use Data Pump with your existing v3 and v4 instances, please upgrade to v5 by following the directions in the RDS User Guide. To learn more about importing and exporting data from your Oracle databases, check out our new import/export guide.

 — Jeff;

Amazon RDS makes it easy to provision and run production databases, and customers running RDS for SQL Server will be pleased to hear they can now use SQL Server Agent to schedule and execute administrative tasks on their databases.

SQL Server Agent is a tool designed to take some of the manual heavy lifting of tuning and maintaining database services off database administrators’ shoulders. For example, you could schedule regular index builds and data integrity checks as part of your regular maintenance program.

You can also schedule and run T-SQL script jobs using the tool, which is useful for a number of uses:

• Merging or transforming data as part of regular reporting runs
• Database clean ups, purging older records or converting logical deletions to physical ones
• Performance tuning

To get started, simply connect your SQL Server Management Studio to your Amazon RDS for SQL Server database. Click on ‘New Job’, and follow the guided instructions. Easy as that.

SQL Server Agent joins the Database Tuning Advisor as an invaluable tool in any SQL Server DBAs kit. You can learn more about viewing the logs of the SQL Server Agent and managing jobs in the ‘Common DBA tasks’ section of the Amazon RDS documentation.

Scheduling options in SQL Server Agent.

Let us know what jobs you’re schedulding with SQL Server Agent, or jump in and spin up your production stack on Amazon EC2 with provisioned IOPS or AWS Elastic Beanstalk.

We’re hiring!

If you’re passionate about bringing tools such as SQL Server and SQL Server Agent to customers across the world, the Amazon RDS team has open positions for software developers, test engineers and product managers. Get in touch on rds-jobs@amazon.com.

~ Matt

We have added three new features to Amazon RDS for Oracle Database to enable you to use it for more of your use cases:

• You can use Amazon RDS for Oracle in Amazon Virtual Private Cloud (VPC)
• You can use Oracle XML DB to store, retrieve, navigate, and query XML documents. 
• You can use Oracle Application Express (APEX) to rapidly build web applications.

Judging from the number of requests that I have received for these features, I expect them to be very popular.

Amazon RDS for Oracle in Amazon VPC
You can now launch Amazon RDS DB Instances running Oracle Database within a Virtual Private Cloud (VPC). Within a VPC, you can define a virtual network topology and customize the network configuration as desired. You can use subnets, routers, and access control lists to create the same type of network that you would find in a traditional data center. Amazon RDS for Oracle makes it easy to set up, operate, and scale Oracle Database deployments in the cloud. You can not only create Multi-AZ Oracle Database deployments with automated failure detection and recovery but also leverage Oracle Enterprise Manager to manage your databases. In addition, all of the other Amazon RDS for Oracle Database functionality is available including backup management, software patching, and easy scaling of compute and storage resources to meet changing needs.

Oracle APEX
Oracle APEX is a rapid web development tool for Oracle database and enables you to create and deploy rich web applications using a web browser and limited programming experience. Both the Runtime and Development environments are available and you can switch between them using our Option Groups functionality.

To start using APEX, you will need to create an RDS for Oracle DB instance (this is the APEX repository). You can also use an existing one if it is running engine version 11.2.0.2.v4 (or newer) of the RDS for Oracle Database. Apply the XMLDB, APEX, and APEX_DEV options via a RDS option group:

You will have to set the password of the APEX_PUBLIC_USER account and then unlock it.

You will also need to install and run the APEX listener (the HTTP server) on your own host or on an Amazon EC2 instance. As part of the configuration process, you will point the listener at your RDS instance.

Oracle XMLDB
Oracle XML DB is a feature of Oracle database that provides native XML storage and retrieval capabilities. It fully supports the W3C XML data model and provides new standard access methods for navigating and querying XML. With Oracle XML DB you get the advantages of relational database technology and the benefits of XML.

In addition to these new updates, we have also included support for Oracle Time Zone so you can change the time zone of your Oracle database. This enables you to consistently store timestamp information when you have users in multiple time zones by setting the time zone datatype TIMESTAMP WITH LOCAL TIMEZONE.

These features are all available at no additional charge in all Regions where Amazon RDS for Oracle is available.

— Jeff and Jinesh;

The I/O Imperative
As I noted earlier this month, modern web and mobile applications are highly I/O dependent, storing and retrieving lots of data in order to deliver a rich and personalized experience.

In order to give you the I/O performance and the flexibility that you need to build these types of applications, we’ve released several new offerings in the last few months:

• For seamless, managed, scaling of NoSQL workloads, we recently introduced DynamoDB, an SSD-backed NoSQL database with read and write times measured in single-digit milliseconds, with very modest variance from request to request. DynamoDB makes it easy to scale up from 20 to 200,000 or more reads and writes per second and to scale back down again based on your application’s requirements. The response to DynamoDB has been incredible, and it has become (as Werner noted) our fastest-growing service.
• Two weeks ago, we launched the first member of the EC2 High I/O family,  the h1.4xlarge instance type, to support time series analysis, NoSQL databases, and mobile and streaming applications requiring low latency access to storage systems delivering tens of thousands of IOPS.  The h1.4xlarge comes with 2 TB of SSD-backed storage on each instance.

We know that you want more options for your I/O intensive applications, and we’re happy to oblige.

Here You Go

Here’s what we are announcing today:

1. A new type of EBS volume called Provisioned IOPS that gives you the ability to dial in the level of performance that you need (currently up to 1,000 IOPS per volume, with more coming soon) . You can stripe (RAID 0) two or more volumes together in order to reach multiple thousands of IOPS.
2. The ability to launch EBS-Optimized instances which feature dedicated throughput between these instances and EBS volumes.

EBS Provisioned IOPS
We released EBS in the summer of 2008. Since that time, our customers have very successfully used EBS to store the persistent data associated with their EC2 instances. We have found that there are certain workloads that require highly consistent IOPS, and others that require more IOPS on an absolute basis. Relational databases certainly qualify on both counts.

As a point of reference, a standard EBS volume will generally provide about 100 IOPS on average, with the ability to burst to hundreds of IOPS on a best-effort basis. Standard EBS volumes are great for applications with moderate or bursty I/O requirements as well as for boot volumes.

The new Provisioned IOPS EBS volume allows you to set the level of throughput that you need, and currently supports up to 1,000 IOPS (for 16K), with higher limits coming soon. For even higher performance, you can stripe multiple Provisioned IOPS volumes together, giving you the ability to deliver thousands of IOPS per logical volume to your EC2-powered application. These volumes deliver consistent performance and are well-suited to database storage, transaction processing, and other heavy random I/O loads. When attached to EBS-Optimized instances, these volumes are designed to deliver within 10% of their provisioned I/O performance 99.9% of the time.

You can create Provisioned IOPS EBS volumes from the AWS Management Console, the command line tools, or via the EC2 APIs. If you use the console, you need only select the Provisioned IOPS volume type and then enter the desired number of IOPS:

Provisioned IOPS volumes are priced at $0.125 per GB of allocated storage per month plus $0.10 per provisioned IOPS per month in US East (Northern Virginia); see the EBS page for more info. By default, each AWS account can create up to 20 TB of Provisioned IOPS volumes with a total of 10,000 IOPS. If you need more of either (or both), simply fill out this form.

You can create a provisioned equivalent of your existing EBS volume by suspending all I/O to your volume, creating a snapshot, and then creating a Provisioned IOPS volume using the snapshot as a starting point.

EBS-Optimized EC2 Instances
For maximum performance and to fully utilize the IOPS provisioned on an EBS volume, you can now request the launch of EBS Optimized EC2 instances. An EBS-Optimized instance is provisioned with dedicated throughput to EBS. The m1.large, m1.xlarge, and m2.4xlarge instance types are currently available as EBS-Optimized instances. m1.large instances can transfer data to and from EBS at a rate of 500 Mbit/second; m1.xlarge and m2.4xlarge instances can transfer data at a rate of 1000 Mbit/second. This is additional throughput, and doesn’t affect other general purpose network throughput already available on the instance.

There is an additional hourly charge for the EBS-Optimized instances: $0.025 for the m1.large and $0.05 for the m1.xlarge and m2.4xlarge instance types.

You can upgrade your EC2 instances to EBS-Optimized instances as follows:

1. Shut down any applications that are running on the instance.
2. Stop the instance.
3. Modify the instance using the ec2-modify-instance-attribute command) and set the EBS-Optimized flag. Change the instance type to one of the supported instance types if necessary.
4. Start the instances

And Here’s Arun
I spoke with Arun Sundaram, a Product Manager on the AWS Storage team, to learn more about these two features. Here’s what he had to say:

 And That’s That
These new features are available for you to use today. Give them a whirl, and let me know what you think!

— Jeff;

PS – The EBS team is hiring! If you are interested, send your resume to ebs-jobs@amazon.com . Open positions include Software Development Manager, Senior Software Development Engineer,  and Director of Product Management.

Ok folks, this one is a mouthful, so bear with me!

The Amazon Relational Database Service for SQL Server now supports the Microsoft SQL Server Database Engine Tuning Advisor. The Advisor will help you to select and create an optimal set of indexes, indexed views, and partitions even if you don’t have an expert-level understanding of the structure of your database or the internals of SQL Server.

You will need to create a representative workload (a set of Transact-SQL statements) to run against the database or databases that you want to tune. The workload can be stored in a file or in a database table. Either way, the workload must be structured in such a way that the Advisor can execute or replay it multiple times. You can, if you’d like, generate the workload by performing a trace against your DB Instance while it is supporting a real or simulated version of your application.

To learn more about how to use the Tuning Advisor, please visit the Common DBA Tasks for Microsoft SQL Server section of the Amazon RDS documentation.

— Jeff;

I’m always happy to be able to talk about AWS options that lower costs and add flexibility for our customers.

To that end, I am pleased to announce the availability of Micro instances for Amazon RDS for the MySQL database engine. The t1.micro RDS instance is a low cost instance type designed for low traffic web applications, test applications and small projects.

As you may already know, Amazon RDS makes it easy to set up, operate, and scale a relational database in the cloud. Our customers have asked us for a lower priced instance type that could satisfy the needs of their less demanding applications. The micro RDS instance allows you to run a fully-featured relational database, starting at just $19 a month (2.5 cents an hour). Micros RDS instances also support Multi-AZ deployments and Read Replicas.

Micro DB instances provide a small amount of consistent CPU resources, and also allow you to burst CPU capacity when additional cycles are available. The Micro DB instance type is available now in all AWS Regions. See the RDS Pricing page for more information on the On Demand and Reserved Instance pricing.

— Jeff;

Users of Amazon RDS for Oracle Database now have access to Oracle Enterprise Manager 11g Database Control, also known as OEM. Starting today, you can enable OEM and use it to manage your RDS for Oracle DB instances with just a few clicks in the AWS Management Console.

OEM Database Control is pre-installed and available at no additional charge for new and existing Amazon RDS for Oracle DB instances and all supported Oracle Editions: Enterprise Edition, Standard Edition, and Standard Edition One, for License Included and Bring Your Own License Customers.

In conjunction with OEM, we are also launching Option Groups. This feature simplifies DB administration by enabling you to save a set of options and their configurations so you can easily apply them to other DB Instances in the future.

To enable OEM for your RDS for Oracle DB Instance, please use the following steps.

First, select Option Groups from the AWS Management Console Navigation Links.

Next, from the Option Groups Screen, Select Create Group, fill in the fields, and then click Yes, Create.

Select your newly  created option group and then click Add Option. Youll see that OEM is selected by default. Choose your desired port and select a security group for the option:

After youve added OEM to your option group, select your DB Instance and choose Modify:

Select your option group from the drop down and decide if you would like your change to take effect immediately, or during your maintenance window:

You can now access OEM for your database by visiting https://your-db-endpoint:1158/em .

To learn more about OEM and Option Groups, please see our documentation and the Amazon RDS for Oracle detail page. If you are new to Amazon RDS, you can get started for free with our sixty day Free Trial.

— Jeff;

You can now launch Amazon Relational Database Service (RDS) MySQL Read Replicas inside of a Virtual Private Cloud (VPC).

Amazon RDS removes the headaches of running a relational database reliably at scale, allowing Amazon RDS customers to focus on innovation for their customers. Read Replicas enables you to elastically scale out beyond the capacity constraints of a single DB Instance for read-heavy database workloads.

You can now create one or more replicas of a given source DB Instance and serve incoming read traffic from multiple copies of your data within a VPC environment. You can create a Read Replica with a few clicks of the AWS Management Console or using the CreateDBInstanceReadReplica API.

Amazon VPC allows you to customize the network configuration to closely resemble a traditional network that you might operate in your own datacenter. I described the process of launching a DB Instance inside of a VPC in an earlier post.

Amazon RDS in VPC enables you to have a DB instance within a private network powering a public web application. The DB instance is on a private subnet, which does not have a public IP address. You can also use Amazon RDS + VPC to run corporate applications that are not intended to be accessed from the Internet.

— Jeff;

We are continuing to simplify the Windows development experience on AWS, and today we are excited to announce Amazon RDS for SQL Server and .NET support for AWS Elastic Beanstalk. Amazon RDS takes care of the tedious aspects of deploying, scaling, patching, and backing up of a relational database, freeing you from time-consuming database administration tasks. AWS Elastic Beanstalk is an easy way to deploy and manage applications in the AWS cloud and handles the deployment details of capacity provisioning, load balancing, auto scaling, and application health monitoring.

Here are the details…

Amazon RDS for SQL Server
We launched the Amazon Relational Database Service (RDS) in late 2009 with support for MySQL. Since then, we have added a number of features, including Multi-AZ, read replicas, and VPC support. Last year, we added support for Oracle Database.

Today we are extending the manageability benefits of Amazon RDS to SQL Server customers. Amazon RDS now supports Express, Web, Standard, and Enterprise Editions of SQL Server 2008 R2. We plan to add support for SQL Server 2012 later this year.

If you are a new Amazon RDS customer, you can use Amazon RDS for SQL Server (Express Edition) under the free usage tier for a full year. After that, you can use the service under multiple licensing models, with prices starting as low as $0.035/hour. Refer to Amazon RDS for SQL Server pricing for more details.

.NET Support for AWS Elastic Beanstalk
Earlier this year, we added support for PHP applications to Elastic Beanstalk alongside the existing support for Java applications.

Today, we are extending Elastic Beanstalk to our Windows developers who are building .NET applications. Elastic Beanstalk leverages the Windows Server 2008 R2 AMI and IIS 7.5 to run .NET applications. You can run existing applications on AWS with minimal changes. There is no additional charge for Elastic Beanstalkyou pay only for the AWS resources needed to store and run your applications. And if you are eligible for the AWS free usage tier, you can deploy and run your application on Elastic Beanstalk for free.

AWS Toolkit for Visual Studio Enhancements
We are also updating the AWS Toolkit for Visual Studio so you can deploy your existing web application projects to AWS Elastic Beanstalk. You can also use the AWS Toolkit for Visual Studio to create Amazon RDS DB Instances and connect to them directly, so you can focus on building your applications without leaving your development environment.

Lets look at how it all works. For a detailed step-by-step walkthrough, visit the AWS Elastic Beanstalk Developer Guide.

Deploy Your Application to AWS Elastic Beanstalk
To get started, simply install the AWS Toolkit for Visual Studio and make sure you have signed up for an AWS account. You can deploy any Visual Studio Web project to AWS Elastic Beanstalk, including ASP.NET MVC projects and ASP.NET Web Forms. As an example, I will use the NerdDinner MVC sample application.

(click for full-sized image)

To deploy to AWS Elastic Beanstalk, right-click the project, and then click Publish to AWS. Provide the details and complete the wizard. This will launch a new Elastic Beanstalk environment and create the AWS resources to run your application. Thats it; NerdDinner is now running on Elastic Beanstalk.

Create and Connect to an Amazon RDS Database Instance
By default, NerdDinner connects to a local SQL Server Express database, so well need to make a few changes to connect it to an Amazon RDS for SQL Server instance. Lets start by creating a new Amazon RDS for SQL Server instance using the AWS Explorer view inside Visual Studio.

(click for full-sized image)

We will also need to create the schema that NerdDinner expects. To do so, simply use the Publish to Provider wizard in Visual Studio to export the schema and data to a SQL script. You can then run the SQL script against the RDS for SQL Server database to recreate the schema and data.

(click for full-sized image)

Update Your Running Application
Now that the Amazon RDS for SQL Server database is set up, lets modify the applications connection string to use it. To do so, you simply modify the ConnectionString.config file in your NerdDinner project and provide the connection details of your RDS instance.

(click for full-sized image)

Finally, you will republish these changes to AWS Elastic Beanstalk. Using incremental deployments, the AWS Toolkit for Visual Studio will only upload the modified file and RDS-backed NerdDinner becomes available a few seconds later.

(click for full-sized image)

I hope that you enjoy these new AWS features!

— Jeff (with lots of help from Saad Ladki of the Elastic Beanstalk team);

The Multi-AZ (Availability Zone) feature of the Amazon Relational Database Service (RDS) replicates database updates across two Availability Zones to increase durability and availability. Amazon RDS will automatically fail over to the standby for planned maintenance and unplanned disruptions:

This feature is now available for all license types supported by Amazon RDS for Oracle Database including Standard Edition and Standard Edition One, and can be enabled with a single click, for new and existing RDS DB Instances:

In order to work properly with a Multi-AZ DB Instance, your application must be able to reconnect to the instance after a failover. We’ve added a new option to the rds-reboot-db-instance command to allow you to trigger a failover. You can also do this from the AWS Management Console.

When automatic failover occurs, your application can remain unaware of what’s happening behind the scenes. The CNAME record for your DB instance will be altered to point to the newly promoted standby. Your client library must be able to close and reopen the connection in the event of a failover.

If you have set up an Amazon RDS DB Instance as a Multi-AZ deployment, automated backups are taken from the standby to enhance DB Instance availability (by avoiding I/O suspension on the primary). The standby also plays an important role in patching and DB Instance scaling. In order to minimize downtime during planned maintenance, patches are installed on the standby and then an automatic failover makes the standby into the new primary. Similarly, scaling to a larger DB Instance type takes place on the standby, followed by an automatic failover.

Multi-AZ deployments also offer enhanced data protection and reliability in unlikely failure modes. For example, in the unlikely event a storage volume backing a Multi-AZ DB Instance fails, you are not required to initiate a Point-in-Time restore to the LatestRestorableTime (typically five minutes prior the failure). Instead, Amazon RDS will simply detect that failure and promote the hot standby where all database updates are intact.

In addition to Multi-AZ support, Amazon RDS for Oracle Database customers can now specify any one of the thirty Oracle-recommended character sets, including Shift-JIS, when creating new database instances.

Visit the Amazon RDS for Oracle Database page for additional information about these new features.

— Jeff;