Category: Amazon RDS

You can now purchase Relational Database Service Reserved DB Instances using the AWS Management Console. Reserved DB Instances allow you to reduce your costs if you plan to run your database for all or most of a one or three year term.

Here’s a tour. There’s a new item Reserved DB Instance item in the navigation area:

You can see all of your Reserved DB Instances, you can see the active or inactive instances, and you can also filter by name:

You can initiate your purchase by clicking on the Purchase Reserved DB Instances button and making your selection from the form. It is important to note that you are purchasing the instance for use within a particular AWS Region.

Of course, you have the opportunity to confirm your purchase before it is final:

This new UI should make it even easier for you to use the Relational Database Service and to benefit from the Reserved DB Instance pricing. What are you waiting for?

— Jeff;

The first AWS Office Hours session focused on Amazon EC2. Although I was scheduled to host the event, a sudden need for dental surgery made room for Jinesh Varia and Steve Riley to step in! Here’s a screen shot from the event (click on it to watch a replay):

From all accounts the event went really well and they’ll be doing another event next month.

The December 8, 2010 AWS Office Hours will focus on the Relational Database Service (RDS) and will take place from 9 AM to 10 AM PST.Attendance is limited, so sign up now (Note: the confirmation page still lists the old date; we’ll fix that ASAP).

You can submit questions in advance via Twitter by tagging them with #rdsofficehours. If someone else has already asked the question that you were planning to ask, re-tweet it and we’ll raise its priority accordingly.

— Jeff;

We’ve made some important updates to the AWS SDK for Java the AWS SDK for PHP, and the AWS SDK for .NET. The newest versions of the respective SDKs are available now.

AWS SDK for Java

The AWS SDK for Java now supports the new Amazon S3 Multipart Upload feature in two different ways. First, you can use the new APIs — InitiateMultipartUpload, UploadPart, CompleteMultipartUpload, and so forth. Second, you can use the SDK’s new TransferManager class. This class implements an asynchronous, higher level interface for uploading data to Amazon S3. The TransferManager will use multipart uploads if the object to be uploaded is larger than a configurable threshold. You can simply initiate the transfer (using the upload method) and proceed. Your application can poll the TransferManager to track the status of the upload.

The SDK’s PutObject method can now provide status updates via a new ProgressListener interface. This can be used to implement a status bar or for other tracking purposes.

We’ve also fixed a couple of bugs.

AWS SDK for PHP

The AWS SDK for PHP now supports even more services. We’ve added support for Elastic Load Balancing, the Relational Database Service, and the Virtual Private Cloud.

We have also added support for the S3 Multipart Upload, and for CloudFront Custom Origins, and you can now stream to (writing) or from (reading) an open file when transferring an S3 object. You can also seek to a specific file position before initating a streaming transfer.

The 1000-item limit has been removed from the convenience functions; get_bucket_filesize, get_object_list, delete_all_objects, delete_all_object_versions, and delete_bucket will now operate on all of the entries in a bucket.

We’ve also fixed a number of bugs.

AWS SDK for .NET

The AWS SDK for .NET now supports the Amazon S3 Multipart Upload feature using the new APIs — InitiateMultipartUpload, UploadPart, CompleteMultipartUpload, etc.as well as a new TransferUtility class that automatically determines when to upload objects using the Multipart Upload feature.

Weve also added support for the CloudFront Custom Origins and fixed a few bugs.

These SDKs (and a lot of other things) are produced by the AWS Developer Resource team. They are hiring and have the following open positions:

• Ruby Software Development Engineer
• Software Development Engineer
• Software Development Engineer – Windows/.NET

— Jeff;

We introduced the DB Engine Version Management feature earlier this year to give you control over the version of MySQL running in each of your DB Instances. You can now access this feature from the AWS Management Console. You can create a new DB Instance using any supported version of MySQL:

You can also upgrade a running instance to a newer version using the Modify Instance operation:

As you can see, you can update your DB Instances to a newer version of MySQL with a couple of clicks. Pretty easy, huh?

— Jeff;

It is always fun to write about price reductions. I enjoy knowing that our customers will find AWS to be an even better value over time as we work on their behalf to lower our own costs and pass the savings along.

Effective October 1, 2010 we are reducing the On-Demand and Reserved pricing for the High Memory Double Extra Large (db.2xlarge) and High Memory Quadruple Extra Large (db.4xlarge) DB Instances. Here are the old and the new On-Demand prices in the US-East Region for standard deployments:

Corresponding discounts have been applied in the other AWS Regions, to Reserved Instances, and to Multi-AZ deployments; see the Amazon RDS page for more information. If you have existing RDS Reserved DB Instances, your hourly usage rate will be lowered to the new rate effective October 1st, 2010.

— Jeff;

It is possible to characterize the workload handled by a relational database in terms of the ratio of reads to writes. Some applications seem to use about the same number of reads and writes. Others write a little bit of data and read a lot. This read-heavy behavior is frequently seen in web applications.

Different scaling techniques are applicable to different workloads. Read-heavy workloads that place too much of a load on a single database deployment can often be accommodated by distributing the reads to one or more “read replicas.” The read replicas track all of the writes made to the master and can provide an increase in aggregate read throughput when properly implemented.

Well, guess what? You can now set up read replicas for the Amazon Relational Database Service (RDS). You can do this for a single or multi-AZ DB Instance deployment. Here’s a block diagram:

All of the writes are directed to the main DB Instance.The Read Replicas track all changes made on the main DB Instance and update their own copies of the data.

You can choose to read from the main DB Instance or from any of the Read Replicas. If you need strict read-after-write consistency (what you read is what you just wrote) then you should read from the main DB Instance.  Otherwise, you should spread out the load and read from one of the Read Replicas. You can make this decision on a query-by-query basis within your application. You will probably want to maintain some sort of registry of available Read Replicas within your application, choosing from among them on a round-robin or randomly distributed basis.

As I noted earlier, the Read Replicas track all of the changes made to the source DB Instance. This is an asynchronous operation (updates are propagated to Read Replicas after they occur on the source DB Instance) and the Read Replicas can sometimes be out of date with respect to the source.This phenomenon is called replication lag and it is a common side-effect of the MySQL replication process. Each Read Replica publishes a Replica Lag metric in Amazon CloudWatch to allow you to see how far it has fallen behind the source DB Instance. You can access this metric via the AWS Management Console and the Amazon CloudWatch APIs, or via the customary “show slave status” MySQL command. When things are running well the lag is generally measured in seconds or a small number of minutes.

Read Replicas are complements, not substitutes, to Multi-AZ deployments.  Because of the replication lag you can expect with Read Replicas and MySQL 5.1s built-in replication, Read Replicas are not best-suited to increasing database write availability or protecting your latest database updates in the event of source DB Instance or Availability Zone failure. Instead, you should run your source DB Instance as a Multi-AZ deployment.  The replication used by Multi-AZ deployments is synchronous, meaning that all database writes are concurrent on the primary and standby and all of your latest updates to the primary should be available on the standby after failover occurs. In addition, with Multi-AZ deployments replication is fully managed.  Amazon RDS automatically monitors for DB Instance failure conditions or Availability Zone failure and initiates automatic failover to the standby in a different Availability Zone if an outage occurs. You can use a Multi-AZ deployment as the source for your Read Replicas, gaining the availability and durability benefits of Multi-AZ, as well as the scaling benefits of Read Replicas.

You can create Read Replicas using the RDS APIs, the RDS command-line tools, or the AWS Management Console.If you use the Amazon RDS APIs, you will call the CreateDBInstanceReadReplica API to create a Read Replica from a specified source DB Instance. For a create to succeed, your source DB Instance must be running MySQL 5.1.50 or later. You can upgrade the DB Instance to the proper version using the ModifyDBInstance API or the Modify Instance menu item in the AWS Management Console. When you create a replica of an existing DB Instance, one of two things happen next:

• For a single AZ DB Instance, the DB Instance pauses briefly while a snapshot is made. The instance then resumes, and the Read Replica is created from the snapshot.
• For a Multi-AZ DB Instance deployment, a snapshot is made from the standby without pausing the primary. The Read Replica is created from the snapshot.

The Read Replica will always have the same MySQL version (e.g. MySQL 5.1.50) as the source DB Instance. Each Read Replica should have at least as much storage and processing power as the source DB Instance. You can scale the processing power and storage of each Read Replica independently of the source DB Instance or of other Read Replicas. As of launch time, each DB Instance can have up to five Read Replicas. Read Replicas are billed at the same rates as standard DB Instances, and standard DB Instance reservations (RDS Reserved Instances) of the same DB Instance class and Region can be applied to Read Replicas.

There are other use cases for a Read Replica beyond increasing effective read traffic capacity. For example, you might want to use a Read Replica for business reporting. This prevents queries by business analysts from impacting the performance of your production DB Instance. For this use case, you might want to add a special index that is only on the Read Replica and is used only for reporting purposes. To do this, you can enable DDL write operations on a given Read Replica by setting a parameter in the Read Replica’s DB Parameter Group. The ability to promote Read Replica into a primary/source DB Instance does not exist at feature launch, but this functionality is coming soon.

It should be fairly easy to create a data-driven auto scaling database cluster using Read Replicas and CloudWatch metrics. At the low end, this cluster would consist of a Small DB Instance running in one Availability Zone with 5 GB of storage. At the high end it would consist of a High Memory Quadruple Extra Large Multi-AZ deployment (primary/secondary pair) DB Instance with 1 TB of storage and 5 associated Read Replicas. That’s quite a range!

— Jeff;

I can’t even leave my neighborhood without getting feature requests! My neighbor Rick was walking his dog past my house and stopped to ask me when we were going to support Reserved Instances for Amazon RDS. Such is life in the Pacific Northwest part of the US.

Hey Rick, we’ve got your RDS Reserved Instances ready now. You can get a lower price and know that an instance has been reserved for your use.

After you make a low, one-time payment, you can use a DB Instance at an hourly rate that is significantly lower than the On-Demand pricing for the same instance class. This will result in a decrease in AWS charges for any application that requires full-time access to a relational database.

When you purchase an RDS Reserved DB Instance, you choose a specific AWS Region and instance class. You do not need to choose an Availability Zone. You can purchase up to 20 Reserved DB instances. If you need more DB Instances, you need to complete the Amazon RDS DB Instance request form. You cannot move a reservation from one Region to another and they always apply to a particular DB instance class, so make your purchase with care.

You can make your purchase using the RDS APIs, or the RDS command-line tools. Here are the commands:

• rds-describe-reserved-db-instances – Returns the list of DB Instance reservations for your account or details for one of your reserved database instances.
• rds-describe-reserved-db-instances-offerings – Returns the list of DB Instance offerings that are available for purchase.
• rds-purchase-reserved-db-instances-offering – Purchases one or more reserved DB Instances.

The one-time fee starts at $227.50 for a Small DB Instance with a one-year term.

— Jeff;

PS – Clean up after your dog, Rick.

I’ve got a pair of related news items for users of the Amazon Relational Database System (RDS):

1. We now support version 5.1.49 of MySQL, with the InnoDB Plugin as the InnoDB storage engine.
2. We now provide a new, optional level of control over when and if your DB Instance(s) are upgraded to new MySQL versions supported by Amazon RDS. This new functionality is called DB Engine Version Management, and this post will help you understand what it can do and how to use it.

MySQL Version 5.1.49 with InnoDB Plugin

Version 5.1.49 of MySQL includes a number of bug fixes and enhancements. For DB Instances running MySQL 5.1.49, the most significant change is the inclusion of InnoDB Plugin as the InnoDB storage engine, replacing InnoDB Builtin. InnoDB Plugin offers (according to the feature page) performance and scalability enhancements (especially on multi-core platforms), fast index creation, data compression, and more. The InnoDB Builtin (InnoBase) is not available in 5.1.49 and will not be available in future Amazon RDS Supported versions.

New DB Instances that you create will now run MySQL 5.1.49 with InnoDB Plugin, unless otherwise specified using the new DB Engine Version Management feature described in the next section.

DB Engine Version Management

With DB Engine Version Management, Amazon RDS gives you additional (yet optional) control over the version of relational database software (i.e. MySQL) powering your DB Instance. The goal of this functionality is to provide you the flexibility to maintain compatibility with specific MySQL versions, test new versions with your application before deploying in production, and perform version upgrades on your own terms and timelines. Lets take a look:

DB Instance Creation

Amazon RDS now allows you to create new DB Instances using any supported version of MySQL. Right now this means version 5.1.45 and 5.1.49, but we plan to support additional minor and major versions (e.g. MySQL 5.5) in the future. There’s a new EngineVersion attribute associated with each DB Instance, and you can specify your desired Version when issuing a CreateDBInstance API call (or the associated command).  New DB Instances will now use MySQL 5.1.49 with InnoDB Plugin by default.  If you still wish to use MySQL 5.1.45 (which uses InnoDB Builtin) for a new DB Instance, you can do so using this new functionality.

Automatic Upgrade Schedule

Your existing DB Instances will be upgraded to the new version of MySQL with InnoDB Plugin during your scheduled maintenance window on the week of October 4, 2010. If you don’t want to upgrade, you need to use ModifyDBInstance to set the AutoMinorVersionUpgrade attribute of your DB instances to “false.”

Find Your Engine

The new DescribeDBEngineVersions API call returns a list of all supported versions of MySQL. You can use it (or the rds-describe-db-engine-versions command) to drive your configuration process. You could also use it to set up a really cool automated testing framework. You could call the function, iterate through the resulting list of versions, and create a new DB Instance for each one. Then you could test your application to make sure that it works with each version of MySQL. Or you could create several DB Instances (one per version) simultaneously and test the same queries against each instance.

Choose to Upgrade Today

With DB Engine Version Management, you can also upgrade any of your existing DB Instances to InnoDB Plugin and the new version of MySQL today by setting the EngineVersion attribute to “5.1.49” using the ModifyDBInstance API call (or the associated command).  Amazon RDS will take care of the rest. This will happen immediately if you set the ApplyImmediately flag when you modify the EngineVersion attribute, or at the next maintenance window for your DB Instance if you don’t.

Learning More

To learn more about DB Engine Version Management and associated API changes, please visit our Forum Post on the subject. You can also read more about the new functionality via our DB Engine Version Management FAQs, which include information about our guidelines for supporting new versions and deprecating old versions.

With these changes, Amazon RDS becomes even more powerful. If you want to verify that your application works with the new version of MySQL, simply create a snapshot of your existing DB Instance, create a new one from the snapshot, and test away.

— Jeff;

We’ve added some very handy new functionality to the RDS tab of the AWS Management Console. Here’s a quick tour.

First, you can create a new DB Instance from any of your existing snapshots. You can click this button:

Or you can right-click on one of your snapshots:

After you provide the parameters, Amazon RDS will create a new DB Instance for you:

As you can see, you can choose to create the new instance in a different Availability Zone, on a different DB instance class, and so forth. You could, for example, create a snapshot of a production database running on one of the larger instance classes and then create a DB Instance using a smaller instance class for some light-duty testing.

Second, you can create a new DB Instance as of any point in time that falls within the backup retention period of any of your existing DB Instances. Again, you can get to this from a button or a right-click on one of your existing DB Instances:

After you provide the parameters, Amazon RDS will create a new DB Instance for you:

If you accidentally break your production database when you check in and run some new code, you can use this feature to create a new instance representing the state of the database as it was immediately preceding the change.

Both of these features will make it even easier for you to use the Relational Database Service.

— Jeff;

By popular demand, the Relational Database Service (RDS) now supports SSL encrypted connections!

We now generate an SSL certificate for each DB Instance. If you need a certificate for an existing instance you’ll need to reboot it using the AWS Management Console, the RDS command-line tools, or the RDS APIs.

Here are a few things to keep in mind:

• SSL encrypts the data transferred “over the wire” between your DB Instance and your application. It does not protect data “at rest.” If you want to do this, you’ll need to encrypt and decrypt the data on your own.
• SSL encryption and decryption is a compute-intensive task and as such it will increase the load on your DB Instance. You should monitor your database performance using the CloudWatch metrics in the AWS Management Console (pictured at right), and scale up to a more powerful instance type if necessary.
• The SSL support is provided for encryption purposes and should not be relied upon to authenticate the DB Instance itself.
• You can configure your database to accept only SSL connections by using the GRANT command with the REQUIRE SSL option. You can do this on a per-user basis so you could, for example, require SSL requests only from users connecting from a non-EC2 host.

You can learn more about this new feature in the RDS Documentation on Database Instances and in the forum post.

— Jeff;