Category: Amazon RDS

If you use Amazon RDS, you probably understand the ease with which you can create read replicas to increase the scalability and performance of your database-backed applications.

Today we are extending this feature to decrease replica creation time, increase snapshot performance, and give you even more read throughput.

Let’s take a closer look…

Parallel Replica Creation
You can now create multiple read replicas in quick succession (limited to a total of five per master). You no longer have to wait for one creation operation to complete before starting the next one. This new feature will save you time and reduce your administrative burden when creating and maintaining multiple read replicas.

Snapshots and Point in Time Recovery
You have always been able to create database snapshots and to perform point in time recovery operations on your master database instance. You can now perform the same operations on read replicas, as long as you are running MySQL 5.6.

Second-Tier Replicas
You can now create second-tier read replicas if you are running version 5.6 of MySQL. You can create a couple of different topologies using this new feature. First, you can use a single first-tier replica to shift the replication load from the primary instance to the replica in the first tier, like this:

Second, you can use multiple first-tier and second-tier replicas to prepare for an extremely high level of read traffic:

You can create up to five first-tier replicas. Each of these can support up to five second-tier replicas. Therefore, one master can support thirty replicas.

In any architecture where replicas are present, you must be aware of the phenomenon known as replication lag. Each of the replicas must obtain the log data from the master and then apply it locally. You can monitor the replication lag (as seen by each replica) in the AWS Management Console:

You can use Provisioned IOPS on the replicas and you can choose between eight distinct DB Instance types, starting from the Micro DB Instance and going all of the way up to the High Memory Quadruple Extra Large.

As always, these features are available now and you can start using them today.

— Jeff;

With AWS being put to use in an ever-widening set of use cases across organizations of all shapes and sizes, the need for additional control over the permissions granted to users and to applications has come through loud and clear.  This need for control becomes especially pronounced at the enterprise level. You don’t want the developers who are building cloud applications to have the right to make any changes to the cloud resources used by production systems, and you don’t want the operators of one production system to have access to the cloud resources used by another.

The Story So Far
With the launch of IAM in the middle of 2010, we gave you the ability to create and apply policies that control which users within your organization were able to access AWS APIs.

Later, we gave you the ability to use policies to control access to individual DynamoDB, Elastic Beanstalk, Glacier, Route 53, SNS, SQS, S3, SimpleDB, and Storage Gateway resources.

Today we are making IAM even more powerful with the introduction of resource-level permissions for Amazon EC2 and Amazon RDS. This feature is available for the RDS MySQL, RDS Oracle, and RDS SQL Server engines.

On the EC2 side, you can now construct and use IAM policies to control access to EC2 instances, EBS volumes, images, and Elastic IP addresses. On the RDS side, you can use similar policies to control access to DB instances, Read replicas, DB Event subscriptions, DB option groups, DB parameter groups, DB security groups, DB snapshots, and subnet groups.

Let’s take a closer look!

Resource-Level Permissions for EC2
You can now use IAM policies to support a number of important EC2 use cases. Here are just a few of things that you can do:

• Allow users to act on a limited set of resources within a larger, multi-user EC2 environment.
• Set different permissions for “development” and “test” resources.
• Control which users can terminate which instances.
• Require additional security measures, such as MFA authentication, when acting on certain resources.

This is a complex and far-reaching feature and we’ll be rolling it out in stages. In the first stage, the following actions on the indicated resources now support resource-level permissions:

• Instances – Reboot, Start, Stop, Terminate.
• EBS Volumes – Attach, Delete, Detach.

EC2 actions not listed above will not be governed by resource-level permissions at this time. We plan to add support for additional APIs throughout the rest of 2013.

We are also launching specific and wild-card ARNs (Amazon Resource Names) for all EC2 resources. You can refer to individual resources using ARNs such as arn:aws:ec2:us-east-1:1234567890:instance/i-i-96d811fe and groups of resources using ARNs of the form arn:aws:ec2:us-east-1:1234567890:instance/*.

EC2 policy statements can include reference to tags on EC2 resources. This gives you the power to use the same tagging model and schema for permissions and for billing reports.

In order to make it easier for you to test and verify the efficacy of your policies, we are extending the EC2 API with a new flag and a couple of new functions.  The new flag is the DryRun flag, available as a new general option for the EC2 APIs. If you specify this flag the API request will perform an authorization determination, but will not actually process the API request (for example, to determine whether a user has permission to terminate an instance without actually terminating the instance). 

In addition, when using API version 2013-06-15 and above, you will now receive encoded authorization messages along with authorization denied errors that can be used in combination with a new STS API, DecodeAuthorizationMessage, to learn more about the IAM policies and evaluation context that led to a particular authorization determination (permissions to the new STS API can be controlled using an IAM policy for the sts:DecodeAuthorizationMessage action).

The final piece of the EC2 side of this new release is an expanded set of condition tags that you can use in your policies. You can reference a number of aspects of each request including ec2:Region, ec2:Owner, and ec2:InstanceType (consult the EC2 documentation for a complete list of condition tags).

Resource Permissions for RDS
You can also use policies to support a set of important RDS use cases. Here’s a sampling:

• Implement DB engine and Instance usage policies to specific group of users. For example, you may limit the usage of the m2.4xl instance type and Provisioned IOPS to users in the Staging users or Production users groups.
• Permit a user to create a DB instance that uses specific DB parameter groups and security groups. For example, you may restrict Web application developers to use DB instances with Web application parameter groups and Web DB Security groups. These groups may contain specific DB options and security group settings you have configured.
• Restrict a user or user group from using a specific parameter group to create a DB instance. For example, you may prevent members of the Test users group from using Production parameter groups while creating test DB instances.
• Allow only specific users to operate DB instances that have a specific tag (e.g. DB instances that are tagged as “production”). For example, you may enable only Production DBAs to have access to DB instances tagged with Production label.

As you might have guessed from my final example, you can now make references to tags on any of the RDS resources (see my other post for information on the newly expanded RDS tagging facility), and you can use the same tags and tagging schema for billing purposes.

As we did for EC2, we have added a set of RDS condition tags for additional flexibility. You can reference values such as rds:DatabaseClass, rds:DatabaseEngine, and rds:Piops in your policies.

For more information, consult the Managing Access to Your Amazon RDS Resources and Database chapter in the RDS documentation.

The AWS Security Blog takes an even more detailed look at this feature and shows you how to use it!

Go For It
These new features are available now and you can start using them today.

— Jeff;

You can now use tags to organize your Amazon RDS resources. Also, as I have noted in my companion blog post, you can reference these tags in IAM policies in order to manage access to RDS resources and to control the actions that can be applied to the resources. You can even use the tags to track costs by grouping expenses for similarly tagged resources.

Taqgable Resources
You can now tag every type of RDS resource (prior to this release you could only tag DB instances):

• DB instances
• Reserved DB instances
• Read replicas
• DB Event subscriptions
• DB option groups
• DB parameter groups
• DB security groups
• DB snapshots
• DB subnet groups

As is the case with other taggable AWS resources, you can create up to ten tags per resource. The tag names and values are simply character strings; AWS does not apply any meaning to them. Tag names can be from 1 to 128 Unicode characters in length. Tag values can be from 1 to 256 Unicode characters in length. Consult the RDS documentation for additional information on the set of acceptable characters for names and values.

Applying Tags
You can apply tags from the AWS Management Console, the Amazon RDS API, and the Amazon RDS Command Line Interface (CLI).

Here’s a screen shot of the console interface for viewing and adding tags:

If you are using the Command Line Interface, check out the rds-add-tag-to-resource command. From the API, use the AddTagsToResource function. In either case, you’ll need to know how to construct Amazon Resource Names (ARNs) for the newly taggable resources. Visit Constructing an Amazon RDS Resource Name for more information on this topic.

To learn more about this new feature, read Tagging Resources in the RDS Documentation.

— Jeff;

You have a plethora of options when you want to run a relational or NoSQL databases on AWS.

On the relational side, you can use the Relational Database Service (RDS) to run a MySQL, Oracle, or SQL Server database. RDS will take care of the scaling, backup, maintenance, patching, and failover for you so that you can focus on your application.

On the NoSQL side, DynamoDB can scale to accommodate any amount of data and any request rate, making it ideal for applications with an unlimited potential to grow.

The AWS Marketplace also contains a very well-curated selection of relational and NoSQL databases and caches.

You also have the option to launch the database of your choice on an Amazon EC2 instance. In order to provide you with the information that you need to do this while taking advantage of all of the flexibility that AWS has to offer, we have worked with our partners to write some very detailed white papers.

The first such white paper, RDBMS in the Cloud: PostgreSQL on AWS, was released a few days ago. This 23 page document was authored by AWS Solutions Architect Miles Ward and the senior staffers at PalominoDB.

The document covers installation, use of SSD storage, and use of EBS. It also addresses the operational side with detailed information about maintenance, backup and restore, storage of backup files, replication, and monitoring. It concludes with a detailed look at security, touching on disk encryption, row-level encryption, and SSL.

Stay tuned for additional white papers in this series!

— Jeff;

I am happy to announce that the Amazon Relational Database Service (RDS) now supports version 5.6 of MySQL.

If you are an existing RDS customer, you know that Amazon RDS for MySQL delivers several important benefits to MySQL customers including ease of deployment, high availability with automatic failure detection and failover, read replicas, push button scalability, automated back-ups, point-in-time restore and automated software upgrades and patching.

New Features in MySQL 5.6
You now have access to a number of powerful new MySQL 5.6 features. Here’s a sampling:

• Crash Safe Read Replicas – MySQL 5.6 improves the reliability of RDS Read Replicas by making the binlog and table data transactionally consistent on the primary database instance. This allows Read Replicas to resume replication automatically after a primary database instance crashes or fails over (in the case of Multi-AZ instances).
• InnoDB Integration with Memcached – You can use the Memcached API to bypass the SQL layer and treat InnoDB data as a key-value store. You can enable Memcached on your Amazon RDS for MySQL instance using a DB Option Group.
• Binary Log Access – You can download and stream binary logs through the native mysqlbinlog tool. This can be useful for a variety of purposes such as syncing data with an on-premises deployment, audit logging, analytics, and debugging of replication errors. You must enable backup retention for the RDS database instance in order to take advantage of this feature.
• Online Schema Changes – You can now initiate multiple, concurrent ALTER TABLE operations on InnoDB tables to add columns or indexes while simultaneously running queries.
• Full Text Indexes – You can build full text indexes (for InnoDB tables) on columns with text based content to speed up searches for words/phrases.

Launch Now
You can get started with MySQL 5.6 today by launching RDS Database Instances from the AWS Management Console:

Upgrading From MySQL 5.5
Upgrading an existing database instance from MySQL 5.5 to MySQL 5.6 is not currently supported. However, we intend to provide this functionality in the near future.

In the meantime, if you would like to port your existing MySQL 5.5 database to MySQL 5.6, you can use mysqldump to export your database from your existing MySQL 5.5 database instance and import it into a new MySQL 5.6 database instance.

For more information, visit the MySQL section of the Amazon RDS User Guide.

— Jeff;

The Amazon RDS team has been rolling out features at a very rapid pace! Today we are giving you the ability to upgrade existing RDS database instances from MySQL 5.1 to MySQL 5.5 using our new Major Version Upgrade feature.

MySQL 5.5 includes several features and performance benefits over MySQL that may be of interest to you including enhanced multicore scaling, better use of I/O capacity, and enhanced monitoring by means of the performance schema. MySQL 5.5 defaults to version 1.1 of the InnoDB Plugin, which improves on version 1.0 (the default for MySQL 5.1) by adding faster recovery, multiple buffer pool instances, and asynchronous I/O.

Today’s RDS release will make it easy for you to upgrade. You simply select the instance in the AWS Management Console, choose the Modify option, and select the latest version of MySQL 5.5 (you cannot upgrade to earlier versions). You can choose to apply the upgrade immediately or during the next maintenance window for the instance. In either case, your instance will be unavailable for a few minutes while the upgrade completes and the instance is rebooted.

Here’s what you need to do to upgrade.

1. Read the MySQL 5.5 release notes and verify that none of the changes will affect your application.
2. Launch a test database instance and verify that your application runs as expected. Amazon RDS makes this easy: you can create a snapshot of your running instance, create a new instance from the snapshot, upgrade it to MySQL 5.5 using the Modify command, and do your testing.
3. Modify your production database.

Read the RDS major version upgrade documentation to learn more.

— Jeff;

I’m happy to announce that we are lowering the price of Amazon RDS (Relational Database Service) database instances, both On-Demand and Reserved.

On-Demand prices have been reduced as much as 18% for MySQL and Oracle BYOL (Bring Your Own License) and 28% for SQL Server BYOL. All of your On-Demand usage will automatically be charged at the new and lower rates effective June 1, 2013.

Reserved Instance prices have been reduced as much as 27% for MySQL and Oracle BYOL. The new prices apply to Reserved Instance purchases made on or after June 11, 2013.

Here is a table to illustrate the total cost of ownership for an m2.xlarge DB Instance for MySQL or Oracle BYOL using a 3-year Reserved Instance:

Although Reserved Instance purchases are non-refundable, we are making a special exception for 1-year RI’s purchased in the last 30 days and 3-year RI’s purchased in the last 90 days. For a limited time, you can exchange recently purchased RI’s for new ones. You’ll receive a pro-rata refund of the upfront fees that you paid at purchase time. If you would like to exchange an RDS Reserved Instance for a new one, simply contact us.

As you may know from my recent blog post, we have made a lot of progress since releasing Amazon RDS just 3.5 years ago. In addition to the recently announced Service Level Agreement (SLA) for Multi-AZ database instances, you have the ability to provision up to 30,000 IOPS for demanding production workloads, encryption at rest using Oracle’s Transparent Data Encryption, and simple disaster recovery using Multi-AZ and read replicas.

— Jeff;

PS – Here’s a full list of AWS price reductions.

The Amazon Relational Database Service (RDS) was designed to simplify one of the most complex of all common IT activities: managing and scaling a relational database while providing fast, predictable performance and high availability.

RDS in Action
In the 3.5 years since we launched Amazon RDS, a lot has happened. Amazon RDS is now being used in mission-critical deployments by tens of thousands of businesses of all sizes. We now process trillions of I/O requests each month for these customers. We’re seeing strong adoption in enterprises such as Samsung and Unilever, web-scale applications like Flipboard and Airbnb, and large-scale organizations like NASA JPL and Obama for America.

RDS Innovation
We have added support for three major database engines (MySQL, Oracle, and SQL Server), expanded support to all nine of the AWS Regions, and added more than 50 highly requested features. Here is a timeline to give you a better idea of how many additions we have made to RDS since we launched it:

Let’s take a closer look at a few of the more important features on this timeline:

• Multiple Database Engines – Support for Oracle Database and Microsoft SQL Server in addition to MySQL.
• Multi-AZ Deployments – This feature enables you to create highly available database deployments with synchronous replication across Availability Zones, automatic failure detection and failover using just a few clicks on the AWS Management Console.
• Read Replicas – This feature makes it easy to elastically scale out beyond the capacity constraints of a single database instance for read-heavy database workloads. You can promote a Read Replica to a master as needed and monitor the replication status directly through the AWS Management Console.
• Provisioned IOPS – Amazon RDS Provisioned IOPS is a storage option designed to deliver fast, predictable, and consistent I/O performance, and is optimized for I/O-intensive, transactional (OLTP) database workloads. You can provision up to 3TB of storage and 30,000 IOPS per database instance.
• Point-and-click Access to Database Logs – You can view and download a variety of database log files to troubleshoot database issues directly using the AWS Management Console.
• DB Notifications via Email and SMS – You can subscribe to receive e-mail or SMS notifications when database events such as failover, low storage, replication state change, and so forth occur.

As I noted above, these innovations are powering some of the worlds most popular applications that are used by millions of users.

General Availability & The New RDS SLA
We’re marking Amazon RDS as “generally available” after adding the highly requested features described above.

With strong customer adoption across multiple market segments, numerous new features, and plenty of operational experience behind us, we now have a Service Level Agreement (SLA) for Amazon RDS, with 99.95% availability for Multi-AZ database instances on a monthly basis. This SLA is available for Amazon RDS for MySQL and Oracle database engines because both of those engines support Multi-AZ deployment. If availability falls below 99.95% for a Multi-AZ database instance (which is a maximum of 22 minutes of unavailability for each database instance per month), you will be eligible to receive service credits. The new Amazon RDS SLA is designed to give you additional confidence to run the most demanding and mission critical workloads dependably in the AWS cloud. You can learn more about the SLA for Amazon RDS at http://aws.amazon.com/rds-sla.

— Jeff;

Amazon RDS for MySQL has long had the ability to create Read Replicas. You can do this with a couple of clicks in the AWS Management Console. Each read replica runs as a slave to the master database instance. Under certain circumstances, replication can stop. This can happen if you cause a replication error by running DML queries on the replica that conflict with the updates made on the master and also under other circumstances.

To help you detect and respond to this situation, Amazon RDS now monitors the replication status of your Read Replicas and sets the Replication State field to Error if replication stops for any reason. You can learn more about the error by inspecting the Replication Error field.

The status is now visible in the AWS Management Console:

If you deem that you can safely skip the error, you can run the CALL mysql.rds_skip_repl_error command. Otherwise, you can delete the Read Replica and create a new one with the same DB Instance Identifier (so that the Endpoint remains the same as that of your old read replica). If a replication error is fixed, the Replication State changes to Replicating.

You can also use Amazon RDS Event Notifications to automatically get notified when you encounter a replication error. You can also monitor the Replication Lag metric and set up a CloudWatch alarm to receive a notification when the lag crosses a particular threshold tolerable by your application.

Learn more about monitoring Read Replicas by visiting the Working with Read Replicas section of the Amazon RDS User Guide.

— Jeff;

Amazon RDS for Oracle Database now supports a pair of important features to help protect your mission-critical data:

• Transparent Data Encryption protects data at rest. It encrypts your data before it is written to storage, and decrypts it after it is read from storage. You can choose to encrypt tablespaces or specific table columns using a number of industry standard encryption algorithms including Advanced Encryption Standard (AES) and Data Encryption Standard (Triple DES).
• Native Network Encryption protects data in motion using the Oracle Net Services. You can choose between AES, Triple DES, and RC4

These features are components of Oracle’s Advanced Security Option (ASO) for Oracle Database 11g Enterprise Edition, available for use on Amazon RDS under the Bring-Your-Own-License (BYOL) model. There is no additional charge for either feature.

Enabling Native Network Encryption
To enable Native Network Encryption, add the NATIVE_NETWORK_ENCRYPTION option to an option group associated with the RDS DB Instance and specify the option settings. The settings are described in the Options for Oracle DB Engine section of the Amazon RDS Documentation and include SQLNET.ENCRYPTION_SERVER (encryption behavior), SQLNET.CRYPTO_CHECKSUM_SERVER (data integrity behavior),  SQLNET.ENCRYPTION_TYPES_SERVER (encryption algorithm), and SQLNET.CRYPTO_CHECKSUM_TYPES_SERVER (checksum algorithm). You must also make the corresponding changes in the sqlnet.ora file on the client in order to be able to connect to the DB Instance.

Enabling Transparent Data Encryption
You can choose to encrypt entire tables (tablespaces) or individual columns.

To enable Transparent Data Encryption, add the TDE option to an option group associated with the RDS DB Instance. Once you choose to enable this option for a DB Instance, it becomes permanent, and cannot be disabled.

For more information on these and other options, take a look at the Options for Oracle DB Engine section of the Amazon RDS Documentation.

Ready Now
This feature is available today and you can start using it now (I never get tired of typing that!).

— Jeff;