AWS Architecture Blog
Selecting the right database and database migration plan for your workloads
There has been a tectonic shift in the approach to hosting enterprise workloads. Companies are rapidly moving from on-premises data centers to cloud-based services. The driving factor has been the ability to innovate faster on the cloud.
Your transition to cloud can be straightforward, but it does go beyond the usual ‘lift-and-shift’ approach. To start with, you’ll need a cloud provider that provides a data store and computing facility. But you’ll be able to grow your business with a provider that has purpose-built resources and a platform that supports innovation. Companies are using higher-level services such as fully managed databases and serverless compute offered by Amazon Web Services (AWS) to get the most out of their cloud adoption journey.
In this blog, we will focus on the database tier of enterprise applications. We will evaluate the reasons for moving to managed or purpose-built databases. Then we’ll discuss in more detail the options you have and how to implement the right database migration plan for your workloads.
Why move to a managed database?
Managed databases, or purpose-built databases, are managed services by AWS that free you to focus on other business and technical priorities. Different types of databases fill certain needs, and so the database tier in an application should never be a one-size-fits-all approach. Based on the kind of application you are developing, you should research managed options that suit your business and enterprise use cases.
| Database Type | Use Cases | AWS Service |
|---|---|---|
| Relational | Traditional applications, enterprise resource planning (ERP), customer relationship management (CRM), ecommerce |
|
| Key-Value | High-traffic web applications, ecommerce systems, gaming applications |
|
| In-Memory | Caching, session management, gaming leaderboards, geospatial applications |
|
| Document | Content management, catalogs, user profiles |
|
| Wide Column | High-scale industrial apps for equipment maintenance, fleet management, and route optimization |
|
| Graph | Fraud detection, social networking, recommendation engines |
|
| Time Series | Internet of Things (IoT) applications, DevOps, industrial telemetry |
|
| Ledger | Systems of record, supply chain, registrations, banking transactions |
|
Table 1. Managed databases by AWS
Managed database features
Manageability. The top priority and most valuable asset that you own as a company is your data. While data remains your key asset, spending time on database management is not the optimum use of your time. Managed services have built-in reliable tooling for multiple aspects of database management, which can help your database operate at the highest level.
Availability and disaster recovery. Most managed databases at AWS are highly available by default. For example, Amazon Aurora is designed to offer 99.99% availability, replicating six copies of your data across three Availability Zones (Figure 1). It backs up your data continuously to Amazon S3. It recovers transparently from physical storage failures; instance failover typically takes less than 30 seconds.
Figure 1. Replication across three Availability Zones with Amazon Aurora DB cluster
With managed databases, you get multiple options to create a highly available and fault tolerant database. Alternatively, if you choose to self-host a database elsewhere you will have to formulate your own disaster recovery (DR) strategy. This takes meticulous DR planning and relies heavily on a constant monitoring solution.
Elasticity and agility: Cloud offers elasticity for scaling your database. You can scale in minutes and spin up and down additional servers and storage size as needed. It offers you flexibility with capacity planning. You can always reassess your database tier to see if it is over or under provisioned.
Self-managed databases on AWS
If I do not need a managed database, should I still use Amazon EC2 to host my database?
Here are some cases when you may find it beneficial to host your database on Amazon EC2 instances:
- You need full control over the database and access to its underlying operating system, database installation, and configuration.
- You are ready to plan for the high availability of your database, and prepare your disaster recovery plan.
- You want to administer your database, including backups and recovery. You must perform tasks such as patching the operating system and the database, tuning the operating system and database parameters, managing security, and configuring high availability or replication.
- You want to use features that aren’t currently supported by managed services.
- You need a specific database version that isn’t supported by AWS managed database service.
- Your database size and performance needs exceed the limits of the managed service.
- You want to avoid automatic software patches that might not be compliant with your applications.
- You want to achieve higher IOPS and storage capacity than the current limits of the managed services.
Can I customize my underlying OS and database environment?
Amazon RDS Custom is a managed database service for applications that require customization of the underlying operating system and database environment. With Amazon RDS Custom, you have access to the underlying EC2 instance that hosts the DB engine. You can access the EC2 instance via secure shell (AWS Systems Manager and SSH) and perform the customizations to suit your application needs.
Choosing which migration plan to implement
The first step in your cloud database migration journey is identifying which database that you want to migrate to. For relational databases, determine the migration strategy. In the majority of the database migrations, you can choose to rehost, replatform, or refactor.
Refer the AWS Prescriptive Guidance for choosing a migration strategy for relational databases.
The next is determining which database migration plan best serves your needs. AWS provides a number of options to help correctly handle your data migration with minimal downtime. Here are the database migration plans that you can consider using for your cloud database adoption journey:
1. AWS-offered AWS Database Migration Service (AWS DMS): AWS Database Migration Service (AWS DMS) is a self-service option for migrating databases. You can use AWS DMS to migrate between homogeneous database types, such as going from one MySQL instance to a new one. You can also use AWS DMS between heterogeneous database types, such as moving from a commercial database like Oracle to a cloud-native relational database like Aurora. Read tutorials about migrating sample data: AWS Database Migration Service Step-by-Step Walkthroughs.
AWS DMS offers minimal downtime, supports widely used databases, supports on-going replication, is offered as a low-cost service, and is highly reliable. If you are looking for an end-to-end service for database migration, consider AWS DMS.
2. AWS Database Migration Service (DMS) with AWS Schema Conversion Tool (SCT): If you are migrating between heterogenous databases, use the AWS Schema Conversion Tool (SCT). It converts the source database schema and the database code objects (like views, stored procedures, and functions) to a format compatible with the target database (Figure 2).
Figure 2. Supported conversions with AWS Schema Conversion Tool
For heterogeneous migrations, consider using AWS DMS with AWS SCT.
3. Database Freedom Program: If you are new to cloud computing or if your database migration plan must be evaluated and validated by industry experts, then try the Database Freedom Program. Database Freedom is a unique program designed to assist customers in migrating to AWS databases and analytics services. They will provide technical advice, migration support, and financial assistance.
You can use the Contact Us page for the Database Freedom program to get in touch with experts that can assist you in your cloud adoption journey.
4. AWS Professional Services and Partners: You may have in-house database expertise, but need end-to-end implementation assistance across different application tiers. Get help from the Professional Services of AWS or connect with the certified network of AWS Partners. AWS Database Migration Service Delivery Partners help customers use AWS DMS to migrate databases to AWS securely, while minimizing application downtime and following best practices.
Conclusion
Migrating to the cloud is a journey that is ever-evolving. To remain focused on your innovations, leverage the managed services of AWS for your migration journey.
I hope this post helps you consider using a managed database service when available, and effectively evaluate and choose the right database migration plan for your enterprise. For any queries, feel free to get in touch with us. We will be happy to help you in your cloud journey.
Happy migrating!
Additional Reading