What is Database Migration?
Database migration is a process that shifts a database from one environment to another. As businesses seek to modernize their applications, they migrate the associated databases from on-prem servers to the cloud.
Migrating databases to the cloud allows companies to increase service reliability, scalability, and performance. In addition, cloud data migration reduces the infrastructural costs of maintaining on-prem data centers.
Moving data volumes to the cloud is important to ensure data integrity, compliance, and business continuity. Consider the following.
Budget
Data migration involves tasks like data auditing, mapping, testing, and post-migration monitoring. Sometimes, the application that accesses the database requires modification before it can read data from the migrated database. Businesses must budget for the time, costs, and resources needed for such tasks.
Downtime
When migrating the database, plan for downtime as you switch between the source and destination target. While zero downtime is impossible, you can reduce business disruption by planning the migration and using appropriate database migration tools. For example, you can test the cloud database long before the switchover to ensure application reliability and data accuracy.
Source and target systems
A homogenous transfer moves data from source to target systems with similar database engines. In contrast, data moving to a different target platform is called a heterogeneous transfer. For example, migrating a MySQL database to a MySQL database is homogenous, while transferring Postgres SQL to Amazon Aurora is heterogenous. Homogenous transfer is more straightforward because the database can be shifted to the destination target without additional manipulation. However, if heterogeneous migration is involved, database architects would have to clean and transform the data before transferring it.
Migration type
Database migration can be of the following types.
- A partial migration moves only a part of the entire dataset from the original database to the cloud.
- A complete database migration transfers all data from the source to the target platform.
After either transfer, the source database is disconnected from the application. Another type of migration is ongoing migration or data replication. It copies data from the source to the destination database indefinitely.
What are the types of database migrations?
Before migrating, consider what type of migration would best suit your use case.
Offline migration
Offline migration is suitable for shifting non-critical databases that can tolerate significant downtime. When you perform offline migration, you must stop applications from writing to the database for a certain period. The time gap allows database architects to lift and shift the database to the cloud and modify the application. Usually, businesses choose offline migration when modernizing legacy systems not supported by cloud migration tools. Likewise, offline database migration is ideal for test and development projects where timely data updates are not critical.
Let’s take the example of migrating an on-premise Oracle database to Amazon RDS for Oracle. Despite being hosted on different infrastructures, both databases share similar engines. Therefore, the data migration process is straightforward, as we describe below.
Step 1 — Establish network connectivity
Connect the on-premise database to the AWS cloud using a secure network. You can set up a virtual private network or use AWS Direct Connect to enable on-premise to cloud connectivity. With AWS Direct Connect, you can directly connect databases without passing them through the public internet.
Step 2 — Stop application write
To prevent unexpected changes to the target database, stop applications from writing to the source database.
Step 3 — Transfer data from on-prem to cloud
Next, deploy native data transfer tools on a cloud server. To reduce costs, we recommend setting up the tools on Amazon EC2. This way, you pay only for the resources used to run the native database tools. Then, the tools are used to extract the data from the source database and restore them to the cloud database.
Step 4 — Perform database switchover
After transferring the entire database to the cloud, validate it to ensure data consistency. Then, redirect all data queries from the application to the new database.
Production workloads migration
Migrating production workload requires a different approach than shifting non-critical databases. Production workloads and their data sources cannot be taken offline. Users must be able to access services as usual, even when data transfer is in progress. Therefore, ample planning and coordination are required to make migration possible without shutting down the application.
Architects apply homegrown scripts or third-party data replication tools to migrate databases for production workloads. The scripts or tools replicate data from the on-prem database to the cloud database. While data replication takes place, the on-prem database remains operational.
Challenges arise when software developers revise the script to accommodate new and ongoing data requirements. In addition, the third-party database tools they use may require updating for successful database migration.
Online database replication
Replicating a database involves writing scripts that set up database schemas in new database engines and then transferring data from the source to the target database. Schemas contain definitions to store and arrange data in a relatable database. Engineers can use third-party data replication tools to create and maintain multiple data copies across different servers. Both approaches are equally challenging because they involve considerable manual intervention, are prone to mistakes, and can be inefficient.
If you want a more efficient and secure way of replicating databases online, consider AWS Database Migration Service (DMS). It is a cloud service that automates database migration across various database engines. Instead of manually scripting the data transfer, you can get DMS to automatically scan the schemas of the original database and create new schemas for the target. This allows developers to focus on innovating applications instead of laborious database management.
Below, we share how to replicate your on-prem database to the cloud.
Step 1 — Inventory existing data
Analyze the databases currently hosted on-premises. To accelerate the process, use AWS DMS Fleet Advisor. This tool analyzes the existing database and analytics server fleet and identifies potential migration paths. It also compiles performance, schemas, usage patterns, and other metrics.
Step 2 — Shift the database to the cloud
Use AWS DMS to migrate the entire database from the on-prem data center to the cloud. In addition, you can turn on DMS’s change data capture (CDC) mode to enable near real-time replication. With CDC, DMS captures data changes as they happen to the original database and applies the changes to all replicated databases.
Database refactoring
In some cases, the database migration strategy involves refactoring or heterogeneous transfer. Businesses refactor databases when modernizing applications for several reasons, including reducing licensing costs, accessing cloud capabilities, and improving data quality. In a heterogeneous migration, information stored in the source database cannot be shifted as is. Data must be cleaned, transformed, and structured per new schemas that the target database supports.
Below are steps that help you refactor a database.
Step 1 — Audit the database
Analyze the on-prem database and identify the schemas used to store the data. Plan how the current data will be stored in the new database. Sometimes, database refactoring involves consolidating several databases into one or distributing data from a single database to multiple databases.
Step 2 — Convert the database schema
Convert the schema compiled from the source database into the required formats for the target database. You can use AWS DMS Schema Conversion to streamline the process.
Note: Database engineers might need to manually create or modify the new schemas, particularly for highly complex datasets.
Step 3 — Initiate database migration
Migrate data from on-prem to cloud database with AWS DMS. To ensure the cloud database captures all changes during migration, turn on the CDC mode. After transferring all data, redirect the application services to the cloud database.
How can AWS support your database migration needs?
Businesses migrate databases to the cloud to reduce infrastructural costs, improve service reliability, and take advantage of other cloud capabilities. Database migration can be as simple as porting an entire database as it is or rebuilding entire schemas to meet new data requirements. Regardless of migration complexities, ensuring data is shifted securely, accurately, and on time is essential. Explore AWS Database Migration Service for managed tools that allow near real-time switchover, database analysis, automated data transfer, and more.