AWS Database Blog

Australia Finance Group’s Journey to Cloud Databases: Migrating from Oracle Exadata to Amazon RDS

Australia Finance Group (AFG) is one of the largest mortgage aggregators in Australia, according to the Mortgage and Finance Association of Australia. AFG migrated their Oracle Siebel Customer Relationship Management (CRM) application to a managed database with Amazon RDS for Oracle. In this blog post, we describe their approach to migrating the database from Oracle Database 11.2 on Oracle Exadata on-premises, to Oracle Database 12.1 on Amazon RDS for Oracle. We also provide tips for successful migrations of large database applications to Amazon RDS.

Background
AFG’s applications were hosted out of two data centers, and more than 80 percent of IT expenditure at AFG was consumed by IT operational costs. AFG wanted to reduce operational costs and free resources for innovation by migrating these applications to the cloud and decommissioning the data centers.

At AFG, the application most critical to the business is the Oracle Siebel CRM. This application helps AFG coordinate communications with customers across multiple channels and lodge home-loan applications electronically. The Oracle platform is used by all 2,800 of AFG’s brokers and their support staff.

For the CRM database, AFG was looking for a solution to reduce operational costs and result in low management, maintenance, and administrative overheads. At the same time, AFG needed a solution that was highly available and scalable.

On-premises setup
In the on-premises setup, the Siebel CRM database (Oracle 11.2) was deployed on an Oracle Exadata X2-2 Quarter rack system. The Exadata Quarter rack had two database servers, each with two six-core Intel Xeon processors and 96 GB RAM, for a total of 24 CPU cores and 192 GB RAM. The Exadata quarter rack also had three Exadata Storage servers, with a total of 36 CPU cores for SQL processing and 1.1 TB of Exadata smart flash cache. The Siebel CRM database size was 1.2 TB.

For disaster recovery, the Siebel CRM database was replicated to another Oracle Exadata quarter rack in a secondary data center using Oracle Data Guard replication.

The following diagram shows this setup.

Oracle Data Gaurd Replication

Choosing between Oracle on Amazon EC2 and Amazon RDS for Oracle
Before migrating the Siebel CRM database to AWS, AFG had to decide between Oracle on Amazon EC2 and Amazon RDS for Oracle. They decided to deploy the Siebel CRM database on Amazon RDS for Oracle instead of Oracle on Amazon EC2 due to the following considerations:

  • Amazon RDS has a lower management and maintenance overhead because it is a managed service.
  • Amazon RDS has a lower administration overhead due to automated patching and simple backup and recovery process.
  • It is relatively easier to implement a Multi-AZ deployment using RDS, compared to a Multi-AZ deployment with Amazon EC2 and Oracle Data Guard.
  • You can easily snapshot an RDS instance to create a copy. This approach is useful if you want to clone your production database to replicate and troubleshoot application issues.
  • With Amazon RDS, you can easily scale up or scale down the compute and memory resource of your instance in a few minutes. This scaling comes in handy if you need to temporarily scale up the database instance while you identify and resolve application performance issues.

Multi-AZ deployment on AWS
The following diagram shows the final Siebel CRM deployment on AWS after migration.

The Siebel application servers and the Siebel database on Amazon RDS were deployed across multiple Availability Zones (AZs). Multi-AZ deployments provide enhanced availability and durability for Amazon RDS DB instances, making them a natural fit for production database workloads.

When you provision a Multi-AZ DB instance, Amazon RDS automatically creates a primary DB instance and synchronously replicates the data to a standby instance in a different AZ. In case of an infrastructure failure (for example, instance hardware failure, storage failure, or network disruption), Amazon RDS performs an automatic failover to the standby instance, which becomes the new primary. Because the endpoint for your DB instance remains the same after a failover, the Siebel servers can connect to the new primary instance and resume operations without any data loss.

To learn how to set up Amazon RDS for Oracle as the database backend of your Siebel CRM application, see Installing a Siebel Database on Oracle on Amazon RDS in the Amazon RDS documentation.

Migrating the database using Oracle Streams
The Siebel CRM database was 1.2 TB in size, and AFG wanted to minimize the downtime during migration. To do so, they took an Oracle Data Pump export of the on-premises database, copied it to Amazon S3 and imported it into the Amazon RDS instance. Then they configured Oracle Streams replication between the on-premises database and the Amazon RDS instance. They used Oracle Streams replication because Amazon RDS for Oracle does not currently support Oracle Data Guard replication, which may have been a more obvious choice.

Before cutover, AFG stopped the production application servers, waited for the remaining changes to be replicated to Amazon RDS, and shut down the on-premises database. Then the Siebel application was installed on Amazon EC2 instances and Siebel Enterprise was configured to use the database on Amazon RDS. Finally they cut over by repointed the DNS to the Siebel application on AWS.

The following diagram shows replication of database changes from the on-premises setup to Amazon RDS using Oracle streams.

Oracle Streams replicates information from one Oracle database to another and can capture both data definition language (DDL) and data manipulation language (DML) changes. The capture process in the source database captures information from the database redo log, converts it into logical change records (LCRs), and stores these in a source queue. The LCRs are propagated from the source queue to a destination queue in the destination database. The apply process takes the LCRs from the destination queue and applies them to the destination database. You can configure Oracle Streams to replicate changes for tables, schemas, tablespaces, or databases. The source database must be in ARCHIVELOG mode and supplemental logging must be enabled.

Performance tuning and sizing the Amazon RDS instance
The simplest way of sizing your Amazon RDS instance might be to find the CPU cores, RAM, disk size, and IOPS used on-premises and select an appropriate Amazon RDS instance based on that. Although this method is easy, it’s not optimal. Tuning and right-sizing your database before migration can help reduce AWS infrastructure costs and also database licensing costs.

In this case, AFG was migrating from an Oracle Exadata machine on-premises to Oracle Enterprise Edition on Amazon RDS. AFG had to identify some of the queries benefiting from Exadata capabilities such as Smart Flash Cache and Smart Scan and tune or optimize them on Amazon RDS. The DBA team and the Siebel team at AFG worked together to identify and tune the database before migration to AWS. The teams took the following steps:

  • They set up a performance test environment on Amazon RDS and imported the full production data.
  • They used a SQL record and playback tool from Ponder Proserve to identify differences in query execution time. They recorded a day’s worth of transactions from the production database on-premises and replayed these transactions on the Amazon RDS instance. To find which queries were running slower, the run time of each of these queries on Amazon RDS was recorded and compared to the run time for each on the production database.
  • To solve issues of slowness, the teams did the following:
    • Individual queries performing poorly were identified and tuned by adding or adjusting the indexes, or by making changes to the Siebel application.
    • The teams migrated from Oracle 11.2 on Exadata to Oracle 12.1 on Amazon RDS for Oracle. They had to adjust the optimizer_features_enable and optimizer_index_cost_adj database parameters to optimize the execution plans and to favor Index scans instead of full table scans.
    • To identify the optimal Amazon RDS instance size, the teams experimented with different Amazon RDS instance sizes and disk options while playing back the recorded queries.

AWS Infrastructure Event Management
AFG took advantage of AWS Infrastructure Event Management (IEM) to help ensure a smooth migration and mitigate risks. AWS IEM assisted AFG in reviewing and fine-tuning the migration plan before cutover and support was available during the migration to help them in resolving issues that occurred post migration.

Summary and lessons learned

This blog post outlines the steps AFG took to migrate their mission-critical Siebel CRM database running on Oracle Exadata to Amazon RDS for Oracle with minimal downtime. While migrating large databases to Amazon RDS, we recommend keeping the following points in mind:

  • Assess your current environment and identify the infrastructure resources available to your database (CPU cores, RAM, disk size, disk IOPS, and so on). While estimating the compute requirements on AWS, factor in the difference in processor performance levels and clock speeds on-premises compared to on For example, the R4 instances on AWS use the Intel Xeon E5-2686 v4 (Broadwell) processors, which are faster than, say, a 5-year-old Intel Xeon processor model on-premises. Also factor in the utilization levels of the hardware on-premises while estimating your compute, storage, and memory requirements on RDS.We recommend using comparative sizing based on existing Oracle Automatic Workload Repository (AWR) reports to size your database. When you are migrating from Exadata, you should also considering the savings due to smart scans and storage indexes while calculating the throughput requirements. For more details on sizing on AWS, see the whitepaper Demystifying the Number of vCPUs for Optimal Workload Performance on the AWS website.
  • Set up a performance test environment on Amazon RDS, and import the full data from on- You can use Oracle Real Application Testing or similar tools to capture transactions from the production database and replay them in the performance test environment. This approach helps identify poorly performing queries that require tuning.
  • Amazon RDS should be your default choice while migrating to AWS. You can consider deploying Oracle on Amazon EC2 when you cannot use Amazon RDS for Oracle due to certain restrictions like the following:
    • The database size is close to or greater than the storage limit for Amazon RDS for Oracle.
    • The CPU, memory, or IOPS required is greater than what is available on the largest RDS instance.
    • You are using an application that requires Sys or System user access.
  • Migrating large databases to Amazon RDS within a small downtime window requires careful planning:
    • We recommend that you take a point-in-time export of your database, transfer it to AWS, import it into Amazon RDS, and then apply the delta changes from on-premises.
    • You can use AWS Direct Connect or AWS Snowball to transfer the export dump to AWS.
    • You can use AWS Database Migration Service (AWS DMS) to apply the delta changes and sync the on-premises database with the Amazon RDS instance. You can also use Oracle Streams replication as described in this blog post, but setting it up is more complicated compared to AWS DMS. When AFG migrated to AWS, AWS DMS was not available, so they chose to use Oracle Streams.

About the Authors

Ashok Shanmuga Sundaram is a partner solutions architect with the Global System Integrator (GSI) team at Amazon Web Services. He works with the GSIs to provide guidance on enterprise cloud adoption, migration and strategy.