Why did my AWS DMS task fail when using Binary Reader for Amazon RDS Oracle?

Last updated: 2019-10-02

Why did my AWS Database Migration Service (AWS DMS) task fail when using Binary Reader for Amazon Relational Database Service (Amazon RDS) for Oracle?

Short Description

During the change data capture (CDC) phase, Oracle provides two methods to read the redo logs: Oracle LogMiner and Binary Reader. The Oracle LogMiner is a SQL interface that accesses the online and archived redo logs, and the Binary Reader is an AWS DMS feature that reads and parses the redo logs directly.

When using Binary Reader for migrations that have a high volume of changes, Binary Reader has less of an impact on the Oracle source database compared to using Oracle LogMiner because the archive logs are copied and then parsed on the replication instance. For migrations that have a high volume of changes, Binary Reader usually has better CDC performance compared to using Oracle LogMiner. Be sure to provision sufficient network bandwidth to avoid network performance bottlenecks.

If the prerequisites aren't met when using Binary Reader, you can receive two types of errors:

  • Permissions errors
  • Extra connection attribute errors

Resolution

Permissions errors

AWS DMS uses the Binary Reader by creating directories on the source database. As a result, the AWS DMS user account must have required privileges to access the source Oracle endpoint and create necessary directories. If AWS DMS doesn't have permissions, you see log entries similar to the following:

Messages
[SOURCE_CAPTURE ]E: OCI error 'ORA-00604: error occurred at recursive SQL level 1 ORA-20900: Invalid path used for directory: /rdsdbdata/log/arch ORA-06512: at "RDSADMIN.RDSADMIN", line 321 ORA-06512: at line 2' [1022307] (oradcdc_bfilectx.c:164)

To resolve these errors, be sure to use the Amazon RDS master user as the AWS DMS user. The directories are created automatically when an AWS DMS task starts running. If the directories aren't created, log in to the Oracle database using the master user. Then, run the following commands to test if these directories can be created:

SQL> exec rdsadmin.rdsadmin_master_util.create_archivelog_dir;
SQL> exec rdsadmin.rdsadmin_master_util.create_onlinelog_dir;

Check the results by querying the all_directories table:

SQL> select directory_path from all_directories where directory_name in ('ONLINELOG_DIR','ARCHIVELOG_DIR');
DIRECTORY_PATH
--------------------------------------------------------------------------------
/rdsdbdata/log/arch
/rdsdbdata/log/onlinelog

After you create the required directories ONLINELOG_DIR and ARCHIVELOG_DIR, restart your AWS DMS task.

Extra connection attribute errors

If you use the Binary Reader, but you're missing the necessary extra connection attributes for your Oracle source, you see the following log entry:

Messages
[TASK_MANAGER ]E: ORA-00604: error occurred at recursive SQL level 1 ORA-20900: Invalid path used for directory: awsdms_dir_test ORA-06512: at "RDSADMIN.RDSADMIN", line 321 ORA-06512: at line 2 ; Invalid RDS Oracle binary reader db settings, replacePathPrefix should be set to TRUE and usePathPrefix should be set to '/rdsdbdata/log/'; Invalid RDS Oracle binary reader db settings, useAlternateFolderForOnline should be set to TRUE; Invalid RDS Oracle binary reader db setting, oraclePathPrefix should not be empty; Invalid RDS Oracle binary reader db settings; Failed while preparing stream component 'st_0_4MGMBIOJCILNOU3UHICCDBCNFQ'.; Cannot initialize subtask; Stream component 'st_0_4MGMBIOJCILNOU3UHICCDBCNFQ' terminated [1020418] (replicationtask.c:2680)

To use the Binary Reader to capture change data for an Amazon RDS Oracle source, add the following extra connection attributes to the source endpoint:

useLogMinerReader=N;useBfile=Y;replacePathPrefix=true;usePathPrefix=/rdsdbdata/log/;useAlternateFolderForOnline=true;oraclePathPrefix=/rdsdbdata/db/ORCL_A/;accessAlternateDirectly=false