How do I create a validation only AWS DMS task?

Short description

AWS DMS allows you to create validation only tasks using either the AWS DMS console or the AWS Command Line Interface (AWS CLI). You can use validation only tasks to validate your data without performing any migration or data replication. When you use validation only tasks, there is no overhead on your existing migration task because the validation itself is decoupled from migration.

There are two types of validation only tasks: Full load validation only tasks, and change data capture (CDC) validation only tasks.

• Full load validation only tasks complete much faster than their CDC equivalent when many failures are reported. But, in full load mode, changes to the source or target endpoint are reported as failures, which can be a disadvantage.
• CDC validation only tasks delay validation based on average latency. They then retry failures multiple times before reporting them. If the majority of the data comparisons result in failures, then a CDC validation only task is very slow, which is a potential disadvantage.

For more information on how you can use validation only tasks, see the Validation only use cases section of Validation only tasks.

Resolution

Note: If you receive errors when running AWS CLI commands, make sure that you’re using the most recent version of the AWS CLI.

Create a validation only task using the AWS DMS console

1.    Open the AWS DMS console, and then from the navigation pane, choose Database migration tasks.

2.    Choose Create task, and then under Task configuration, enter the details for your task.

3.    Under Migration type, choose Replicate data changes only.

4.    Under Task settings, select the JSON editor, and then change these settings:

"EnableValidation": true,
"ValidationOnly": true,

5.    For Migration Type, choose one of these options:

• For a full load validation only task, choose Migrate existing data.
• For a CDC validation only task, choose Replicate data changes only.

6.    Under Migration task startup configuration, choose Manually later. This allows you to verify the task settings before starting the task.
Note: The default TargetTablePrepMode is set to DO_NOTHING. If TargetTablePrepMode has been modified, set the TargetTablePrepMode to DO_NOTHING.

7.    Choose Create task.

Create validation only task using AWS CLI

1.    For Linux and Windows environments, run the create-replication-task command to create a validation only task. You can also specify a cdc-start-time, which might be useful if you need to start validation from a specific timestamp. See these examples:

Linux:

aws dms create-replication-task --replication-task-identifier validation-only-task --replication-task-settings '{"FullLoadSettings":{"TargetTablePrepMode":"DO_NOTHING"},"ValidationSettings":{"EnableValidation":true,"ValidationOnly":true}}' --replication-instance-arn
arn:aws:dms:us-east-1:xxxxxxxxxxx:rep:ABCDEFGH12346 --source-endpoint-arn arn:aws:dms:us-east-1:xxxxxxxxxxxx:endpoint:KSXGO6KATGOXBDZXKRV3QNIZV4 --target-endpoint-arn arn:aws:dms:us-east-1:xxxxxxxxxxxxxxx:endpoint:7SIYPBZTE2X3CZ7FPN7KKOAV6Q --migration-type
cdc --cdc-start-time "2022-06-08T 00:12:12" --table-mappings file://Table-mappings.json

Windows:

aws dms create-replication-task --replication-task-identifier validation-only-task --replication-task-settings '{"FullLoadSettings":{"TargetTablePrepMode":"DO_NOTHING"},"ValidationSettings":{"EnableValidation":true,"ValidationOnly":true}}' --replication-instance-arn

2.    Open the AWS DMS console, and then from the navigation pane, choose Database migration tasks.

3.    Confirm that the task that you created with the AWS CLI is created.

4.    From the Overview details section, expand the Task settings (JSON), and then confirm that these settings are in place:

"EnableValidation": true,
"ValidationOnly": true,
"TargetTablePrepMode": "DO_NOTHING",

These examples create a CDC validation only task. Use the same settings for a full load validation only task, but change the --migration-type to full-load when you run the create-replication-task command.

---

Related information

Task settings example

Limitations