How do I modify the error handling task settings for an AWS DMS task?

Last updated: 2019-09-27

What are the error handling task settings for AWS Database Migration Service (AWS DMS)? How do I modify error handling task settings for my replication task?

Short Description

Use error handling settings to change the behavior of a replication task during change data capture (CDC) when it encounters an error. Error handling settings can manage errors at the data level, table level, based on the data manipulation language (DML), or based on environmental errors. Environmental errors include connectivity issues between the replication instance and the source or target, task failures due to resource contention on the replication instance, source/target not available etc. Because environmental issues can be intermittent, AWS DMS reserves capability for automatic resumes, in case of a failure.

See the following examples of common error handling task settings and their use cases:

  • TableErrorPolicy defines the action that AWS DMS takes when there is an error related to general data or metadata for a specific table. The default value for this settings is SUSPEND_TABLE. For example, if you migrate 10 tables from a source to a target using AWS DMS, and one of the tables has a metadata error, then AWS DMS suspends the table with the error. Then, the task continues migrating the remaining nine tables. Modify this setting to use STOP_TASK to stop the task if there are errors.
  • DataTruncationErrorPolicy defines the action that AWS DMS takes when data is truncated during migration. The default value for this setting is LOG_ERROR, which means that the error is logged in the task logs and the table migration continues. You can modify this setting to use IGNORE_RECORD, LOG_ERROR, SUSPEND_TABLE, and STOP_TASK. The difference between IGNORE_RECORD and LOG_ERROR is that IGNORE_RECORD increments DataErrorEscalationCount. This is the maximum number of errors that can occur in the data for a table or task before DataErrorEscalationPolicy is triggered.
  • RecoverableErrorCount defines the maximum number of attempts that AWS DMS makes to resume a task automatically if an environmental error occurs. The default value of this setting is -1, which allows AWS DMS to attempt to restart the task indefinitely. Set this to 0 if you never want AWS DMS to attempt to restart a task.

For a complete list of the available settings, see Error Handling Task Settings.

Resolution

Modify error handling task settings using the AWS Command Line Interface (AWS CLI):

1.    Open the AWS DMS console.

2.    Choose Database migration tasks from the navigation pane.

3.    If the task that you want to modify is running, select the task, choose Actions, and then choose Stop.

4.    After the task is stopped, choose the name of your task.

5.    From the Overview details section, expand the Task settings (JSON) section.

6.    Copy and paste the task settings into a JSON editor.

7.    Edit the settings under ErrorBehavior. The following example settings set FailOnTransactionConsistencyBreached to true and RecoverableErrorCount to 10.

"ErrorBehavior": {
        "DataErrorPolicy": "LOG_ERROR",
        "DataTruncationErrorPolicy": "LOG_ERROR",
        "DataErrorEscalationPolicy": "SUSPEND_TABLE",
        "DataErrorEscalationCount": 0,
        "TableErrorPolicy": "SUSPEND_TABLE",
        "TableErrorEscalationPolicy": "STOP_TASK",
        "TableErrorEscalationCount": 0,
        "RecoverableErrorCount": 10,
        "RecoverableErrorInterval": 5,
        "RecoverableErrorThrottling": true,
        "RecoverableErrorThrottlingMax": 1800,
        "ApplyErrorDeletePolicy": "IGNORE_RECORD",
        "ApplyErrorInsertPolicy": "LOG_ERROR",
        "ApplyErrorUpdatePolicy": "LOG_ERROR",
        "ApplyErrorEscalationPolicy": "LOG_ERROR",
        "ApplyErrorEscalationCount": 0,
        "ApplyErrorFailOnTruncationDdl": false,
        "FullLoadIgnoreConflicts": true,
        "FailOnTransactionConsistencyBreached": true,  
        "FailOnNoTablesCaptured": false

8.    Save the JSON file, and note the Task ARN and file path to use later.

9.    Install and configure the latest version of the AWS CLI, if you haven't already.

10.    Run the modify-replication-task command with the Task ARN and the file path that you noted in step 8.

Linux example

aws dms modify-replication-task --replication-task-arn arn:aws:dms:us-east-1:112345679012:task:123eadf45gtJCVU2IBPzasetg5 --replication-task-settings file://your-task-settings.json

Windows example

aws dms modify-replication-task --replication-task-arn arn:aws:dms:us-east-1:112345679012:task:123eadf45gtJCVU2IBPzasetg5 --replication-task-settings file://"c:\temp\your-task-settings.json"

11.    Return to the AWS DMS console, and choose Database migration tasks from the navigation pane.

12.    Confirm that the task status is Modifying.

13.    Choose the name of your task.

14.    From the Overview details section for your task, expand Task Settings (JSON), and confirm that the ErrorBehavior settings are updated.


Did this article help you?

Anything we could improve?


Need more help?