How do I set up detailed debug logging for my AWS DMS task?

Last updated: 2021-02-10

What types of logging are available for AWS Database Migration Service (AWS DMS) tasks? How do I enable detailed debug logging for an AWS DMS task?

Resolution

Note: If you receive errors when running AWS Command Line Interface (AWS CLI) commands, make sure that you’re using the most recent AWS CLI version.

Enable debugging for an AWS DMS task

Note: You can now enable detailed debugging using the AWS DMS console when you create or modify a DMS task. To do this, access the JSON editor in your Task settings. For LogComponents, change the Severity, as required.

To monitor your AWS DMS task, you can modify the logging task settings and specify which component activities and severity values are logged. The following example uses the most detailed level of debug logging: LOGGER_SEVERITY_DETAILED_DEBUG. Detailed debug logging requires a large amount of storage space. If you are using ongoing replication, also known as change data capture (CDC), the log entries might consume the rest of your storage space.

1.    Open the AWS DMS console, and then choose Database migration tasks in the navigation pane.

2.    Choose your task.

3.    From the Overview details section, expand the Task Settings (JSON) section, and then copy and paste the Logging section of task settings into a JSON editor.

Note: If your AWS DMS task is running, you can modify only the Logging task setting. If you want to modify any other task setting, stop the task before modifying it.

4.    Edit the Severity setting of each log component that you want to enable detailed debug logging on by replacing LOGGER_SEVERITY_DEFAULT (or any other setting) with LOGGER_SEVERITY_DETAILED_DEBUG. The SOURCE_UNLOAD and TARGET_LOAD log components are used for full load tasks only. SOURCE_CAPTURE and TARGET_APPLY are used for ongoing replication (CDC) tasks.

Example

{
    "Logging": {
        "EnableLogging": true,
        "LogComponents": [{
                "Id": "SOURCE_UNLOAD",
                "Severity": "LOGGER_SEVERITY_DEFAULT"
            },
            {
                "Id": "TARGET_LOAD",
                "Severity": "LOGGER_SEVERITY_DEBUG"
            },
            {
                "Id": "SOURCE_CAPTURE",
                "Severity": "LOGGER_SEVERITY_DEFAULT"
            },
            {
                "Id": "TARGET_APPLY",
                "Severity": "LOGGER_SEVERITY_DETAILED_DEBUG"
            },
            {
                "Id": "TASK_MANAGER",
                "Severity": "LOGGER_SEVERITY_DEFAULT"
            }
        ],
        "CloudWatchLogGroup": "dms-tasks-xxxx",
        "CloudWatchLogStream": "dms-task-xxxxx"
    }
}

5.    Save the JSON file, and note the Task ARN and file path for use later.

6.    Install and configure the latest version of the AWS CLI.

7.    Run the modify-replication-task AWS CLI command, using your Task ARN and the path of the file that you saved in step five.

Linux example

aws dms modify-replication-task --replication-task-arn arn:aws:dms:us-east-1:123456789012:task:2PVREMWNPGYJCVU2IBPTOYTIV4 --replication-task-settings file://your-task-settings.json

Windows example

aws dms modify-replication-task --replication-task-arn arn:aws:dms:us-east-1:123456789012:task:2PVREMWNPGYJCVU2IBPTOYTIV4 --replication-task-settings file://"c:\temp\your-task-settings.json"

Note: To find your Task ARN, open the AWS DMS console, and choose Database migration tasks from the navigation pane. The Task ARN appears in the Overview details section. For more information, see Constructing an Amazon Resource Name (ARN) for AWS DMS.

8.    Open the AWS DMS console, and choose Database migration tasks from the navigation pane.

9.    Confirm that the task status is Modifying.

10.    From the Overview details section for your task, under Task Settings, confirm that the Severity setting is LOGGER_SEVERITY_DETAILED_DEBUG for each log component that you modified.

The next time that you run this task, you'll get detailed debug logs.

Modify an existing task to enable debugging

You can also modify an existing task to enable detail debugging.

1.    Modify the AWS DMS task.

2.    Confirm that the Enable CloudWatch logs checkbox is selected.

3.    Change the Severity of the five standard loggers in the drop-down list, as required.

Note: You can enable detailed debugging for all AWS DMS loggers, not only the five standard ones available in the AWS DMS console. To do this, Modify the task then from the Task settings, in the JSON editor, for LogComponents, change the Severity as required.