How can I verify if my AWS DMS migration task is stuck or making progress?

Last updated: 2021-02-16

My AWS Database Migration Service (AWS DMS) task is stuck or it is not progressing. How do I troubleshoot why my task is not progressing?

Short description

Although AWS DMS tasks are rarely stuck, they can sometimes be slow to progress. Follow the steps in this article to check if the data from your DMS task is migrated from source to target.

Resolution

Check the status of your AWS DMS task

Use the following steps below to check the status of your DMS task.

  1. Log in to the AWS DMS console.
  2. From the navigation pane, choose the Database migration tasks.
  3. Review the status of your task. The status should be as follows:
    • During the full load phase, your task status should be Running.
    • During change data capture (CDC) phase or ongoing replication phase of a CDC-only task, your task status should be Replication ongoing.
    • During a full load and CDC, your task status should be Load complete, replication ongoing.

Monitor Amazon CloudWatch logs

Check the migration task by monitoring the Amazon CloudWatch logs.

  1. Log in to the AWS DMS console.
  2. From the navigation pane, choose Database migration tasks, and then select your task.
  3. Choose View CloudWatch logs. This redirects you to the AWS CloudWatch console, where you can monitor the logs for your task.

Note: When viewing your logs, choose Retry to refresh the logs and display the latest information with the timestamp. If you don’t see a new message in log after five minutes, proceed to the next step.

Refresh the table statistics of your DMS task

Refresh the Table statistics of your AWS DMS task.

  1. Log in to the AWS DMS console.
  2. From the navigation pane, choose Database migration tasks, and then select your task.
  3. Choose Table statistics.

During a full load, you can see an increase in the Full Load rows value and a change in the Load state value. During ongoing replication (CDC), you can see an increase in DMLs (Inserts, Updates, and Deletes) and DDLs.

If you have a test database that has little activity, you might not see any change in your task logs or in the table statistic counters.

Monitor CloudWatch metrics for rows unloading and applying

  1. Log in to the AWS DMS console.
  2. From the navigation pane, choose Database migration tasks, and then choose your task.
  3. Choose CloudWatch metrics. This action re-directs you to the CloudWatch Console.

During the full load phase of the DMS task, monitor the following metrics using CloudWatch:

  1. Choose Full load from the dropdown list in the CloudWatch console.
  2. Monitor the FullLoadThroughputRowsSource metric. This metric gives a detailed picture of the rate at which AWS DMS can unload source data into the replication instance during the full load phase.
  3. Monitor the FullLoadThroughputRowsTarget. This metric shows the rate at which the rows are leaving the replication instance to commit to the target.

During the CDC phase, monitor the following metrics using CloudWatch:

  1. Choose CDC from the drop down list in the CloudWatch console.
  2. Monitor the CDCThroughputRowsSource metric. This metric gives a detailed picture of the rate at which changes are captured from the source and moved to the replication instance during the CDC phase.
  3. Monitor the CDCThroughputRowsTarget. This metric shows the rate at which the changes are moved from the replication instance to the target.
  4. Monitor the CDCLatencySource metric. This metric shows the latency between source and replication instance in seconds.
  5. Monitor the CDCLatencyTarget metric. This metric shows the latency between replication instance and target in seconds.

You can also query the record count on the target at specific intervals to confirm that the data is being migrated to the target. This interval varies based on the load on the source, target, and replication instances, and the amount of data that is in a single record.

If you see no latency on your DMS task, and no new log message appears in the task log, then enable debug logging. Make sure to do this while the task is running, and then monitor the Amazon Cloud Watch logs of the DMS task.

Note: It's a best practice to enable debug logging for a short time only while you are actively troubleshooting the task. If you enable debug logging for longer, the replication instance disk space can fill up quickly and impact running tasks on the DB instance.


Did this article help?


Do you need billing or technical support?