How can I use Data Pipeline to copy tables from one database table to another?

Last updated: 2022-05-23

I want to use AWS Data Pipeline to copy a table to a different database. For example, I want to copy an Amazon Relational Database Service (Amazon RDS) for MySQL table or Amazon RDS for PostgreSQL table to Amazon Redshift.

Resolution

Copy an Amazon RDS for MySQL table to Amazon Redshift

Create a pipeline using one of the following Data Pipeline templates:

For more information about these templates, see Amazon RDS to Amazon Redshift templates.

Copy an Amazon RDS for PostgreSQL table to an Amazon Redshift table

Use the PostgresqlRdsToRedshift scripts from the AWSLabs GitHub repository. Before you begin, modify the postgresqlRDS-to-Redshift-definition.json file with the following:

  • Your database
  • Your Amazon Elastic Compute Cloud (Amazon EC2) instance
  • The path to your hosted shell script in Amazon Simple Storage Service (Amazon S3)

Note: Be sure to modify the JSON file according to your use case.

Copy an Amazon RDS for MySQL table to an Amazon RDS for PostgreSQL table

Use the MySqlRdsToPostgreSqlRds scripts from the AWSLabs GitHub repository. Before you begin, modify the mysqlRDS-psqlRDS-copy-using-shell-definition.json file with the following:

  • Your database
  • Your EC2 instance
  • The path to your hosted shell script in Amazon S3

Note: Be sure to modify the JSON file according to your use case.

Define a pipeline in JSON format

Did this article help?

Submit feedback

Do you need billing or technical support?

Contact AWS Support