How can I use Data Pipeline to copy tables from one database table to another?

Last updated: 2020-05-22

I want to use AWS Data Pipeline to copy a table to a different database. For example, I want to copy an Amazon Relational Database Service (Amazon RDS) MySQL table or Amazon RDS PostgreSQL table to Amazon Redshift.

Resolution

Copy an Amazon RDS MySQL table to Amazon Redshift

Create a pipeline using one of the following Data Pipeline templates:

For more information about these templates, see Amazon RDS to Amazon Redshift Templates.

Copy an Amazon RDS PostgreSQL table to an Amazon Redshift table

Use the PostgresqlRdsToRedshift scripts from the AWSLabs GitHub repository. Before you begin, modify the postgresqlRDS-to-Redshift-definition.json file with the following:

  • Your database
  • Your Amazon Elastic Compute Cloud (Amazon EC2) instance
  • The path to your hosted shell script in Amazon Simple Storage Service (Amazon S3)

Copy an Amazon RDS MySQL table to an Amazon RDS PostgreSQL table

Use the MySqlRdsToPostgreSqlRds scripts from the AWSLabs GitHub repository. Before you begin, modify the mysqlRDS-psqlRDS-copy-using-shell-definition.json file with the following:

  • Your database
  • Your EC2 instance
  • The path to your hosted shell script in Amazon S3

Define a Pipeline in JSON Format

Did this article help you?

Yes
No

Anything we could improve?

Let us know

Need more help?

Contact AWS Support