How do I issue a bulk upload to a DynamoDB table?
Last updated: 2020-09-01
How do I bulk upload data to an Amazon DynamoDB table?
Use one of the following options to upload data to DynamoDB in bulk.
Use the BatchWriteItem API operation to issue multiple PutItem calls simultaneously. You can also use parallel processes or threads in your code to issue multiple parallel BatchWriteItem API calls to make the data load faster.
AWS Data Pipeline
If the data is in Amazon Simple Storage Service (Amazon S3), you can use Data Pipeline to export to DynamoDB. Data Pipeline automates the process of creating an Amazon EMR cluster and exporting your data from Amazon S3 to DynamoDB in parallel BatchWriteItem requests. When you use Data Pipeline, you don't have to write the code for the parallel transfer. For more information, see Importing data from Amazon S3 to DynamoDB.
To upload data to DynamoDB with Amazon EMR and Apache Hive:
- Create an EMR cluster:
For Release, choose emr-5.30.0 or later.
For Applications, choose an option that includes Hive.
- Create an external Hive table that points to the Amazon S3 location for your data.
- Create another external Hive table, and point it to the DynamoDB table.
- Use the INSERT OVERWRITE command to write data from Amazon S3 to DynamoDB. For more information, see Importing data to DynamoDB.
AWS Database Migration Service (AWS DMS)
You can use AWS DMS to export data from a relational database to a DynamoDB table. For more information, see Using an Amazon DynamoDB database as a target for AWS Database Migration Service.