Category: AWS Data Pipeline

AWS Data Pipeline helps you to reliably process and move data between compute and storage services running either on AWS on on-premises. The pipelines that you create with Data Pipeline’s graphical editor are scalable and fault tolerant, and can be scheduled to run at specific intervals. To learn more, read my launch post, The New […]

The AWS Data Pipeline lets you automate the movement and processing of any amount of data using data-driven workflows and built-in dependency checking. Today we are making the Data Pipeline more flexible and more useful with the addition of a new scheduling model that works at the level of an entire pipeline. This builds upon […]

Two of the most frequent feature requests for Amazon DynamoDB involve backup/restore and cross-Region data transfer. Today we are addressing both of these requests with the introduction of a pair of scalable tools (export and import) that you can use to move data between a DynamoDB table and an Amazon S3 bucket. The export and […]

As we often do, we launched AWS Data Pipeline in a single AWS Region (US East (Northern Virginia) to be precise). It is now available in the following Regions as well: US West (Oregon) EU (Ireland) Asia Pacific (Sydney) Asia Pacific (Tokyo) AWS Data Pipeline has always supported data flows between Regions. With today’s release, […]

AWS Data Pipeline (see my introductory blog post for more information) is a web service that helps you to integrate and process data across compute and storage services at specified intervals. You can transform and process data that is stored in the cloud or on-premises in a highly scalable fashion without having to worry about […]

If you use Amazon DynamoDB, you may already be using AWS Data Pipeline to make regular backups to Amazon S3, or to load backup data from S3 to DynamoDB. Today we are releasing a new feature that enables periodic copying of the data in a DynamoDB table to another table in the region of your […]

Update (May 2023) – AWS Data Pipeline service is in maintenance mode and no new features or region expansions are planned. To learn more and to find out how to migrate your existing workloads, please read Migrating workloads from AWS Data Pipeline. The AWS Data Pipeline is now ready for use and you can sign […]

Update (May 2023) – AWS Data Pipeline service is in maintenance mode and no new features or region expansions are planned. To learn more and to find out how to migrate your existing workloads, please read Migrating workloads from AWS Data Pipeline. Data. Information. Big Data. Business Intelligence. It’s all the rage these days. Companies […]