AWS DataSync News – S3 Storage Class Support and Much More

AWS DataSync helps you to move large amounts of data into or out of the AWS Cloud (read my post, New – AWS DataSync – Automated and Accelerated Data Transfer, to learn more). As I explained in my post DataSync is a great fit for your Migration, Upload & Process, and Backup / DR use cases. DataSync is a managed service, and can be used to do one-time or periodic transfers of any size.

Newest Features
We launched DataSync at AWS re:Invent 2018 and have been adding features to it ever since. Today I would like to give you a brief recap of some of the newest features, and also introduce a few new ones:

• S3 Storage Class Support
• SMB Support
• Additional Regions
• VPC Endpoint Support
• FIPS for US Endpoints
• File and Folder Filtering
• Embedded CloudWatch Metrics

Let’s take a look at each one…

S3 Storage Class Support
If you are transferring data to an Amazon S3 bucket, you now have control over the storage class that is used for the objects. You simply choose the class when you create a new location for use with DataSync:

You can choose from any of the S3 storage classes:

Objects stored in certain storage classes can incur additional charges for overwriting, deleting, or retrieving. To learn more, read Considerations When Working with S3 Storage Classes in DataSync.

SMB Support
Late last month we announced that AWS DataSync Can Now Transfer Data to and from SMB File Shares. SMB (Server Message Block) protocol is common in Windows-centric environments, and is also the preferred protocol for many file servers and network attached storage (NAS) devices. You can use filter patterns to control the files that are included in or excluded from the transfer, and you can use SMB file shares as the data transfer source or destination (Amazon S3 and Amazon EFS can also be used). You simply create a DataSync location that references your SMB server and share:

To learn more, read Creating a Location for SMB.

Additional Regions
AWS DataSync is now available in more locations. Earlier this year it became available in the AWS GovCloud (US-West) and Middle East (Bahrain) Regions.

VPC Endpoint Support
You can deploy AWS DataSync in a Virtual Private Cloud (VPC). If you do this, data transferred between the DataSync agent will not traverse the public internet:

The VPC endpoints for DataSync are powered by AWS PrivateLink; to learn more read AWS DataSync Now Supports Amazon VPC Endpoints and Using AWS DataSync in a Virtual Private Cloud.

FIPS for US Endpoints
In addition to support for VPC endpoints, we announced that AWS DataSync supports FIPS 140-2 Validated Endpoints in US Regions. The endpoints in these regions use a FIPS 140-2 validated cryptographic security module, making it easier for you to use DataSync for regulated workloads. You can use these endpoints by selecting them when you create your DataSync agent:

File and Folder Filtering
Earlier this year we added the ability to use file path and object key filters to exercise additional control over the data copied in a data transfer. To learn more, read about Excluding and including specific data in transfer tasks using AWS DataSync filters.

Embedded CloudWatch Metrics
Data transfer metrics are available in the Task Execution Details page so that you can track the progress of your transfer:

Other AWS DataSync Resources
Here are some resources to help you to learn more about AWS DataSync:

• Migrating Data to AWS: Understanding Your Options (AWS Online Tech Talk).
• Cloud Data Migration.
• AWS DataSync User Guide.
• AWS DataSync API Reference.
• AWS DataSync Quick Start In-cloud Transfer and Scheduler.
• Migrating storage with AWS DataSync (blog post).
• Migrating hundreds of TB of data to Amazon S3 with AWS DataSync (blog post).

— Jeff;