AWS Storage Blog

Category: Migration & Transfer Services

AWS DataSync Featured Image 2020

How London Stock Exchange Group migrated 30 PB of market data using AWS DataSync

London Stock Exchange Group (LSEG) has 30 PB of Tick History-PCAP data, which is ultra-high-quality global market data that is based on raw exchange data, timestamped to the nanosecond. An additional 60 TB is generated every day. LSEG sought to migrate their data from Wasabi cloud storage, LSEG was looking for a new solution to […]

AWS DataSync Featured Image 2020

Transferring data in Amazon S3 between AWS GovCloud (US) Regions and commercial AWS Regions using AWS DataSync

AWS users who need to comply with the most stringent US government security and compliance requirements operate their workloads in AWS GovCloud (US), which is architected as a separate partition providing network and identity isolation. A common use case for AWS GovCloud (US) users is to operate in both AWS GovCloud (US) Regions and commercial AWS Regions, such as […]

AWS DataSync

Transfer customer managed SSE-KMS encrypted objects across AWS accounts and Regions using AWS DataSync

Update (7/12/2024): Post updated to clarify that you must use a fully qualified Amazon Resource Name (ARN) when specifying your customer managed KMS keys. Some organizations have requirements to manage their own data encryption keys, both in general and during data transfer processes. In addition, when considering data transfer solutions (not just for encrypted data), […]

AWS Transfer Family Featured Image

Architecting secure and compliant managed file transfers with AWS Transfer Family SFTP connectors and PGP encryption

Users in industries such as financial services and healthcare regularly exchange files with their external business partners containing sensitive and regulated datasets, such as Personal Identifiable Information (PII) and financial records. These file transfers often happen over the Secure File Transfer Protocol (SFTP) and encrypting files using Pretty Good Privacy (PGP) before transfer is often […]

AWS Transfer Family Featured Image

Using Amazon Cognito as an identity provider with AWS Transfer Family and Amazon S3

In highly regulated industries, securely exchanging files business-to-business is a crucial business practice. When building out a Managed File Transfer (MFT) environment, it is common to consider using a third-party identity solution for authenticating users. This approach offers simplicity for businesses that already use an identity service, allowing them to maintain identities for a variety […]

AWS Transfer Family Featured Image

Implementing least privilege access in an AWS Transfer Family workflow

Architecting secure data transfer workloads is critical for today’s businesses. Customers need to be sure that each end user can only access the minimally appropriate set of files and folders once authenticated to AWS Transfer Family. There are multiple Identity and Access Management (IAM) roles necessary when designing these authentication and access controls, and customers […]

AWS DataSync Featured Image 2020

Replicate objects Using AWS DataSync with Amazon S3 compatible storage on Snowball Edge

Users have successfully leveraged Amazon Web Services (AWS) Snow Family to transfer petabytes of data between on premises and AWS Regions since its launch in 2015 with the AWS Snowball device. Increasingly, users are not just migrating data with the AWS Snow Family but now are using AWS Snowball Edge Compute Optimized devices to host […]

AWS DataSync Featured Image 2020

Migrating Oracle Cloud Infrastructure Object Storage to Amazon S3 using AWS DataSync

Users face challenges in their digital transformation journey involving the migration of data across various platforms, on-premises file systems, and other cloud services. When using other cloud providers, scenarios arise where the seamless transfer of data becomes essential. Whether executing a one-time data transfer or integrating it into a scheduled workflow, minimizing business downtime is […]

AWS DataSync Featured Image 2020

Implementing AWS DataSync with hundreds of millions of objects

Moving large volumes of data across your hybrid cloud environments can seem like a daunting task, especially when dealing with a litany of requirements that arise when working within the technical limits of network, storage, compute, and operating system layers, both on-premises and in the cloud. Users face additional challenges when balancing their Recovery Time […]

AWS DataSync Featured Image 2020

Automate data synchronization between AWS Outposts racks and Amazon S3 with AWS DataSync

Many organizations generate large quantities of data locally, including digital imagery, sensor data, and more. Customers require local compute and storage to ingest and enable real-time predications based on their data, and often preprocess this data locally before transferring to the cloud to unlock additional business value such analysis, reporting, and archiving. Automating transfers to […]