Category: Technical How-to

Today, Amazon S3 hosts over 100 trillion objects and regularly peaks at tens of millions of requests per second. As the set of use cases for S3 has expanded, our customers have asked us for new ways to regulate access to their mission-critical buckets and objects. For example, with a data lake hosted on Amazon […]

There are many factors to consider when migrating data from on premises to the cloud, including speed, efficiency, network bandwidth and cost. A common challenge many organizations face is choosing the right utility to copy large amounts of data from on premises to an Amazon S3 bucket. I often see cases in which customers start with a free […]

AWS customers sometimes host AWS Transfer Family endpoints in network address translation (NAT) architectures. One common reason to host the AWS Transfer endpoint behind a NAT is to protect the server with a firewall offered by an AWS Marketplace partner. With the SFTP protocols, there are generally no major issues with using NAT architectures and […]

Many organizations use single sign-on (SSO) to ensure security and compliance, streamline access management, and improve their user experience. Some organizations require all corporate applications, workloads, and solutions to use SSO for authentication, including their disaster recovery (DR) solution. You can integrate CloudEndure Disaster Recovery, offered by AWS, into your identity provider’s (IdP) SSO using […]

UPDATE 9/8/2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details. Monitoring end-user activity and access to data is core to any modern data security strategy. As customers migrate workloads to the cloud, logging end-user accesses of customer data is a key component of internal security policies and is required to meet […]

Update (6/14/2022): The “Copying objects across accounts” section has been updated to reflect the new Amazon S3 Object Ownership feature, an S3 bucket-level setting that you can use to disable access control lists (ACLs) and take ownership of every object in your bucket. You no longer need to configure your cross-account AWS DataSync task to […]

A challenge for many enterprises with data at the scale of petabytes is managing and taking actions on their data to migrate, improve efficiency, and drive down costs through automation. Amazon S3 buckets can hold billions of objects and exabytes of data, letting you build your applications with the ability to grow and scale as […]

Object storage gives you virtually unlimited scale, which helps you grow your business without being concerned with managing the infrastructure to support your data. Managing millions to billions of objects in Amazon S3 can be difficult, inefficient, and time consuming if you don’t take steps to automate the management of this data at scale. Data […]

UPDATE (2/10/2022): Amazon S3 Batch Replication launched on 2/8/2022, allowing you to replicate existing S3 objects and synchronize your S3 buckets. See the S3 User Guide for additional details. The need for data synchronization in Amazon S3 comes up in a number of scenarios for customers – enabling a new geographic region for end users, […]

Earlier this year, AWS Transfer Family launched support for Amazon EFS. With this update, you can grant your end users access to files stored in Amazon EFS file systems using any of the protocols supported by Transfer Family. However, if you use Okta as the identity provider, there are additional considerations during setup. For example, […]