AWS Storage Blog

Category: Advanced (300)

Amazon S3 Archive Storage Classes

Identify cold objects for archiving to Amazon S3 Glacier storage classes

Update (02/13/2024): Consider Amazon S3 Lifecycle transition fees that are charged based on the total number of objects being transitioned, the destination storage class (listed on the Amazon S3 pricing page), as well as the additional metadata charges applied. You can use the S3 pricing calculator to estimate the total upfront and monthly costs by […]

AWS DataSync Featured Image 2020

Derive insights from AWS DataSync task reports using AWS Glue, Amazon Athena, and Amazon QuickSight

Update (9/22/2023): Step 6b updated to automatically detect and update the Amazon Athena table schema when crawler detects large data transfer values reported in bytes that would consume the table’s maximum integer value while storing data. As customers scale their migration of large datasets with millions of files across multiple data transfers, they are faced […]

Amazon S3 featured image - new

How Continental uses Mountpoint for Amazon S3 in autonomous driving development – accelerating simulation performance by 20%

Continental and AWS have been collaborating to create the Continental Automotive Edge (CAEdge) framework – a modular hardware and software environment that connects the vehicle to the cloud and features virtual workbenches. The platform includes a virtual workbench offering numerous options to develop, supply, and maintain software-intensive system functions. It supports a wide range of automotive […]

Amazon S3 featured image - new

Optimizing performance of Apache Spark workloads on Amazon S3

This blog covers performance metrics, optimizations, and configuration tuning specific to OSS Spark running on Amazon EKS. For customers using or considering Amazon EMR on EKS, refer to the service documentation to get started and this blog post for the latest performance benchmark. Performance is top of mind for customers running streaming, extract transform load […]

Amazon FSx for OpenZFS

Sharing data on Amazon FSx for OpenZFS across Linux and Windows clients

Many organizations need a high-performance shared file system that they can access simultaneously from Linux and Windows, despite different permission models across the platforms. For example, a media and entertainment enterprise may render workloads mutually on Linux and Windows clients. These customers may use mechanisms like “User Mapping” to make sure that their Windows clients can […]

Authorize NFS clients outside of AWS with AWS IAM Roles Anywhere

Securely storing and authorizing access to data in the cloud is a top priority. One challenge faced by organizations is developing a consistent authorization experience to grant access to data for hybrid architectures. Workloads running on AWS can access data stored on services like Amazon Elastic File System (Amazon EFS) using AWS Identity and Access […]

AWS Transfer Family Featured Image

Detect malware threats using AWS Transfer Family

Securely sharing files over SFTP, FTP, and FTPS is a staple within many business-to-business (B2B) workflows. Across industries, companies use file transfer to transmit inventory, invoice, and compliance information. It is critical for companies to make sure that shared files do not have any malicious content that could compromise their systems. Guaranteeing the shared files […]

Amazon FSx for OpenZFS

Configuring the auto-expansion of Amazon FSx for OpenZFS with Amazon CloudWatch and AWS Lambda

Today’s demanding workloads such as database, rendering farm, analytics and ML workloads, have increasingly demanding IO requirements. These workloads need a reliable storage infrastructure that provides sufficient storage capacity, IOPS, and throughput. As customers move more workloads to the cloud, they want to benefit from the agility and performance capabilities of the cloud as their […]

AWS DataSync Featured Image 2020

Migrate on-premises data to AWS for insightful visualizations

When migrating data from on premises, customers seek a data store that is scalable, durable, and cost effective. Equally as important, BI must support modern, interactive, and fast dashboards that can scale to tens of thousands of users seamlessly while providing the ability to create meaningful data visualizations for analysis. Visualization of on-premises business analytics […]

AWS DataSync Featured Image 2020

How to accelerate your data transfers with AWS DataSync scale out architectures

Do you ever wonder how you can keep up with incoming requests for increased storage capacity without having to expand data center footprint, increase utility spend, and continually handle hardware refresh cycles? Customers are looking to free up space from on-premises storage systems or other clouds, whether it is for existing archival datasets, transitioning their […]