AWS Storage Blog

Protect on-premises VMware infrastructure with NetApp BlueXP Disaster Recovery, Amazon Elastic VMware Service, and Amazon FSx for NetApp ONTAP

Your VMware workloads contain critical data that drives business decisions and powers your operations. Maintaining the availability and resilience of your data is a top priority where potential disasters such as ransomware threats, catastrophic hardware failures, and natural calamities, can lead to costly downtime and data loss. To address these challenges, all businesses require strategic […]

Amazon S3 featured image 2023

Building multi-writer applications on Amazon S3 using native controls

Organizations managing data lakes often require additional infrastructure to support concurrent writes from multiple applications. Traditional approaches require external systems for coordination, adding infrastructure overhead, costs, and potential performance bottlenecks. Developers typically implement client-side locking mechanisms using databases or dedicated lock services, resulting in complex multi-step workflows. Amazon S3 offers capabilities to address these concurrent […]

Architecting scalable checkpoint storage for large-scale ML training on AWS

The exponential growth in size and complexity of foundation models (FMs) has created unprecedented infrastructure demands across compute, networking, and storage resources. Storage systems, in particular, face intense requirements for throughput, latency, and capacity. In machine learning (ML) model training, these storage demands are particularly evident in checkpointing—a critical reliability mechanism that periodically saves and […]

Amazon S3 Tables

Query Amazon S3 Tables from open source Trino using Apache Iceberg REST endpoint

Organizations are increasingly focused on addressing the growing challenge of managing and analyzing vast data volumes, while making sure that their data teams have timely access to this data to enable rapid insights and decision-making. Data analysts and scientists need self-service analytics capabilities to build and maintain data products, often involving complex transformations and frequent […]

SAN boot your Amazon EC2 enterprise environments from Amazon FSx for NetApp ONTAP

Traditionally, many enterprises and organizations with on-premises infrastructure have used boot-from-SAN (Storage Area Network) rather than using locally attached storage. Booting from SAN offers centralized management and backup of boot volumes, supports high availability through multipathing, and enables greater flexibility by allowing systems to boot from pre-configured OS images hosted on a shared storage array […]

Amazon S3 Tables

From raw to refined: building a data quality pipeline with AWS Glue and Amazon S3 Tables

Organizations often struggle to extract maximum value from their data lakes when running generative AI and analytics workloads due to data quality challenges. Although data lakes excel at storing massive amounts of raw, diverse data, they need robust governance and management practices to prevent common quality issues. Without proper data validation, cleansing processes, and ongoing […]

Enable item-level search and recovery for Amazon EC2 with AWS Backup

Users often use backups to help recover data after a disaster or security incident. However, what is often overlooked is the need to restore data due to an operational incident such as a data corruption event or deleted file. The ability to identify files and directories within a backup and restore them is an important […]

AWS DataSync Featured Image 2020

Automate data transfers and migrations with AWS DataSync and Terraform

In today’s data-driven world, organizations face the challenge of efficiently managing and consolidating vast amounts of information from diverse sources. Whether it’s for analytics, machine learning (ML), or other business-critical applications, the ability to seamlessly transfer and organize data is crucial. However, this process can be complex, time-consuming, and prone to errors when done manually. […]

Amazon FSx for NetApp ONTAP

Best practice configuration of Amazon FSx for NetApp ONTAP for Microsoft SQL Server workloads

When it comes to running enterprise workloads, Microsoft SQL Server sits at the heart of many of these solutions providing the data needed to drive important business decisions. However, what is often not considered is the storage service underpinning the delivery of that critical data, and with the right tools and management applied, how that […]

Amazon S3 featured image 2023

University of California Irvine backs up petabytes of research data to AWS

Editor’s note: AWS is not responsible for UCI’s public GitHub repo linked in this post, which has been provided so that interested parties can explore the solution described in this post in more detail. The University of California, Irvine (UCI) is a public land-grant research university with troves of research data stored on servers in […]