AWS Storage Blog

Tag: Amazon FSx for Lustre

Choosing the right storage for cloud native CI/CD on Amazon Elastic Kubernetes Service

Building and testing software is a resource-intensive operation that usually involves a fleet of very powerful servers waiting in the wings for build jobs. With the rise of cloud native continuous integration/continuous development (CI/CD) systems on Kubernetes (i.e., Tekton, Jenkins X), we’re seeing a shift from the large (and often over-provisioned) static fleet of build […]

Best practices for monitoring Amazon FSx for Lustre clients and file systems

Lustre is a high-performance parallel file system commonly used in workloads requiring throughput up to hundreds of GB/s and sub-millisecond per-operation latencies, such as machine learning (ML), high performance computing (HPC), video processing, and financial modelling. Amazon FSx for Lustre provides fully managed shared storage with the scalability and performance of the popular Lustre file […]

Accelerating GPT large language model training with AWS services

GPT, or Generative Pre-trained Transformer, is a language model that has shown remarkable progress in various vertical industries. This technology has been used to generate human-like text in fields such as finance, healthcare, legal, marketing, and many others. In finance, GPT is being used to analyze financial data, generate reports, and assist with decision-making. In […]

Amazon FSx for Lustre

Protecting your high-performance file systems with Amazon FSx for Lustre

As companies shift high-performance workloads toward cloud solutions, data storage and data protection go side-by-side. Many companies have both internal and external security rules and regulations they must adhere to when storing their data. Amazon FSx for Lustre offers fully managed, scalable file systems for fast-processing workloads, providing secure, shared access to your users. In […]

High-performance cloud storage comes of age with Amazon FSx for Lustre

The rapid maturation of cloud tools for high-performance workloads in the past several years has made it possible for household names like T-Mobile, Toyota, and Rivian to move their high-performance analytics and AI/ML environments to the cloud. These are hugely data-intensive workflows that many companies five years ago believed would never be able to be […]

Optimizing SAS Grid on AWS with Amazon FSx for Lustre

Many customers run complex analytics and high performance SAS-based applications on premises using the SAS Grid platform to perform large-scale analytics. Customers with a strategy to move to open-source or cloud-native solutions will often consider refactoring applications to Python or R to lower their total cost of ownership, however refactoring these applications as part of […]

Automate file retrieval from S3 Glacier when using FSx for Lustre

Customers are running modern high performance computing (HPC) workloads on AWS to process large datasets cost-effectively and optimize performance. These workloads typically rely on a high-performance file system to provide low latency and high throughput access to the data. A performant file system is important so that the file storage can keep up with the […]

Spend less while increasing performance with Amazon FSx for Lustre data compression

Many customers associate a performance cost with data compression, but that’s not the case with Amazon FSx for Lustre. With FSx for Lustre, data compression reduces storage costs and increases aggregate file system throughput. As organizations continue to build applications faster than ever, the amount of data that organizations must store grows rapidly. Organizations need […]

AWS Backup 2021 blog image

Cross-Region and cross-account backups for Amazon FSx using AWS Backup

Today, AWS Backup is announcing cross-Region and cross-account backup (CRAB) support for Amazon FSx. AWS Backup offers a cost-effective, fully managed, policy-based service that further simplifies data protection at scale. AWS Backup also helps you support your regulatory compliance obligations and meets your business continuity goals. Amazon FSx makes it easy and cost-effective to run […]

Customer innovation during the COVID-19 pandemic

To call 2020 a challenging year would be a considerable understatement. The prolonged period of the pandemic has made a significant impact on our daily lives, creating a “new normal.” What used to be normal, connecting face-to-face to conduct business and socialize with friends and family, has turned even more digital. This was inevitable and […]