AWS Storage Blog

Category: Compute

Optimizing SAS Grid on AWS with Amazon FSx for Lustre

Many customers run complex analytics and high performance SAS-based applications on premises using the SAS Grid platform to perform large-scale analytics. Customers with a strategy to move to open-source or cloud-native solutions will often consider refactoring applications to Python or R to lower their total cost of ownership, however refactoring these applications as part of […]

Amazon EBS

AWS SAN in the Cloud: Millions of IOPs and tens of GB/s to any Amazon EC2 instance

Over the years, traditional on-premises applications have relied on ever more powerful (and expensive) storage arrays to scale application IO performance and provide a single server with millions of IOPs. Meanwhile, AWS was perfecting the art of linear, elastic horizontal scalability of storage performance in the cloud. Until now, it’s been difficult to meet the […]

AWS Transfer Family Featured Image

Enabling user self-service key management with AWS Transfer Family and AWS Lambda

Customers who use the AWS Transfer Family service are typically exchanging files with their business partners who provide them with SSH public keys. In a large-scale deployment of the AWS Transfer Family service, public key management eventually becomes a time-consuming task to refresh expired keys and rotate keys for security. When using custom identity providers (custom IdP), […]

Amazon S3 Object Lambda

Automate the configuration of Amazon S3 Object Lambda to process data as its retrieved

Customers often have multiple applications consuming information from a single dataset stored in Amazon S3, but each application requires different variations to meet their specific needs. For example, one application may require that data be in a redacted CSV format, while another application requires unredacted information. As another example, a stock image website may require […]

Amazon EBS

Optimizing SAP HANA’s persistence layer with Amazon EBS gp3 volumes

SAP HANA is an in-memory, relational database which enterprises rely on to run their mission critical Enterprise Resource Planning (ERP) systems and analytical applications. As SAP HANA is an in-memory database, you may wonder why the storage layer is relevant. A key point is that memory is volatile. When you write data to a HANA […]

Johnson & Johnson reduces analysis time by 35% with their data science platform using Amazon EFS

Johnson & Johnson data science and business experts collaborate to incorporate science into healthcare solutions, including medical device and diagnostic technologies, consumer healthcare products, and pharmaceuticals. Johnson & Johnson needed storage to share and perform analytics on their data science workbench for genomics neuroscience, R&D, and drug discovery. A massively scalable solution was required to […]

CloudEndure-AWS-Storage-Blog

Enhanced disaster recovery monitoring with CloudEndure and Amazon CloudWatch

Update (March 4, 2022): Updated Amazon CloudWatch events section to utilize Amazon EventBridge rules instead, allowing you to further customize your serverless event architecture. AWS is also deprecating the requests module in botocore to improve flexibility and performance, and added the ability to use Layers within AWS Lambda to continue using this module. Code has […]

Amazon S3 Batch Operations featured image

Copying objects greater than 5 GB with Amazon S3 Batch Operations

Update (3/4/2022): Added support for Glacier Instant Retrieval storage class. Update (4/19/2022): Included the copy destination prefix parameter in the Amazon CloudFormation template. Update (10/26/2022):Added performance guidance and best practices, and included template optimized for copying objects restored from archive to a different storage class. A large number of customers store their data in Amazon […]

Achieve higher database performance using Amazon EBS io2 Block Express volumes

I think it’s safe to say that every organization that runs its own IT environment has applications running on a database. Managing database servers is a critical job, as any performance hiccups on the database could affect many users and the workloads that they execute. Many enterprises see negative impact on the performance of their […]

AWS Backup 2021 blog image

How to retain resource tags in AWS Backup restore jobs

UPDATE (2/3/2022): Source code extended to support AWS Backup for Amazon S3. UPDATE (6/24/2022): Source code updated to read tags from recovery points inline with changes to AWS Backup boto3 APIs. Update (2/23/2023): Fixed bug in source code related to support for Amazon S3 backup restores. AWS customers use tags to track ownership of resources, […]