AWS Storage Blog

Category: Expert (400)

AWS DataSync Featured Image 2020

Event-driven data transfer to container-shared storage on AWS

Businesses using data lake solutions built on Amazon S3 often want their data science teams to have access to that same data for machine learning or analytics projects deployed on tools like RStudio Server and Shiny. To do so, they can easily deploy these tools in the cloud using Amazon ECS or Amazon EKS serverless containers with AWS Fargate, and can access […]

Read More

Optimizing storage management with Amazon FSx for Lustre storage quotas

Data-intensive use cases across industries are getting larger each year. Use cases that require multi-user file systems, such as user shares for data science or computational engineering, can benefit from cloud solutions to help manage file storage consumption. In this blog post, I walk you through the Amazon FSx for Lustre storage quota feature, discussing […]

Read More
AWS Storage Gateway Featured Image

Implementing a backup solution with AWS Storage Gateway

Backups are the insurance policy for our data. We hope to never use them, but if the time comes when we need them, they had better be there for us. At a high level, there are two different variations: local and offsite. A local backup can be as simple as copying data to another physical […]

Read More

Clustered storage simplified: GFS2 on Amazon EBS Multi-Attach enabled volumes

One of the design patterns for high availability of mission critical applications is the use of shared storage. This architectural pattern enables you to access storage from multiple hosts simultaneously, making your applications resilient to node failures. Customers with demanding transaction processing systems, analytics workloads, or high performance computing clusters need highly available, high-performance storage […]

Read More

Persistent storage for container logging using Fluent Bit and Amazon EFS

UPDATE 9/8/2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details. Logging is a powerful debugging mechanism for developers and operations teams when they must troubleshoot issues. Containerized applications write logs to standard output, which is redirected to local ephemeral storage, by default. These logs are lost when the container is terminated […]

Read More

Automating shadow copies configuration on Amazon FSx for Windows File Server

We often hear from customers that they love the self-service, fine-grained file restore capability of the shadow copies feature on Amazon FSx for Windows File Server Filer Server (Amazon FSx). Our customers also relay to us the convenience of being able to schedule shadow copies on file systems using PowerShell commands. However, enabling Windows shadow […]

Read More

Instant data sharing with Multi-Attach enabled Amazon EBS volumes

Mission-critical applications relying on highly performant and low latency access to data (such as business intelligence platforms) require resilient and highly available (HA) architecture. Implementing active-active and active-standby architectures with shared storage in cloud environments is now possible with introduction of EBS Multi-Attach enabled EBS volumes. Multi-Attach, coupled with Cluster Volume Manager (CVM), offers functionality […]

Read More

Using Amazon FSx for Windows File Server with Azure ADDS domain

Customers are using Amazon FSx for Windows File Server for a variety of use cases that need shared file storage that is accessible over the industry-standard Server Message Block (SMB) protocol, and that is integrated with Microsoft Active Directory. You can use Amazon FSx with any Microsoft Active Directory that is hosting your user identities, […]

Read More
Amazon S3

Auditing Amazon S3 encryption methods for object uploads in real time

Encryption of data at rest is increasingly necessary for maintaining compliance and providing another layer of protection for data against unauthorized access. Amazon S3 offers multiple methods for server-side encryption (SSE) of new objects in your bucket. As not all encryption options are equal, customers often ask how to track and control the method of […]

Read More
AWS Transfer Family Featured Image

Using Okta with multi-factor authentication for AWS Transfer for SFTP

Enterprises often use SFTP to provide third parties like vendors, partners, or offsite laboratories access to their data lakes for things like uploads, downloads, or distributing data exports to clients. AWS Transfer Family provides a fully managed SFTP (now expanded to offer FTPS and FTP) service for Amazon S3. The service frees you from managing […]

Read More