Category: Expert (400)

Businesses using data lake solutions built on Amazon S3 often want their data science teams to have access to that same data for machine learning or analytics projects deployed on tools like RStudio Server and Shiny. To do so, they can easily deploy these tools in the cloud using Amazon ECS or Amazon EKS serverless containers with AWS Fargate, and can access […]

Data-intensive use cases across industries are getting larger each year. Use cases that require multi-user file systems, such as user shares for data science or computational engineering, can benefit from cloud solutions to help manage file storage consumption. In this blog post, I walk you through the Amazon FSx for Lustre storage quota feature, discussing […]

Backups are the insurance policy for our data. We hope to never use them, but if the time comes when we need them, they had better be there for us. At a high level, there are two different variations: local and offsite. A local backup can be as simple as copying data to another physical […]

One of the design patterns for high availability of mission critical applications is the use of shared storage. This architectural pattern enables you to access storage from multiple hosts simultaneously, making your applications resilient to node failures. Customers with demanding transaction processing systems, analytics workloads, or high performance computing clusters need highly available, high-performance storage […]

UPDATE 9/8/2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details. Logging is a powerful debugging mechanism for developers and operations teams when they must troubleshoot issues. Containerized applications write logs to standard output, which is redirected to local ephemeral storage, by default. These logs are lost when the container is terminated […]

We often hear from customers that they love the self-service, fine-grained file restore capability of the shadow copies feature on Amazon FSx for Windows File Server Filer Server (Amazon FSx). Our customers also relay to us the convenience of being able to schedule shadow copies on file systems using PowerShell commands. However, enabling Windows shadow […]

Mission-critical applications relying on highly performant and low latency access to data (such as business intelligence platforms) require resilient and highly available (HA) architecture. Implementing active-active and active-standby architectures with shared storage in cloud environments is now possible with introduction of EBS Multi-Attach enabled EBS volumes. Multi-Attach, coupled with Cluster Volume Manager (CVM), offers functionality […]

Customers are using Amazon FSx for Windows File Server for a variety of use cases that need shared file storage that is accessible over the industry-standard Server Message Block (SMB) protocol, and that is integrated with Microsoft Active Directory. You can use Amazon FSx with any Microsoft Active Directory that is hosting your user identities, […]

Encryption of data at rest is increasingly necessary for maintaining compliance and providing another layer of protection for data against unauthorized access. Amazon S3 offers multiple methods for server-side encryption (SSE) of new objects in your bucket. As not all encryption options are equal, customers often ask how to track and control the method of […]

Enterprises often use SFTP to provide third parties like vendors, partners, or offsite laboratories access to their data lakes for things like uploads, downloads, or distributing data exports to clients. AWS Transfer Family provides a fully managed SFTP (now expanded to offer FTPS and FTP) service for Amazon S3. The service frees you from managing […]