AWS Storage Blog
Category: Advanced (300)
Allowing external users to securely and directly upload files to Amazon S3
Organizations are often required to store files, images, and other digital assets in a repository. In many cases, the source of these files are partners or individuals who are not connected to internal systems and requires corporate authentication in order to upload the files. Customers traditionally use servers to handle file uploads, which can use […]
Using AWS DataSync to move data from Hadoop to Amazon S3
You want to leverage cloud scalability, increase cost efficiency by paying only for utilized storage, decouple big data storage from processing, and increase capabilities for data analytics and machine learning using AWS. But how do you move your Hadoop cluster? To accelerate this transition, AWS DataSync recently launched support for moving data between Hadoop Distributed […]
Restoring on-premises applications to AWS from Amazon EBS Snapshots created by EBS direct APIs
Incremental, point-in-time copies of data can be a secure and cost effective tool anchoring disaster recovery, data migration, and compliance solutions. Amazon EBS Snapshots are how EBS customers leverage point-in-time copies of their data stored on AWS, and you can use Snapshots on premises too. In December 2019, AWS introduced Amazon EBS direct APIs, providing […]
Simplify data migrations using an AWS DataSync agent on Linux KVM Hypervisor
UPDATE (1/19/2023): Some readers who followed the steps in this blog post to deploy an AWS DataSync agent on the KVM platform ran into issues, either because the hypervisor host does not support virtualization or it is not enabled on the platform. Therefore, I have added the steps to verify whether the hypervisor host supports […]
Running WordPress on Amazon EKS with Amazon EFS Intelligent-tiering
A large percentage of websites today rely on Content Management Systems (CMS) which provide content creators, who may have little to no experience in web development, with the ability to easily publish their content to a website for distribution to their end users. By far, the most popular CMS platform today is WordPress. More developers […]
Monitor Amazon S3 activity using S3 server access logs and Pandas in Python
Monitoring and controlling access to data is often essential for security, cost optimization, and compliance. For these reasons, customers want to know what data of theirs is being accessed, when it is being accessed, and who is accessing it. With more data to monitor, large amounts of data can make it more challenging to granularly […]
Point-in-time restore for Amazon S3 buckets
Enterprises store increasing quantities of object data for use cases like data lakes, document management systems, and media libraries. Performing point-in-time restores for large datasets can be challenging, as existing approaches with full-restore from backup are time consuming and expensive. Alternatively, restoring individual objects to previous versions is prone to errors and delays the restore […]
Mount Amazon EFS file systems cross-account from Amazon EKS
Many customers use multiple AWS accounts managed by AWS Organizations to create security and cost boundaries around business units, projects, or applications. AWS Organizations helps you centrally manage and govern your environment as you grow and scale your AWS resources. In some cases, an application in one AWS account must access data in another. As […]
Enabling user self-service key management with AWS Transfer Family and AWS Lambda
Customers who use the AWS Transfer Family service are typically exchanging files with their business partners who provide them with SSH public keys. In a large-scale deployment of the AWS Transfer Family service, public key management eventually becomes a time-consuming task to refresh expired keys and rotate keys for security. When using custom identity providers (custom IdP), […]
Enabling multiprotocol workloads with Amazon FSx for NetApp ONTAP
In an enterprise environment, it is common that both Linux and Windows workloads require access to the same data. For example, electronic design automation (EDA) and media and entertainment workloads often involve Windows users accessing data produced by Linux compute instances. Concurrent access to this data requires multiprotocol access: NFS for Linux workloads and SMB […]