AWS Storage Blog
Category: Intermediate (200)
Run queries up to 9x faster using Trino with Amazon S3 Select on Amazon EMR
Customers building data lakes continue to innovate in the ways that they store and access their data. For these customers, performance is critical, particularly when they are accessing large amounts of data. For example, data scientists, data analysts, and data engineers running queries from open source frameworks like Trino want to accelerate access to their […]
Restoring archived objects at scale from the Amazon S3 Glacier storage classes
Every organization around the world has archival data. There is a data archiving need not only for companies that have been around for a while, but also for digital native businesses. Workloads such as medical records, news media content, and manufacturing datasets, often store petabytes – or billions of objects indefinitely. The vast majority of […]
Using available Amazon EFS security features while migrating files with AWS DataSync
When performing an online data migration, an important requirement is often security in transit. When evaluating migration options, you should consider if the tools available can provide encryption of data in flight, to help prevent unauthorized users from reading your data. Amazon Elastic File System (EFS) provides the ability to encrypt data in transit by […]
How TMAP Mobility transferred 2.4 PB of Hadoop data using AWS DataSync
Launched in 2002, TMAP Mobility is Korea’s leading mobility platform, with 20 million registered users and 14 million monthly active users. TMAP provides navigation services based on a wide range of real-time traffic information and data. Previously, the Data Intelligence group at TMAP Mobility operated a mobility-data platform based on a Hadoop Distributed File System […]
Optimizing Amazon FSx for Windows File Server performance with new metrics
Storage administrators—whether they are managing user and departmental shares, high-availability databases like SQL Server, or business applications—need to understand how their workloads are performing. Understanding optimal throughput capacity, storage capacity, and storage type for your file systems helps ensure high performance and enables you to right-size your file storage to optimize cost. In addition to […]
How Amagi uses Amazon S3 Glacier Instant Retrieval to optimize media storage costs
Amagi is a global leader in SaaS technology providing end-to-end cloud-managed live and on-demand video infrastructure for TV and Over-the-Top (OTT) media services for over 700 playout and 2,000 ad-insertion channels across 40 countries. Amagi enables TV networks and content owners to launch, manage, distribute, and monetize live, linear, and on-demand channels across cable, OTT, […]
SAN: A million IOPs in AWS from Amazon FSx NetApp ONTAP
There are use cases where applications demand the highest IOPS and throughput to achieve strict service level requirements. You may have seen how it’s possible to leverage the immense horizontal storage scalability of Amazon Elastic Block Store (EBS) into concentrated vertically scaled storage performance in my recent blog “SAN in the Cloud: Millions of IOPs […]
Manage Amazon S3 storage costs granularly and at scale using S3 Intelligent-Tiering
Cost-effective data storage is critical when building and scaling data lakes that manage and hold growing datasets. By choosing the right storage architecture, customers are empowered to quickly experiment and migrate to AWS. Amazon S3 Intelligent-Tiering is a storage class that allows customers to optimize storage costs automatically when data access patterns change without performance […]
Mounting Amazon S3 to an Amazon EC2 instance using a private connection to S3 File Gateway
Customers rehosting applications in the cloud that deal with large files and unstructured data can benefit by utilizing object storage from a performance, scalability, and cost perspective, as compared to block or file storage. If a legacy or COTS (commercial-off-the-shelf) application being migrated doesn’t inherently support object storage services like Amazon S3, it may be […]
How Jemena approached data migration using AWS DataSync and shared VPCs
Organizations starting their cloud migration journey must make several design choices about their AWS architecture. Some of these design choices relate to organizational structure, the number of AWS accounts, Virtual Private Cloud (VPC) options, and other details. Depending on these upfront choices, the tooling and approach to migrate data from an on-premises system to AWS […]