AWS Storage Blog

Tag: Amazon Simple Storage Service (Amazon S3)

Amazon S3

MemQ by Pinterest: An efficient, scalable, cloud-native publish/subscribe system

The Logging Platform at Pinterest powers all data ingestion and transportation at Pinterest. At the heart of the Pinterest Logging Platform are distributed pub/sub systems that help our customers transport, buffer, and consume data asynchronously. Pub/sub messaging, is a form of asynchronous service-to-service communication used in serverless and microservices architectures. In a pub/sub model, any […]

Read More
AWS DataSync Featured Image

How to securely share application log files with third parties

What do we do when our applications fail, and we must provide instance-level log data to external entities for troubleshooting purposes? It’s best to limit direct human interaction with our production resources, so we often see temporary access provided for a fixed period. For highly regulated industries, the approval process for production access can be […]

Read More
Amazon S3 Glacier Storage Classes

Modernizing NASCAR’s multi-PB media archive at speed with AWS Storage

The National Association for Stock Car Auto Racing (NASCAR) is the sanctioning body for the No. 1 form of motor sports in the United States, and owns 15 of the nation’s major motorsports entertainment facilities. About 15 years ago NASCAR began to collect all the video, audio, and image assets from over the last 70+ […]

Read More
Amazon S3 Replication featured image

Considering four different replication options for data in Amazon S3

UPDATE (2/10/2022): Amazon S3 Batch Replication launched on 2/8/2022, allowing you to replicate existing S3 objects and synchronize your S3 buckets. See the S3 User Guide for additional details. As your business grows and accumulates more data over time, you may need to replicate data from one system to another, perhaps because of company security […]

Read More
Amazon S3 Glacier Storage Classes

Best practices for archiving large datasets with AWS

As companies grow, they often find themselves managing an ever-increasing amount of data. Customers often need to retain backups for business continuity or disaster recovery, as well as records for compliance and audits. In addition, some customers may need to retain backups to create a centralized repository of information that is heterogeneous in nature, with […]

Read More
Amazon S3

Monitor Amazon S3 activity using S3 server access logs and Pandas in Python

Monitoring and controlling access to data is often essential for security, cost optimization, and compliance. For these reasons, customers want to know what data of theirs is being accessed, when it is being accessed, and who is accessing it. With more data to monitor, large amounts of data can make it more challenging to granularly […]

Read More
Amazon S3 Glacier Storage Classes

Collecting, archiving, and retrieving surveillance footage with AWS

Video feeds and still images from judiciary locations are considered critical forms of evidence in the court of law. These locations can be police stations and government offices or even civil locations of importance like banks and hospitals. As governments, particularly in smart cities rely upon video surveillance, it is critical to design a cost […]

Read More
Amazon S3

Point-in-time restore for Amazon S3 buckets

Enterprises store increasing quantities of object data for use cases like data lakes, document management systems, and media libraries. Performing point-in-time restores for large datasets can be challenging, as existing approaches with full-restore from backup are time consuming and expensive. Alternatively, restoring individual objects to previous versions is prone to errors and delays the restore […]

Read More
S3 Intelligent-Tiering featured image

How CineSend manages their media content using S3 Intelligent-Tiering

Is your organization managing terabytes (or even petabytes) of data stored as objects across hundreds if not thousands of buckets on Amazon S3? What are the chances that the access patterns and application requirements for all of these objects are the same? For most companies out there – slim to none. We operate in a […]

Read More
Amazon S3 Storage Lens featured image

Access your Amazon S3 Storage Lens metrics in AWS Partner applications

Managing cloud storage at scale requires the right metrics and tools to maintain visibility into your storage footprint, and uncover opportunities to reduce storage costs or apply data protection best practices. With the launch of S3 Storage Lens in November 2020, customers gained access to the first cloud storage analytics solution offering organization-wide visibility into […]

Read More