AWS Storage Blog

Tag: Amazon Simple Storage Service (Amazon S3)

S3 cost optimization

Optimize storage costs by analyzing API operations on Amazon S3

The demand for data storage has increased with the advent of a fast-paced data environment – creating, sharing, and replicating data at a large scale. Most organizations are looking for the optimal way to store their data cost-effectively, giving them everything they need from their data but without breaking the bank. Cloud storage provides flexible […]

Read More
S3 Security

Reduce encryption costs by using Amazon S3 Bucket Keys on existing objects

As more organizations look to operate faster and at scale, they need ways to meet critical compliance requirements and improve data security. Encryption is a critical component of a defense in depth strategy, and when used correctly, can provide an additional layer of protection above basic access control. However, workloads that access millions or billions […]

Read More
Amazon EFS serverless featured image

Analytical processing of millions of cell images using Amazon EFS and Amazon S3

Analytical workloads such as batch processing, high performance computing, or machine learning inference often have high IOPS and low latency requirements but operate at irregular intervals on subsets of large datasets. Typically, data is manually copied between storage tiers in preparation of processing, which can be cumbersome and error-prone. Given this, IT teams want to […]

Read More
Amazon S3

Allowing external users to securely and directly upload files to Amazon S3

Organizations are often required to store files, images, and other digital assets in a repository. In many cases, the source of these files are partners or individuals who are not connected to internal systems and requires corporate authentication in order to upload the files. Customers traditionally use servers to handle file uploads, which can use […]

Read More
Amazon S3 Object Lambda

How Trend Micro uses Amazon S3 Object Lambda to help keep sensitive data secure

Does your application handle data that is uploaded by hundreds of thousands of end users? Is that same underlying data then shared across the same magnitude of users? Being able to scan data for malware before it’s returned to an application helps keep sensitive data secure, provides protection regardless of when the data was initially […]

Read More
Amazon S3 Glacier Storage Classes

Compressing and archiving logs to the Amazon S3 Glacier storage classes

In distributed architectures, there is often a need to preserve application logs, and for AWS customers preservation is often done via an Amazon S3 bucket. The logs may contain information on runtime transactions, error/failure states, or application metrics and statistics. These logs are later used in business intelligence to provide useful insights and generate dashboards, […]

Read More
AWS DataSync Featured Image 2020

Using AWS DataSync to move data from Hadoop to Amazon S3

You want to leverage cloud scalability, increase cost efficiency by paying only for utilized storage, decouple big data storage from processing, and increase capabilities for data analytics and machine learning using AWS. But how do you move your Hadoop cluster? To accelerate this transition, AWS DataSync recently launched support for moving data between Hadoop Distributed […]

Read More

A gene-editing prediction engine with iterative learning cycles built on AWS

NRGene develops cutting-edge genomic analytics products that are reshaping agriculture worldwide. Among our customers are some of the biggest and most sophisticated companies in seed-development, food and beverages, paper, rubber, cannabis, and more. In the middle of 2020, NRGene joined a consortium of companies and academic institutions to build the best-in-class gene-editing prediction platform to […]

Read More
Amazon S3

MemQ by Pinterest: An efficient, scalable, cloud-native publish/subscribe system

The Logging Platform at Pinterest powers all data ingestion and transportation at Pinterest. At the heart of the Pinterest Logging Platform are distributed pub/sub systems that help our customers transport, buffer, and consume data asynchronously. Pub/sub messaging, is a form of asynchronous service-to-service communication used in serverless and microservices architectures. In a pub/sub model, any […]

Read More
AWS DataSync Featured Image

How to securely share application log files with third parties

What do we do when our applications fail, and we must provide instance-level log data to external entities for troubleshooting purposes? It’s best to limit direct human interaction with our production resources, so we often see temporary access provided for a fixed period. For highly regulated industries, the approval process for production access can be […]

Read More