AWS Storage Blog
Tag: Amazon Simple Storage Service (Amazon S3)
How to move and store your genomics sequencing data with AWS DataSync
Genomics data is expanding at a rate exceeding Moore’s law according to the National Human Genome Research Institute. As more sequencing data is produced and researchers move from genotyping to whole genome sequencing, the amount of data produced is outpacing on-premises capacity. Organizations need cloud solutions that help manage data movement, storage, and analysis. The […]
Optimize storage costs by analyzing API operations on Amazon S3
The demand for data storage has increased with the advent of a fast-paced data environment – creating, sharing, and replicating data at a large scale. Most organizations are looking for the optimal way to store their data cost-effectively, giving them everything they need from their data but without breaking the bank. Cloud storage provides flexible […]
Reduce encryption costs by using Amazon S3 Bucket Keys on existing objects
As more organizations look to operate faster and at scale, they need ways to meet critical compliance requirements and improve data security. Encryption is a critical component of a defense in depth strategy, and when used correctly, can provide an additional layer of protection above basic access control. However, workloads that access millions or billions […]
Analytical processing of millions of cell images using Amazon EFS and Amazon S3
Analytical workloads such as batch processing, high performance computing, or machine learning inference often have high IOPS and low latency requirements but operate at irregular intervals on subsets of large datasets. Typically, data is manually copied between storage tiers in preparation of processing, which can be cumbersome and error-prone. Given this, IT teams want to […]
Allowing external users to securely and directly upload files to Amazon S3
Organizations are often required to store files, images, and other digital assets in a repository. In many cases, the source of these files are partners or individuals who are not connected to internal systems and requires corporate authentication in order to upload the files. Customers traditionally use servers to handle file uploads, which can use […]
How Trend Micro uses Amazon S3 Object Lambda to help keep sensitive data secure
Does your application handle data that is uploaded by hundreds of thousands of end users? Is that same underlying data then shared across the same magnitude of users? Being able to scan data for malware before it’s returned to an application helps keep sensitive data secure, provides protection regardless of when the data was initially […]
Compressing and archiving logs to the Amazon S3 Glacier storage classes
In distributed architectures, there is often a need to preserve application logs, and for AWS customers preservation is often done via an Amazon S3 bucket. The logs may contain information on runtime transactions, error/failure states, or application metrics and statistics. These logs are later used in business intelligence to provide useful insights and generate dashboards, […]
Using AWS DataSync to move data from Hadoop to Amazon S3
You want to leverage cloud scalability, increase cost efficiency by paying only for utilized storage, decouple big data storage from processing, and increase capabilities for data analytics and machine learning using AWS. But how do you move your Hadoop cluster? To accelerate this transition, AWS DataSync recently launched support for moving data between Hadoop Distributed […]
A gene-editing prediction engine with iterative learning cycles built on AWS
NRGene develops cutting-edge genomic analytics products that are reshaping agriculture worldwide. Among our customers are some of the biggest and most sophisticated companies in seed-development, food and beverages, paper, rubber, cannabis, and more. In the middle of 2020, NRGene joined a consortium of companies and academic institutions to build the best-in-class gene-editing prediction platform to […]
MemQ by Pinterest: An efficient, scalable, cloud-native publish/subscribe system
The Logging Platform at Pinterest powers all data ingestion and transportation at Pinterest. At the heart of the Pinterest Logging Platform are distributed pub/sub systems that help our customers transport, buffer, and consume data asynchronously. Pub/sub messaging, is a form of asynchronous service-to-service communication used in serverless and microservices architectures. In a pub/sub model, any […]