AWS Storage Blog

Category: Technical How-to

Implementing sensor workflows using AWS Snowcone and AWS IoT Greengrass

In our first blog of this series, we covered using an IoT device to store the sensor data on an Amazon EC2 instance running on AWS Snowcone. That use case covered collecting data from sensors in locations such as a factory floor or a mine with austere network connectivity. There are other instances where you […]

Video transcoding at the edge with AWS Snowcone

A customer doing video analysis in remote locations has the following problem: they must capture high-resolution video in the field and then transfer that data to a durable, highly available data store in the cloud for long-term storage. They also want to keep copies of video files in the remote location so that they can […]

Amazon S3

Automate Amazon S3 Versioning using AWS Config rules

Different enterprises and organizations have different data compliance requirements and regulations that they must adhere to for legal, security, safety, and best practice reasons. Historically, customers with data in Amazon S3 have manually performed remediation actions on non-compliant buckets. This includes writing and maintaining scripts running on regular intervals to check for non-compliant S3 buckets […]

Amazon S3

Limit access to Amazon S3 buckets owned by specific AWS accounts

Customers use Amazon S3 to store and protect data for a range of use cases, including data lakes, enterprise applications, backup, and archive. Amazon S3 provides easy-to-use management features, fine-grained access controls, strong consistency, and durability to meet a range of business, organizational, and compliance requirements. A common data loss prevention requirement is ensuring that […]

Amazon S3

Using job tags to manage permissions for Amazon S3 Batch Operations jobs

As organizations grow their use of AWS, they often find that a variety of teams and applications begin to use the data stored in Amazon S3. While customers love the agility benefits of this, they also seek to govern their data’s security, productivity, and cost. Earlier this year we announced support for job tags with […]

re:Invent

Automating copying encrypted Amazon EBS snapshots across AWS accounts

UPDATE: The screenshots of the Amazon DLM interface in this post were updated on 7/14/2021. Many customers have disaster recovery (DR) best practices that require them to copy their Amazon EBS snapshots to an isolated account in a different Region and encrypt those snapshots with a different key. In doing so, customers are able to […]

re:Invent

Running Kubernetes cluster with Amazon EKS Distro across AWS Snowball Edge

AWS Snowball Edge customers are running applications for edge local data processing, analysis, and machine learning using Amazon EC2 compute instances on Snowball Edge devices in remote or disconnected locations. Customers use Snowball Edge devices in locations including, but not limited to, cruise ships, oil rigs, and factory floors with no or limited network connectivity. […]

Add storage to your high-performance file system with a single click

Many organizations have on-premises, high-performance workloads burdened with complex management and scalability challenges. Scaling data-intensive workloads on-premises typically involves purchasing more hardware, which can slow time to production and require high upfront investment. The agility to scale compute and storage resources to meet business needs is one of the reasons why our customers choose to […]

Amazon S3

Building a central asset register with Amazon S3 Inventory

UPDATE 7/12/2022: Amazon SQS policy updated to support every AWS Region (step 3 in the architecture diagram) in the central.yml template. Many AWS customers store millions of objects in their Amazon S3 buckets, due to the scalability, durability, and performance that S3 provides. Customers compelled to build an information asset register for compliance reasons or […]

Persistent storage for container logging using Fluent Bit and Amazon EFS

UPDATE 9/8/2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details. Logging is a powerful debugging mechanism for developers and operations teams when they must troubleshoot issues. Containerized applications write logs to standard output, which is redirected to local ephemeral storage, by default. These logs are lost when the container is terminated […]