AWS Storage Blog

Category: Compute

AWS DataSync Featured Image 2020

FORMULA 1 transfers race car data into AWS using AWS DataSync

At FORMULA 1, time is always of the essence, whether that be on-track at our Event Technical Centre or back at our Media & Technology Centre in Kent, UK. As the Cloud Team Lead at FORMULA 1, I am responsible for defining, designing, and implementing workloads across the cloud estate, including security, platform, project, networking, […]

re:Invent

AWS re:Invent recap: Extend Amazon S3 to on-premises environments with AWS Outposts

While we could not meet with you in person at re:Invent this year, we hope that you enjoyed joining us for our first ever virtual re:Invent – enabling you to get the full re:Invent experience from home, and for free. In this blog post, I highlight Amazon S3 on Outposts, launched on September 30, 2020 […]

AWS Transfer Family Featured Image

Securing AWS Transfer Family with AWS Web Application Firewall and Amazon API Gateway

AWS Transfer Family is a fully managed, serverless file transfer service for Amazon S3 and Amazon EFS. The service provides you with the flexibility to authenticate your file transfer client users using credentials stored in an identity provider (IdP) of your choice. You can achieve this by integrating an Amazon API Gateway endpoint backed by an AWS Lambda function that […]

AWS DataSync Featured Image 2020

Post-process your transferred data with file-level logging in AWS DataSync

AWS DataSync is an online data transfer service that simplifies, automates, and accelerates moving data between on-premises storage systems and AWS Storage services, as well as between AWS Storage services. Customers use DataSync for a number of use cases, including migrating active datasets to AWS, archiving data to free up on-premises storage capacity, replicating data […]

Implementing sensor workflows using AWS Snowcone and AWS IoT Greengrass

In our first blog of this series, we covered using an IoT device to store the sensor data on an Amazon EC2 instance running on AWS Snowcone. That use case covered collecting data from sensors in locations such as a factory floor or a mine with austere network connectivity. There are other instances where you […]

AWS re:Invent recap: Modernize your applications with Amazon EFS

Development teams are modernizing their applications by adopting containers, serverless, and microservices-based architectures. As containers are transient in nature, long-running applications can benefit from keeping state in durable file storage. At the same time, serverless computing enables you to be more agile while spending less time dealing with the security, scalability, and availability of your […]

re:Invent

Running Kubernetes cluster with Amazon EKS Distro across AWS Snowball Edge

AWS Snowball Edge customers are running applications for edge local data processing, analysis, and machine learning using Amazon EC2 compute instances on Snowball Edge devices in remote or disconnected locations. Customers use Snowball Edge devices in locations including, but not limited to, cruise ships, oil rigs, and factory floors with no or limited network connectivity. […]

Amazon S3

Building a central asset register with Amazon S3 Inventory

UPDATE 7/12/2022: Amazon SQS policy updated to support every AWS Region (step 3 in the architecture diagram) in the central.yml template. Many AWS customers store millions of objects in their Amazon S3 buckets, due to the scalability, durability, and performance that S3 provides. Customers compelled to build an information asset register for compliance reasons or […]

Persistent storage for container logging using Fluent Bit and Amazon EFS

UPDATE 9/8/2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details. Logging is a powerful debugging mechanism for developers and operations teams when they must troubleshoot issues. Containerized applications write logs to standard output, which is redirected to local ephemeral storage, by default. These logs are lost when the container is terminated […]

Building an IoT solution at the edge with AWS Snowcone

UPDATE: The second blog post in this two-post series was published on January 5, 2020. Internet of Things (IoT) applications, like other applications, require edge solutions to operate in austere conditions with limited network connectivity or limited infrastructure. IoT applications at the edge can span numerous uses, like automation, optimization, and intelligent manufacturing to name […]