AWS Architecture Blog

Category: Amazon Simple Storage Service (S3)

Unlocking the future of video data: March Networks cloud storage on AWS

Enterprise video surveillance is operating at an unprecedented scale as organizations across retail, banking, quick-service restaurants (QSR), convenience stores, and transportation networks generate petabytes of video data across thousands of distributed locations. As retention requirements grow and organizations seek to extract more operational insights from video, traditional on-premise storage models are becoming increasingly difficult and […]

Specification-driven composition for flexible data workflows

Specification-driven composition addresses a common scalability bottleneck in data pipelines. Data pipelines often start as simple scripts, but as they grow, you duplicate transformation logic and small changes cascade across multiple workflows. Copying and modifying data transformation logic across scripts leads to workflows that become difficult to manage at scale. Tracking what each pipeline does […]

Modernizing financial analytics with Amazon SageMaker Unified Studio

Avanse Financial Services, India’s leading education loan providers, migrated to a cloud-native lakehouse architecture using Amazon SageMaker Unified Studio, which unified their data engineering, analytics, and artificial intelligence (AI) workflows in a single governed environment on AWS. In this post, we walk through their migration journey so you can adapt their approach to your own environment.

Automating contract intelligence with Doczy.ai™ on AWS

In this post, we show you how Doczy.ai™ uses generative AI on AWS to automate contract intelligence at scale, transforming unstructured documents into structured, actionable insights, so organizations can automate critical business processes and unlock the full value of their data.

AWS Cloud architecture diagram for the PACIFIC platform showing a multi-layered system. At the top, a PACIFIC Web Client connects to an Identity & Authorization layer containing Amazon Cognito, AWS IAM, and AWS Secrets Manager. Traffic flows through AWS WAF to an Application Load Balancer within a VPC, which distributes requests to Amazon ECS (AWS Fargate) hosting four containerized microservices: core-modules, integration-module, pcf-exchange-module, and edc-dtr-module. These modules connect to Amazon RDS for relational database storage and Amazon S3 for object storage. External integrations at the bottom include BASF Product Carbon Footprint Services, an EDC/DTR Service Provider, and the Catena-X Automotive Network. The diagram illustrates a secure, microservices-based architecture for automotive industry carbon footprint data exchange.

PACIFIC enables multi-tenant, sovereign product carbon footprint exchange on the Catena-X data space using AWS

This post explores how PACIFIC enables multi-tenant, sovereign PCF exchange on the Catena-X data space using Amazon Elastic Container Service (Amazon ECS) on AWS Fargate, Amazon Cognito, and AWS Identity and Access Management (IAM) to deliver measurable environmental impact and competitive advantage in a carbon-conscious marketplace.

Figure 2: Aigen modernized architecture

How Aigen transformed agricultural robotics for sustainable farming with Amazon SageMaker AI

In this post, you will learn how Aigen modernized its machine learning (ML) pipeline with Amazon SageMaker AI to overcome industry-wide agricultural robotics challenges and scale sustainable farming. This post focuses on the strategies and architecture patterns that enabled Aigen to modernize its pipeline across hundreds of distributed edge solar robots and showcase the significant business outcomes unlocked through this transformation. By adopting automated data labeling and human-in-the-loop validation, Aigen increased image labeling throughput by 20x while reducing image labeling costs by 22.5x.

This diagram shows the AWS architecture of Santander's Catalyst platform that provides AI capabilities to teams across the company.

Digital Transformation at Santander: How Platform Engineering is Revolutionizing Cloud Infrastructure

Santander faced a significant technical challenge in managing an infrastructure that processes billions of daily transactions across more than 200 critical systems. The solution emerged through an innovative platform engineering initiative called Catalyst, which transformed the bank’s cloud infrastructure and development management. This post analyzes the main cases, benefits, and results obtained with this initiative.

BASF Digital Farming builds a STAC-based solution on Amazon EKS

This post was co-written with Frederic Haase and Julian Blau with BASF Digital Farming GmbH. At xarvio – BASF Digital Farming, our mission is to empower farmers around the world with cutting-edge digital agronomic decision-making tools. Central to this mission is our crop optimization platform, xarvio FIELD MANAGER, which delivers actionable insights through a range […]

AWS architecture showing protein vector processing workflow with ECR, Lambda, and LanceDB

A scalable, elastic database and search solution for 1B+ vectors built on LanceDB and Amazon S3

In this post, we explore how Metagenomi built a scalable database and search solution for over 1 billion protein vectors using LanceDB and Amazon S3. The solution enables rapid enzyme discovery by transforming proteins into vector embeddings and implementing a serverless architecture that combines AWS Lambda, AWS Step Functions, and Amazon S3 for efficient nearest neighbor searches.

From virtual machine to Kubernetes to serverless: How dacadoo saved 78% on cloud costs and automated operations

In this post, we walk you step-by-step through dacadoo’s journey of embracing managed services, highlighting their architectural decisions as we go.