AWS Architecture Blog

Category: Analytics

Let's Architect

Let’s Architect! Designing systems for stream data processing

Harnessing the potential of streaming data processing offers the opportunity to stay at the forefront of industries, make data-informed decisions with agility, and gain invaluable insights into customer behavior and operational efficiency.

Reusable ETL framework architecture

Use a reusable ETL framework in your AWS lake house architecture

Data lakes and lake house architectures have become an integral part of a data platform for any organization. However, you may face multiple challenges while developing a lake house platform and integrating with various source systems. In this blog, we will address these challenges and show how our framework can help mitigate these issues. Lake […]

Technical architecture for implementing multi-lingual semantic search functionality

Content Repository for Unstructured Data with Multilingual Semantic Search: Part 2

Leveraging vast unstructured data poses challenges, particularly for global businesses needing cross-language data search. In Part 1 of this blog series, we built the architectural foundation for the content repository. The key component of Part 1 was the dynamic access control-based logic with a web UI to upload documents. In Part 2, we extend the […]

Topic fanout pattern based on Amazon SQS and Amazon SNS

Best practices for implementing event-driven architectures in your organization

Event-driven architectures (EDA) are made up of components that detect business actions and changes in state, and encode this information in event notifications. Event-driven patterns are becoming more widespread in modern architectures because: they are the main invocation mechanism in serverless patterns. they are the preferred pattern for decoupling microservices, where asynchronous communications and event […]

End-to-end data-processing architecture

Temporal data lake architecture for benchmark and indices analytics

Financial trading houses and stock exchanges generate enormous volumes of data in near real-time, making it difficult to perform bi-temporal calculations that yield accurate results. Achieving this requires a processing architecture that can handle large volumes of data during peak bursts, meet strict latency requirements, and scale according to incoming volumes. In this post, we’ll […]

Serverless data archiving and retrieval

Reduce archive cost with serverless data archiving

For regulatory reasons, decommissioning core business systems in financial services and insurance (FSI) markets requires data to remain accessible years after the application is retired. Traditionally, FSI companies either outsourced data archiving to third-party service providers, which maintained application replicas, or purchased vendor software to query and visualize archival data. In this blog post, we […]

Let's Architect

Let’s Architect! Open-source technologies on AWS

We brought you a Let’s Architect! blog post about open-source on AWS that covered some technologies with development led by AWS/Amazon, as well as well-known solutions available on managed AWS services. Today, we’re following the same approach to share more insights about the process itself for developing open-source. That’s why the first topic we discuss […]

Architecture for AWS Clean Rooms Scope 3 collaboration

Managing data confidentiality for Scope 3 emissions using AWS Clean Rooms

Scope 3 emissions are indirect greenhouse gas emissions that are a result of a company’s activities, but occur outside the company’s direct control or ownership. Measuring these emissions requires collecting data from a wide range of external sources, like raw material suppliers, transportation providers, and other third parties. One of the main challenges with Scope […]

AWS IoT Greengrass stream manager sending data directly to Kinesis

Creating scalable architectures with AWS IoT Greengrass stream manager

Designing a scalable, global, real-time, distributed system to process millions of messages from a variety of critical devices can complicate architectures. Collecting large data streams or image recognition from the edge also requires scalable solutions. AWS IoT Core is designed to handle large numbers of Internet of things (IoT) devices sending a few messages per […]

Warm standby with managed services

Disaster Recovery Solutions with AWS managed services, Part 3: Multi-Site Active/Passive

Welcome to the third post of a multi-part series that addresses disaster recovery (DR) strategies with the use of AWS-managed services to align with customer requirements of performance, cost, and compliance. In part two of this series, we introduced a DR concept that utilizes managed services through a backup and restore strategy with multiple Regions. […]