AWS Architecture Blog

Category: Amazon Managed Streaming for Apache Kafka (Amazon MSK)

Let's Architect

Let’s Architect! Designing systems for stream data processing

Harnessing the potential of streaming data processing offers the opportunity to stay at the forefront of industries, make data-informed decisions with agility, and gain invaluable insights into customer behavior and operational efficiency.

Topic fanout pattern based on Amazon SQS and Amazon SNS

Best practices for implementing event-driven architectures in your organization

Event-driven architectures (EDA) are made up of components that detect business actions and changes in state, and encode this information in event notifications. Event-driven patterns are becoming more widespread in modern architectures because: they are the main invocation mechanism in serverless patterns. they are the preferred pattern for decoupling microservices, where asynchronous communications and event […]

Let's Architect

Let’s Architect! Open-source technologies on AWS

We brought you a Let’s Architect! blog post about open-source on AWS that covered some technologies with development led by AWS/Amazon, as well as well-known solutions available on managed AWS services. Today, we’re following the same approach to share more insights about the process itself for developing open-source. That’s why the first topic we discuss […]

Mainframe data integration conceptual flow

Mainframe data integration: Using mainframe data to build cloud native services with AWS

For International Women’s Day and Women’s History Month, we’re featuring more than a week’s worth of posts that highlight female builders and leaders. We’re showcasing women in the industry who are building, creating, and, above all, inspiring, empowering, and encouraging everyone—especially women and girls—in tech. Many companies in the financial services and insurance industries rely […]

Figure 2. Simulated RLN architecture in the AWS Cloud

Scaling DLT to 1M TPS on AWS: Optimizing a Regulated Liabilities Network

SETL is an open source, distributed ledger technology (DLT) company that enables tokenisation, digital custody, and DLT for securities markets and payments. In mid-2021, they developed a blueprint for a Regulated Liabilities Network (RLN) that enables holding and managing a variety of tokenized value irrespective of its form. In a December 2021 collaboration with Amazon […]

Figure 2. Technical workflow for Meshify IoT architecture

How Meshify Built an Insurance-focused IoT Solution on AWS

The ability to analyze your Internet of Things (IoT) data can help you prevent loss, improve safety, boost productivity, and even develop an entirely new business model. This data is even more valuable, with the ever-increasing number of connected devices. Companies use Amazon Web Services (AWS) IoT services to build innovative solutions, including secure edge […]

Figure 1 - Architecture representing an IoT ingestion pipeline

Field Notes: Deliver Messages Using an IoT Rule Action to Amazon Managed Streaming for Apache Kafka

With IoT devices scaling up rapidly, real-time data integration and data processing has become a major challenge. This is why customers often choose Message Queuing Telemetry Transport (MQTT) for message ingestion, and Apache Kafka to build a real-time streaming data pipeline. AWS IoT Core now supports a new IoT rule action to deliver messages from […]

Figure 3. Replay Architecture

Amazon MSK Backup for Archival, Replay, or Analytics

Amazon MSK is a fully managed service that helps you build and run applications that use Apache Kafka to process streaming data. Apache Kafka is an open-source platform for building real-time streaming data pipelines and applications. With Amazon MSK, you can use native Apache Kafka APIs to populate data lakes. You can also stream changes to […]