Let’s Architect! Architecting a data mesh
Data architectures were mainly designed around technologies rather than business domains in the past. This changed in 2019, when Zhamak Dehghani introduced the data mesh. Data mesh is an application of the Domain-Driven-Design (DDD) principles to data architectures: Data is organized into data domains and the data is the product that the team owns and offers for consumption.
A data mesh architecture unites the disparate data sources within an organization through centrally managed data-sharing and governance guidelines. Business functions can maintain control over how shared data is accessed because data mesh also solves advanced data security challenges through distributed, decentralized ownership.
This edition of Let’s Architect! introduces data mesh, highlights the foundational concepts of data architectures, and covers the patterns for designing a data mesh in the AWS cloud with supporting resources.
Let’s explore a video introduction to data lakes, lake houses, and data mesh. This resource explains how to leverage those concepts to gain greater data insights across different business segments, with a special focus on best practices to build a well-architected, modern data architecture on AWS. It also gives an overview of the AWS cloud services that can be used to create such architectures and describes the fundamental pillars of designing them.
Knowing what a data mesh architecture is, here is a step-by-step video from re:Invent 2022 on designing one. It covers a use case on how GoDaddy considered and implemented data mesh, in addition to:
- The fundamental pillars behind a well-architected data mesh in the cloud
- Finding an approach to build a data mesh architecture using native AWS services
- Reasons for considering a data mesh architecture where data lakes provide limitations in some scenarios
- How data mesh can be applied in practice to overcome them
- The mental models to apply during the data mesh design process
Now let’s explore data accessibility as it relates to data mesh architectures.
Amazon DataZone is a new AWS business data catalog allowing you to unlock data across organizational boundaries with built-in governance. This service provides a unified environment where everyone in an organization—from data producers to data consumers—can access, share, and consume data in a governed manner.
Here is a video to learn how to apply AWS analytics services to discover, access, and share data across organizational boundaries within the context of a data mesh architecture.
Feeling inspired to build? Hands-on experience is a great way to learn and see how the theoretical concepts apply in practice.
This workshop teaches you a data mesh architecture building approach on AWS. Many organizations are interested in implementing this architecture to:
- Move away from centralized data lakes to decentralized ownership
- Deliver analytics solutions across business units
Learn how a data mesh architecture can be implemented with AWS native services.
See you next time!
Thanks for exploring architecture tools and resources with us!
Next time we’ll talk about monitoring and observability.