[SEO Subhead]
This Guidance demonstrates how to streamline data access management through the integration of Amazon DataZone and Jira ticketing systems. The included code repository contains stacks that provision an event rule and the required identity and access management roles and policies to facilitate interaction with both Jira and Amazon DataZone. This foundational infrastructure allows you to integrate an Amazon DataZone subscription workflow with a Jira project, enabling data stewards to accept or reject access to data requested in Amazon DataZone directly from the Jira interface. With its extensible design, you can incorporate additional components for notifications on other Amazon DataZone events as well as integrate this Guidance with other ticketing systems as your workflows evolve.
Note: [Disclaimer]
Architecture Diagram
[Architecture diagram description]
Step 1
A data consumer searches for a data asset in the Amazon DataZone portal and submits a subscription request to access a data asset.
Step 2
Amazon DataZone produces an event, which is then captured through an event rule in Amazon EventBridge. This event triggers the execution of an AWS Step Functions target.
Step 3
Step Functions adds tasks to an Amazon Simple Queue Service (Amazon SQS) queue.
Step 4
The AWS Lambda function Create Get Issue polls a task from the Amazon SQS queue and subsequently creates content for the Jira issue based on the captured event.
Step 5
The Lambda function Create Get Issue gets Jira credentials from AWS Secrets Manager and creates the issue in the Jira project.
Step 6
Producers change the status of the issue in the Jira project board to “Accepted” or “Rejected.”
Step 7
Step Functions triggers the Lambda function Create Get Issue to retrieve the status of the issue and waits for the status to change. Once the status changes, Step Functions triggers the execution of the Lambda function DataZone Subscription Update.
Step 8
The Lambda Function DataZone Subscription Update assumes an AWS Identity and Access Management (IAM) role that is a member of an Amazon DataZone project of the target data.
Step 9
The Lambda Function DataZone Subscription Update communicates with Amazon DataZone to update the status of the subscription request
Step 10
If the status of the issue in Jira was changed to “Accept,” the consumer accesses the data.
Step 1
A data consumer searches for a data asset in the Amazon DataZone portal and submits a subscription request to access a data asset.
Step 2
Amazon DataZone produces an event, which is then captured through an event rule in Amazon EventBridge. This event triggers the execution of an AWS Step Functions target.
Step 3
Step Functions adds tasks to an Amazon Simple Queue Service (Amazon SQS) queue.
Step 4
The AWS Lambda function Create Get Issue polls a task from the Amazon SQS queue and subsequently creates content for the Jira issue based on the captured event.
Step 5
The Lambda function Create Get Issue gets Jira credentials from AWS Secrets Manager and creates the issue in the Jira project.
Step 6
Producers change the status of the issue in the Jira project board to “Accepted” or “Rejected.”
Step 7
Step Functions triggers the Lambda function Create Get Issue to retrieve the status of the issue and waits for the status to change. Once the status changes, Step Functions triggers the execution of the Lambda function DataZone Subscription Update.
Step 8
The Lambda Function DataZone Subscription Update assumes an AWS Identity and Access Management (IAM) role that is a member of an Amazon DataZone project of the target data.
Step 9
The Lambda Function DataZone Subscription Update communicates with Amazon DataZone to update the status of the subscription request
Step 10
If the status of the issue in Jira was changed to “Accept,” the consumer accesses the data.
Well-Architected Pillars
The AWS Well-Architected Framework helps you understand the pros and cons of the decisions you make when building systems in the cloud. The six pillars of the Framework allow you to learn architectural best practices for designing and operating reliable, secure, efficient, cost-effective, and sustainable systems. Using the AWS Well-Architected Tool, available at no charge in the AWS Management Console, you can review your workloads against these best practices by answering a set of questions for each pillar.
The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.
-
Operational ExcellenceRead the Operational Excellence whitepaper
Amazon DataZone simplifies data management for both end-users and operational teams. For example, Amazon DataZone offers a user interface for subscribing to data assets and emits EventBridge events for subscription requests, while Step Functions provides transparency into the current state of the subscription workflow. Additionally, AWS X-Ray assists in the troubleshooting of both the Lambda functions and the Step Functions.
-
SecurityRead the Security whitepaper
Amazon DataZone implements an authorization mechanism for data asset access, including subscription requests. In addition, IAM policies narrow access to the principle of least privilege. Finally, communication with the external Jira instance uses HTTPS and token authentication. By incorporating these security-centric services, this Guidance can help safeguard the confidentiality and availability of the data and the systems involved.
-
ReliabilityRead the Reliability whitepaper
EventBridge and Step Functions support the subscription workflow, with built-in retry mechanisms to help ensure a resilient architecture. Additionally, by decoupling the components, Amazon SQS is used to provide resilience for the communication between the Lambda functions and the Jira system. This approach focuses on building workloads that perform their intended functions correctly and consistently, while also enabling the ability to recover quickly from failures to meet demands.
-
Performance EfficiencyRead the Performance Efficiency whitepaper
The capabilities of Amazon DataZone include the structured and streamlined allocation of IT and computing resources, addressing key areas such as the selection of optimized resource types and sizes, performance monitoring, and maintaining efficiency. The architecture further employs EventBridge, Step Functions, and Amazon SQS to support the subscription workflow with built-in retry mechanisms and decoupled components. This holistic approach focuses on workload performance, consistency, and the ability to quickly recover from failures, while also addressing distributed system design, recovery planning, and adaptability to changing requirements.
-
Cost OptimizationRead the Cost Optimization whitepaper
Amazon DataZone helps you establish a data mesh architecture within a significantly reduced timeframe compared to traditional methods. Moreover, instead of maintaining the subscription workflow state in databases, the state is managed within the Step Functions and Amazon SQS services, helping to avoid the operational overhead and costs associated with managing and scaling database infrastructure.
Furthermore, the use of serverless services, such as EventBridge, Step Functions, Lambda, and Amazon SQS, allows for a pay-per-use model and enables this Guidance to scale up or down based on demand without the need to provision and maintain dedicated infrastructure.
-
SustainabilityRead the Sustainability whitepaper
The Guidance helps you obtain a comprehensive understanding of the environmental impacts of the utilized services, quantifying these impacts throughout the entire workload lifecycle, and applying design principles and best practices to reduce these impacts. Specifically, it uses EventBridge, Step Functions, Amazon SQS, and Lambda, which eliminate the need for persistent storage or always-on Amazon Elastic Compute Cloud (Amazon EC2) instances. Instead, the compute resources are only used on demand, and the state is maintained within Step Functions and Amazon SQS, contributing to a more environmentally conscious and efficient cloud-based approach.
Related Content
[Title]
Disclaimer
The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.
References to third-party services or organizations in this Guidance do not imply an endorsement, sponsorship, or affiliation between Amazon or AWS and the third party. Guidance from AWS is a technical starting point, and you can customize your integration with third-party services when you deploy the architecture.