AWS Business Intelligence Blog
Automate your Amazon QuickSight assets deployment using the new Amazon EventBridge integration
Business intelligence (BI) and IT operations (BIOps) teams often have the need to automate and accelerate the deployment of BI assets to make sure of business continuity. We heard that you wanted an automated, scalable way to deploy, back up, or replicate Amazon QuickSight assets at scale so that BIOps teams within your organization can work in an agile manner.
We are launching QuickSight assets Create, Update, and Delete (CUD) events using Amazon EventBridge. With EventBridge, developers can respond automatically to events in QuickSight such as a new dashboard creation or update. These events are delivered to EventBridge in near real time. Developers can write simple rules to indicate which events are of interest to them and what actions to take when an event matches a rule. By subscribing and responding to QuickSight events in EventBridge, customers can automate their workflows, such as continuous deployment, replication, and backups. This launch covers Create, Update, and Delete events for
- Dashboards
- Shared folders
- VPC connections
- Analyses
Feature overview
All events payloads have account ID, Region, and TimeStamp in UTC. The following table shows the events with event specific details.
Asset type | Event | Common details | Event specifc details |
Dashboard | QuickSight dashboard creation successful | DashboardId, VersionNumber | . |
QuickSight dashboard creation failed | Errors | ||
QuickSight dashboard update successful | . | ||
QuickSight dashboard update failed | Errors | ||
QuickSight dashboard published version updated | . | ||
QuickSight dashboard deleted | DashboardId | ||
Analysis | QuickSight analysis creation successful | AnalysisID | . |
QuickSight analysis creation failed | Errors | ||
QuickSight analysis deleted | . | ||
Folders | QuickSight folder created | FolderId | ParentFolderArn (If applicable) |
QuickSight folder updated | . | ||
QuickSight folder deleted | . | ||
QuickSight folder membership updated | MembersAdded[], MembersRemoved[] | ||
VPC Connection | QuickSight VPC connection creation successful | VPCConnectionId, AvailabilityStatus | . |
QuickSight VPC connection creation failed | . | ||
QuickSight VPC connection update successful | . | ||
QuickSight VPC connection update failed | . | ||
QuickSight VPC connection deletion successful | . | ||
QuickSight VPC connection deletion failed | . |
Use case
Let’s consider a fictional company, AnyCompany, that owns healthcare facilities across the globe. They have set up a development QuickSight account for authors to create and update QuickSight assets and a separate production account. Additionally, in some cases, due to data residency regulation, they have to maintain the same assets across multiple Regions. AnyCompany is scaling its business and they want to automate deployment within and across multiple QuickSight accounts and back up QuickSight assets as soon as there are changes in an asset lifecycle.
AnyCompany has the following key deployment and backup requirements:
- Continuous Deployment: AnyCompany wants to automate the deployment of QuickSight assets from their development to their production account. The production accounts for some teams are in different Regions whereas for others they’re within the same Region. AnyCompany wants to automate this process of deployment. AnyCompany uses QuickSight shared folders for the DevOps team to pick up any updated assets. When an author is done with their changes, they update the dashboard in a specific shared folder. Until now, the author had to inform the DevOps team that the dashboard was ready for deployment, which is manual and not efficient.
- Automated Backup: As AnyCompany rolls out critical dashboards for business, it needs to make sure that dashboards are highly available . As part of their strategy, AnyCompany wants to maintain a backup of assets to restore in case of disasters. Until now, AnyCompany has been backing up assets on a fixed schedule. They want to reduce the delay between a change in an asset and backup.
Continuous deployment
When an author is done with their changes, they update the dashboard in a specific shared folder. In case the dashboard is no longer needed in production, they simply remove the dashboard from the folder. DevOps has created a rule in EventBridge to capture and send folder membership update events to a Lambda function, which then uses QuickSight export and import APIs to deploy the updated dashboard from the development account to the production account.
DevOps chooses the following event and creates a rule to send it to a Lambda function
QuickSight folder membership updated
Creating the EventBridge rule
- Open the Amazon EventBridge console and in the navigation pane, choose Rules.
- Choose Create rule. Enter a name and description for the rule. For example, enter
QuickSightAssetChangeRule
. - Select default Event bus.
- Choose Rule with an event pattern, and then choose Next.
- For Event source, choose AWS events or EventBridge partner events.
- Scroll down to the Creation method section.
- Choose Custom pattern (JSON editor).
- In the Event pattern text box, enter the following code snippet and choose Next.
- For Target types, choose AWS service and Lambda function.
- For Function, choose the Lambda function that you created and then choose Next.
- In Configure tags, choose Next and then choose Create rule.
Once the Lambda function gets invoked, it uses the following QuickSight APIs to export the updated dashboard from the development account and import it into the production account. Details of how to use these APIs can be found in the QuickSight Developer Guide and Automate and accelerate your Amazon QuickSight asset deployments using the new APIs.
Export APIs
You can use the following APIs to initiate, track, and describe the export jobs that produce the bundle files from the source account. A bundle file is a zip file (with the .qs extension) that contains assets specified by the caller and, optionally, all dependencies of the assets.
- StartAssetBundleExportJob – Use this asynchronous API to export an asset bundle file.
- DescribeAssetBundleExportJob – Use this synchronous API to get the status of your export job. When successful, this API call response will have a presigned URL to fetch the asset bundle.
- ListAssetBundleExportJobs – Use this synchronous API to list past export jobs. The list will contain both finished and running jobs from the past 15 days.
Import APIs
These APIs initiate, track, and describe the import jobs that take the bundle file as input and create or update assets in the destination account.
- StartAssetBundleImportJob – Use this asynchronous API to start an import of an asset bundle file.
- DescribeAssetBundleImportJob – Use this synchronous API to get the status of your import job.
- ListAssetBundleImportJobs – Use this synchronous API to list past import jobs. The list will contain both finished and running jobs from the past 15 days.
Automated backup
AnyCompany deploys business-critical dashboards, and it’s important for them to have proper backup and version control processes. In addition to running scheduled export jobs at regular intervals, they also want to back up critical assets as soon as they’re updated. Along with deployment, they back up the exported files. DevOps at AnyCompany has created EventBirdge rules to capture dashboard update events, and when the event gets triggered, they use the assets exported API StartAssetBundleExportJob to export and back up the dashboard in their preferred storage system. They create the EventBridge rule using the same steps as mentioned before and use the following event pattern. This rule will only be triggered when the dashboard with the specified ID is updated.
Conclusion
QuickSight asset events using EventBridge eliminate manual processes or polling for updates and provide a way to automate deployment processes across multiple environments. This post illustrates use cases where you can apply these events for automation. For more information, refer to Amazon QuickSight and the QuickSight Developer Guide.
If you have any questions or feedback, please leave a comment. For additional discussions and help getting answers to your questions, check out the QuickSight Community.
Join the Quicksight Community to ask questions, share answers, learn with others, and explore additional resources.
About the Authors
Mayank Agarwal is a product manager for Amazon QuickSight, AWS’ cloud-native, fully managed BI service. He focuses on embedded analytics and developer experience. He started his career as an embedded software engineer developing handheld devices. Prior to QuickSight he was leading engineering teams at Credence ID, developing custom mobile embedded device and web solutions using AWS services that make biometric enrollment and identification fast, intuitive, and cost-effective for Government sector, healthcare and transaction security applications.
Arun Santhosh is a Senior Solution Architect for Amazon QuickSight. Arun started his career at IBM as a developer and progressed on to be an Application Architect. Later, he worked as a Technical Architect at Cognizant. Business Intelligence has been his core focus in these prior roles as well.