AWS Business Intelligence Blog

Accelerate migration from traditional BI tools to Amazon QuickSight with generative AI and Storm Reply

This is a guest post written by Francesco Ongaro, Piero Ciffolillo, Alessandro Dovico and Matteo Lanati of Storm Reply and Marcello Tava and Toaha Umar of BMW Group.

Business intelligence (BI) and data visualization are vital for making quick, informed decisions. They provide insights and visibility into processes, performance, costs, and customer behavior, enabling businesses to adapt swiftly to market changes and improve operational efficiency across various areas.

However, the presence of multiple and diverse traditional on-premise solutions can often hinder innovation due to slow functional evolution, data fragmentation, scalability issues, and high maintenance and licensing costs.

To address these challenges, the BMW Group is continuously exploring cloud-native BI solutions. By adopting these tools, the BMW Group aims at overcoming the obstacles posed by traditional systems and solving scalability and operating cost issues.

In this post, AWS premier consulting partner Storm Reply and the BMW Group share how they collaborated to migrate BMW Group’s traditional on-premises BI tools to Amazon QuickSight. By using advanced code automation, generative AI, and an AWS serverless architecture, the team simplified the BMW Group’s BI processes, reduced manual efforts, and achieved scalability. The post outlines the specific challenges encountered during the migration and explains how they were overcome through a collaborative approach, using cloud-centered solutions.

The results of the pilot, achieved in just 6 weeks, highlighted the potential of the solution as a standard migration tool. This led the BMW Group to extend its cooperation with Storm Reply to industrialize the solution at scale.

Key challenges in the migration process

BMW Group faced the significant task of migrating a very large number of on-premises dashboards. Transitioning traditional BI tools to a modern, cloud-centered, fully managed solution like QuickSight requires substantial effort and expertise.

Key challenges include manual data transformation, converting dashboards and functionalities (such as calculated fields) between different tools, and providing a smooth transition without disrupting ongoing operations while maintaining a similar user experience.

Streamlined migration to QuickSight for enhanced BI

For this pilot, we selected a number of dashboards hosted on an internal BMW Group data exchange platform for business analysts This platform is visited by more than 10,000 unique internal BMW Group employees yearly and contains more than 100 dashboards. It is currently based on Tableau on-premises servers.

The pilot goals were to demonstrate that following aspects:

  • The user experience would be adequate and similar to Tableau
  • The solution would offer a high level of automation to enable large-scale migration
  • Manual code conversion would be minimal, facilitated by an upfront complexity analysis report, thanks to generative AI
  • The solution would be generic enough to be applied to other BI tools in the future

To achieve these goals, we began to define a modular approach, described later in this post, that addresses the main challenges. We developed conversion libraries and scripts to automate most tasks, using generative AI to assess the complexity of the transformation and translate tool-specific configurations that would otherwise require significant manual effort. Our goal was to create an automated, agnostic solution that could be scaled and evolved to accommodate other BI tools besides Tableau.

Solution overview

We started by designing a migration workflow, keeping in mind the importance of defining a block strategy, useful for future evolutions.

The workflow, depicted in the following diagram, begins with the export of a Tableau file describing the component of a dashboard.

The file is then analyzed by our generative AI complexity analysis tool, which estimates the time it takes a developer to perform the relevant manual transformation tasks. This is important to understand how much time and resources we could save through this automated solution.

The next step is automation, where the conversion from Tableau to QuickSight takes place.

The QuickSight API is invoked by passing the results obtained in the previous step.

Finally, the dashboards are ready and available in QuickSight and can be audited by the end-user.

Complexity analysis

Within the complexity analysis component, described in the following diagram, we developed a parsing script that takes the Tableau file and extracts all the parameters of a specific dashboard, such as how many charts are present, the type of charts, the number of fields calculated, and so on.

The results are stored in a JSON file that is then sent to a large language model (LLM) along with appropriate prompts. The result of the complexity analysis tool is a complexity report.

The solution is based on Amazon Bedrock, and we used Anthropic’s Claude LLM family.

Automation

The automation component consists of two different streams, as shown in the following diagram. The first stream, at the top, starts parsing and analyzing the Tableau file and extracts relevant information about the main components, such as charts, labels, colors, and more.

The results are fed into an intermediate file, which we designed. This allows us to have a solution agnostic to BI platforms, which can be used in the future to translate other different BI tools. From there the intermediate file is processed by an analysis script that creates a QuickSight model.

This model is then merged with the result from the lower branch, where the Tableau input functions reside, which are language-specific expressions used to change the appearance of the dashboard.

The resulting data flows into an LLM that performs a crucial task: translating the calculated fields from Tableau to QuickSight, because they use a different syntactic language.

Finally, the results of the combinations between the QuickSight calculated fields and the analysis model are sent to the QuickSight API, which creates all the dashboards with their filters, labels, colors, and so on.

Pilot migration outcomes and insights

During this pilot, we focused on five types of graphs using anonymized infotainment data from the BMW Group:

  • Line chart
  • Horizontal bar chart
  • Vertical bar chart
  • Table chart
  • Map chart

After developing the components outlined in the previous sections, we executed our migration workflow. The results of the conversion are illustrated in the following figures, where we compare the initial dashboard (on the left) with the QuickSight dashboard (on the right), which was automatically generated by our solution. All data displayed is examples or mock-up data without real meaning.


Tableau Line Chart
QuickSight Line Chart
(Automatically Generated)

Tableau Horizontal Bar Chart

QuickSight Horizontal Bar Chart
(Automatically Generated)

Tableau Vertical Bar Chart
QuickSight Vertical Bar Chart
(Automatically Generated)

Tableau Table Chart
QuickSight Table Chart (Automatically Generated)

Tableau Map Chart

QuickSight Map Chart (Automatically Generated)

What’s next

At this stage, the joint team of Storm Reply and BMW Group aims to expand the range of chart types available, improve complexity analysis, and enhance accuracy through generative AI, covering complex data correlation and transformation, making migrations even more efficient.

Conclusion

The pilot project, conducted over 6 weeks, successfully demonstrated the effectiveness of automating the migration of traditional BI dashboards to QuickSight. By using advanced automation scripting and generative AI, we achieved a significant reduction in migration efforts, up to 80%, compared to traditional manual methods. This approach enabled us to convert complex data visualizations and charts from existing BI tools into the specific syntax and format required by QuickSight, achieving a high preservation rate of 90% of the original data after conversion without manual intervention.

The complexity analysis tool played a key role in estimating migration efforts, enabling informed planning and resource allocation. By identifying potential challenges early, we improved the overall efficiency of the project. In addition, the insights gained from this analysis are critical in defining an accurate business case for this type of project, providing valuable data to assess its feasibility.

Overall, this project highlights the transformative potential of combining automation and generative AI in modernizing BI processes, enabling organizations to use data more effectively and make informed decisions quickly.


About the Authors

Francesco Ongaro is a Senior Manager and Business Unit Manager at Storm Reply, based in Munich, Germany. He graduated with honors from the University of Bologna and was a Visiting Researcher at the Network Research Laboratory of the University of California, Los Angeles. An experienced leader with over 10 years in AWS, he manages large teams and drives business growth for German and Italian companies with a focus on innovation and customer success. He likes snowboarding, hiking, and traveling with his wife, and spending times with good friends.

Marcello Tava is Product Owner for GenAI Solutions for IT at the department of Artificial Intelligence of the BMW Group. He graduated at Milan Polytechnic University in Aerospace Engineering and holds a PhD in Numerical Optimization of System Design from the University of Tokyo. He is the author of more than 20 patents in the fields of car navigation systems, route optimization, and data processing. He is devoted to developing IT solutions that can add more fun and reduce the workload of his fellow colleagues. He is member of the German Watercolor Society and loves playing the piano and hiking in the South Tyrol mountains.

Piero Ciffolillo is a Consultant at Storm Reply, based in Munich, Germany. He has an academic background in physical engineering and data science. He has a proven track record in providing innovative infrastructure services and designing generative AI solutions in the cloud. In his free time, Piero enjoys playing football, performing as a bass guitarist in an amateur band, and cooking.

Alessandro Dovico is a consultant for Storm Reply and is based in Turin, Italy. He has a background as a full-stack developer and data visualization expert for Amazon QuickSight. He has contributed to numerous projects involving data visualization, cloud architecture, and full-stack development, using AWS services to create seamless, robust platforms. In his free time, he enjoys reading and watching movies.

Toaha Umar is an AI Engineer at BMW, specializing in generative AI within the BMW AI Solutions team. He has a strong background in generative AI, MLOps, AWS solution architecture, and project management, and previously led the AI Task Force at Storm Reply. In addition to his professional commitments, Toaha has over 7 years of experience in leadership positions in non-profit organizations, including IEEE and TUM.ai, Europe’s leading AI student initiative. He is passionate about mentoring young professionals and contributing to the AI community. In his free time, he enjoys reading, cooking, and traveling, and loves exploring novel technologies along with culinary experiences.

Matteo Lanati is a Senior Consultant at Storm Reply Germany in Munich. He has an academic background in telecommunications and over 10 years of experience as a system administrator and in DevOps. He has contributed to multiple projects on topics such as infrastructure as code, automation, migration to Kubernetes, and architecture design based on AWS services. In his free time, he likes reading and climbing.

About Storm Reply

Storm Reply is the AWS expert company within the Reply network. Storm has been an AWS premier consulting partner since 2014 and was named the AWS System Integrator of the Year (EMEA) in 2022 and 2023. Using several AWS competencies and with a strong and trusted relationship with AWS, Storm helps leading enterprises, higher mid-markets, and digital natives use the ever-growing AWS ecosystem effectively and target-oriented. They have a proven track record in industry-tailored cloud adoption and migrations and state-of-the-art software development, resulting in generative AI and IoT innovations all across the DACH market and global automotive industry.