AWS Business Intelligence Blog

Amazon QuickSight empowers business users on the automotive production floor with near real-time digital process boards

This post was jointly authored with André Schmidt of BMW Group.

Digital process dashboards serve as essential tools for process leaders and management to monitor and analyze standardized key performance indicators (KPIs), such as scrap rate per machine or number of system malfunctions per shift. In BMW Group‘s manufacturing plants, these dashboards play an integral role in daily status meetings and are used by process leaders at the production line, giving them visibility into the production processes they own.

Creating, updating, and operating dashboards efficiently presents several challenges that require innovative solutions and expertise to overcome:

  • Each process leader has unique dashboard requirements, necessitating collaboration with the IT team. This led to delays due to limited IT capacity and back-and-forth communication.
  • As a consequence, process leaders often built dashboards without IT involvement using various tools and services, resulting in a complex application landscape that was difficult to maintain and operate.
  • Production-floor data was siloed in production databases, making it challenging to access the data needed for the dashboards.
  • Some analytical systems were connected directly to the production-floor production databases, posing a risk of downtime on the production floor.
  • Inconsistent KPI calculations across different dashboards caused misalignment between teams.

To address these challenges, BMW Group launched the Shop Floor Business Intelligence (SFBI) initiative. The goal was to empower process leaders to create and manage dashboards without IT involvement, based on standardized KPIs calculated from data in a central data lake and a standardized tool set and architecture.

In this post, we explore how BMW Group used AWS analytics services, AWS Partner solutions, and Amazon QuickSight to build the SFBI solution.

Why QuickSight?

Using QuickSight, users can create and share interactive dashboards and visualizations without extensive technical expertise, enabling self-service dashboard creation by process leaders.

  • Ease of embedding – QuickSight offers seamless embedding capabilities so users can integrate dashboards and analytics directly into their existing digital process dashboard web application (T-Cube), providing a cohesive user experience.
  • Cost-effectiveness – The QuickSight pay-per-session pricing model aligns with BMW Group’s cost optimization goals because they only pay for actual usage rather than fixed licensing fees.
  • Integration with Amazon Web Services (AWS) – SFBI is built on top of BMW Group’s existing AWS based Cloud Data Hub (CDH) data lake, and the deep integration of QuickSight with AWS services streamlines the overall analytics process.
  • Security and compliance – The robust security features of QuickSight, including row-level security and integration with AWS Identity and Access Management (IAM), help maintain data privacy and comply with guidelines and regulations.

By using QuickSight and the AWS analytics services, BMW Group was able to build the SFBI solution and empower process leaders to create and manage dashboards without IT involvement, based on standardized KPIs calculated from data in a central data lake and a standardized tool set and architecture.

Solution overview

The SFBI solution was built with QuickSight as the business intelligence (BI) layer, providing an intuitive interface for process leaders to create and manage dashboards without IT involvement. The seamless embedding capabilities of QuickSight equips users to integrate dashboards and analytics directly into their existing digital process dashboard web application (T-Cube), offering a cohesive user experience. The following figure shows a QuickSight dashboard seamlessly embedded into T-Cube.

The BI layer interacts with two primary data sources: Amazon Athena and Snowflake. Amazon Athena is used to access precalculated KPIs and other data stored in the CDH, BMW Group’s company-wide data lake. For KPIs that require on-the-fly calculations based on user-set filters and parameters, BMW Group uses user-defined functions (UDFs) in their Snowflake data warehouse. To avoid data duplication and reduce complexity, Snowflake directly accesses external Apache Iceberg tables stored in the CDH, providing the computational power necessary for fast KPI calculations and a satisfying user experience in Amazon QuickSight.

The semantic layer of the data lake uses Apache Iceberg tables, which enable efficient update and delete operations. This makes sure that KPI calculations are always executed on a consistent snapshot of the raw data tables. The use of Apache Iceberg is particularly important for data sources ingested through Kafka streams, where rows need to be updated rather than simply inserted.

Moving upstream, AWS Glue extract, transform, and load (ETL) capabilities are employed to clean, validate, and standardize data from the source layer before moving it into the semantic layer of the data lake. The source layer consists of JSON files containing all Kafka records ingested from manufacturing and logistics systems.

At the data ingestion level, the BMW Group standardized the process through a Confluent Kafka streaming layer for SFBI. This makes sure that near real-time data flow from manufacturing and logistics systems into the data lake. By using the CDH, manufacturing and streaming data can be integrated and enriched with over a thousand other data assets, enabling innovative dashboard experiences and relating manufacturing data to other parts of the business. Refer to the following figure for an illustration of the architecture.

Conclusion

By using AWS analytics services, integrating with their existing Cloud Data Hub (CDH) and using a Snowflake data warehouse, BMW Group has successfully addressed the challenges of standardizing KPI calculations for production floor operational reporting. By using QuickSight and embedding with their existing T-Cube solution, business users are now able to create new dashboards based on standardized KPIs and integrate with other data assets in the CDH to generate insights that weren’t possible before. By empowering business analysts with the tools to derive insights from data, BMW Group has paved the way for informed decision-making, ultimately driving its competitive edge in the fast-paced automotive landscape.


About the Authors

Florian Seidel is a Global Solutions Architect specializing in the automotive sector at AWS. He guides strategic customers in harnessing the full potential of cloud technologies to drive innovation in the automotive industry. With a passion for analytics, machine learning, AI, and resilient distributed systems, Florian helps transform cutting-edge concepts into practical solutions. When not architecting cloud strategies, he enjoys cooking for family and friends and experimenting with electronic music production.

André Schmidt is the Product Manager of the Shopfloor Business Intelligence Platform at BMW Group. He is one of the founders of the SFBI product, where he is engaged as a Lead Architect focusing on the overall architecture and the benefits for business. André is interested in all facets of software development and cloud technologies. Outside the office, he cherishes family time and has a keen interest in craft.

Vineet Singh is a Principal Solutions Architect at Amazon Web Services with over 20 years of data analytics expertise. Specializing in data analytics and AI/ML, he helps customers unlock business value. Passionate about transformative technology, he collaborates with customers to formulate and solve complex problems in data analytics, AI, and machine learning, delivering impactful solutions that address their unique challenges.

Carlo Degli Atti Panzeri is a Senior Go-to-Market Specialist at AWS, where he guides automotive and manufacturing customers through their digital transformation journey to become data-driven organizations and make well-informed decisions. He helps OEMs shape their data strategies and grow their businesses with the adoption of AWS analytics services and solutions.