How BMW Group optimizes software builds using AWS VM Import/Export and Amazon EBS Snapshot Copy

The BMW Group (BMW) is a global manufacturer of premium automobiles and motorcycles, covering the brands BMW, BMW Motorrad, MINI, and Rolls-Royce. Since 2022, BMW has started using AWS to scale its capacity to handle vast increases in needed compute and storage capacity to build and test automated driving features, such as adaptive cruise control, parking assist, and piloted driving, as well as embedded automotive systems, such as infotainment or telematics. In 2023, BMW announced its continuous partnership with AWS to build BMW’s advanced driver assistance system (ADAS), a new cloud-based system that allows BMW’s development teams to respond more quickly to customer demands and build next-level features, such as the all-new BMW iDrive, to be released in 2025 with the BMW Neue Klasse models.

Figure 1: Stephan Durach, BMW SVP Connected Company, at AWS re:Invent 2023.

Virtual Machine Import/Export (VMIE) enables customers to import virtual machine (VM) images from their existing virtualization environment to Amazon Elastic Compute Cloud (Amazon EC2) and then export them back. This enables you to migrate applications and workloads to Amazon EC2, copy your VM image catalog to Amazon EC2, or create a repository of VM images for backup and disaster recovery. With Amazon Elastic Block Store Snapshots (EBS Snapshots), customers can create point-in-time snapshots of Amazon EBS volumes and copy these snapshots from one AWS Region to another, or within the same AWS Region. Using EBS Snapshots and its copy capability, you can scale your workloads across different AWS Regions, back up your data and logs across geographical locations at regular intervals, and encrypt or re-encrypt a snapshot with an AWS managed key or customer managed key for data retention and auditing needs.

In this post, we describe how VMIE and EBS Snapshots work together to assist BMW in streamlining its software development cycle. We discuss leveraging incremental EBS Snapshot copies to reduce the time for copying snapshots, which saves on data transfer and storage costs. You can use this solution when faced with a similar need in your development cycle.

Challenges for optimizing at scale

In recent years, BMW has been setting up a common software development platform, CodeCraft, which is part of its overall “Software Factory.” During BMW’s development cycles, this software development platform executes at peak more than 110,000 continuous integration and continuous delivery (CI/CD) builds daily, using over 60,000 virtual CPUs. Zuul-CI is the CI/CD tool of choice because it offers project gating based on speculative merges, preventing broken branches. These builds can be very complex, relying on up to thousands of direct and transitive dependencies. Frequently, it takes several hours to generate artifacts up to several hundred GBs, causing two major challenges:

1. For any missing or outdated dependency, the required version must be downloaded on-demand from the central artifact repository during pipeline execution. This results in a longer build execution, making the system wait instead of doing useful work. By having cached dependencies read locally, we can accelerate pipeline execution. For each minute saved in pipeline execution time on average, we can reduce the daily Amazon EC2 peak usage by over 1,800 hours (1 minute x 110,000 pipeline executions). Assuming the m6i.4xlarge instance is used for these workloads, BMW is saving almost $1,700 per day ($0.92 per hour in Frankfurt x 1,833 hours) for every minute saved during pipeline execution.
2. As BMW aims to deliver higher-level autonomous driving functions, the demand for the software development platform (SDP) is expected to grow substantially. All these new initiatives are managed and tracked with a strict timeline. To meet these milestones, the development team has a limited window each night to update CI/CD build images with the latest dependencies and prepare pipelines for the next day’s new commits.

BMW leverages VMIE and Amazon EBS to streamline its development cycle

To address these challenges, BMW uses VMIE and Amazon EBS to improve its development cycle efficiency. The process begins with EC2 instances executing nightly tasks on BMW’s SDP environments in Frankfurt, Stockholm, and Dublin simultaneously. These EC2 instances are preconfigured to create Amazon Machine Images (AMIs) for different CI/CD pipelines aiming to produce embedded automotive systems such as ADAS, telematics, and infotainment systems. These EC2 instances pull in all necessary build and test dependencies, which are then stored within EBS volumes attached to an EC2 instance. The EBS volumes function as the primary storage for disk images that contain the compiled dependencies and environment configurations. In addition, BMW is able to capture the precise state of the volume at a specific moment by using EBS Snapshots on these disk images. The snapshots are reliable data points for backup and essential for disaster recovery and data replication. These EBS Snapshots are taken to create the AMIs, having all the latest dependencies “baked in.” With new changes pushed from engineers to the Git repo, future CI/CD pipeline executions leverage these new AMIs.

Figure 2: Interaction between the daily Amazon EC2 AMI creation process and the usage of those AMIs in the CI pipeline.

When BMW needs to deploy its environments across multiple AWS Regions and accounts, it relies on the VMIE service. The service facilitates the import of disk images from Amazon Simple Storage Service (Amazon S3) to create new snapshots within the Amazon EC2 environment. In parallel, VMIE works in conjunction with EBS Snapshot Copy to transfer these snapshots from a VMIE service account to a BMW development account. BMW is able to create AMIs from the snapshots after they are in its development account. By using the combination of VMIE and EBS Snapshot Copy, the BMW development teams gain access to consistent, updated AMIs in these different AWS Regions. Having updated AMIs readily available in different AWS Regions is crucial for resilience and for enabling engineers to have a short iteration cycle on their commits.

Figure 3: Diagram to illustrate the VMIE and EBS Snapshot copy operations

Leveraging incrementality for more benefits

While the combined use of VMIE and EBS Snapshot Copy provides unique benefits, the transferred snapshots are full (non-incremental) because each snapshot imported and exported by VMIE possesses unique identifiers that are essential for creating new AMIs. Currently, customers copy snapshots directly within or across AWS Regions to fulfill various needs, such as disaster recovery, workload migration, or launching applications in different geographical areas. To optimize this process, we recommend the adoption of incremental snapshot copies. To enable incremental snapshot copies, we save only the blocks on the device that have changed since the most recent snapshot copy. This reduces the time needed to copy snapshots and saves on data transfer and storage costs by not duplicating data. When you copy a snapshot across AWS Regions or accounts, the copy is an incremental copy if the following conditions are met:

• The snapshot was previously copied to the destination AWS Region or account
• The most recent snapshot copy still exists in the destination AWS Region or account
• All copies of the snapshot in the designated AWS Region or account are either unencrypted or were encrypted using the same AWS Key Management Service (AWS KMS) key

Conclusion

In this post, we provide an overview of how VMIE and EBS Snapshot Copy support BMW to streamline the development cycle to meet its objectives. The use of VMIE and EBS Snapshot Copy can help customers back up data and migrate applications from one account to another across different AWS Regions, or within the same AWS Region, while saving on cost and time. To learn more about VMIE and EBS Snapshot Copy, visit:

• What is VM Import/Export?
• Copy an Amazon EBS Snapshot