Hyperparameter optimization for quantum machine learning with Amazon Braket

This post was contributed by Jean-Michel Lourier, Tanvi Singhal, David Sauerwein, and Fabian Furrer

We’ve been working to implement a hybrid quantum-classical algorithm for machine learning that includes hyperparameter optimization (HPO) on Amazon Braket, the AWS service for quantum computing. This involves iteratively tuning the free parameters during training to find the most performant quantum machine learning (QML) algorithm.

This approach presents two major challenges though:

1. How can we ensure that the experiments are reproducible and well tracked?
2. How can we use resources in the most cost-effective and time-efficient manner during the iterative development process?

These are common problems in the machine learning community, too, which means we can leverage their experience to address these issues for hybrid quantum-classical algorithms. We discussed solutions for the first challenge in an earlier post.

Today, we’ll address the second challenge and show how to construct a cost-effective development cycle for training and evaluating a hybrid quantum-classical algorithm for image classification.

We’ll use Amazon Braket notebooks and Amazon Braket Hybrid Jobs, employing simulators and quantum devices with hyperparameter optimization. The quantum machine learning (QML) algorithm we consider is based on the image classification methodology introduced by Huang et al. and our implementation draws from the quantum transfer learning published by PennyLane.

Developing QML with HPO

For the image classification we use a variational quantum algorithm which is a type of hybrid quantum-classical algorithm that combines quantum circuits with classical optimization.

In a variational quantum algorithm, you start with a parameterized quantum circuit, typically consisting of a series of quantum gates with adjustable parameters. The aim is to find the optimal set of parameters that solves your specific problem. This approach is akin to traditional machine learning, where developers iterate through various architectures and parameter configurations to identify the most effective solution.

A key consideration in this process is deciding how to use resource-intensive (but highly effective) tools like GPUs to expedite the discovery of the optimal solution.

For this post, we show how to build a three-step development cycle using Amazon Braket resources to tackle the main challenges of optimizing hybrid quantum-classical QML algorithms with HPO:

1. Ideation in Amazon Braket notebook: At the beginning of algorithm development, it’s important to be able to test new ideas quickly at small scale. Using the Amazon Braket SDK in Jupyter notebooks, we can explore ideas by running classical simulations for small numbers of qubits in a Braket notebook – and iterate quickly.
2. Scaling with Hybrid Jobs and running HPO: After establishing the basic architecture of the algorithm, we conduct multiple parallel experiments to identify the ideal hyperparameters within a given quantum noise model. Using Amazon Braket Hybrid Jobs, we initiate multiple asynchronous jobs on dedicated devices. The classical component of the algorithm runs on a dedicated EC2 instance, while the quantum computations are carried out on the Amazon Braket-managed density-matrix simulator, DM1. This enables us to scale the algorithm up to 17 qubits, enabling the discovery of the optimal hyperparameters. Alternatively, quantum computations can be executed on Amazon Braket’s local density matrix simulator for cost-effective HPO in a hyperparameter space with up to 12 qubits.
3. Verification on Braket quantum processing unit (QPU): Finally, we verify the performance of the identified parameters on a real QPU. Even though we include simulated noise in the previous steps, results on a real quantum device are still likely to vary, so we need to verify them.

Going through the development workflow will help you to gain several key insights and skills:

• Dataset reduction: We demonstrate techniques to reduce datasets, making the example feasible for quantum devices, ensuring readers can replicate the process effectively.
• Model-training: Learn how to train a hybrid classical-quantum model for computer vision using Amazon Braket.
• Simulator variants: Understand the nuances of running model training with a local simulator and a high-performance simulator to gauge performance differences.
• Limitations exploration: Explore the inherent limitations when training a quantum neural network (QNN) and gain insights into what to expect in real-world applications.
• Efficient testing: Discover efficient methods for testing a QNN – crucial for model validation.
• Estimating costs: Understand how to measure and estimate the costs for invoking Amazon Braket managed simulators or devices.

Today, our primarily aim is to conceptually demonstrate the implementation process, rather than focusing on training models with high predictive power. However, the techniques and principles we present here can lay a foundation for more specialized and application-specific quantum machine learning projects.

Grasping the concepts and practical steps outlined in this post will let you tailor and apply these strategies later to your specific use cases and domains.

Quantum image classification

Transfer learning is a well-established technique for training artificial neural networks. A pre-trained model is a saved network that was previously trained on large datasets. In the case of image processing, the pre-trained model is typically on a large-scale image-classification task. You can either re-use the pre-trained model as-is, or use transfer learning to customize it to a given task. The key is that the pre-trained model has learned representations of many generic objects in the visual world. You can then take advantage of these learned feature maps without having to start from scratch by training a large model on a large dataset.

Here we look into a classification task to distinguish between images of bees and ants. We’ve shown some example images in Figure 1.

Figure 1: Example images used to train the QML algorithm [Transfer Learning for Computer Vision Tutorial].

While in general it’s not yet clear if a QML algorithm has the potential to improve performance, it does open the space for new modelling approaches that you can evaluate using the development approach we’ll discuss in this post.

Solution overview

Figure 2 shows a simple solution for executing hyperparameter optimization (HPO) for QML on an Amazon Braket notebook instance.

For each combination of hyperparameters, we call the Hybrid Jobs API for training and evaluating the quantum image classifier. In the Hybrid Jobs execution process, the quantum computational component is executed on either a managed simulator or a Braket quantum processing unit (QPU), depending on the solution’s development stage. When the HPO completes, the notebook retrieves the results from an Amazon Simple Storage Service (Amazon S3) bucket.

We’ll cover the various development stages, along with detailed discussions on the Amazon Braket Hybrid Jobs – and the HPO process – later in this post.

Figure 2: Solution architecture to train and test the QML algorithm.

Ideation in Braket notebooks at small scale

Braket offers a technology-agnostic API to build, test, run and analyze quantum algorithms. We leveraged this feature to test our implementation at small scale before progressing to a high-performance simulator and a real quantum device. This allows for cost- and time-effective testing and debugging, before executing our algorithm at full scale.

Initially, we test the QML algorithm with a local simulator – that is, we execute the algorithm on the notebook instance using local (classical) compute resources. Because of the relatively limited resources of the notebook instance, we execute the training and evaluation of the quantum neural network at very small scale. We reduced the size to fit on 4 qubits, which you can see in the example notebook later.

Scaling out with Hybrid Jobs and running HPO

Hybrid Jobs

To scale out the solution in a next step, we execute the QML training and evaluation using Amazon Braket Hybrid Jobs. Hybrid Jobs are designed to execute iterative algorithms that require classical and quantum resources.

The algorithm is deployed within a classical container environment, from which quantum resources are requested as specific quantum tasks. This offers three key advantages. First, it allows for the cost-effective use of quantum resources on a ‘pay-as-you-go’ basis. Second, the quantum tasks submitted by a hybrid job benefit from higher priority queueing to the same QPU. This allows executing the iterative algorithm invoking classical and quantum resources effectively. Third, the quantum and classical resources can be scaled beyond the limit of the local resources of notebook instance, which means the QNN can be scaled out.

In the previous section, we trained a small scale QNN on four qubits. As a next step, we train a model using the Braket DM1 high-performance simulator using a hybrid job. This lets us target a greater number of qubits which isn’t feasible to compute with the local simulator. Additionally, DM1 allows for simulating noise. As noise, we choose a depolarizing quantum channel that replaces (with a certain probability) the state with a completely mixed state. In our notebook example, we assume a depolarizing rate of 1% for single-qubit gates, and 9% for two-qubit gates.

We use this setup for optimizing the hyperparameters of the quantum image classifier. Once we find the optimal hyperparameters, we train the classifier and evaluate it on a real quantum device but with the optimal parameters so we can benchmark the results with real noise influence.

Hyperparameter optimization (HPO)

HPO is an important technique in machine learning that involves tuning the hyperparameters of a model to maximize its performance on a given task.

In quantum machine learning, we use HPO to optimize the hyperparameters of a quantum circuit to improve its performance on a specific task.

HPO for QML involves searching through a space of possible hyperparameters to find the optimal configuration for a given task. This space can include hyperparameters like the number of qubits, the depth of the quantum circuit, and the strength of the gate operations. We can do this by:

1. Using classical optimization algorithms, like Bayesian optimization or genetic algorithms, to search through the hyperparameter space. We can use these algorithms to evaluate the performance of a given hyperparameter configuration and adjust the search based on the results.
2. Using reinforcement learning (RL) algorithms to optimize the hyperparameters. RL algorithms can learn to adjust the hyperparameters based on the feedback received from the environment, which can lead to more efficient and effective optimization.

HPO for QML is an active area of research, with many new techniques and algorithms being developed to improve its effectiveness and efficiency. It has the potential to significantly improve the performance and scalability of quantum machine learning algorithms, making them more practical for real-world applications.

In our solution, we leveraged the open-source Ray Tuner for a distributed search for optimal hyperparameters. Ray Tuner integrates many different hyperparameter search frameworks. We use the open-source package HyperOpt as a search framework, enabling us to use tree-structured Parzen estimators (TPEs) to search the hyperparameter space shown in Table 1 efficiently.

This hyperparameter space includes configurations with up to 12 qubits. We can simulate configurations with up to 12 qubits using the local density matrix simulator in Amazon Braket. However, here we show an implementation on DM1 which also allows for scaling beyond the hyperparameter search space shown in Table 1 in terms of number of qubits. Hence, we use DM1 which can simulate circuits of up to 17 qubits.

TPE approximates a response surface for the target metric based on trial results and selects promising sample candidates based on that value. This allows for an efficient, distributed search in the hyperparameter space. We optimize for cross entropy loss on a validation dataset and find the lowest validation loss of 0.67 for the configuration shown in Table 2.

Table 1: Hyperparameter space for HPO.

Table 2: Optimal configuration based on HPO.

Amazon Braket Cost Tracker allows for tracking statistics of the involved measurement shots and quantum tasks. We used the Braket pricing page to estimate the costs of the HPO trials and for predicting the costs of the verification step discussed in the next section.

Our HPO with eight samples from the hyperparameter space incurred a cost for all trials of around $20. This excludes costs for the Amazon Elastic Compute Cloud (amazon EC2) instances that provided the classical compute. Running the QML algorithm with the configuration described in Table 3 incurred Amazon Braket charges of around $1.30 and required 8800 measurement shots and 352 quantum tasks.

You can find more details in the accompanying example notebook.

Validation on Braket QPU

From the QML HPO, we identified a promising configuration which we’ve detailed in Table 3.

We conducted the HPO using simulated noise on the DM1 simulator – of course, the outcomes on a real quantum device might differ. To validate the results of the HPO, we tested the best configuration on an Braket QPU. Braket offers a technology-agnostic interface to the quantum hardware, and that allows us to choose the best fitting hardware for our use case. We chose the QPU Lucy from Oxford Quantum Circuits (OQC), because it’s one of the most cost-efficient options and offers the required resource, which in this case was 5 qubits.

We estimated the costs for running the best configuration on OQC Lucy by means of the statistics gathered with Braket Cost Tracker from the HPO runs. Running the best configuration required 352 quantum tasks and 8800 shots. Using Braket pricing again, we estimated the costs for the run on the Braket QPU to be $0.30/shot x 8800 shots + $0.30/task x 352 tasks = $108.68.

This shows the cost-efficiency of our approach to run a QML HPO on a simulator first, and validating it on a QPU after, rather than executing the QML HPO directly on a QPU. The run on OQC Lucy resulted in a validation loss of 0.63, which is fairly aligned with the result we obtained using Braket DM1 with a validation loss of 0.67.

Prerequisites

For this walkthrough, you should have the following prerequisites:

• An AWS account
• Permissions to provision AWS Braket.

Cost estimate

The cost of deploying this solution as a proof of concept is projected in Table 3. We assume you’ll use the managed notebook for execution and exploration for about 12 hours.

Table 3: Cost estimation

Deployment and execution

To deploy this and get going:

1. Navigate in Amazon Braket console to the London (eu-west-2) region.
2. Create a Braket notebook (NotebooksCreate notebook Instance) and select t3.2xlarge as the instance type while keeping remaining parameters as your default.
3. Clone the provided example notebook to the notebook instance from:

https://github.com/aws-samples/hpo-for-qml/blob/main/quantum_transfer_learning_with_hpo.ipynb

4. Follow the instructions in the notebook quantum_transfer_learning_with_hop.ipynb.

Cleaning up

To avoid incurring future charges, delete the Amazon Braket notebook Instance, and the Amazon S3 bucket associated with that instance.

Conclusion

In this post, we explored the development of a cost-efficient QML HPO (Quantum Machine Learning Hyperparameter Optimization) algorithm using Amazon Braket.

Initially, we implemented and tested a scaled-down version of the QML algorithm on a local simulator. This facilitates rapid development cycles in the early stages. Secondly, we run an HPO to find a promising configuration of the full-scale QML algorithm on Braket DM1 managed simulator. That allowed us to identify good hyperparameters in a cost-efficient way.

We simulated noise in the HPO, and then ran the best HPO configuration on a real Braket QPU with real noise impact to validate the results.

If you want to learn about QML on Amazon Braket we recommend diving deeper into our other posts on Tracking quantum experiments with Amazon Braket Hybrid Jobs and Amazon SageMaker Experiments or Using Quantum Machine Learning with Amazon Braket to Create a Binary Classifier.