High Performance Cloud Computing Supports Disease Prevention

The Walter and Eliza Hall Institute of Medical Research, the oldest medical research institute in Australia, undertakes research across a range of areas including breast, ovarian, and blood cancers, type 1 diabetes, rheumatoid arthritis, coeliac disease, and malaria. More than 60 clinical trials based on discoveries made at the institute are underway. These include trials of vaccines for type 1 diabetes, coeliac disease, malaria, and trials of a new class of anti-cancer agents for treating patients with leukemia.

Recently, the Systems Biology and Personalised Medicine Division (SBPM) at the Walter and Eliza Hall Institute enlisted the help of DiUS and AWS to accelerate the processing and analysis of embarrassingly parallelizable data and image sets. (An embarrassingly parallel workload or problem is one where little or no effort is needed to separate the problem into a number of parallel tasks and scale these independently).

In a pilot study, SBPM was interested in exploring the use of cloud computing to reduce the time it took to analyze high-resolution microscopy data. To solve this challenge, the institute enlisted the help of DiUS to discover and validate a new approach for data analysis, leveraging a cloud-based capability on AWS. SBPM also asked DiUS to help prototype a platform that would support the orchestration of High Performance Computing (HPC).

Starting small, DiUS facilitated an ideation session to better understand the scientists’ needs and define an overarching solution. The session quickly identified the need for on-demand and short-lived computing clusters, customized and templated for each laboratory. With the scope defined, DiUS partnered with SBPM technical analysts and AWS scientific computing specialists to develop a cost-effective, scalable, and secure solution.

The team built a platform in the microscopy laboratory to enable image scientists to perform initial exploratory analysis locally on their workstations, then seamlessly synchronize the local data sets to AWS, and perform detailed analysis in the cloud. Upon completion, results are transferred back to the scientists’ workstations. This approach has reduced typical image processing times from seven hours to less than one. With AWS, scientists can quickly analyze massive data pipelines, store petabytes of data, and share their results with collaborators around the world, focusing on science rather than servers.

Meanwhile, in the proteomics laboratory, a similar setup enabled scientists to auto-scale R based analysis of mass-spectrometer genomics data. Beyond just computation acceleration, these early successes show that it is feasible to augment on-site HPC with on-demand computing. Innovations like these have enabled the Walter and Eliza Hall Institute to remain at the forefront of medical research and contemplate future research directions using these newly proven scientific computing and research capabilities.

Read the full case study here and learn more about the technology behind the project here.

Discover more about AWS and scientific computing at https://aws.amazon.com/government-education/scientific-computing/