AWS Public Sector Blog
Cloud powers faster, greener, and more collaborative research, according to new IDC report
Collaboration is key to solving the most challenging problems facing humanity. From drug discovery to climate change, scientists from across institutions, disciplines, and countries are working together to innovate new breakthroughs—and their ability to do so is accelerated by cloud technology. According to a new IDC report, the cloud is helping researchers conduct research faster than ever before by reducing data analysis and processing times, and is allowing researchers around the world to collaborate on solving universal problems. The IDC report, titled “Cloud Powering Faster, Greener, and More Collaborative Research,” estimates that cloud-based infrastructure compute spending by educational institutions will increase more than 28% annually through 2025, to surpass $5 billion.
The report—which reflects perspectives from researchers in Australia, Canada, the US, Colombia, and the UK—highlights five main benefits for researchers: 1) scalability and data processing capabilities that reduce time to get to insights; 2) efficiency gains from reducing capital expense, procurement, and maintenance burdens; 3) access to a wide network of applications and computing resources; 4) research democratization; and 5) increased research collaboration. In addition to the positive impact on research, IDC also forecasts that continued adoption of cloud computing globally could prevent environmental emission of more than 1 billion metric tons of CO2 from 2021 through 2024, almost equivalent to removing the 2020 CO2 emissions of Germany and the U.K. combined (IDC Worldwide CO2 Emissions Savings from Cloud Computing Forecast, 2021-2024).
With Amazon Web Services (AWS) and AWS Partners, researchers are analyzing massive data pipelines, storing petabytes of data, and advancing research using transformative technologies like artificial intelligence (AI), machine learning (ML), and quantum computing. This is resulting in breakthroughs spanning healthcare and the environment. The following AWS customers and partners use the cloud to conduct innovative new research:
Accelerating COVID-19 vaccine research
A researching team from University of British Colombia, Canada is using the cloud to advance COVID-19 vaccine research. “To securely process and move the data from our existing data model, we applied the pipeline and then soon after we started using the cloud to process datasets at an enormous scale of over a million libraries a day. We analyzed 5.7 million samples and found 130,000 new species (a 10x increase) of RNA viruses in 11 days. Traditional ways would have required 8 to 10 months to run this analysis,” one researcher said.
Sorbonne University is conducting research to better understand the molecular functioning of the COVID-19 virus to develop drug treatments. The outcomes of the research will be a public data set that will be shared with the community and could help identify potent drugs. The project relies on the power of the national supercomputing center (GENCI) and of AWS Cloud to perform high performance computing (HPC) simulations, modeling different proteins involved in the SARS-Cov-2 and Cov-1 viruses.
Compute for climate, genomics, and more
Researchers at the University of New South Wales Sydney (UNSW) used AWS to innovate methods to detect ephemeral water floods, common in drylands like Australia’s Murray-Darling Basin. This problem has been exacerbated by climate change and requires, according to the United Nations, “an integrated view on water, the biosphere, and environmental flows… to devise sustainable agricultural and economic systems that will allow us to decelerate climate change, protect us from extremes, and to adapt to the unavoidable at the same time.”
UNSW researchers used large clusters on AWS to run analysis on detecting and reducing the impact of floods. They are innovating the use of new data sources from the existing NASA’s Landsat-8 (L8) satellite and two European Space Agency Sentinel-2 (S2) satellites as input to ML-based mapping of surface water and flooding dynamics. This new method supports real-time information on flood occurrence, predicting floods with 99% accuracy.
When UNSW needed to create a lab for protein structure predication, they were facing a six month-plus wait for the server hardware to be procured. With AWS, they had the environment up and running within two weeks. They also found that AWS supported their unpredictable demand as researchers could run models when they needed, rather than their previous methods of having to book time on the university’s shared high performance computing (HPC) cluster, where it could sometimes take months for an opening to become available.
Computing power is helping researchers in Australia analyze the human genome. Dr. Denis Bauer and her team at Australia’s government scientific research agency, Commonwealth Scientific and Industrial Research Organization (CSIRO), tackles one of the biggest challenges in bioinformatics. They use serverless infrastructure on AWS, demonstrating its ability to handle both data and compute-intensive tasks. Genomic information is frequently multiple gigabytes per subject and quickly scales to terabytes when thousands of subjects are involved. “Traditional database schemas don’t scale to cater for such large cohort sizes and are not economical enough to handle bursting workloads sustainably,” Dr. Bauer from CSIRO said. “We achieved a 300-fold cost saving where instead of spending thousands of dollars to support clinical genomics, AWS and serverless allows critical data to be shared with the world for less than the cost a cup of coffee per day.”
Data processing at scale
The Structural Biology Research Center (SBRC) at the High Energy Accelerator Research Organization (KEK) in Japan worked with Fusic Co., Ltd., an Advanced AWS Consulting Partner, to create a HPC environment built on AWS to analyze large sets of data gathered from their research. KEK is an inter-agency research institute that studies the laws of nature and the basic structure of matter, including use of particle accelerators to speed particles to near light speed, and SBRC studies three-dimensional structures of macromolecules by using synchrotrons and other facilities to understand life activity.
Fusic Co., Ltd. created a menu of data analysis tasks on AWS. This menu provided researchers with the flexibility to customize their own analysis and access data specific to their work. Fusic Co., Ltd. built a solution that was linked and automated to create a seamless user experience.
While KEK and Fusic Co., Ltd. continue to improve the solution’s performance, KEK believes they have achieved their vision of a data analysis service used for experiments. KEK and Fusic Co., Ltd.’s success has led to an effort to broaden their impact across the field of research. This includes connecting their hybrid cloud environment to the Science Information NETwork (SINET), an academic information platform for universities and researchers across the globe. Through SINET, researchers outside of KEK can also access to KEK’s application for data analysis application as well as calculation resources prepared by KEK on AWS.
Increasing efficiency and streamlining grants administration
In the US, researchers at a large Ivy League university needed an accessible, readable, and user-friendly visualization of their grant data. While their grant administration program provided full research administration, the researchers needed a more simplified view into their grant funds and associated expenditures, without having to log into a new, external system.
Working with the institution, Pariveda Solutions, Inc., an AWS Premier Consulting partner, developed a prototype and built a new, lightweight application on AWS with services including AWS Lambda, Amazon DynamoDB, and AWS AppSync to give them a seamless view into their research grant data. Today the tooling Pariveda built on AWS has scaled up and is utilized by over 3,000 researchers at the institution. By spending less time on grant logistics, researchers are able to dedicate more time to solving some of the world’s most pressing challenges.
Researchers today are leveraging computational and data science to accelerate their research. If you are an institution or a researcher seeking to understand what tools can help you meet your mission, please reach out to us at aws-wwps-edu-intl@amazon.com.
Visit the cloud technology for academic research hub for more information on how AWS powers research across the globe.
Subscribe to the AWS Public Sector Blog newsletter to get the latest in AWS tools, solutions, and innovations from the public sector delivered to your inbox, or contact us.
Please take a few minutes to share insights regarding your experience with the AWS Public Sector Blog in this survey, and we’ll use feedback from the survey to create more content aligned with the preferences of our readers.