Simplifying access to cloud resources for researchers: CloudBank

Many of the fundamental breakthroughs such as detecting gravitational waves, genetic editing, changing the course of the Ebola outbreak and tracking down the origin of mass (the Higgs boson discovery) were made possible not only due to the strong federal support for research but also via advancements in the underlying computing and information technologies. Today, there is an exponential increase in data volume from simulations, sensors, observatories, satellites, and large scientific instruments such as the Vera Rubin Observatory (few hundreds of petabytes), High Luminosity Large Hadron Collider (anticipated to enter the “exabyte” era), and ITER, a fusion energy device (multiple exabytes). These are making it impossible to gain insights from a single computing center if not by a single country.

Computing architectures are also evolving at a rapid pace, mainly in response to the changing needs and capabilities, driven by innovations in artificial intelligence (AI) and machine learning (ML) and its use in a range of applications and sectors, including healthcare, energy, finance, and security. The processing responsibilities and capabilities being carried out by hardware accelerators such as graphics processing units (GPUs), field-programmable gate arrays (FPGAs), and application-specific integrated circuits (ASICs) are unprecedented. In addition, with the National Quantum Initiative Act enabling advances in quantum computing and the Presidential Executive Order emphasizing leadership in AI, there is the opportunity to rethink the entire computing stack for this emerging technology.

Initially thought of as the “operating system of the Internet,” cloud computing provides several benefits including easy access to a heterogeneous set of hardware and software resources, scalability on demand, and cost savings. Cloud computing has gained a widespread adoption not only in the IT industries but also in many sectors, such as banking, finance, education, health, utilities, airlines, retail, and telecom, since the launch of Amazon Elastic Compute Cloud (Amazon EC2) in August 2006. Using Amazon Web Services (AWS), researchers have created the largest high-performance cluster in the cloud to study natural language processing by using over 1.1 million virtual CPUs (vCPUs) on Amazon EC2 Spot Instances running in a single AWS Region demonstrating scalability and elasticity of the infrastructure. The Children’s Hospital of Philadelphia and Edico Genome have set the Guinness World Record for the fastest time to analyze 1,000 human genomes using Amazon EC2 F1 instances (FPGAs)—two hours and twenty-five minutes, accelerating time-to-science by multiple orders of magnitude (processing a single human genome would generally take more than 40 hours on a traditional CPU-based system). Researchers at the IceCube Experiment just completed the largest cloud simulation in history using more than 51,500 cloud-based NVIDIA GPUs distributed across AWS, Google Cloud Platform (GCP), and Microsoft (Azure) in 28 cloud regions across three continents (North America, Europe, and Asia), which was orchestrated using HTCondor. Carnegie Mellon researchers are transforming education by teaching quantum integer programming using three different quantum processing technologies such as superconducting qubits (Rigetti), gate-based ion traps (IONQ), and quantum annealing (D-wave) via Amazon Braket.

NSF Cloud Access Announcement and CloudBank

To better support the growing use of cloud computing resources with increasing data- and compute-intensive research and education workloads, the National Science Foundation’s (NSF) Directorate for Computer and Information Science and Engineering (CISE) announced the Cloud Access solicitation in September 2018. This historic effort was based on recommendations from the National Academies of Sciences, Engineering, and Medicine, the CISE research and education community (Enabling Computer and Information Science and Engineering Research and Education in the Cloud), experience from public-private cloud partnerships, and a community-inspired vision for future advanced computing infrastructure (Future Cyberinfrastructure: Rethinking NSF’s Computational Ecosystem for 21st Century Science and Engineering).

The NSF, through its competitive merit review process, selected the CloudBank team, led by the University of California (UC), San Diego, University of Washington (UW), and UC Berkeley in partnership with Strategic Blue, to simplify access to public clouds across computer science research and education in the United States. Researchers that use CloudBank gain access to advanced hardware resources such as CPUs, GPUs, FPGAs, ASICs, and quantum processing units (QPUs). In addition, CloudBank offers proposal assistance, facilitated cloud access and account management, monitoring and resource usage optimization, and eliminates university overhead/indirect costs, and provides curated training materials, classroom, and help desk support. CloudBank Enterprise is also available to all institutions without the NSF grant process.

“Public cloud has become an important resource for computer science research and education, but with the rapid growth in the diversity of resource offerings, users increasingly encounter pain points to adoption that limit the potential of these resources in their work,” said SDSC Director Michael Norman, principal investigator for the project. “CloudBank will address these pain points by providing ‘on-ramp’ support that helps researchers overcome challenges such as managing cost, translating and upgrading research computing environments to an appropriate cloud platform, and learning cloud-based technologies that accelerate and expand research.”

In August 2020, the NSF announced the first CloudBank associated award to Vanessa Frias-Martinez, associate professor at the University of Maryland, to design, develop, and deploy a privacy-preserving, inclusive public transit toolkit to assess the quality of service across socioeconomic statuses in Baltimore City using AWS.

How to apply for CloudBank

So far, the NSF has released three CloudBank-eligible solicitations (NSF 20-591, CISE Core Programs; NSF 20-592, Cyberinfrastructure for Sustained Scientific Innovation; and NSF 20-563, Cyber-Physical Systems), with possibly up to 650 awards across a wide array of domains. To apply, in addition to NSF requirements that are standard for all proposals and also specific to these competitions (e.g., a Data Management Plan), applicants for CloudBank should include a supplementary document that covers (a) the public cloud provider that will be used; (b) anticipated annual and total costs for accessing the desired cloud computing resources, based on pricing currently available from the public cloud computing providers; and (c) a technical description of the requested cloud computing resources. Check out examples of the Data Management and Supplementary Document.

To learn more about CloudBank, watch this on-demand presentation, and visit the NSF FAQ and/or CloudBank FAQ pages. Please contact us or email help@cloudbank.org if there is anything we can do to help accelerate your research.