Category: Education


Alces Flight: Build your self-service supercomputer in minutes

We are excited to announce that Alces Flight has made hundreds of science and HPC applications available in the AWS Marketplace, making it easy for any researcher to spin up any size High Performance Computing (HPC) cluster in the AWS Cloud.

In the past, researchers often had to wait in line for the computing power for scientific discovery. With Alces Flight, you can take your compute projects from 0 to 60 in a few minutes. Researchers and scientists can quickly spin up multiple nodes with pre-installed compilers, libraries, and hundreds of scientific computing applications. Flight comes with a catalog of more than 750 built-in scientific computing applications, libraries and versions, covering nearly everything from engineering, chemistry and genomics, to statistics and remote sensing.

After designing and managing hundreds of HPC workflows for national and academic supercomputing centers in the United Kingdom, Alces built and validated HPC workflows tailored to researchers and automated applications built for supercomputing centers. They are now making that catalog available in the AWS Marketplace.

Take Flight today with AWS & Alces Flight

The Alces Flight portfolio of Software-as-a-Service (SaaS) offerings provides researchers with on-demand HPC clusters or ‘self-service supercomputers’ in minutes. With hundreds of science and HPC apps available in the AWS Marketplace, Alces Flight helps you automatically create a research-ready environment that complements Amazon Elastic Block Store (EBS), Amazon Simple Storage Service (Amazon S3), and Amazon Elastic Compute Cloud (Amazon EC2) resources in the AWS Cloud.

Researchers get:

  • Instant access to a popular catalog of hundreds of HPC and scientific computing apps that rival those available at supercomputing centers.
  • Access to HPC clusters available for use any time, from anywhere, at virtually any scale, and at the lowest possible cost with Amazon EC2 Spot Instances pricing.
  • Instant on-demand access to powerful scientific computing tools that make global collaboration easy.

Read more about this announcement on Jeff Barr’s blog here. And get started and see the Alces Flight portfolio of apps in the AWS Marketplace.

The Birds in the Cloud: How the University of Oklahoma Uses NEXRAD Data to Study Birds

The Next Generation Weather Radar Network (NEXRAD) is familiar to most of us as a backdrop for the weather report on the local TV news, but this radar array does more than track weather. During the spring and fall, NEXRAD detects vast “clouds” of migrating birds. This phenomenon is often an annoyance to meteorologists, but it turned out to be an opportunity for ornithologists at the University of Oklahoma (OU).

Dr. Eli Bridge who works at the Oklahoma Biological Survey at OU and his colleagues are using NEXRAD data to study birds and other flying animals. They can quantify nighttime migration of birds in the spring and fall by looking at radar reflectivity data where there is clear air. For one of their projects, they are trying to compile radar data to estimate the size of Purple Martin Roosts.

Purple Martins form large, dense aggregations during the late summer where there may be as many as 50,000 birds spending the night in two or three trees. When they leave these roosts at sunrise they make characteristic ring-shaped reflectivity patterns in scans from nearby radars. Eli is trying to use the radar data to develop an index of roost size (number of birds) and then compare roosts across years to see how things like drought and land use practices affect these regional bird populations.

In order to process radar data and work with it in various ways, they needed to find ways to get access to unfiltered, raw radar data. One option was to make requests for individual scans, which included making the request, waiting, receiving the response, and only getting a little bit of the data back at a time. A much better option would be to have an archive available to avoid making one-off requests.

“One of the biggest challenges in our work was simply obtaining large chunks of radar data to work with.” Eli said. “We had to make individual requests for lots of radar scans and then compile them on external hard drives or servers at OU (which costs us money). Having NEXRAD on AWS is a major help to us. I can download a radar scan in about 2-4 seconds. So there’s really no need for us to store raw data anymore.”

By looking at the entire NEXRAD archive (which extends back to the early 1990s), Eli and his team are able to study how birds are responding to droughts, climate change, environmental change, and seasonal queues.

“No way could we do this if we had to request data sets one by one. In short, we can work through large amounts of data quickly since NEXRAD is on AWS. Now the door is open to us and we have to figure out how to process all of it,” Eli said.

Now that all of the NEXRAD data is available, the heavy lifting aspect has shifted. You can start thinking much bigger. Having something like a continental-scale mosaic of all the NEXRAD radar scans over a 10-minute period available as raw data (not just a report or an image) would be the evolution of this kind of big data. This is something you can only do because it’s in the cloud and it is widely shared.

Learn more about NEXRAD on AWS in this blog post.

 

Radar data generated by the birds

 

The Evolution of High Performance Computing: Architectures and the Cloud

A guest blog by Jeff Layton, Principal Tech, AWS Public Sector

In High Performance Computing (HPC), users are performing computations that no one ever thought would be considered. For example, there are researchers performing a statistical analysis of the voting records of the Supreme Court, sequencing genomes of humans, plants, and animals, creating deep learning networks for object and facial recognition so that cars and Unmanned Aerial Vehicles (UAVs) can guide themselves, searching for new planets in the galaxy, looking for trends in human behavioral patterns, analyzing social patterns in user habits, targeting advertisement development and placement, and thousands of other applications.

From lotions to aircrafts, the products and services that are connected with HPC touch us each and every day, and we often don’t even realize it.

A great number of these applications are coming from the use of the massive amount of data that has been collected and stored. This is true of classic HPC applications or new HPC applications, such as deep learning that need massive data sets for learning and large stat sets for testing the model. These are very data-driven applications and their scale is getting larger every day.

A key feature of this “new” HPC is that it needs to be flexible and scalable to accommodate these new applications and the associated sea of data. New applications and algorithms are developed each year and their characteristics can vary widely, resulting in the need for increasingly diverse hardware support and new software architectures.

The cloud allows users to dynamically create architectures as they are needed, using the right amount of compute power (CPU or GPU), network, databases, data storage, and analysis tools. Rather than the classic model of fitting the application software to the hardware, the cloud allows the application software to define the infrastructure.

The cloud has a number of capabilities that map to the evolving nature of HPC, including:

  1. Scale and Elasticity
  2. Code as Infrastructure
  3. Ability to experiment

Scale and Elasticity

Thousands upon thousands of compute resources, massive storage capacity, and high-performance network resources are available worldwide via the cloud.

Combining scale and elasticity creates a capability for HPC cloud users that doesn’t exist for centralized shared HPC resources. If resources can be provisioned and scaled as needed and there is a large pool of resources, then waiting in job queues are a thing of the past. Each HPC user in the cloud can have access to their own set of HPC resources, such as compute, networking, and storage resources for their own specific applications with no need to share the resources with other users. They have zero queue time and can create architectures that their applications need.

Code as Infrastructure

Cloud computing also features the ability to build or assemble architectures or systems using only software (code), in which software serves as the template for provisioning hardware. Instead of having to assemble physical hardware in a specific location and manage such things as cabling, cabling labels, switch configuration, router software, and patching, HPC in the cloud allows the various components to be specified by writing a small amount of code, making it easy to expand or contract or even re-architect on-the-fly.

Code as infrastructure addresses the classic HPC problem of inflexible hardware and architecture. However, if a classic cluster architecture is needed, then that can be easily created in the cloud. If a different application needs a Hadoop architecture or perhaps a Spark architecture, then those too can be created. Only the software changes.

Ability to Experiment

As HPC continues to evolve, new applications are being developed that take advantage of experimentation, test, and iteration. These applications may involve new architectures or even re-thinking how the applications are written (re-interpretation). Having access to modular, fungible resources as a set of building blocks that can be configured and reconfigured as-needed is crucial for this new approach.

This will become even more important as HPC moves forward because the new wave of applications are heavily oriented toward massive data. Pattern recognition, machine learning, and deep learning are examples of these new applications and being able to create new architectures will allow these applications to flourish and develop based on the scale and flexibility of the cloud and corresponding economics.


 

See how HPC is used for open data and scientific computing here: www.aws.amazon.com/scico and www.aws.amazon.com/opendata. And check out Jeff’s previous blog The Evolution of High Performance Computing.

AWS Public Sector Summit Countdown: Learn how NGA, DOJ, and NREL are Innovating with Cloud Computing

The agenda is now live for the AWS Public Sector Summit!  Governments, educational institutions, and nonprofits from around the world are coming to DC to share their journey to the cloud with you. You can expect to hear updates on new services and offerings from AWS tech experts, as well as insight from your peers in deep dive sessions and interactive panel discussions on:

  • Using AWS to Meet Requirements for HIPAA, FERPA, and CJIS
  • Next Generation Open Data Platforms
  • Policy as a Strategic Enabler for Cloud Adoption
  • Hybrid Architectures: It’s Not All or Nothing
  • Security Updates
  • Adoption Models: How Different Organizations are Approaching Cloud Adoption

Don’t miss hearing from leaders across the public sector, including:

  • Sue Gordon, Deputy Director, National Geospatial-Intelligence Agency (NGA)
  • Prad Prasoon, Business Technology Strategist, American Heart Association (AHA)
  • Jay Haque, Director of Development Operations and Enterprise Computing, The New York Public Library
  • Debbie Brodt-Giles, Digital Assets Supervisor, National Renewable Energy Laboratory (NREL)
  • Adrian Farley, CIO, CA Department of Justice (DOJ)

These are just some of the leaders who will be sharing their perspectives. View session details, including more featured speakers, titles, and abstracts here.

We will also be announcing the City on a Cloud Innovation Challenge winners during the keynote on June 21. And we are thrilled to have AWS CEO Andy Jassy joining us at this year’s Summit – don’t miss hearing his insights on June 21!

Will you be joining us? Register now for the complimentary event!

Rapidly Recover Mission-Critical Systems in a Disaster

Due to common hardware and software failures, human errors, and natural phenomena, disasters are inevitable, but IT infrastructure loss shouldn’t be.  With the AWS cloud, you can rapidly recover mission-critical systems while optimizing your Disaster Recovery (DR) budget.

Thousands of public sector customers, like St Luke’s Anglican School in Australia and the City of Asheville in North Carolina, rely on AWS to enable faster recovery of their on-premises IT systems without unnecessary hardware, power, bandwidth, cooling, space, and administration costs associated with managing duplicate data centers for DR.

The AWS cloud lets you back up, store, and recover IT systems in seconds by supporting popular DR approaches from simple backups to hot standby solutions that failover at a moment’s notice. And with 12 regions (and 5 more coming this year!) and multiple AWS Availability Zones (AZs), you can recover from disasters anywhere, any time. The following figure shows a spectrum for the four scenarios, arranged by how quickly a system can be available to users after a DR event.

These four scenarios include:

  1. Backup and Restore – This simple and low cost DR approach backs up your data and applications from anywhere to the AWS cloud for use during recovery from a disaster. Unlike conventional backup methods, data is not backed up to tape. Amazon Elastic Compute Cloud (Amazon EC2) computing instances are only used as needed for testing. With Amazon Simple Storage Service (Amazon S3), storage costs are as low as $0.015/GB stored for infrequent access.
  2. Pilot Light – The idea of the pilot light is an analogy that comes from gas heating. In that scenario, a small flame that’s always on can quickly ignite the entire furnace to heat up a house. In this DR approach, you simply replicate part of your IT structure for a limited set of core services so that the AWS cloud environment seamlessly takes over in the event of a disaster. A small part of your infrastructure is always running simultaneously syncing mutable data (as databases or documents), while other parts of your infrastructure are switched off and used only during testing. Unlike a backup and recovery approach, you must ensure that your most critical core elements are already configured and running in AWS (the pilot light). When the time comes for recovery, you can rapidly provision a full-scale production environment around the critical core.
  3. Warm Standby – The term warm standby is used to describe a DR scenario in which a scaled-down version of a fully functional environment is always running in the cloud. A warm standby solution extends the pilot light elements and preparation. It further decreases the recovery time because some services are always running. By identifying your business-critical systems, you can fully duplicate these systems on AWS and have them always on.
  4. Multi-Site – A multi-site solution runs on AWS as well as on your existing on-site infrastructure in an active- active configuration. The data replication method that you employ will be determined by the recovery point that you choose, either Recovery Time Objective (the maximum allowable downtime before degraded operations are restored) or Recovery Point Objective (the maximum allowable time window whereby you will accept the loss of transactions during the DR process).

Learn more about using AWS for DR in this white paper. And also continue to learn about backup and restore architectures, both using partner products and solutions, that assist in backup, recovery, DR, and continuity of operations (COOP) at the AWS Public Sector Summit in Washington, DC on June 20-21, 2016. Learn more about the complimentary event and register here.

 

Bring Your Own Windows 7 Licenses for Amazon Workspaces

Guest post by Len Henry, Senior Solutions Architect, Amazon Web Services

Amazon WorkSpaces is our managed virtual desktop service in the cloud. You can easily provision cloud-based desktops and allow users to access your applications and resources from any supported device. The Bring Your Own Windows 7 Licenses (BYOL) feature of Amazon Workspaces furthers our commitment to providing you with lower costs and greater control of your IT resources.

If you are a Microsoft Volume License license-holder with tools and processes for managing Windows desktop solutions, you can reduce the cost for your WorkSpaces (up to 16% less per month) and you can use your existing Desktop image for your Workspaces. Let’s get started.

Architectural Designs

Your WorkSpaces can access your on-premises resources when you extend your network into AWS. You can also extend your existing Active Directory into AWS. This white paper describes how you achieve connectivity and the images below take you through different points of connection.

Figure 1 Amazon WorkSpaces when using an AWS Directory Service and a VPN Connection

Figure 2 Amazon WorkSpaces when using an AWS Directory Service and a Direct Connect

As a part of the implementation, you will create a Dedicated VPC.  You will also create a Dedicated Directory Service (the Dedicated Directory option will not be present until the WorkSpaces team enables the BYOL account). You can use AWS Workspaces with your existing Active Directory or one of the AWS Directory Services.

You can extend your Active Directory into AWS by deploying additional Domain controllers into the AWS cloud or using our managed Directory Service’s AD Connector feature to proxy your existing Active Directory. We provide you with specific guidance on how to extend your on-premises network here. You can use our Directory Service to create three types of directories:

  1. Simple AD:  Samba 4 powered Active Directory compatible directory in the cloud.
  2. Microsoft AD: Powered by Windows Server 2012 R2.
  3. AD Connector:  Recommended for leveraging your on-premises Active Directory.

Your choice of Directory Service depends on the size of your Active Directory and your need for specific Active Directory features. Learn more here.

With BYOL, you use your 64 bit Windows 7 Desktop Image on hardware that is dedicated to you. We use your image to provision WorkSpaces and validate that it is compatible with our service.

Typical milestones (and suggested stakeholders) for your implementation:

You provide estimates to us of your initial and expected growth of active WorkSpaces.  AWS selects resources for your WorkSpaces based on your needs.  Your BYOL WorkSpaces are deployed on dedicated hardware to allow you to use your existing software license. Tools and AWS features include:

  • OVA – You provide images for BYOL in the OVA industry standard format for Virtual Machines. You can use any of the following software to export to an OVA: Oracle VM VirtualBox, VMWare VSphere, Microsoft System Center 2012 Virtual Machine Manager, and Citrix XenServer.
  • VM Import – You will use VM import in the AWS Command Line Interface (CLI) (AWS CLI).  You execute import image after your OVA has been imported into Amazon Simple Storage Service (Amazon S3).
  • VPC Wizard – You will create several VPC resources for your BYOL VPC. The VPC Wizard can create your VPC and configure public/private subnets and even a hardware VPN.
  • AWS Health Check Website – You can use this site to check if your local network meets the requirements for using WorkSpaces. You also get a suggestion for the region you should deploy your WorkSpaces in.

A proof of concept (POC) with public bundles will give your team experience using and supporting WorkSpaces.  A POC can help verify your network, security, and other configurations. By submitting a base Windows 7 image, you reduce the likelihood of your customizations impacting on-boarding. You can customize your image after on-boarding and you can have regularly scheduled meetings with your AWS account team to make it easier to coordinate on your implementation.

With WorkSpaces, you can reduce the work necessary to manage a Virtual Desktop Infrastructure solution. This automation can help you to manage a large number of users. The Workspaces API provides you commands for typical WorkSpaces use cases: creating a WorkSpace, checking the health of a WorkSpace, and rebooting a WorkSpace. You can use the WorkSpaces API to create a portal for managing your WorkSpaces or for user self-service.

In order to ensure that you are ready to get started with BYOL, please reach out to your AWS account manager, solutions architect, or sales representative, or create a Technical Support case with Amazon WorkSpaces. Please contact us to get started using BYOL here.


Learn more about WorkSpaces and other enterprise applications at the complimentary AWS Public Sector Summit in Washington, DC June 20-21, 2016.

Amazon Web Services Pledges Training and Certifications for Veterans

Amazon pledged to offer 10,000 service members, transitioning veterans, and military spouses over $7m in Amazon Web Services (AWS) trainings. This pledge is part of Joining Forces, the First Lady and Dr. Jill Biden’s initiative that works hand in hand with the public and private sectors to ensure that service members, veterans, and their families have the tools they need to succeed throughout their lives. By offering cloud computing training, we are excited to help veterans transition more easily into the civilian workforce and help develop the skills necessary to pursue jobs in the high-demand cloud computing space.

With the dramatically increasing demand for employees skilled in cloud computing, AWS is providing an academic gateway for the next generation of IT and cloud professionals from the military. These training pledges include:

  • Free membership to AWS Educate, Amazon’s global initiative to provide students and educators with the resources needed to greatly accelerate cloud-related learning endeavors and to help power the entrepreneurs, workforce, and researchers of tomorrow.  This includes $50 in credits for AWS cloud services, training courses like AWS Tech Essentials, a wide library of cloud content, and access to our collaboration portal.
  • Free access to over 90 labs on AWS services and solutions, as well as certification prep labs.
  • Eligibility for AWS Certification exam reimbursement from the Department of Veterans Administration under the GI Bill’s education provision.

Amazon is also committed to training 25 wounded warriors at AWS Boot Camps for functional roles in cloud computing and commercial companies operating in the tech space and hiring 25,000 veterans and military spouses over the next five years. Learn more in the Amazon blog here.

At Amazon, we are extremely committed to military service members and their spouses. And we want to thank all active and retired military members for their service and look forward to working with transitioning veterans.

Learn more about AWS Educate here.

 

The Evolution of High Performance Computing

A guest blog by Jeff Layton, Principal Tech, AWS Public Sector

The High Performance Computing (HPC) world is evolving rapidly. New workloads, such as pattern recognition, speech, video, and text processing, speech and facial recognition, deep learning, machine learning, and genomic sequencing, are being executed on HPC systems. The main motivation behind this evolution is economic and technical. As HPC systems became more powerful, agile, and less costly, they can be used for applications that have never had access to high scale, low cost infrastructure.

The cloud has accelerated this evolution because it is scalable and elastic, allowing self-service provisioning of one to thousands of processors in minutes. As a result, HPC users are coming to AWS with new and expanding application requirements and are seeing reduced time-to-results, faster speed to deployment, greater architectural flexibility, and reduced costs. Cloud computing is pushing HPC at the pace of computing innovation as users benefit from advances in microprocessors, GPUs, networking, and storage.

The cloud and the evolving HPC world

The HPC world has a need for more processing capability, which is driving HPC system development. The current HPC architecture, the cluster, was created for a common architecture and operating system that had price-performance benefits far beyond proprietary systems. Clusters with commodity processors were then doing production work for a number of companies and labs, which led to the explosion of clusters in HPC.

Clusters have come a long way and have greatly increased access to HPC resources at an affordable price. This includes both embarrassingly parallel applications and tightly coupled applications.

Issues with traditional HPC fixed architectures

The HPC cluster architecture is a relatively fixed architecture with a set of servers (nodes). Each server has a small amount of internal storage (if any at all), connected by a dedicated network, using software tools to manage user requests for resources. It is rare for any changes to be made to the system, such as adding nodes, processor upgrades, additional node storage, network topology, or technology changes. Once put in place, the vast majority of dedicated cluster systems never change architecture.

The rise of the Hadoop architecture, which addresses a large class of HPC problems, makes this inflexibility an even greater challenge. The Hadoop architecture (also known as the Map-Reduce architecture) calls for nodes with a lot of local storage and only uses TCP networks. The typical on-premises HPC system uses the smallest, least expensive, but reliable drive in each node. For Hadoop workloads, customers often procure a separate system specifically designed for Hadoop workloads. Employing this strategy would create two HPC architectures with conflicting configurations. However, this is unnecessary when cloud computing is the platform, as both rely on commodity systems, dynamically created clusters, and software stacks that are purpose-built for the needs of particular problems.

The cloud allows you to go beyond thinking that HPC is only about clusters and that all applications must adapt that model. If you have a new architecture in mind for your application or your workflow, you can simply and easily create it in the cloud.

Do you want to use a combination of containers and microservices for your application? The AWS Cloud allows you to construct what you need with some very simple code. If the architecture doesn’t work as well as you wanted, then you just turn off the system and stop paying for it.

Learn more about HPC with AWS in this video below.

In future blogs, I’ll discuss some of the pain points of HPC beyond architectural rigidity and how the cloud addresses them. Stay tuned! In the meantime, learn more about HPC with AWS here: https://aws.amazon.com/hpc/

Villanova University Scales Website After Their Buzzer Beater Win in the NCAA Men’s Basketball Championship

Congratulations to the Villanova Wildcats for their heart-stopping win in the NCAA Men’s Basketball Tournament! Victories like that impact not just the team, but the entire institution: the students, fans, alumni, and staff. Behind the scenes, schools must ramp everything from security to IT as they work to support their team for the big game.

Whether it is a major sporting event or the start of a new semester, the university website should be reliable and scalable. While owning servers and hosting a website on campus is an option, that strategy can make peak usage periods challenging since institutions must purchase excess hardware that is unused most of the year.

In 2012, Villanova chose another path.  They moved their Adobe Experience Manager website to the cloud on AWS with help from AWS Managed Services Partner ICF Olson.  While they owned onsite servers starting in 2009, they discovered the most difficult piece of running a website was staffing. “We would train staff in the new technologies, but it was difficult to retain the website operations because their technology skills were so in demand,” said Gabe Monteleone, Assistant Vice President, University Applications and Information Systems at Villanova University. “That triggered the decision to look to Amazon Web Services and the cloud.”

This was Villanova’s first foray into the cloud and the team “hasn’t looked back.” Based on Amazon CloudWatch metrics, Villanova.edu scaled from less than 1,000 website requests on the day of the NCAA Championship game to over 225,000 requests after Villanova won the tournament.

The decision to move to the cloud included a review of the costs of onsite hosting plus staff.  In this case, Villanova saves money, sees improved overall performance, and sees a benefit for their employees. “Performance and uptime have improved and we haven’t had any noticeable downtime since 2012,” continued Monteleone. “It is easy for us to maintain the site and still do development on the platform.  Our team focuses on high-value projects that include coding and working with end users, rather than maintenance.”

Monteleone recommends that IT teams consider the full picture when planning to move their website to the cloud.  For example, budgeting changes when a university moves to a utility model.  Schools can quickly spin up more compute power around big events (rather than purchase hardware), but they need to plan for those spikes in the budgeting cycle.

As for the future, the applications team sees future opportunities for cloud services.  But for today, they are happy with the win.


 

Interested in learning more about launching your university’s website in the cloud? Hear more about Villanova’s experience from Gabe Monteleone at Ellucian Live in Denver.  He will join AWS on a panel discussion called Cloud Adoption in Higher Education on April 18 at 4:45PM.   Or, for more information, visit: aws.amazon.com/websites.

High Performance Cloud Computing Supports Disease Prevention

The Walter and Eliza Hall Institute of Medical Research, the oldest medical research institute in Australia, undertakes research across a range of areas including breast, ovarian, and blood cancers, type 1 diabetes, rheumatoid arthritis, coeliac disease, and malaria. More than 60 clinical trials based on discoveries made at the institute are underway. These include trials of vaccines for type 1 diabetes, coeliac disease, malaria, and trials of a new class of anti-cancer agents for treating patients with leukemia.

Recently, the Systems Biology and Personalised Medicine Division (SBPM) at the Walter and Eliza Hall Institute enlisted the help of DiUS and AWS to accelerate the processing and analysis of embarrassingly parallelizable data and image sets. (An embarrassingly parallel workload or problem is one where little or no effort is needed to separate the problem into a number of parallel tasks and scale these independently).

In a pilot study, SBPM was interested in exploring the use of cloud computing to reduce the time it took to analyze high-resolution microscopy data. To solve this challenge, the institute enlisted the help of DiUS to discover and validate a new approach for data analysis, leveraging a cloud-based capability on AWS. SBPM also asked DiUS to help prototype a platform that would support the orchestration of High Performance Computing (HPC).

Starting small, DiUS facilitated an ideation session to better understand the scientists’ needs and define an overarching solution. The session quickly identified the need for on-demand and short-lived computing clusters, customized and templated for each laboratory. With the scope defined, DiUS partnered with SBPM technical analysts and AWS scientific computing specialists to develop a cost-effective, scalable, and secure solution.

The team built a platform in the microscopy laboratory to enable image scientists to perform initial exploratory analysis locally on their workstations, then seamlessly synchronize the local data sets to AWS, and perform detailed analysis in the cloud. Upon completion, results are transferred back to the scientists’ workstations. This approach has reduced typical image processing times from seven hours to less than one. With AWS, scientists can quickly analyze massive data pipelines, store petabytes of data, and share their results with collaborators around the world, focusing on science rather than servers.

Meanwhile, in the proteomics laboratory, a similar setup enabled scientists to auto-scale R based analysis of mass-spectrometer genomics data. Beyond just computation acceleration, these early successes show that it is feasible to augment on-site HPC with on-demand computing. Innovations like these have enabled the Walter and Eliza Hall Institute to remain at the forefront of medical research and contemplate future research directions using these newly proven scientific computing and research capabilities.

Read the full case study here and learn more about the technology behind the project here.

Discover more about AWS and scientific computing at https://aws.amazon.com/government-education/scientific-computing/