By spinning up hundreds or even thousands of nodes on AWS and getting results in hours vs weeks, our scientific researchers have a lot more freedom to ask questions that weren’t even possible before. The speed is important, but equally important is the additional intellectual curiosity this enables for researchers.
Lance Smith Associate Director of IT, Celgene

Celgene is a global biopharmaceutical company that develops drug therapies for cancer and inflammatory disorders. Headquartered in New Jersey and employing more than 8,000 globally, the company is committed to improving the lives of patients through the delivery of innovative treatments. To accomplish this, Celgene is engaged in more than 300 clinical trials at major medical centers around the world.

In the pharmaceutical industry a failed clinical trial may result in product failure which can exceed a billion dollars in total expense to pharmaceutical companies. To avoid this scenario, Celgene’s scientific researchers take extreme measures to analyze compounds and focus only on those with a high probability of success. “Our goal is to go into trials with a high level of confidence in the hypotheses we develop. Our high-performance computing workflow provides the analytic data to give us that confidence. Failing fast saves the company significant amounts of time and money, so speed and agility in our analytic workflows are critical.”

To create efficiencies for its pharmaceutical researchers, the Celgene Research and Early Development (R&ED) team wanted to improve its high-performance computing (HPC) workflows. Celgene decomposed the workflows performed by its scientific researchers looking for ways to automate some process steps to free up more time for the researchers’ critical tasks. “The efficiency of our research team is critical to our success as a business. Even small gains in research staff productivity can have a significant impact on cost and time to market.”

Scalability and self-service were also necessary. Providing the research staff the tools to be self-sufficient and to scale their resources on demand was a requirement for Celgene.    

The Celgene R&ED division chose Amazon Web Services (AWS) as its cloud technology provider in 2014, initially as a result of the performance, scalability and security offered by AWS. Celgene moved two of its critical HPC workloads to AWS, which resulted in improved collaboration, increased business process agility, improved scalability, and lower total cost of ownership.

Based on the success that migrating these initial workloads to the cloud delivered to the business, Celgene sought to migrate additional applications to AWS. With over more than 1000 applications and all of them running in standard on-premises IT environments where data availability and integrity is critical, Celgene needed an enterprise-class platform that provided an easy migration path to running in the cloud. “These applications are mission-critical and cannot be modified, so we needed a way to easily move them to the cloud without disruption to the business or refactoring of the applications.”

Celgene chose Amazon Elastic File Service (Amazon EFS) to store file data for all of its R&D workloads and many of its critical applications, including its computational chemistry and genomics-computing HPC workloads. “Amazon EFS provides a standard file system interface that allowed us to migrate our applications to the cloud without any modification to the application. Also, being a managed file service, we no longer had to manage and administer the file system resources we had in-house, freeing up time for IT Ops to focus on items that impact our business.”

Celgene runs many HPC workloads on hundreds of Amazon Elastic Compute Cloud (Amazon EC2) instances and uses Amazon Simple Storage Service (Amazon S3) and Amazon S3 Glacier to store petabytes of genomic data. “Some of our genomic files are very large in size, even after compression, so we need the robust storage capabilities of Amazon S3 and Amazon Glacier,” says Smith.

As a research organization, Celgene’s R&ED division manages thousands of instruments on premises, which will continue to exist in physical labs with the research teams. In many instances, tests that run on lab instruments are done with unique and irreplaceable tissue samples, so the tests and the resulting data can literally never be reproduced. For long-term, durable retention of this data, Celgene uses Amazon S3, using AWS Storage Gateway and AWS Direct Connect for data transport.

After migrating many of its critical research applications to AWS, Celgene adopted additional AWS Cloud services to streamline business workflows and further improve organizational efficiency and agility. “Since our data is centralized in AWS, we can utilize additional AWS services to improve the agility of our business. For example, we use Turbot, a guardrails platform that allows for controlled, monitored, and automated access to AWS resources. The solution provides Celgene with enhanced DevOps CI/CD automation capabilities by using AWS CloudFormation and Terraform to enforce hundreds of security and compliance policies across various AWS services and OS environments. The platform allows Celgene’s Cloud Infrastructure team to focus less on managing a cloud platform and be more supportive to their researchers, helping them automate and architect HPC simulations across thousands of nodes in a secure, yet flexible manner.

Using AWS, Celgene scientists have dramatically reduced the time it takes to complete HPC jobs needed for cancer drug research. “For our informatics researchers, computational jobs on AWS can be reduced to hours, compared to weeks or months when we were managing our HPC cluster on premises,” says Smith. As a result, researchers can run many more queries. “By spinning up a few hundred nodes on AWS and getting results in less than a day, our scientific researchers have a lot more freedom to ask questions that weren’t even possible before,” Smith says. “Speed is important, but equally important is the additional intellectual curiosity this enables for researchers. They can ask scientific questions they were afraid or unable to ask before because of hardware limitations or time constraints.”

The company is also using the AWS Cloud to simplify and improve collaboration. “With our physical environment, research collaboration was not possible,” says Smith. “On premises isn’t as agile as our researchers want to be. Quotes, standards, finance, capital procurement, delivery, installation, and configuration take months in an enterprise setting. And because research collaborations can start up spontaneously, the result is completely un-forecasted demand for IT. Historically Celgene has been an industry leader in scientific collaboration; AWS allows us to expand our partnerships into the informatics and computational space. Using AWS, we enable teamwork by providing isolated access for researchers from different organizations. Each collaborator can upload petabyte-scale data and work together in a common, secure area.” This collaborative capability has helped Celgene expand its research with the public sector. “We have many collaborative projects with prominent research universities, and we see that growing exponentially,” says Smith.

To learn more about how AWS can help you manage your HPC cluster, visit our AWS High Performance Computing details page. To learn more about using AWS as a life-sciences organization, visit the Life Sciences detail page.