Standard Chartered Cuts Risk Grid Costs 60% on Amazon EC2 Spot Instances

When you think of industries taking maximal advantage of the cloud, financial services might not be the first that comes to mind. Banks' legacy core systems, deployed on premises and shaped by extensive governance and regulatory requirements, are hardly accelerators of cloud adoption. Addressing the considerations of key stakeholders takes time, as well. Even so, most banks are at least taking first steps into the cloud, and some are already running significant workloads there.

Case in point: Standard Chartered, a leading worldwide provider of corporate finance and investment banking (CFIB), wholesale, treasury, and retail banking services. In 2015, Standard Chartered shifted onto Amazon Web Services (AWS), tripling its compute capacity. A risk-analysis grid was the first workload to go live on AWS and benefited from the elasticity and pay-as-you-go pricing. Then, in 2017, the company slashed compute costs a further two-thirds by moving the grid onto Amazon Elastic Compute Cloud (Amazon EC2) Spot Instances.

"Grid computing is an ideal workload for the cloud, so working with AWS technology was the easy part," says Richard Davis, Global Head of Technology Services for Standard Chartered.

Many uncertain factors affect the value of investment portfolios, so market-risk analysis is one of the most important behind-the-scenes activities at banks like Standard Chartered. Without clear risk pictures, banks are unable to price trades, hedge effectively, or maintain appropriate capital ratios. Also, banks must comply with regulatory regimes that grow increasingly complex—and require more frequent risk reporting—with each passing year.

To run the complex computer modeling necessary to establish comprehensive risk pictures, Standard Chartered uses grid computing, in which tens of thousands of cores are architected to execute largely independent calculations over many hours. Such calculations must be run daily to satisfy the Fundamental Review of Trading Book (FRTB) and other regulatory reporting requirements. Risk calculations would also ideally be rerun as soon as possible after any number of worldwide political, regulatory, business, or even weather events.

In other words, risk-analysis grid computing has a ravenous appetite for resources. Standard Chartered projected that effective operation of the risk grid that was then still running on premises would soon outpace both budgetary and floorspace constraints. “We decided that the best long-term strategy would be to bring this workload to the cloud,” says Davis. “We selected AWS because of its leading position in the market, as well as for the breadth and maturity of services we were looking for."

Standard Chartered manages its grid with a provisioning solution based on AWS Lambda, which executes serverless functions in response to events, and Amazon Relational Database Service (Amazon RDS), which maintains the state of the Amazon EC2 capacity lifecycle. Because this solution can start or stop tens of thousands of concurrent cores in minutes, the company pays for grid resources only when calculations are running.

Simply by shifting this workload onto Amazon EC2 On-Demand Instances in 2015, Standard Chartered had already achieved significant savings. Those savings increased in 2017, when the company started running the grid almost entirely on Amazon EC2 Spot Instances. Because Spot Instances enable customers to take advantage of unused AWS compute capacity at substantially lower costs than other Amazon EC2 instance types, the move only strengthened Standard Chartered’s already strong case for the cloud.

"Elasticity and cost reduction were the primary drivers in our business case for the cloud, and Amazon EC2 On-Demand Instances already ticked those boxes," says Davis. "So it was great news when we realized that Spot Instances would further reduce our grid infrastructure costs by more than 60 percent."

Because the available amount of unused Amazon EC2 compute power is not constant, Spot Instances can be interrupted on 2 minutes' notice. Workloads that tolerate such interruptions well include certain big data processes, stateless web apps, and—as it happens—precisely the sort of high-performance batch processing needed for risk-analysis computing grids.

"Financial services market-risk calculations are actually a perfect fit for Spot Instances, because they're stateless," says Davis. "They do have to restart when they are resubmitted to the next vCPU, but they're otherwise unaffected by interruptions. We realized we could get our costs even lower if we broke the calculations into smaller chunks that could complete in less time."

Now that Standard Chartered has ready access to such affordable compute resources, the company is realizing even more value with them. “Using Amazon EC2 Spot Instances, we can experiment and test more in non-production environments due to the lower cost of runs,” says Davis. “For example, we can check performance of different hardware types with a single click, as opposed to the traditional months-long testing processes, making it even easier to provide our customers with the best-performing services.”

Although the risk grid is the most critical workload Standard Chartered is running on AWS, it's far from the only one. The company now uses 70 times more compute resources on AWS than it had access to on premises and, since 2018, has taken a cloud-first approach to all new software development. Standard Chartered is speeding up the release of new features to customers with an AWS-based CI/CD DevOps platform, which sees more than 240,000 builds a month.

Standard Chartered considers companywide cloud adoption key to its goal of making banking simpler, faster, and more convenient for customers. Deploying the risk grid in the cloud has laid a strong foundation. "We put a lot of work into a cloud governance framework that will enable bringing more and more workloads onto the cloud," Davis says. "We've been taking advantage of AWS training and we have more and more teams interested in building cloud-native apps every day. The cloud is an essential part of our strategy to keep moving in the direction banking must go."

To learn more, visit aws.amazon.com/financial-services/grid-computing.