SBI Securities Migrates a 13-Billion Dollar Online Trading System to AWS, Processing 100 Million Daily Visits

Since launching an internet trading service in October 1999, SBI Securities has been a leader in the Japanese online securities market with a customer-centric management philosophy. The company decided to migrate its domestic stock trading system to AWS because it expected a dramatic expansion of its customer base following the elimination of domestic stock trading fees.

In 2019, SBI Securities announced its neo securitization initiative, which included the Zero Revolution to eliminate domestic stock trading fees in online transactions from the end of September 2023. Additionally, the Japanese government’s new NISA (Nippon Individual Savings Account) system was set to launch in January 2024. While those developments promised rapid growth in number of accounts and transaction volume, there were concerns about capacity expansion with conventional on-premises environments. “With on-premises, it took about 1 year to procure hardware and other resources, and we risked being unable to handle the rapid increase in transaction volume. Therefore, we decided to migrate to AWS to shorten the expansion period,” says Ty Hahn, Executive Officer and Head of the Corporate IT Department at SBI Securities.

Before migrating to AWS, SBI Securities conducted preliminary verification and confirmed the benefits in terms of cost and service expansion. “Considering hardware procurement, maintenance, and operation costs, we determined that migrating to AWS would provide cost benefits. The agility to flexibly construct development and verification environments was also attractive as we aimed to accelerate future service development,” says Keiji Muto, Executive Officer and IT Planning Manager at SBI Securities.

For the above reasons, when migrating the domestic stock exchange system to the cloud, SBI Securities adopted AWS, which had a proven track record with the company’s backup site and sub-channels. “We had been using AWS for years, so having in-house knowledge was our primary advantage. We also valued the broad portfolio and the position of AWS as a leader among cloud providers,” says Atsushi Iwamoto of SBI Simplex Solutions (SSS), a strategic IT partner within the SBI Group.

The project started in November 2022 and migration to AWS was completed 16 months later in March 2024. That migration period was less than half that of on-premises implementations. The migration targets were not limited to front-end applications, but also spanned a wide range of components such as cache, database, business logic, batch processing, and order processing infrastructure. “In addition to the main transaction system, we faced the challenge of migrating the backend database from on-premises to AWS,” recalls Yukihiro Fujita of SSS.

A key feature of the project was the use of AWS Cloud Development Kit (AWS CDK). SBI Securities used AWS CDK to code infrastructure resources and implement all changes to the production environment through a CI/CD pipeline, improving work quality and increasing agility. As a tool that empowers developers to describe the AWS environment using common programming languages, AWS CDK can provision AWS resources through AWS CloudFormation, contributing to faster implementation and reduced risk of human error.

“The infrastructure development team had previously coded all infrastructure, including on-premises, to build environments quickly and with high quality,” says Hahn. “It was natural that we would also need to code for AWS, so we adopted AWS CDK, which allows us to program from among multiple options.”

To increase availability, the company adopted a multi-AZ configuration that operates across multiple Availability Zones (AZs). For availability testing, SBI Securities used AWS Fault Injection Service (AWS FIS) to simulate failures in its production environment, implementing chaos engineering to increase fault tolerance.

“While we had conducted simulated failure tests before, this was not possible in the infrastructure area managed by AWS,” says Iwamoto. “When building a mission-critical system, we felt that testing in these areas was essential. Upon learning about the launch of AWS FIS, we decided to adopt it. As a result, we were able to repeat simulations of unpredictable AZ failures and achieve a smooth transition between AZs.”

For peripheral systems linked to domestic stock exchange systems, SBI Securities  performed connection tests to confirm latency and potential failures, safeguarding availability.

“The domestic stock trading system is linked to over 500 servers on AWS and on-premises,” says Fujita. “We conducted tests to withstand rapid increases in traffic while prioritizing all connections.”

Throughout the project, the AWS account team and solutions architects collaborated closely with the SBI Securities team, empowering both companies to overcome various challenges.

“The AWS team provided in-depth advice even on unexpected issues,” recalls Hahn. “There were instances where they dug deep together with the service team at the American headquarters of AWS to find solutions.”

After migrating to AWS, the number of the company's comprehensive securities accounts reached 11 million by the end of September 2023, when the Zero Revolution began. Since the launch of the new NISA, the customer base has continued to expand at an accelerated rate, reaching 13 million accounts on July 16, 2024. The domestic stock trading system now handles 100 million visits per day, processes approximately 3.6 million transactions daily, and manages a daily transaction volume exceeding 13 billion dollars. The new system, combining excellent scalability and flexibility with low latency, responds quickly to the rapid increase in visits and supports stable transactions.

“The number of visits has been growing at a pace that exceeds expectations. Responding to rapid business growth, our improved agility allows immediate capacity expansion, significantly increasing our confidence,” says Muto.

With infrastructure coding using AWS CDK, the capacity expansion period has been reduced to less than half the time required for on-premises systems.

“Currently, it's less than half, but as we advance automation in various areas, including testing, we aim to eventually reduce the expansion period to one quarter of the original time,” says Hahn.

Looking to the future, the company is using the expertise acquired through this project to plan for migrations of foreign stock, investment trust, and bond trading systems, as well as backend business systems, to AWS by around 2026.

The advancement of development environments remains an ongoing priority. In addition to exploring efficiency improvements using generative artificial intelligence, the company is actively promoting the development and recruitment of personnel with AWS expertise.

“Our engineering capabilities have significantly improved through in-house development, with SSS playing a central role. However, we still need to expand our pool of personnel skilled in AWS services, who can handle  code development using AWS CDK and drive cloud-native transformation,” notes Muto. “We appreciate the ongoing support from AWS and look forward to further enhancements in training programs to facilitate the stable operation of our mission-critical securities systems.”