Testing Big Projects Using Distributed Load Testing on AWS with Sony Music Solutions Inc.

Established in 1987, SMS, which is part of Sony Music Group, provides one-stop comprehensive entertainment solutions in Japan, offering 360-degree services across all business domains. From recording studios to concert venues to fan club operations, SMS provides expert services for designing, marketing, and merchandising music and video products. The company also securely develops and operates hundreds of websites.

Before launching its web services, SMS performs load and performance testing to make sure that the websites can seamlessly handle estimated traffic. The company previously used a third-party service for large-scale load testing, but the process required requesting services and waiting as long as several weeks to complete the test. As the number of SMS’s high-load projects increased, the company started exploring in-house load testing solutions to make its processes more cost-efficient and to meet shorter deadlines.

In 2022, SMS learned about Distributed Load Testing on AWS, which automates software application testing at scale and at load to help developers identify potential performance issues before application release. SMS was already on AWS; the company had built its infrastructure using AWS services such as Amazon Elastic Compute Cloud (Amazon EC2)—which provides secure and resizable compute capacity for virtually any workload. “We knew that Distributed Load Testing on AWS could meet our needs for in-house, large-scale load testing while keeping costs down,” says Hajime Tachimori, SysOps engineer at SMS.

SMS began using Distributed Load Testing on AWS in August 2022. Rolling out the solution was quick and simple because it is deployed as an AWS CloudFormation template, which speeds up cloud provisioning with infrastructure as code. “We were able to start using Distributed Load Testing on AWS very quickly and easily because we only needed to deploy the AWS CloudFormation template, and the deployment was completed in minutes,” says Tachimori. “Moreover, no customization or creation of additional templates was necessary.”

Now, SMS uses Apache JMeter, an open-source tool, to create test scenarios for typical website operations, such as heavy traffic or product selection and purchasing on an ecommerce site. Next, it compresses the scenarios and uploads them to Distributed Load Testing on AWS. SMS then sets the load test configuration by choosing the duration of the test and determining how many containers to launch using Amazon Elastic Container Service (Amazon ECS), a fully managed container orchestration service that empowers developers to more efficiently deploy, manage, and scale containerized applications. Finally, the company runs the test and uses AWS services to check the results, which involves confirming resource status, response times, and error counts. SMS can now complete a process that previously took weeks in as little as 3 days. A key reason for selecting Distributed Load Testing on AWS was to further boost engineering productivity because the solution uses Amazon ECS alongside AWS Fargate, a serverless, pay-as-you-go compute engine that gives SMS’s teams the capability to focus on building applications without managing servers.

Through this efficient, in-house load testing process, SMS can quickly calculate the load from test results and determine how many Amazon EC2 servers it needs to run for specific events, such as spikes in ticket purchasing ahead of a concert. “Our previous solution was time-consuming and laborious,” says Kazuki Kikuchi, a cloud engineer at SMS. “The biggest advantage of Distributed Load Testing on AWS is that you don’t have to worry about adjusting the number of servers for an event.”

SMS is also now better equipped to identify and resolve bottlenecks, which significantly enhances the company’s capability to handle a growing number of access requests. In fact, since implementing Distributed Load Testing on AWS, the company has increased the maximum requests per minute it can manage from 0.74 million to 2 million—a performance improvement of more than 170 percent.

In addition, SMS saved time and money by implementing Distributed Load Testing on AWS. “Once we decide on the load to test, we only need to set the number of containers accordingly,” says Kikuchi. “That leads to a significant reduction in time and effort.” On top of the time the company saves by performing tests in house instead of through a third party, SMS team members can also work more cost-efficiently. Using Distributed Load Testing on AWS, the company can carry out its testing for as little as a few dollars per load test. As a result, SMS has reduced costs by more than 90 percent per test.

In 1 week, SMS realized its goal of bringing complex performance and load testing processes in house, which helped the company increase performance to meet its growing needs while reducing costs.

Moving forward, the company plans to further streamline its testing process through automatic scheduling. In addition, SMS will expand its testing options both by developing its own tests and by collaborating with other companies.

“Using Distributed Load Testing on AWS offered great advantages,” says Kikuchi. “The load testing that had long been challenging became simple to carry out.”