AWS Lambda had a striking effect. Image processing that used to take several hours was finished in a little over 10 seconds. We were also able to reduce costs to about one twentieth of those for performing the same processing on premises. The concept of running out of resources doesn't apply to AWS Lambda at all. This is something you can't replicate with an on-premises solution.
Daisuke Agata Technical Director, Business Division No. 6, Square Enix Co., Ltd.

Square Enix's corporate philosophy is to spread happiness across the globe by providing unforgettable experiences. The company provides high-quality entertainment content, services, and products. Its flagship video-game titles include Dragon Quest, Final Fantasy, and Tomb Raider. Dragon Quest, one of the company's key game franchises, began as game software for the Nintendo Entertainment System in 1986. With Dragon Quest X, it has evolved into a type of online game known as an MMORPG (Massively Multiplayer Online Role-Playing Game). For Dragon Quest X, in which hundreds of thousands of people play in the same world, Square Enix has built infrastructure designed to provide a good play experience to customers at all times.

Dragon Quest X is connected to a portal site and smartphone app that provide players with the ability to exchange messages, make purchases, and manage characters in the game. It also features a function for taking in-game screenshots, which customers can view and manage from the portal site. This function has proven very popular, and customers take a huge number of screenshots, especially during events held in the game that coincide with occasions such as Christmas or New Year's Eve.

When an in-game screenshot is taken, the image is sent to a server where processing—such as thumbnail creation and the addition of a copyright watermark—occurs. "This image processing is quite resource intensive," says Daisuke Agata, technical director at Square Enix. That means there is a spike in server load when a lot of people take screenshots at the same time, such as during events. Usually 200 to 300 images are received for processing each minute, but during New Year's Eve in-game events, this can climb as high as 6,000 images per minute.

"The screenshots are processed sequentially server side. During the spikes that occur a few times each year, it would sometimes take three or four hours before image processing finished and customers could view their screenshots," Agata says. "Accelerating this process and boosting customer satisfaction was one of the challenges Square Enix faced in growing its customer base."

square-enix_image

In-game screenshots taken by customers.

To accelerate image processing, Square Enix first looked into increasing the number of threads processed by preparing a new server with a specification of around 64 cores. "This was because we thought boosting parallel processing capability would be more effective than increasing the speed of processing individual images," says Agata.

After several internal discussions, the company decided it would be more efficient to move to the cloud instead of introducing a new server. Kojiro Kubo, of the Square Enix Information Systems Department, recommended the AWS Cloud, believing it would be hard to justify introducing an expensive server simply to handle spikes that only occur a few times each year. "In particular, I believed using AWS Lambda to execute code on the AWS infrastructure would eliminate the need to perform image processing in our environment, while still enabling integration with on-premises servers," Kubo says. "I also thought AWS had a low barrier to entry and seemed easy to use, so I recommended it."

When a screenshot is taken, the binary image data are first uploaded to Amazon Simple Storage Service (Amazon S3). An Amazon S3 event then triggers AWS Lambda for image processing. After this, queues containing the binary image data output from Amazon Simple Queue Service (Amazon SQS) are imported into the on-premises servers to save the processed image data.

The in-game screenshot function is expected to cause even larger spikes as customer numbers grow. "With AWS Lambda, we can theoretically scale up without limit. This was another important consideration in the decision-making process," says Agata.

Square Enix prepared samples and tested them on AWS Lambda beginning in summer 2015. "We made it possible to run our image processing on AWS Lambda. We then spent time on testing, such as figuring out what to do about integration with our on-premises servers," says Agata.

Because availability and reliability are critical for the screenshot feature, Square Enix also tested backup plans. Following these tests, the company completed migrating to AWS Lambda in December 2015.

jp_diagram_square-enix_1024x406

Using AWS Lamba, Square Enix has reduced the amount of time it takes for a customer to be able to view their screenshots. "Customers wouldn't be satisfied if it took an hour for screenshots to be viewable after taking them. So we set a goal of 10 minutes to complete processing, and we then performed tests based on a rough workload of 18,000 images per minute, which is triple the 6,000 images per minute expected at peak times," says Agata.

At a New Year's Eve event after migration to AWS Lambda, screenshots were taken at a rate of more than 6,000 per minute. "AWS Lambda had a striking effect. Image processing that used to take several hours was finished in a little over 10 seconds. We were also able to reduce costs to about one twentieth of those for performing the same processing on premises," says Agata.

Adopting AWS also enabled Square Enix to reduce both infrastructure and labor costs. "Eliminating the labor associated with operations, maintenance, and server replacement also reduced human resources costs," says Kubo.

Information about Dragon Quest X is often sent to customers via push notifications, and this processing fluctuates significantly depending on the day and time. Consequently, Square Enix is looking into using Amazon Simple Notification Service (Amazon SNS) for notifications. It is also considering adopting Amazon CloudFront for content delivery.

"In the future we would also like to implement proper certificate handling to reduce the risks associated with providing web-based services. In light of that, we are considering adopting AWS Certificate Manager (ACM), which can handle SSL/TLS certificate provisioning and management," Kubo says.

"We'd like to continue successfully integrating our on-premises environment while we proactively investigate the migration of services that would be more efficient in the cloud," says Agata.

square-enix_photo2

Kojiro Kubo (left), Square Enix Information Systems Department, and Daisuke Agata (right), Square Enix technical director.

To learn more about how AWS Lambda can help you run serverless data processing, visit our Serverless Computing details page.