AWS Case Study: So-net Entertainment

In the internet advertising business, customer event logs are critical to understanding customer behavior patterns, such as the frequency and amount of time spent on different pages and trending search terms. So-net has been managing a fleet of on-premise servers to store and analyze their customer logs. However, So-net’s data volume has surged with the rapid growth of its business, drastically increasing capacity requirements and slowing log processing time. Given the importance of timely insights to the business, these delays quickly became unacceptable.

In March 2010, So-net started a development project team to build a solution to handle their exploding data. “It was not practical to efficiently store and analyze 3.5 TB of yearly log data with our on-premise infrastructure,” said Takahiro Yasuda, the leader of the development team. “Moving to the cloud allowed us to cost-effectively meet these rapidly increasing capacity requirements while limiting our upfront commitment, providing the flexibility to scale up and down based on changing needs, and lowering costs by only charging us for what we used. Further, all of this could be done without having to rely on external professionals to build and manage the infrastructure. We chose AWS because we'd had positive experiences working with them and knew that no other vendor could offer their level of rapid innovation.”

After a two-month test and evaluation period, the development team started uploading data into Amazon Simple Storage Service (Amazon S3). In April 2011, So-net uploaded 5 TB into S3 and officially launched its information analytics system. Using Amazon Elastic MapReduce to efficiently process their data, So-net can now analyze up to twelve months of log data faster than it took to do a single month previously. Both Amazon Simple Storage Service (S3) and Amazon Elastic MapReduce have also brought huge ROI benefits to the company. So-net no longer incurs capital expenses and their operational costs are around JPY 500,000 per month, a huge savings compared to the tens of millions of Japanese Yen that would have been required to purchase and maintain on-premise infrastructure.

So-net has experienced further cost savings by leveraging unused EC2 capacity through Spot Instances. Akikazu Takeuchi, a Senior Research Fellow at So-net, “Running Amazon Elastic MapReduce on Spot Instances has reduced the cost of our ad hoc data analysis by approximately 50%"

Furthermore, AWS Identity and Access Management (IAM) has allowed So-net to easily manage the complex security and resource management requirements of its internal users.

So-net's system diagram

To meet the demands of an ever expanding business, So-net’s development team is continuously working to improve their analytics systems. “With Amazon Web Services, we no longer spend time worrying about underutilized infrastructure and maintenance and can focus on differentiating through our customer offerings,” said Takahiro Yasuda. “Rapid access to critical customer insights has allowed us to differentiate our business in the face of a highly competitive and rapidly changing industry.”

To learn more about So-net, visit http://www.so-net.ne.jp/corporation/en/overview.html.

July 30, 2011