Asahi Shimbun Uses Amazon DocumentDB (with MongoDB Compatibility) as Content Master Database to Overhaul Official App, Increases Use by 20% Due to Improved Functionality
Headquartered in Osaka, Japan, The Asahi Shimbun Company produces a wide range of media in formats such as the national newspaper The Asahi Shimbun and the news site Asahi Shimbun Digital. The company built an app delivery infrastructure using Amazon Web Services (AWS) when overhauling the official Asahi Shimbun Digital app in April 2020. For its content master database, which stores past articles, Asahi Shimbun selected Amazon DocumentDB (with MongoDB compatibility), a fast, scalable, highly available, and fully managed document database service that supports MongoDB workloads. By utilizing Amazon DocumentDB, which has high performance and availability and is easy to maintain, the company reduced operating costs while maintaining flexibility for its article category searches and other areas.
By introducing managed services and serverless architecture from AWS, we were able to obtain a development environment that eliminates the creation of source code and has no overhead."
Assistant Manager, Customer Experience Department
Digital Innovations Division, The Asahi Shimbun Company
Identifying the Need for a Content Master Database to Easily Extract Data from Documents
Founded as a newspaper in 1879, The Asahi Shimbun Company continues its evolution into a general media business that enriches lives under its corporate motto of “Let’s think together and create together.” Asahi Shimbun Digital is one of the world's oldest online news sites, dating back to the early days of the internet in 1995. The site has approximately 3.75 million registered users between the browser version and app version, with approximately 200 million monthly page views and about 60,000 daily active users as of May 2020.
Since its release in 2012, the Asahi Shimbun Digital app seen many iterations. However, over that time, many users have asked for the app to be made easier to use. According to Katsuji Moriyasu, assistant manager of the Customer Experience Department, Digital Innovation Division, “That is why we decided to overhaul the official app for the first time in around 6 years in order to improve the UI and UX. We re-examined our development system, which had been outsourced to vendors, and chose to proceed by bringing things in-house.”
When aiming to improve the UX, there were issues with the design of article delivery in the app infrastructure, which had been built using some AWS services, include Amazon Simple Storage Service (Amazon S3) and Amazon CloudFront. Tetsuya Nishikawa of the Customer Experience Department, Digital Innovations Division explained the situation: “In the past, we had used Amazon S3 for the article master database, which stored articles by specific periods and genres in .zip files using batch processing. And we were only able to deliver these endpoints to the app through Amazon CloudFront via the Web API. This meant that past articles could not be pinpointed when searching, and we were restricted from calling up articles that met user characteristics. In order to resolve this problem, we decided to introduce a content master database that could extract document data in the new app and in which functionality could be developed easily. ”
Evaluating Read Performance and Selecting Amazon DocumentDB as a Fully Managed Service
The company considered a NoSQL database for its content master database, focusing on MongoDB and examples of its successful implementation to ensure there was thorough documentation. The Asahi Shimbun Company ultimately chose Amazon DocumentDB, which became available in the AWS Asia Pacific (Tokyo) Region in May 2019, as its MongoDB-compatible database. The company had three main reasons for choosing Amazon DocumentDB: high performance, scalability, and availability; low maintenance as a fully managed service; and its easy creation of indexes. Motoki Okamoto of the Customer Experience Department, Digital Innovations Division explains: “Amazon DocumentDB has high read browsing performance and maintains high performance that can withstand millions of requests. In addition, we determined that since it is a fully managed service, we are freed from operations such as scaling, backup and patch application, which would lead to reduced development and operation costs.”
When implementing Amazon DocumentDB, The Asahi Shimbun Company conducted a proof-of-concept (POC), which took place over approximately three weeks from late September 2019, to verify processes from article registration in the database through retrieval using the API. After optimizing performance in the POC, the company started on development, which was completed by December 2019. The production environment was opened in March 2020 after going through standalone tests and front-end integration testing, and the new Asahi Shumbun Digital app was released on April 13, 2020.
“We chose a serverless architecture using AWS Lambda, AWS Step Functions and Amazon Kinesis Data Streams for the batch processing that stored the articles in the database. The API was containerized in AWS Fargate and managed using Amazon Elastic Container Service (Amazon ECS)," says Okamoto.
Internal development on the backend was performed by 12 engineers across three teams covering batch processing, APIs, and infrastructure. The teams avoided proprietary logic and stuck with a policy of completing the project using AWS managed services while following the data flow. “We were especially able to reduce the amount of effort writing scripts by using AWS Step Functions that support the creation of workflows," says Nishikawa. "We also received expert advice on batch processing and the use of events from AWS solution architects, so the in-house development proceeded smoothly.”
Dramatic Improvement in Past Article Search, New Targeting Capabilities
By using Amazon DocumentDB, the new Asahi Shimbun Digital app dramatically improved search capabilities and can now return past articles that match user interests in search. Compared to the structure of the old app, the company has reduced operating costs by 30 percent. "As long as we’re within the memory capacity limits, we can use Amazon DocumentDB at the same pricing even if data increases. Moving forward, as we increase the volume of stored articles, we will be able to stay within the memory capacity limits as long as we handle text data, and the fact that there will be continued service improvements while maintaining the fixed pricing is a major advantage," says Okamoto.
For the development environment as well, it is now possible to run short improvement cycles since the team can conduct standalone tests on the local environment, verification environment, and others. By bringing the process in-house, The Asahi Shimbun Company no longer has a "black box" system and expects to make significant improvements in development speed. “By introducing managed services and serverless architecture from AWS, we were able to obtain a development environment that eliminates the creation of source code and has no overhead. Engineers are quickly adding new functions, and you can feel the advantages of bringing the work in-house, including the ability to fix bugs," says Moriyasu.
Migrating App to a Cloud-Native Architecture for a More Personalized User Experience
Overhauling the official Asahi Shimbun Digital app made it so articles could be found more intuitively and enhanced usability in the form of significantly improved launch and response times. As a result, usage frequency increased by approximately 20 percent, with daily average app launches growing from 1.9 to 2.3. In the future, the company wants to continue improve by moving to a fully cloud-native architecture to create an even more personalized user experience.
“We believe that, in the end, making daily progress on improvements leads to strengthened functionality and reduced running costs. We will make that progress on improvements by continuing to have AWS provide new capabilities, including new services," says Moriyasu.
In the meantime, the Asahi Shimbun Company is discovering vast potential in Amazon Kinesis Data Streams, which it's chosen for additional app development, and is planning the implementation of new functions utilizing stream data. According to Nishikawa, “We are considering further utilizing AWS services to deliver greater peace of mind to customers, including providing functions that utilize real-time data such as location and health care information as well as quality improvements that utilize user login data.”
About The Asahi Shimbun Company
Benefits of AWS
- Significant improvement to searchability of past articles
- 30% reduction in monthly operation costs for extracting past articles
- Shortened improvement cycle and reduced debugging time
- Improved development speed by bringing the development system in-house
- Approximately 20% improvement to frequency of use for the app
AWS Services Used
Amazon DocumentDB (with MongoDB compatibility) is a fast, scalable, highly available, and fully managed document database service that supports MongoDB workloads.
AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume.
AWS Step Functions
AWS Step Functions is a serverless function orchestrator that makes it easy to sequence AWS Lambda functions and multiple AWS services into business-critical applications. Through its visual interface, you can create and run a series of checkpointed and event-driven workflows that maintain the application state.
Amazon Kinesis Data Streams
Amazon Kinesis Data Streams (KDS) is a massively scalable and durable real-time data streaming service. KDS can continuously capture gigabytes of data per second from hundreds of thousands of sources such as website clickstreams, database event streams, financial transactions, social media feeds, IT logs, and location-tracking events.
Click here ≫ for the case study on building a prototype “article and image search system” for editors at The Asahi Shimbun Company