The Pokémon Company Migrates to AWS Purpose-Built Databases
Launched in Japan in 1996, Pokémon has become one of the most popular entertainment brands worldwide. People of all ages—hundreds of millions of them—enjoy training and battling Pokémon creatures through the Pokémon Trading Card Game and the Pokémon GO mobile game.
In Bellevue, Washington, engineers at The Pokémon Company International (TPCi) are responsible for making sure Pokémon fans outside of Asia can track their achievements through full-featured user accounts. Known as the Pokémon Trainer Club, the solution not only must deliver a seamless experience any time a “trainer” wants to sign on but must also comply with the Children's Online Privacy Protection Act and General Data Protection Regulation.
“Our user profile system is largely two sets of data: users for Pokémon GO and users for the Pokémon Trading Card Game,” says Jeff Webb, development manager at TPCi. “User profiles from different play types require different processes and maintenance, but they all live in the Pokémon Trainer Club system. Because of the complexity and volume of data we’re dealing with, database scalability and reliability are absolutely critical to delivering a great user experience.”
After the launch of Pokémon GO in 2016, the number of users requiring access to this system increased to more than 300 million in two years. The company decided to migrate to a fully managed solution to free up time and resources to focus on strategic initiatives.
Using Amazon Aurora, we went from 300 nodes to 30, and we are no longer paying for database licenses. Our monthly database cost has dropped by tens of thousands of dollars each month."
Development Manager, The Pokémon Company International
Battle of the Databases
Originally, the Pokémon Trainer Club was supported by a third-party NoSQL document database, which contained all user profile data and changelogs for more than 300 million Pokémon GO users. Database shards contained the master database of Pokémon creatures, Trading Card Game online user data, and data from in-person “organized play” groups. Memcached provided fast access to user session data. The company hosted its database and memory-caching system on Amazon Elastic Compute Cloud (Amazon EC2), with Elastic Load Balancing managing traffic and Amazon Simple Storage Service (Amazon S3) for object storage. “We chose our previous NoSQL database because of its fast lookups,” notes Webb. “However, we found out that achieving the speed we wanted required maintaining many indices and managing many complex nodes—more than 300 at one point.”
Keeping nodes running smoothly was challenging. When index or database nodes became unavailable, it increased latency. If a Memcached node went down, users couldn’t log in at all. When primary data nodes failed, the team had to drop everything to rebuild them while the system operated on backup nodes. After years of daily management issues, exacerbated by the massive influx of Pokémon GO users, TPCi decided to migrate to Amazon Web Services (AWS) for its fully managed database services.
“We were happy with the performance and reliability of Amazon EC2, Elastic Load Balancing, and Amazon S3,” recalls Webb. “That’s why we decided to investigate AWS managed database services.” The team performed extensive testing to find a solution that would meet its security, scale, performance, and resilience needs, ultimately choosing Amazon Aurora with PostgreSQL compatibility. “Because the authentication requests make up the majority of TPCi traffic, we selected Amazon Aurora PostgreSQL for its enterprise-grade security features, including encryption at rest and in transit, plus its reliability in a multi-region environment.”
The team also decided to move user caching to Amazon ElastiCache, taking advantage of both Redis and Memcached. Redis queues tasks for new users, so they can be prompted to complete post-authentication tasks such as accepting terms and conditions. Memcached helps keep tickets live so that existing users’ sessions are not interrupted when new users join. The team migrated global configuration and time-to-live (TTL) data to Amazon DynamoDB, a key-value database that delivers single-digit millisecond performance at scale. And they also moved event data, such as login history and user change audit records, to Amazon S3.
Reduced Cost, Increased Capacity, and Zero Downtime
The non-relational data stores were migrated first, followed by the migration of user data and database shards to Amazon Aurora. The project took only nine months. “We spent the first few months creating use cases and pressure tests,” says Webb. “Then we relationalized all the data and deconstructed JSON data. We completed the migration with no downtime for our authentication process, which is remarkable considering that 300 people log in to the Pokémon Trainer Club every second from all around the world.”
Since the migration, TPCi has significantly reduced the time and money spent on databases. “Using Amazon Aurora, we went from 300 nodes to 30, and we are no longer paying for database licenses,” says Webb. “Our monthly database cost has dropped by tens of thousands of dollars each month. Most importantly, we had 168 hours of downtime or performance degradation in the six months prior to migration. Since then, we’ve had zero.”
The migration has delivered other benefits as well. “Using the built-in TTL settings in Amazon DynamoDB, we can track when a user exceeds the maximum login attempt threshold and deny entry,” says Webb. “The result has been a 90 percent reduction in bot login attempts, which frees up system resources for legitimate users and reduces our need to overscale.”
Taking advantage of the fully managed Amazon ElastiCache service significantly reduces the overhead of managing Memcached nodes. User session data and ticketing states stored in the cluster speed up the authentication process and eliminate the need for a user to maintain a connection to a single server.
Less time managing databases frees time for innovation. The company is planning to use serverless computing, containers, and microservices for even greater efficiency and performance. As its services continue to improve, TPCi empowers millions of people around the world to enjoy the Pokémon universe on demand.
To learn more, visit aws.amazon.com/gametech/databases.
About The Pokémon Company International
The Pokémon Company International, a subsidiary of The Pokémon Company in Japan, manages the property outside of Asia and is responsible for brand management, the Pokémon Trading Card Game, home entertainment, and the official Pokémon website.
Benefits of AWS
- Cuts monthly costs by tens of thousands of dollars
- Reduces number of nodes from 300 to 30
- Reduces bot login attempts by 90%
- Eliminates database licensing costs
- Experiences zero hours of downtime or performance degradation after migration
- Handles up to 300 logins per second
AWS Services Used
Amazon Aurora is a MySQL and PostgreSQL-compatible relational database built for the cloud that combines the performance and availability of traditional enterprise databases with the simplicity and cost-effectiveness of open source databases.
Amazon ElastiCache allows you to seamlessly set up, run, and scale popular open-Source compatible in-memory data stores in the cloud. Build data-intensive apps or boost the performance of your existing databases by retrieving data from high throughput and low latency in-memory data stores. Amazon ElastiCache is a popular choice for real-time use cases like Caching, Session Stores, Gaming, Geospatial Services, Real-Time Analytics, and Queuing.
Amazon DynamoDB is a key-value and document database that delivers single-digit millisecond performance at any scale. It's a fully managed, multiregion, multimaster, durable database with built-in security, backup and restore, and in-memory caching for internet-scale applications.
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. This means customers of all sizes and industries can use it to store and protect any amount of data for a range of use cases, such as websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics.
Organizations of all sizes across all industries are transforming and delivering on their missions every day using AWS. Contact our experts and start your own AWS Cloud journey today.