E-commerce buyers want to pay for goods and services easily and securely. Amazon.com enables such payments with Amazon Wallet Service (Wallet), which every Amazon customer around the world interacts with when making purchases on Amazon or visiting the My Wallet page.
Wallet enables customers to place and pay for their orders. It stores the relationships between customers and their payment methods, including credit cards and bank accounts. Wallet is also the service responsible for the secure storage and use of customers’ payment methods for most business transactions on Amazon.com.
While making as many as five billion transactions daily, Amazon customers add new payment methods, subscribe to services such as Amazon Prime, and use 1-Click Ordering. Amazon needed a way to simplify scaling and storage management for Wallet while keeping costs in check and performance consistently high.
Wallet originally ran on an Oracle database, horizontally partitioned and replicated with standby servers and automatic failover. By 2016, Wallet was using 60 database instances. To support this rapid growth, Amazon had to scale Oracle vertically and horizontally on a regular basis, each time requiring three months of work from a team of engineers and database administrators—all without downtime. Growing and supporting Oracle ultimately required three to six months of software-development engineering time per year.
The Wallet team implemented an architecture that used an Oracle Streams replication solution, but this approach did not meet the required service levels. Replication monitoring and management increased operational load for engineering and database administrator teams.
To reduce the cost and operational complexity of operating the Wallet database, the Wallet team decided to move to Amazon DynamoDB on Amazon Web Services (AWS). DynamoDB is a nonrelational database that delivers reliable performance at any scale. DynamoDB eliminated the need to have a specialized database administrator team because the service is fully managed: It allowed the Wallet team to focus on development and not operations. Additionally, AWS CloudFormation templates helped the team define and manage the database infrastructure as code.
The migration to DynamoDB was relatively straightforward because of the well-architected nature of Wallet, but some changes were necessary. The Wallet team was moving the application from a SQL (relational) database to a NoSQL (nonrelational) database, so the team added a new application interface layer to connect the business logic and the database storage protocols. The associated use cases now rely on independent operations, which eliminated the need for cross-table transactions.
Using the auto scaling feature of DynamoDB, the Wallet team reduced administrative effort involved in scaling as well as the cost of operating the database. The Wallet team enabled Time To Live (TTL) to automatically remove records that are no longer used, reducing storage cost. Additionally, point-in-time recovery and on-demand backup and restore significantly reduced costs associated with high availability. To satisfy more sophisticated queries and data aggregation, the team connected the Wallet database to Amazon Redshift.
Wallet must work all the time—even during a migration. Amazon sought to eliminate any impact on customer transactions, maintain complete data consistency, and avoid a code or database schema “freeze” that would interrupt the adoption of new features.
To meet these goals, the migration was performed in three steps. First, in parallel with Oracle persistence, the Wallet team created the DynamoDB persistence layer. Operations were executed on both databases at once, one for the customer transaction and one for verification.
Second, the team migrated all records from Oracle to DynamoDB. The dual read/write setup guaranteed that all new records were created in both databases, meaning that only missing records had to be migrated. The team identified the instances in which transactions failed in one of the databases and recovered them to ensure full consistency.
Third, after the team verified that all records were successfully migrated, Amazon switched the DynamoDB database to the primary role. After running DynamoDB as the primary database for some time, the team disconnected the Oracle database. The migration process met all the team’s requirements, including zero downtime and zero customer impact, with more than 10 billion records migrated from eight Oracle tables to six DynamoDB tables.
DynamoDB performed operations much faster than the Oracle solution and reduced average latency by 50 percent. Maximum transactions per second increased by 40 percent per host, reducing the number of hosts required to process the same volume. The team reduced the time invested in database scaling and management by more than 90 percent.
These efficiencies allow the Wallet team to focus on building new features and capabilities rather than managing infrastructure. Higher performance and scalability mean Amazon can continue to deliver fast, seamless customer experiences even as transaction volumes continue to grow.
Learn more about Amazon DynamoDB, AWS CloudFormation, and Amazon Redshift.