Scaling Amazon.com’s Transactional Subledger using Amazon DynamoDB

Across its many business entities, Amazon processes more than 20 billion financial transactions each month, including accounts receivable, accounts payable, royalties, amortizations, remittances, payments, and cash.

The Financial Ledger and Accounting Systems Hub (FLASH) is a suite of microservices that ingests these financial transactions, performs complex and business-critical functions to substantiate the general ledger, and generates financial statements such as balance sheet, cash flow, and income.

Over the years, FLASH grew to require more than 90 Oracle databases, totaling more than 120 terabytes of data and growing. “Simply maintaining and administering the database technology took hundreds of hours per month,” says Marisamy Krishnan, senior technical program manager at Amazon.

Hardware provisioning for the on-premises solution was complicated and inefficient, costing the FLASH team nearly 100 hours annually. At one point, some of the FLASH services were running on the largest available Oracle-certified hardware.

With transaction volumes growing by a double-digit percentage each year and twofold spikes in volume during peak sales periods, Amazon needed to find a solution that was more scalable and easier to manage. In 2017, the FLASH organization decided to migrate all its Oracle databases to AWS services.

While migrating, the FLASH team had to ensure compliance with Sarbanes-Oxley (SOX) guidelines and tax audits. “We have more than 20 years of financials, subledger details, and other data that is critical to reporting and audits,” says Phani Bhamidipati, senior manager of software development at Amazon. “It had to be accessible during and after the migration.”

They reviewed AWS services with the Amazon security team and created service templates using AWS CloudFormation that met data-protection requirements. To ensure accurate transfer of historical data, AWS Database Migration Service performed row-by-row validations. The team added random sampling of data to the tests for additional verification. To ensure SOX compliance, the FLASH team sent recommendations to service owners on access-control mechanisms and encryption parameters. The team used Amazon Simple Storage Service (Amazon S3) for inexpensive, long-term storage of relational and nonrelational data.

Re-architecting FLASH and adopting highly available and reliable AWS services significantly improved database performance. Most of the critical services that were moved to Amazon DynamoDB saw a 40 percent reduction in latency, despite handling twice the number of transactions. At the same time, costs dropped by 70 percent. Overall, the annual database operating costs for FLASH have remained the same despite provisioning higher capacity on AWS.

Additionally, the move reduced administrative overhead by 70 percent, enabling the engineering team to spend its time improving service performance through activities such as query optimization and performance analysis.

The elastic capacity of preconfigured database hosts on AWS eliminated the administrative overhead required to manage scaling. AWS services have abstracted the hardware away from the engineers, allowing them to focus on optimizing configurations. Using Amazon DynamoDB auto scaling enabled the team to reduce cost by dynamically responding to traffic spikes.

Amazon.com is the world’s leading online retailer and the pioneer of customer reviews, 1-Click shopping, personalized recommendations, Prime, AWS, Kindle, Alexa, and many more products and services.

 

Organizations of all sizes across all industries are transforming their businesses and delivering on their missions every day using AWS. Contact our experts and start your own AWS journey today.