Amazon Adopts Amazon Aurora for Inventory Database
Amazon delights customers with a vast selection of products and fast, often free, shipping. However, to deliver a simple, seamless shopping experience requires a massive amount of infrastructure and technology behind the scenes. Some of Amazon’s largest warehouses occupy nearly one million square feet, employ thousands of associates, and can hold millions of different inventory items.
The Amazon Fulfillment Technologies (AFT) team builds and maintains the company’s Warehouse Management Systems. Among these are Inventory Management Services (IMS), which facilitate warehouse processes, including inbound and outbound shipments, item picking, sorting, packaging, and inventory storage. These are essential for on-time delivery of customer orders.
“After migrating to AWS Aurora, provisioning additional capacity is achieved through a few simple mouse clicks.”
Brent Bigonger, Senior Database Administrator, Amazon
AWS Services Used
Amazon is guided by four principles: customer obsession rather than competitor focus, passion for invention, commitment to operational excellence, and long-term thinking. Customer reviews, 1-Click shopping, personalized recommendations, Prime, Fulfillment by Amazon, AWS, Kindle Direct Publishing, Kindle, Fire tablets, Fire TV, Amazon Echo, and Alexa are some of the products and services pioneered by Amazon.
- Reprovisions in minutes instead of days
- Reduced scaling effort by 95%
- Completed migration with minimal downtime
- Reprovisions in minutes instead of days
AWS Services Used
Databases Bursting at the Seams
Unfortunately, IMS was starting to outgrow its technical foundations. It used a pair of on-premises Oracle databases for each warehouse. Given the large maintenance burden of an unmanaged Oracle database, a large team of database engineers were dedicated to maintenance activities such as OS patching and upgrades. AFT was also challenged by complicated and inefficient hardware provisioning.
“Implementing and scaling hundreds of on-premises instances across the database fleet consumed all database team member resources for many weeks and was a relatively low value-add task for database administrators," says Brent Bigonger, senior database engineer at Amazon. “We knew we could increase efficiency and free up time if we used a managed database service in the AWS cloud.”
Choosing the Cloud
In May 2017, AFT decided to migrate the system to Amazon Aurora. A MySQL- and PostgreSQL-compatible database built for the cloud, Aurora provides the performance and availability of Oracle at approximately one-tenth the cost. Amazon Aurora is fully managed by Amazon Relational Database Service (Amazon RDS), which automates time-consuming administration tasks such as hardware provisioning, database setup, patching, and backups.
PostgreSQL was selected as the database because it could handle complex write transactions at speeds comparable to the Oracle system. “Our writes have a lot of components,” says Michael Wu, principal software development engineer at Amazon. “They include things like location, current state, and financial operations. The PostgreSQL-compatible edition of Amazon Aurora offers high performance for these scenarios.”
High-Stakes Migration, Managed
Once Amazon Aurora was selected, the team began planning the migration. AFT took a replatforming approach, preserving existing service architectures and database schemas to minimize complexity and disruption. Replatforming allowed more than 200 external services to migrate their dependencies with minor code changes.
Automation was essential to enabling such a large-scale migration. “Manual deployment simply wouldn’t work,” says Wu. “We wanted to migrate as quickly as possible by building mechanisms that would allow teams to manage the migration effectively.”
AWS Database Migration Service was used to migrate data from the Oracle databases to the PostgreSQL databases running on Amazon Aurora. Thanks to the modular nature of AWS, it took a team averaging three engineers only three months to complete the automation framework.
Because of the need for always-on operations in the fulfillment centers, the downtime window for the migration in each center was only two hours. Thanks to the automation built in the first stage, the team maintained a 100 percent migration success rate. “It’s always challenging to migrate a large system in a short time frame, but with AWS Database Migration Service, we were able to far exceed the initial expectations,” says Anthony Girolamo, senior software development manager at Amazon.
Keeping Up with Business Growth
The elastic capacity of preconfigured database hosts on AWS eliminated much of the administrative overhead required to scale the system. The transformation has been significant. “In the Oracle world, a seemingly simple change such as scaling from a medium to a large database instance required provisioning hardware, standing up primary and standby databases, and managing failover during transitions, which could take a full day for each instance,” says Bigonger. “Not to mention the fact that we were using specialized hardware that had to be ordered months in advance. After migrating to Amazon Aurora, provisioning additional capacity is achieved through a few simple mouse clicks or API calls.” The scaling effort has been reduced by as much as 95 percent.
High availability is another key benefit of Amazon Aurora. “Previously, if we lost a piece of hardware, it would impact our ability to maintain redundancy for up to a day,” says Bigonger. “With Amazon Aurora, reprovisioning happens automatically in just minutes, ensuring data is always fully protected.”
With the performance of Amazon Aurora, the team is no longer limited by the input/output operations the database instance can handle. “During the process, we shed software ‘technical debt,’ making our software systems better and enabling greater agility,” says Wu. “Our engineers can kickstart projects by spinning up their own test instances whenever they need to. It took the complexity out of the process.”
Migrating IMS to Amazon Aurora resulted in a range of benefits—the most important of which is dependable fulfillment for customers. “Using Amazon Aurora, we can keep up with Amazon’s business growth,” says Girolamo. “That means customers get products faster—which is why we do what we do.”
More Amazon Stories
Amazon's Cloud Journey on AWS
Amazon.com migrated 5,000 databases from Oracle to AWS, cutting its annual database operating costs by more than half and reducing the latency of most critical services by 40 percent. The company leverages Amazon DynamoDB to scale Prime Video and Amazon SageMaker to indentify misplaced inventory.
How Amazon is Achieving Database Freedom Using AWS
Migrating to AWS has cut Amazon's annual database operating costs by more than half. Database-administration and hardware-management overhead have been greatly reduced, and cost allocation across teams is much simpler than before.
Amazon.com Case Study
Amazon.com is the world's largest online retailer. In 2011, Amazon.com switched from tape backup to using cloud-based Amazon S3 for backing up the majority of its Oracle databases. By using AWS, Amazon.com was able to eliminate backup software and experienced a 12X performance improvement, reducing restore time from around 15 hours to 2.5 hours in select scenarios.
Amazon Reduces Infrastructure Costs on Visual Bin Inspection by a Projected 40% Using Amazon SageMaker
Amazon Fulfillment Technologies migrated from a legacy custom solution for identifying misplaced inventory to Amazon SageMaker, reducing AWS infrastructure costs by a projected 40 percent per month and simplifying its architecture.
Amazon Prime Video Uses AWS to Deliver Solid Streaming Experience to More Than 18 Million Football Fans
Amazon Video selected AWS Elemental, an AWS company that combines deep video expertise with the power and scale of the cloud to empower media companies to deliver premium video experiences to consumers.
Prime Video Boosts Scale and Resilience Using Amazon DynamoDB
As part of its strategy to build a platform that could scale to meet projected needs for at least 10 years, Amazon decided to migrate Prime Video to AWS. The migration would replace CQS with a suite of 12 microservices, built using a range of AWS services that included Amazon DynamoDB, AWS Lambda, and Amazon Simple Queue Service (Amazon SQS).
Amazon Robotics Uses Amazon SageMaker to Enable ML Inferencing at Scale
Amazon Robotics developed a sophisticated machine learning model using Amazon SageMaker to streamline inventory management, halving inferencing costs and improving inventory stow rate.
Amazon Advertising Case Study
Amazon Advertising Engineering and Development (AED) increased throughput and avoided a three-year project to rebuild performance monitors and management tools by shifting data held in Oracle databases to Amazon RDS. AED builds, manages, and scales the technologies that undergird Amazon's programmatic advertising offerings.
Amazon.com Buyer Fraud Service Gains Scalability, Cuts Costs in Half Using AWS
Amazon Transaction Risk Management Services (TRMS) migrated more than 100 on-premises Oracle databases to AWS. Because the Buyer Fraud Service is a critical application and must operate at 99.995 percent availability, TRMS decided to use PostgreSQL-compatible Amazon Aurora as the new platform to host its databases.
Amazon CloudWatch Delivers Metrics to Customers Faster, Saves Millions Annually Using Amazon DynamoDB TTL
Amazon CloudWatch (CloudWatch) is a service used by AWS customers to monitor their cloud resources and applications running on AWS.
Scaling Amazon.com’s Transactional Subledger using Amazon DynamoDB
Across its many business entities, Amazon uses the Financial Ledger and Accounting Systems Hub (FLASH) to process more than 20 billion financial transactions each month. In 2017, the FLASH organization decided to migrate all its Oracle databases to AWS services.
Amazon Speeds Desktop Deployment for 25,000 Global Workers Using Amazon WorkSpaces
Amazon's Client Engineering team uses Amazon WorkSpaces to give global workers fast, restricted access to corporate resources, onboard new users in minutes, and save over $17 million annually.
Learn more about Amazon Aurora.