Amazon Reduces Infrastructure Costs on Visual Bin Inspection by a Projected 40% Using Amazon SageMaker
Amazon Fulfillment Technologies (AFT) designs, develops, and operates fulfillment technology solutions for Amazon fulfillment centers (FCs), from automated Amazon Robotics FCs to micro-/pop-up nodes worldwide. AFT must monitor millions of global shipments annually to deliver on Amazon’s promise that an item will be readily available to a customer and will arrive on time. To operate at Amazon’s scale, AFT’s internal visual bin inspection (VBI) team had a proprietary legacy computer vision–based software solution that scanned millions of images across its network of FCs to identify misplaced inventory. Deploying, testing, training, and maintaining the solution were expensive and time consuming. Building new capabilities to overcome those limitations would take several months of development effort.
The VBI team turned to an Amazon Web Services (AWS) solution. The team migrated to Amazon SageMaker, a fully managed service that provides developers and data scientists with the ability to build, train, and deploy machine learning (ML) models quickly. This helped the VBI team achieve a simplified architecture by reducing the technical costs with less infrastructure, enabling them to achieve faster and easier ML model deployments and relieve their development team of the responsibility of system maintenance. The VBI team can now quickly scale with each new FC to perform inspections to support Amazon’s customer promise.
“We will cut infrastructure costs by 40 percent with batch inference—just by using the features that came with Amazon SageMaker.”
Senior Engineer, Amazon Fulfilment Technologies
Solving Past System Deficiencies Using Amazon SageMaker
The VBI team builds autonomous counting systems that use computer vision, ML, and image-processing algorithms to analyze images across its network of FCs worldwide. Initially, the VBI team ran its own solution on Amazon Elastic Compute Cloud (Amazon EC2) C4 Instances. This solution did not support piloting new models—that is, it did not enable the team’s new models to handle requests alongside the old ML model nor did it allow the team to test it using real data and without risking service disruptions. Consequently, the VBI team had to develop ML models offline and validate and test them manually, which often took 3–6 months. The system also did not provide the capability to divert volume to new model launches. And inference—the process of making predictions using a trained model—was limited to CPU-based single input predictions, so the VBI team couldn’t reduce costs with graphics processing units (GPUs) or batch execution.
The VBI team was drawn to Amazon SageMaker because it solved the inefficiencies of the team's legacy software solution while reducing the amount of software and infrastructure that needed to be maintained. The fully managed service enabled the software development team to focus on core competencies instead of system management.
Cutting Workload Using Amazon SageMaker
The VBI team developed the new solution on Amazon SageMaker over 12 weeks; during that time, the team containerized the inference code and built infrastructure for automating deployment. By comparison, the VBI team needed 8–12 months to develop its legacy solution. Then the VBI team conducted performance testing for another 2 months and completed its migration in April 2020 with no disruption to normal operations.
Due to the migration, Lalat Nayak, senior engineer for the VBI team at AFT, projects: "We will cut infrastructure costs by 40 percent with batch inference—just by using the features that come with Amazon SageMaker.” The team also gained a simplified system, trading in 1,000 CPU instances for one endpoint that automatically scales a fleet of fewer than 100 GPU instances. That transition to GPUs has cut the time-to-predict latency by 50 percent. The single Amazon SageMaker endpoint, which handles traffic from sensor towers across North America, Europe, and Japan, processes millions of bin images per day.
When AFT launches FCs in new regions, it now expects to deploy the Amazon SageMaker solution in 1–2 weeks, versus 1–2 months with its legacy solution. “Relying on Amazon SageMaker to host the model gives us the ability to decide whether we use the same model for all the warehouses or just some,” explains Nayak. “It helps us run experiments freely because we can release a feature for just a handful of warehouses and then expand to the rest after successful testing.”
Amazon SageMaker saves the software development engineer team about 1 month per year in maintaining infrastructure and software. It also enabled the VBI team to remove Amazon Simple Queue Service (Amazon SQS), a fully managed message-queuing service, and Amazon Simple Storage Service (Amazon S3), an object storage service, from its ML pipeline, which will save AFT 40 percent per month on AWS costs. Amazon SageMaker has scaled to handle demand spikes automatically, whereas the software development engineer team previously would have had to order more hardware, delaying implementation until its delivery.
Amazon SageMaker also reduced the load on the research wing of the VBI team, which can now launch models in pilot mode and roll out a new model in about 2 weeks, with no overhead needed to handle volume increases. And because Amazon SageMaker is ML-framework agnostic, the VBI team can layer whichever framework it wants on top of it. Converting from Caffe to Apache MXNet a few years ago took the VBI team 6–8 months, but Nayak anticipates a future conversion will be faster and simpler on Amazon SageMaker: “We want to be able to seamlessly transition to whatever technology is working best without investing a lot of time in integrating that technology into our framework. Amazon SageMaker does that for us.”
Delivering on the Amazon Customer Promise
Using Amazon SageMaker, AFT’s VBI team reduced cost and the overhead on software developers while maximizing the efficiency of its solution. The VBI team expects that by the end of 2020, the Amazon SageMaker model will be used in all of its FCs and processing millions images in each center. With the ability to efficiently monitor its FCs to quickly find misplaced inventory, AFT can better fulfill Amazon’s promise that customers will receive their packages on time.
To learn more, visit https://aws.amazon.com/sagemaker/.
Amazon is guided by four principles: customer obsession rather than competitor focus, passion for invention, commitment to operational excellence, and long-term thinking. It has pioneered customer reviews, 1-Click ordering, personalized recommendations, Prime, Fulfillment by Amazon, AWS, Kindle Direct Publishing, Kindle, Fire tablets, Fire TV, Amazon Echo, and Amazon Alexa.
Benefits of AWS
- Developed a new solution in 12 weeks
- Launches new models in 2 weeks compared to 3–6 months
- Processes millions of images daily
- Projects savings of 40% per month on AWS spend by removing unnecessary infrastructure services
- Cut time-to-predict latency by 50% using GPUs instead of CPUs
- Released 1,000 Amazon EC2 CPU instances in exchange for one endpoint on Amazon SageMaker
- Saves the software development engineer team 1 month per year in system maintenance
AWS Services Used
Amazon SageMaker helps data scientists and developers to prepare, build, train, and deploy high-quality machine learning (ML) models quickly by bringing together a broad set of capabilities purpose-built for ML.
Amazon EC2 C4 Instances
Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides secure, resizable compute capacity in the cloud. It is designed to make web-scale cloud computing easier for developers.
Companies of all sizes across all industries are transforming their businesses every day using AWS. Contact our experts and start your own AWS Cloud journey today.