AWS Step Functions Customer Testimonials

More than 7 billion people depend on farms for healthy and affordable food, and Granular’s mission is to continually innovate new ways to help farmers use technology to run stronger businesses. To deliver on this demand, Granular provides software and analytics tools that help make farming more profitable, efficient, and sustainable. Their platform includes a farm management software suite, a farmland real estate analysis tool, and precision agronomy software for ag retailers, and supports growers farming many millions of acres of crops annually across the United States, Canada and Australia.

Granular has had a long history of expertise in processing high density geospatial sensor data from agriculture machinery. Looking to improve customer experience and retention, Granular wanted to develop a high performance, high speed, scalable technique for running translations of agricultural machine data so that their other products could consume that data as soon as possible. In the past, Granular used home grown Windows solutions for managing the workflow and processing of these massive data sets, but they now wanted to write additional components using Kubernetes and AWS Lambda for easier scaling and management. Realizing that writing their own workflow automation tools would take focus away from business problems, they looked to AWS Step Functions to provide a serverless workflow solution.

Using Step Functions, Granular simplified their core processing framework with an easy to understand, self-documenting workflow and improved the scaling and management of their existing application. “We were able to implement a scalable approach to running code through multiple technology stacks, and can now manage the state of our workflows across a mixed platform environment using Linux, Windows, AWS Lambda, and Amazon EC2,” says Zac Oler, Principal Software Engineer at Granular.

Xylem is a leading global water technology company committed to developing innovative technology solutions to the world’s water challenges. To perform analytics, Xylem was using a Hadoop-based, on-premises data lake to store massive amounts of electric, water, and gas metering data. However, this solution limited their data analysis capabilities and was not flexible enough to allow for the addition of new projects. Data processing was taking four to five hours each day, and if they added new customers, up to several days.

By moving the Xylem Data Lake to an AWS-based platform leveraging AWS Batch, AWS Step Functions, and AWS Lambda, the Xylem data team was able to dramatically expand their data analytics capabilities. Xylem feeds each input file to a Step Functions state machine, which then validates the file, tracks metadata and finally, launches and monitors an AWS Batch job that processes the data. Each state machine runs independently and in parallel. The AWS Batch jobs allocate containers automatically, and Xylem can now parse thousands of files simultaneously, without having to worry about jobs competing for resources. As a result, Xylem has reduced data processing time from 20 hours to only two hours – regardless of the size of the dataset.

Coinbase is a secure digital currency exchange platform that makes it easy to buy, sell, and store cryptocurrency like Bitcoin, Ethereum, and Litecoin. As of 2017, Coinbase is the largest bitcoin broker in the world and serves users in 33 countries. Coinbase has a high performing engineering organization. They deploy hundreds of times per day across hundreds of projects, release new features from idea to launch in under an hour, and sustain low failure rates. This is possible because most of their change management and deployment processes are automated and their engineers have adopted a DevOps culture.

A key part of that automation is their secure, automated deployment pipeline, Odin. Odin takes a description of a project release and then safely and securely launches it into AWS. Coinbase automated Odin using AWS Step Functions, allowing them to deploy 12-factor-applications to AWS easily, reliably and securely. Step Functions provided the tools for reliability and end-to-end visibility out of the box, with a serverless operating model that doesn’t require any infrastructure to be provisioned or managed. Designing Odin as a Step Functions state machine lets Coinbase visually monitor the progress of the deployment, and audit each deployment step by step. With new insight into the reasons why services were failing, Coinbase was able to improve their rate of successful mission critical deployments from 90% to 97%. The visibility provided to engineers allowed them to diagnose and resolve their own issues quickly, and reduced the number of support tickets related to unsuccessful deployments.

Read more of Graham's blogs or view the Coinbase Odin deployer to AWS for 12 Factor applications on GitHub.  

Cox Automotive Inc. makes buying, selling, owning and using cars easier. To support digital advertising, they wanted to increase the volume and quality of their machine learning prediction models. However, they were concerned that their models might drift over time, leading to poor decisions. Injecting Data Scientist oversight into the model retraining process was the best solution, but the challenge was to to do so without slowing down the Decision Science team.

Cox Automotive decided to automate their ML model delivery workflow with AWS, including a model approval pipeline coordinated by AWS Step Functions. Once models are retrained using Amazon SageMaker, the output artifacts and diagnostics are uploaded to Amazon S3, which triggers Step Functions to run a model approval workflow. An email is sent to a Data Scientist, who reviews the model and clicks either “Approve” or “Reject”. If approved, Step Functions sends the artifacts to a destination S3 bucket along with an Amazon SNS notification indicating that the new model is ready for consumption.

Investing in automation has given both teams time to work on higher value activities. Cox Automotive now has a reusable, automated process that allows them to spend more time making more accurate models, and less time building deployment pipelines.