AGCO Lowers Costs, Boosts Speed, and Increases Retention Using Amazon Kinesis Services
AGCO Corporation (AGCO), a global provider of agricultural solutions, manufactures industrial farming equipment and cutting-edge digital products designed to revolutionize farms. Its goal of data-driven precision farming solutions means enabling farmers to manage sustainable, productive grain and livestock farms and to maximize yield and profitability. AGCO required modern technology to enable the scale of data tracking and storage necessary for smart farming. For that, it turned to Amazon Web Services (AWS).
The company began its AWS journey with a 2015 migration, and since 2019 it has been focusing on modernizing with microservices, automation, and serverless architecture. AGCO’s former third-party black-box data system limited how the company could use its data. Crucially, it didn’t have enough data stream retention time to account for the long growth and investment cycles in farming. By replacing that system with an orchestration of AWS services, including three Amazon Kinesis services, the company could get timely insights and react quickly to new information. AGCO dramatically increased data retention time and data-retrieval speed, and it did so while reducing costs, which enables its digital-first strategy and serves as a cornerstone for future capabilities.
Amazon Kinesis services provide the building blocks we need at a lower cost. We get a 1:3 or 1:4 return because they cover 60% or more of what we were paying before."
Product Development Manager, AGCO
Replacing Its Legacy System with a Cost-Efficient Architecture
Farmers are increasingly using digital technologies to run their farms more sustainably, and AGCO is building products to help farmers manage their operations more efficiently and profitably. “We build farms where edge computing runs the show: the ventilation, the lighting, the heating. They’re all hooked into the building controller and feeding the cloud,” explains Paul Thornhill, product development manager at AGCO. “The cloud collects and analyzes the data, then sends the results back to the building, which adjusts the ventilation, lighting, and heating accordingly. It reports everything to the farmer.” By automating that process, farmers can not only save time but also make sure that temperature, humidity, and other parameters are always set at the optimal levels to encourage growth without wasting money on electricity.
On its legacy system, AGCO couldn’t enable farmers to compare growth cycles. It had a retention limit of 750 million records, which was only sufficient for a period of about 6 months—barely enough for even one 185-day cycle of bringing a pig from birth to market. Because of the 180,000 records each customer stores per day and the challenge of data retention, AGCO was actually spending more money than it was earning to onboard new customers.
AGCO needed technology that would collect all data from its smart farming equipment and then retain and analyze the data to provide insights and real-time machine diagnostics. Most options to accomplish that goal are expensive and time consuming, but Amazon Kinesis services can do the job by offering managed services and automatic scaling, seamlessly operating in AGCO’s existing AWS infrastructure.
Reducing Cost and Improving Speed
In January 2020, AGCO went live with a serverless data pipeline built using three Amazon Kinesis services. Ingesting and storing data from the AGCO farming equipment is Amazon Kinesis Data Streams (Amazon KDS), a scalable and durable real-time data streaming service that continuously captures gigabytes of data per second from hundreds of thousands of sources. Amazon KDS can then feed equipment data into Amazon Simple Storage Service (Amazon S3), an object storage service. Also capturing equipment data is Amazon Data Firehose, a simple way to reliably load streaming data into analytics services, data lakes, and data stores, which AGCO shapes for particular customer use cases. A downstream Amazon Data Firehose makes data available for use with other AGCO products. Using these tools, AGCO’s system moves 1,200 data points per minute and, when tested, could automatically scale to 10,000 data points per minute.
Once the data is ingested, it can either be stored or go directly to Amazon Managed Service for Apache Flink, which transforms and analyzes streaming data in real time using Apache Flink, an open-source analytics framework. Using this service results in valuable insights for farmers. “We want to give farmers data-driven knowledge about how the farm runs,” says Thornhill. “Then we want to give them insights to improve the production process and more real-time diagnostic feedback about each machine and how to adjust it to match best practices and benchmarks.”
Using Amazon Kinesis services, AGCO retains 1.5 billion records at a lower cost, resulting in 78 percent cost savings. It can consistently load that data to a screen in 600 ms—a process which previously took 8–30 seconds. AGCO also gained more data storage at a drastically reduced cost by using Amazon S3.
Another cost saver for AGCO was the ability to end its third-party contracts. “Even though there’s now more labor on our side to process the data, it’s a better ratio than when we were paying someone else to do it,” Thornhill says. “Amazon Kinesis services provide the building blocks we need at a lower cost. We get a 1:3 or 1:4 return because they cover 60 percent or more of what we were paying before.”
AGCO layers Amazon Kinesis services on top of the existing infrastructure to reduce its complexity and manage the underlying service. As a result, AGCO only needs one person to manage the infrastructure rather than three to five. “Even with the amount of data that we’re currently moving and the complexity of its infrastructure, the pipeline has run nearly hands-free all year,” says Thornhill.
Increasing Productivity and Profitability for Farmers
AGCO plans to use AWS to build true edge communication through an Internet of Things core instead of a hand-built gateway. Using machine learning to analyze weather data, for example, would enable farms to adjust temperature and humidity in a barn minutes before a weather change rather than the usual 2 hours after. With the potential to adjust an environment without involving the farmer, this system would lead to increased production.
To further improve animal welfare, AGCO plans to develop a solution for real-time monitoring of sows. This will enable farmers to make quick, data-based adjustments to the sows’ environments to improve their well-being and increase productivity and profitability.
Using AWS, AGCO improved service for its agricultural customers by lowering costs and increasing data retention and retrieval speed. “Data collected and delivered to our farmers using AWS tools is driving better decisions and animal health,” Thornhill says. AGCO plans to continue using AWS to adapt to the changing world of agriculture.
About AGCO Corporation
Headquartered in the United States and operating worldwide, AGCO Corporation is an innovator of agricultural solutions. Its goal is to create sustainable, productive grain and livestock farms through data.
Benefits of AWS
- Decreased costs by 78%
- Increased data retention to 1.5 billion records and growing
- Reduced data extraction time from 8–30 seconds to 600 milliseconds
- Manages data pipeline with one person rather than 3-5 people
AWS Services Used
Amazon Kinesis Data Streams (Amazon KDS) is a massively scalable and durable real-time data streaming service.
Amazon Data Firehose
Amazon Data Firehose is the easiest way to reliably load streaming data into data lakes, data stores, and analytics services.
Amazon Managed Service for Apache Flink
Set up and integrate data sources and destinations with minimal code, continuously process data with subsecond latencies, and respond to events in real time.
Amazon Simple Storage Service (Amazon S3)
Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance.
Organizations of all sizes across all industries are transforming their businesses and delivering on their missions every day using AWS. Contact our experts and start your own AWS journey today.