Understanding the cost of production in agriculture: How Perennia leverages AWS IoT

What does it truly cost to run an orchard, a vineyard, a farm? How do you take into account all of the inputs from planting, fertilization, irrigation, pest management, harvesting costs, regulatory assessment fees, labor costs, equipment costs, and overhead costs compared to yields and returns? It is an enormous amount of information to keep track of and often involves multiple tracking methods and end-of-season calculations. Perennia Food and Agriculture is a Nova Scotia-based provincial development agency with the mission to support growth transformation and economic development in Nova Scotia’s agriculture, seafood, food and beverage sectors. Perennia recognized the challenge that the agriculture industry is facing when it comes to having a clear understanding of their cost of production, and they decided to do something about it.

“We understand that each farm is different. Greenhouses are different from an apple farm, which is different from a dairy farm” said Grant MacDonald, Executive Director of Software Development at Perennia. “After a long week, working 7 days a week, the last thing a farmer really wants to do is manually enter data entries into an application. We wanted to help collect data easily and simply give them all of the data in the cloud so that they can know the cost of running their business.”

How did Perennia get started with AWS and the “Cost of Production” build?

The Cost of Production project was a new concept for Perennia, inclusive of building a team to make the application. Hiring talented builders to grow their expertise with AWS and their knowledge of agriculture with Perennia was the solution. The team knew they wanted to pull data from the edge and store data in the cloud, and selected AWS as their strategic cloud provider. Perennia quickly embarked on learning about AWS services, including the services they could use to address their Internet of Things (IoT) needs, and worked hand-in-hand with the AWS RnD prototyping team on developing a Cost of Production application prototype.

What problem does the Cost of Production application solve for agriculture businesses?

The Cost of Production acts as an aggregation point for data that is ingested from a variety of edge devices including power sensors, fuel sensors, GPS tracking, and water monitoring. At a high level, the application ingests data from a number of data feeds. This data is then combined with accounting data to compare the usage to the actual spend. All the information coming into the application is being connected through AWS IoT Core, as well as other AWS services. The user gets access to a dashboard with a responsive web design where they can view all of their data in one place. The comprehensive web design of the application enables an integrated view of the operation with the ability to access their data on any device, such as a computer, smartphone, or tablet. The user can review the data, analyze reports, and make intelligent business decisions based on the information displayed. Alerts can be set to notify the users of pending or active issues such as low fuel alerts, error codes on the tractor, device power usage being higher than normal, etc. This enables the employee, or the farm manager to take actions and make decisions in or near real time. It also enables the operation to have a clear understanding of their costs in season, and what they’re spending in each area of their business. All of this information enables the user to find ways to increase efficiency, reduce their carbon footprint, and optimize their business operations overall.

What is AWS IoT Core?

AWS IoT Core (https://aws.amazon.com/iot-core/) is a managed cloud service that lets connected devices easily and securely interact with cloud applications and other devices. AWS IoT Core can support billions of devices and trillions of messages, and can process and route those messages to AWS endpoints and to other devices reliably and securely. With AWS IoT Core, your applications can keep track of and communicate with all your devices, all the time, even when they aren’t connected.

AWS IoT Core also makes it easy to use AWS and Amazon services like AWS Lambda, Amazon Kinesis, Amazon S3, Amazon SageMaker, Amazon DynamoDB, Amazon CloudWatch, AWS CloudTrail, Amazon QuickSight and Alexa Voice Service to build IoT applications that gather, process, analyze, and act on data generated by connected devices, without having to manage any infrastructure.

Cost of Production reference architecture

The reference architecture for the Cost of Production can be broken down into three main components: Ingestion, Transformation, and Data Display.

Ingestion

To understand the cost of production, the producers must capture data from devices that monitor critical information such as fuel levels or temperatures. For this application, Perennia and AWS knew that the application would be ingesting data from IoT devices that were deployed in areas of low and limited connectivity. For this reason, LoRaWAN devices were selected for the reference architecture (LoRaWAN is a low power, wide area networking protocol designed to wirelessly connect battery operated “things” to the internet through regional, national, or global networks). These devices connect to localized gateways that are out in the field. These devices securely send messages to a network server running in the cloud, which then passes the message to AWS IoT Core. The messages being sent to AWS IoT Core are fully encrypted, and the network server is authenticated using X.509 certifications. Once the messages are in AWS IoT Core, the final part of this ingestion process pushes these messages into Amazon Kinesis Firehose.

Transformation

To move data from the edge to end user application readiness, a combination of services were used including Amazon Kinesis Firehose. Amazon Kinesis Firehose is the easiest way to reliably load streaming data into data lakes, data stores, and analytics tools. Amazon Kinesis Firehose enables sub-second low latency applications but in understanding the farming application the edge devices send data at a frequency of 5 minute intervals. Amazon Kinesis Firehose allows for messages to be batched together before writing to Amazon S3. This architectural pattern is part of a well-architected approach to a cost effective application by reducing the number of puts to Amazon S3 thereby reducing the overall cost of the build. Another advantage of Amazon Kinesis Firehose is that in combination with AWS Lambda, it enabled transformation on received messages. This pattern enabled both distillation of critical events and data enrichment, curating only the most important information for the end user applications.

Data Display

The final component of the architecture is the end-user interaction. To maximize flexibility, Amazon OpenSearch Service (September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service) was selected allowing for rapid aggregations of the costing information in a variety of different ways and configurations to provide the greatest flexibility for end-user interactions. For example, an application user  should be able to narrow down total fuel costs from a custom date range (e.g., Aug 9^th to Sept 12th) or review the total energy cost of a specified building over the last month broken into 1 hour intervals. Amazon OpenSearch Service enables rapid user driven interactions and data aggregations coupled with easy to understand visualizations. Amazon OpenSearch Service was easy for Perennia to stand up from the AWS Management Console, enabling the cluster to be stood up in a matter of minutes and to be run as a fully managed service. The end user application interacts with the Amazon API Gateway which interacts with Amazon OpenSearch Service to display the data into the Cost of Production application.

Note: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service.

How long did the build take?

The process of creating the prototype, from setting the requirements to the actual build of the prototype, took just under six weeks to accomplish. Additionally, the timeline for the prototype moving to a production-ready application will equate to less than six months.

What is next for Perennia’s Cost of Production team?

In less than a year, Perennia has gone from building a team and no AWS usage and knowledge to becoming AWS super users. Not only did they leverage AWS services for their Cost of Production build, but they are building new applications leveraging some of the same AWS services. Perennia expressed that having the AWS infrastructure in place for the Cost of Production application made it easy to expand that usage into other builds. Additionally, the serverless computing and pay-as-you-go pricing made it easy for them to build a high performing, cost efficient application.