How Deloitte is Improving Animal Welfare with AI at the Edge Using AWS Panorama

By Pavel Bugneac and Willem van Hove, Consultants – Deloitte
By Vincent Bolwerk, Manager – Deloitte
By Mohsen Damshenas, Partner Solutions Architect – AWS
By Gordon Heinrich, Partner Solutions Architect – AWS

Deloitte

Together with its partners, Deloitte Netherlands works to avoid preventable animal suffering that may take place in slaughterhouses by significantly improving the effectiveness of camera surveillance.

This is accomplished by taking advantage of computer vision at the edge using AWS Panorama, a machine learning (ML) appliance and software developer kit (SDK) that allows organizations to bring computer vision to their on-premises cameras.

By doing so, Deloitte and its partners are tackling the increasing need to monitor how animals are being handled in slaughterhouses, achieving this with the lowest possible latency.

Although camera surveillance (CCTV) increases the possibility to detect welfare deviations in slaughterhouses, the thousands of hours of resulting video footage pose significant challenges. In most cases, only a small, random fraction of the video footage can be reviewed and, therefore, most video footage remains unseen.

Deloitte’s AI4Animals solution processes camera footage to detect potential breaches of animal welfare protocol in real time, such as human/animal interaction, potentially injured animals and signs of life. Based on this, notifications are sent to users and presented through a dashboard to be reviewed and escalated if required. This enables slaughterhouse employees to assess the video images, take corrective actions, and track the follow-up of those actions.

Deloitte is an AWS Premier Tier Services Partner and Managed Cloud Services Provider (MSP). Deloitte is one of the largest professional services firms in the world and a leader in digital transformation strategy.

Deloitte puts cognitive technology into real-world action, working with clients to create artificial intelligence-driven products that put business at the cutting edge.

Benefits of AI4Animals

The continuous interaction between humans and animals in slaughterhouses can lead to animal welfare deviations which can occur in different forms. The AI4Animals solution is capable of detecting these welfare deviations in order to improve the conditions of animals in slaughterhouses.

Deviations detected by AI4Animals include:

• Animal staying behind: The solution identifies one or more animals staying behind for an extended period of time while the rest of the group continues to move. This may indicate lameness, exhaustion, and/or injuries.
• Bottlenecks: This occurs when a large group of animals is stuck in the runway leading to the slaughterhouse, which can cause significant stress for the animals and lead to injury.
• Human movement: Employees walking directly in front of the animals can be another source of stress for the animals.
• Signs of life and consciousness: The solution detects signs of life or consciousness when the animals should not be alive.
• Mobile stunner usage: A mobile stunner is used to render the animals unconscious. The use of the mobile stunner and whether it has been applied according to protocol is monitored.

By detecting these deviations, the AI4Animals solution enables slaughterhouses to considerably improve animal welfare. Unnecessary sources of stress caused by incorrect animal handling are identified and corrective measures are tracked in the application.

Similarly, injured animals are quickly detected and can be assisted, thus reducing the amount of suffering.

Figure 1 shows how the AI4Animals algorithm detects animals, people, and objects and how they interact. Their movements can be visualized, as can be seen on the right where lighter dots indicate detection further back in time. Movements are analyzed to determine potential issues such as a pig staying behind.

Figure 1 – AI4Animals detecting and visualizing pigs movements.

While the end result of the solution is improving animal welfare in slaughterhouses, there are several challenges and obstacles which can potentially hinder a successful integration of the solution in slaughterhouse processes.

Towards Computer Vision on the Edge

A successful implementation of any artificial intelligence (AI) framework into production comes with a myriad of potentially crippling challenges.

One of the critical aspects of animal welfare is swift intervention, which raises the necessity for real-time notifications. These are necessary to provide prompt feedback to the employees and establish a more animal-friendly environment.

Using cloud computing for real-time analysis doesn’t always work in practice. Many slaughterhouses are in rural areas where a stable and high bandwidth network connection is not available.

Using AWS Panorama enables AI4Animals to bring computer vision to on-premises cameras to make predictions locally. The Panorama device can ingest multiple video streams and make real-time predictions by running computer vision models on the Panorama appliance’s powerful NVIDIA GPU.

The on-premises availability of computational resources eliminates time lag caused by sending data to the cloud and back, reducing overall latency of the solution. The solution allows real-time insights and immediately sends out notifications when specific animal welfare deviations are found.

Another important aspect of handling animal welfare video data is privacy and security. This is crucial as the data involved is highly sensitive and personal data has to be handled responsibly.

Using AWS Panorama, raw and potentially sensitive data no longer needs to be sent to the cloud for analysis. This enables slaughterhouses to eliminate the data locality challenges, thereby improving the data security and making it easier to adhere to privacy regulations.

The raw videos may contain recognizable images of the employees, license plates, and other sensitive data. With the data now being analyzed on site, only anonymized insights of edge analysis or small anonymized clips are shared over the internet.

Additionally, the edge processing solution leads to a significant reduction of costs of data transfer and computational resources by reducing the data sent to the cloud and by performing the computations on AWS Panorama.

This enables AI4Animals to detect potential animal welfare deviations in real time at low cost. The solution provides computational power close to the data while seamlessly integrating with all AWS services in the cloud.

AI4Animals Solution Overview

The AI4Animals solution architecture can be split into three main blocks: edge computing, web application, and model training.

For a seamless integration, these blocks are powered by specialized AWS resources which can easily interact with the Panorama devices to ensure compatibility, as shown in Figure 2. Below, we will dive deeper into how each block functions.

Figure 2 – AI4Animals architecture diagram.

Edge Computing

This block consists of the on-premises cameras and the AWS Panorama device. The different camera streams are connected to the Panorama device, which runs several AI models and performs predictions in real time.

The output of these models are sent to the Amazon Simple Storage Service (Amazon S3) bucket for selected video fragments, and to the Amazon DynamoDB database as deviation metadata store.

Moreover, to facilitate real-time intervention the Panorama devices use Amazon Simple Notification Service (Amazon SNS) which sends notifications to users in case of critical deviations.

The model inference results are stored in a DynamoDB database, and this NoSQL database offers the agility and speed best suited for this type of data.

Finally, deviations are written to the Amazon Aurora database to be presented in the dashboard. Aurora is a fast relational database that automatically scales with demand.

The associated videos containing the animal welfare deviations are stored in an Amazon S3 bucket which allows easy streaming to the web application for review.

Web Application

In order to ensure proper corrective measures are undertaken, such as a change in process or retraining, the relevant stakeholders require an interface to analyze and follow up on deviations detected by AWS Panorama.

For this reason, a web application was developed which is hosted using AWS Fargate and connected to the different data stores. The dashboard streams the video from the S3 datastore and displays associated metadata which are stored in the Aurora database.

Users have an overview of daily deviations which they use to follow up on animal welfare deviations detected by the models. These follow-up actions can range from talking to an employee that did not follow protocol to retraining staff to improve specific processes.

The data is visualized using Amazon Athena and Amazon QuickSight to help the decision making and reporting.

Additionally, users can provide feedback on the quality of the models by reviewing the detected deviations. This way, it can be ensured the AI models are regularly refined and high accuracy is maintained.

Model Training

The backbone of the training setup is the Amazon SageMaker environment in which the computer vision models are maintained. This machine learning service allows for relabeling and training of deep neural network models.

New models are trained using powerful GPU training jobs, optimized for inference on the Panorama device with Amazon SageMaker Neo, and subsequently deployed after proper evaluation and testing.

Summary

Deloitte’s AI4Animals solution aims to minimize animal suffering in slaughterhouses by accurately detecting, consistently reviewing, and following up on any potential deviations which may induce suffering.

The technical implications of building such a solution are intricate. Overcoming a number of inherent practical challenges was made possible by using AWS Panorama, a state-of-the-art edge computing device. By combining Panorama with other AWS services, users are able to review animal welfare deviations and define follow-up actions, while also providing feedback that can be used to further refine AI models.

While AI4Animals focuses on animal welfare, the surrounding architecture of this solution can be applied to myriad other industries. If you are interested in finding out more about the AI4Animals solution, please reach out to Deloitte Netherlands, or read more on the AI4Animals solution page.

.

.

Deloitte – AWS Partner Spotlight

Deloitte is an AWS Premier Tier Consulting Partner and MSP. Through a network of professionals, industry specialists, and an ecosystem of alliances, they assist clients in turning complex business issues into opportunities for growth, helping organizations transform in the digital era.

Contact Deloitte | Partner Overview

*Already worked with Deloitte? Rate the Partner

*To review an AWS Partner, you must be a customer that has worked with them directly on a project.