SafetyVisor: Protecting Against COVID-19 with Computer Vision and AWS

By Nicholas Burden, Senior Technical Evangelist at TensorIoT

As COVID-19 has swept across the globe, it has radically challenged businesses in every industry. TensorIoT has responded to these events by innovating to provide companies with powerful tools to drive success under extreme circumstances.

To help safeguard workplaces from the pandemic, TensorIoT developed SafetyVisor, a suite of machine learning (ML) tools that can operate independently or in tandem with existing business infrastructure to monitor safety gear usage (like masks) and social distancing.

SafetyVisor’s computer vision models are designed to work with your existing cameras, and the entire solution is built utilizing a flexible architecture to facilitate easy deployment and use.

In this post, we will delve into using computer vision for a previously unexplored usage. We’ll also explain how TensorIoT developed SafetyVisor on Amazon Web Services (AWS) and explore all of the different ways that using SafetyVisor protects employees and customers.

TensorIoT is an AWS Advanced Consulting Partner with AWS Competencies in Machine Learning, Internet of Things (IoT), Industrial Software, Retail, and Travel and Hospitality. TensorIoT also has service delivery validations for AWS IoT Core and AWS IoT Greengrass.

The AWS Foundation

TensorIoT’s SafetyVisor was built on AWS using a foundation of AWS IoT Greengrass, AWS IoT Core, AWS Lambda, and Amazon DynamoDB.

These elements help SafetyVisor pull data from users’ interconnected cameras to check for adherence to user-selected safety protocols, while AWS IoT Events and Amazon Simple Notification Service (SNS) enable dynamic alerting.

The underlying models themselves were trained on Amazon SageMaker while Amazon Cognito, Amazon API Gateway, and Amazon Simple Storage Service (Amazon S3) bring the secure dashboards to life.

Figure 1 – SafetyVisor is built on AWS.

Face Mask Detection

SafetyVisor grew as the logical extension of existing TensorIoT projects to help businesses enable safer working environments.

Early models were trained to detect conventional job-site Personal Protective Equipment (PPE) such as hardhats, safety goggles, and high-vis attire. The original solution enabled the enforcement of specific PPE policies on a camera-by-camera basis and alerted supervisors when violations occurred.

To address emerging pandemic concerns, the TensorIoT model was retrained to recognize the newest piece of critical safety gear—the face mask.

With the expanded concept of PPE, SafetyVisor can automatically generate alerts whenever a worker without a face mask is detected. The alert includes a secure link to view an image of the non-compliant employee to enable any appropriate intervention or follow-up discussion.

Social Distancing Compliance

Social distancing is the most powerful and widely employed tool against the spread of COVID-19. By maintaining increased distance in social settings, the risk of infection decreases, contributing to safer working environments for employees.

However, social distancing measures are only effective with consistent implementation, making compliance monitoring essential for protecting worker health and lowering potential employer liability.

While existing camera systems enable operators to screen for violations, monitoring large numbers of cameras and employees in real-time presents a challenge, one perfectly suited for computer vision.

By employing conventional object detection, examining the distance between each person’s feet, and scaling the measurement to account for depth and scene geometry, SafetyVisor can determine relative spacing in real-time.

Alerts are automatically generated based on a customizable distance threshold and include an image record of the policy violation with specific offenders clearly highlighted.

Occupancy Tracking

In accordance with social distancing protocols, many businesses need to follow lower occupancy limits in public areas and enforce such limits more stringently than ever before.

By employing computer vision, all human movement across a business’s threshold can be carefully monitored, maintaining an overall count of total occupancy. This enables real-time occupancy tracking to help businesses better monitor and prevent health risks and code violations.

Guided Cleaning

Another business challenge posed by COVID-19 has been the sanitary upkeep of high-traffic spaces. Lobbies, cafeterias, and other large public spaces require additional cleaning efforts to safeguard the health of employees and customers alike.

While repeatedly deep cleaning the entire space is tempting, businesses must account for labor, materials, and the disruption caused by the cleaning activities themselves.

SafetyVisor leverages computer vision to monitor human traffic over time, meaning that high-traffic areas can be visually identified, mapped, and monitored. Whenever a customizable threshold is reached, a cleaning alert can be generated for an area.

This allows for data-driven, targeted cleaning efforts that conserves time and resources without compromising the health and safety of people using the facilities.

Thermal Screening

One of the most direct tools in the fight against COVID-19 is temperature screening, which can help prevent employees from virus exposure by an asymptomatic individual.

Since a high fever is the most common symptom, screening individuals for the increased temperature caused by fever provides an effective line of defense. However, measuring individual temperatures at scale is a serious challenge for large companies.

As an add-on that leverages specialized cameras which offer highly sensitive infrared measuring capabilities, SafetyVisor can passively measure temperature in seconds with an accuracy within 0.2 degrees Fahrenheit. These solutions can be deployed at entrances, reception desks, time clocks, or anywhere fever screening offers improved protection.

Compliance Framework

Beyond the immediate use cases the SafetyVisor solution addresses, the central problem behind many COVID-19 response efforts is one of compliance. With myriad policies impacting how businesses must conduct themselves and manage their customers, the ability to document their efforts and their successes is a challenge on its own.

By meticulously logging the activity of each model, compliance logs are generated automatically and provide documented evidence of the effectiveness of the measures employed.

With thermal screening in particular, facial recognition can be incorporated to connect each screening back to an established identity. This generates a screened attendance log that can be compared with work schedules to ensure full employee participation.

Putting it All to Work

The heart of the SafetyVisor system is the collection of machine learning models, designed to run on the edge. As part of the solution, special edge hardware will be deployed at each location to interface with local camera systems.

Additional configuration work may be required to make video available from certain Network Video Recorders (NVRs) or Video Management Systems (VMS) platforms. Most systems—as well as simple local network cameras with accessible real-time streaming protocol (RTSP) feeds—can be monitored with minimal setup.

Once deployed, the solution requires almost no maintenance, allowing businesses to focus on their business while computer vision safeguards their operation.

Conclusion

SafetyVisor was initially conceived and programmed as a method of using computer vision to monitor the use of personal protective equipment (PPE) in industrial worksites to ensure worker compliance with OSHA regulations.

The flexibility of building on SafetyVisor on AWS enabled TensorIoT to perform a rapid pivot and adapt their work on detecting PPE to detect mask usage and social distancing.

This pandemic has changed the business landscape, creating a need for an adaptable set of tools to protect the safety of workers and customers alike. TensorIoT makes things intelligent, and the creation of the smart computer vision SafetyVisor provides critical solutions to help businesses protect employee and customer health during a global crisis.

Thanks to AWS and TensorIoT, this health protection service has already been deployed to help restaurants and other businesses protect human health while keeping their businesses growing.

The content and opinions in this blog are those of the third-party author and AWS is not responsible for the content or accuracy of this post.

.

.

TensorIoT – AWS Partner Spotlight

TensorIoT is an AWS IoT Competency Partner that was founded on the instinct that the majority of compute is moving to the edge and all things are becoming smarter.

Contact TensorIoT | Partner Overview

*Already worked with TensorIoT? Rate the Partner

*To review an AWS Partner, you must be a customer that has worked with them directly on a project.