Unleash Live Employs AWS and Nvidia GPU Integration for Scalable, Live-streaming AI Object Recognition

What do emergency responders, critical infrastructure asset managers, and miners have in common? All are people who must make decisions quickly in complex situations—often from remote and dispersed locations.

And they are not alone. For people that work in insurance, mining, infrastructure, construction, and other sectors where employees are often disconnected and isolated from their co-workers, many must often make decisions locally without the collaboration of a head office or other key stakeholder. This decentralized decision-making process can strain a large company’s ability to maintain effective oversight, leading to decisions that don’t align with the company’s overall strategy and effectively hurt the bottom line.

Unleash Live CEO Hanno Blankenstein started his company in 2015 to eliminate this disconnect. By applying AI analytics to live streaming, Unleash Live is able to speed up the decision making process between the field and the office and help both field operatives and office workers make better choices together.

How it works

“The introduction of drones, IP cameras, and other IoT devices has led to a higher degree of visibility of operational activities across dispersed assets,” says Blankenstein. “However, timely access to this data and the extraction of insights continues to haunt companies and their management.” “Enterprises lose millions of dollars through stoppages, security breaches, ineffective communication, and data islands,” he adds.

For large companies in sectors such as insurance, mining, infrastructure, and construction—where locations can be remote and dispersed across a country or continents—its employees’ disconnect and isolation often force them to make decisions locally without consulting a head office or other key stakeholders.

This decentralized decision-making process can strain a large company’s ability to maintain effective oversight, leading to decisions that don’t align with the company’s strategy and effectively hurt its bottom line.

Unleash Live applies a high-resolution AI live streaming to improve business operations. “We believe video is the most powerful digital sensor,” says Blankenstein. “Combined with the technology of high-performance drones, connected IP cameras and real-time satellite imagery, companies with expensive facilities or dispersed assets now have a powerful tool for fast high-quality data assessment.”

Further, Unleash Live’s live stream allows users to remotely monitor work sites, building sites, and fixed assets. AI algorithms then enhance the analysis of this live stream and make all captured video and imagery available for review in the secure media library. 3D modeling, point cloud models, clipping and sharing of media, and exporting data into engineering tools provide additional analysis.

The engineering challenge

Using AI in a live stream is an extremely memory hungry and processing intense and at an enterprise level, stream encryption, authentication, and routing are challenging to say the least. Many-to-one-to-many video streaming from drones, robots, and IP cameras only come back to one account and submitting streams out to authorized audiences is a daunting task.

At the moment, ingesting, processing, compute, and presentation AI-enhanced video in the lowest latency possible in HD to anywhere in the world has never been done before. User media library management with video trimming initiates cloud operations and viewing results and achieving compatibility across any modern mobile, tablet, and laptop device to achieve a seamless user experience in the field has been virtually impossible to achieve.

The Solution: AWS batch scaling of memory and GPU intensive AI jobs, 0 to infinite scale

Unleash Live’s engineers worked with AWS to reduce latency and increase the scalability of the video streaming pipeline. They now apply AI models to their live stream pipeline through RTMP streams connecting to ELB, while autoscaling groups provide resiliency achieved by horizontal scaling of streaming instances. The instances types match the non-functional requirements of each streaming service: T2 and C5 instances for RAW streaming and P2 or G3 instances for AI streaming.

RAW and AI streaming instances are operating communication channels in an isolated VPC while each instance writes video chunks to a shared EFS volume. For low-latency distribution of video chunks, a browser’s stream player connects to Unleash Live’s global CDN which is fetching video chunks from EFS volume mounted to a dedicated origin T2 instance. Adaptive streaming using Dash and HLS ensures availability across any browser and player. To top it off, Unleash Live writes all HD full videos and stereo audio to the user’s secure S3 account, making it fully searchable by tags created with Amazon ReKognition and archived into Amazon Glacier over time.

The Unleash Live future

Unleash Live is aiming to forever change secure AI media streaming and open up new opportunities for AI video innovation in many industrial and entertainment sectors. It’s also working hard to expand this capability into real-time and predictive analytics, real-time responses back to the IoT devices, and more immersive VR and AR experiences.

“It’s time to bring on the future of AI,” says Blankenstein. “The future of collaborative digital services is here now.”