AWS Physical AI Blog

Edge Impulse and AWS: Combining Edge Inference with Cloud Intelligence for Physical AI

Introduction

Edge intelligence is transforming manufacturing and logistics. Cameras and sensors now instrument every zone of a facility: loading docks, production lines, staging areas, aisles between racking. These devices produce continuous streams of operational data, yet most of it flows into storage without producing a timely, actionable answer. A warehouse operator looking for a specific forklift or staging container still has to radio a colleague or walk the floor. The data exists; the ability to query it in natural language, in real time, does not.

Edge Impulse, a Qualcomm company, and AWS address this gap with a hybrid edge-to-cloud architecture that combines on-device object detection with cloud-based natural language intelligence. The result is a conversational interface for asset tracking: warehouse operators can ask “Where is the forklift?” and receive immediate, accurate responses drawn from distributed edge devices across their facility. This blog explains the solution architecture and walks through its implementation.

The Challenge

Workers on factory floors and in warehouses frequently lose productive time searching for mobile assets like forklifts, staging containers, and portable tools. These delays compound across shifts, leading to missed production targets, underutilized equipment, and increased operational costs that grow proportionally with facility size.

Existing tracking solutions introduce their own friction. Radio Frequency Identification (RFID) tags are prone to spoofing and blind to untagged items. Global Positioning System (GPS) trackers do not work indoors. Bluetooth Low Energy (BLE) beaconing systems require dedicated infrastructure and ongoing calibration. Centralized camera networks demand high-bandwidth video streams, strain connectivity in large or segmented facilities, and create single points of failure when the network goes down.

The user experience of current systems creates an additional barrier. Accessing asset location data typically requires specialized dashboards, technical queries, or dedicated operators, none of which align with how frontline workers naturally communicate. The result is low adoption, delayed responses, and continued reliance on manual search or radio calls.

As facilities scale camera deployments, the fundamental challenge shifts from data capture to information extraction. Architectures that rely solely on cloud inference face growing pressure from latency requirements, bandwidth costs, and dependency on uninterrupted connectivity. For time-sensitive operations, intelligence needs to run closer to where the data is generated while still leveraging the cloud for orchestration and natural language understanding.

Edge Impulse on AWS

Edge Impulse and AWS have created a hybrid Physical AI solution that balances real-time edge inference with cloud orchestration, designed for industrial asset tracking and object detection use cases. This solution addresses the Edge Intelligence capability within the AWS Physical AI framework, combining local perception with cloud-based natural language understanding. By running inference directly on edge devices rather than routing raw sensor data to the cloud, the architecture minimizes latency, reduces bandwidth consumption, and ensures operational continuity even in connectivity-constrained environments.

Physical AI: Edge Intelligence

As organizations build automation and autonomous systems, edge intelligence is foundational. It provides the real-time inference and sensor processing that lets systems perceive, understand, reason about, and act in the physical world without round-trip cloud dependencies. For edge use cases, inference runs locally for speed and reliability; when deeper understanding is needed, the system reaches to the cloud for more capable models. Cloud connectivity remains essential for telemetry, orchestration, and model lifecycle management, but the critical inference workload stays local. The solution described in this blog exemplifies Edge Intelligence by:

  • Real-time Perception: Edge Impulse’s object detection models run directly on edge devices, delivering fast inference without cloud dependencies and eliminating the latency penalty associated with centralized processing.
  • Local Intelligence: Vision Language Models (VLMs) deployed at the edge provide contextual object descriptions and semantic understanding. In this solution, the edge VLM is Qwen2-VL 8B, an open-source model quantized to INT8 precision for efficient execution within edge hardware memory and compute constraints. Operators can access VLM outputs through direct device connection or through cloud-mediated queries.
  • Cloud Orchestration: Amazon Bedrock AgentCore, backed by Amazon Nova Lite on Amazon Bedrock, aggregates responses from distributed edge devices and provides a natural language query interface. Amazon Nova Lite was selected for its low latency and cost efficiency in orchestration tasks that require structured output generation rather than long-form reasoning. AgentCore was selected because the workflow requires long-running task orchestration across an undefined number of edge devices with variable response times, a pattern suited to a managed agent runtime rather than discrete AWS Lambda functions with AWS Step Functions. Workers can ask questions without understanding the underlying system topology.
  • Connectivity and Lifecycle Management: AWS IoT Core manages the device fleet, collects telemetry, delivers over-the-air model updates, and maintains reliable Message Queuing Telemetry Transport (MQTT)-based edge-to-cloud communication across the facility. AWS IoT Core’s point-to-point messaging capability supports targeted commands to individual devices, reducing messaging costs and improving delivery visibility.

Two-Stage Cascade Inference

At the core of the edge inference strategy is a two-stage cascade pattern designed to balance always-on efficiency with contextual understanding.

Figure 1: Two-Stage Cascade Inference

Stage 1: Initial Detection: A lightweight, application-specific model runs continuously on each edge device (camera, motion sensor, or microphone) at low power. It acts as a gatekeeper, detecting objects or events of interest and narrowing context so that Stage 2 is invoked only when relevant activity is present.

Stage 2: Contextual Analysis: When Stage 1 triggers a detection event, a secondary model activates to provide deeper semantic understanding. This could be a Vision Language Model (VLM), visual anomaly detection, or optical character recognition (OCR) model depending on the use case. In this solution, Stage 2 uses Qwen2-VL 8B, an open-source vision language model quantized to INT8 precision for efficient execution on edge hardware with constrained memory and compute budgets. Stage 2 can run on the same edge device, a local edge server, or in the cloud depending on computational requirements and latency constraints.

This cascade pattern keeps devices focused on continuous, power-efficient monitoring while reserving higher-compute resources for reasoning tasks only when needed. Running Stage 2 only on confirmed detections preserves battery life on portable devices, reduces thermal load to extend hardware lifespan, and minimizes compute costs across large-scale deployments with hundreds of always-on sensors.

Solution Architecture

This solution combines edge-based perception with cloud orchestration across a distributed fleet of camera devices. At the edge, each device runs a self-contained inference stack: an Edge Impulse YOLO Pro model for real-time object detection (Stage 1) and a local Qwen2-VL 8B model (INT8 quantized) for contextual labeling (Stage 2), all managed within an AWS IoT Greengrass runtime. In the cloud, Amazon Bedrock AgentCore, backed by Amazon Nova Lite on Amazon Bedrock, provides the natural language interface. AWS IoT Core manages device connectivity and fleet telemetry. Edge Impulse Studio supports continuous model improvement through retraining on production data: retraining compute runs on Edge Impulse’s cloud infrastructure (containerized training jobs), while AWS provides the data pipeline via Amazon Simple Storage Service (Amazon S3) for image storage and Amazon EventBridge rules to initiate retraining through the Edge Impulse API. The two layers communicate over MQTT via AWS IoT Core, with Amazon Simple Queue Service (Amazon SQS) and Amazon EventBridge providing event-driven orchestration between user queries and edge responses. Amazon CloudFront delivers the mobile web frontend over HTTPS to workers on the warehouse floor.

Figure 2: Solution Architecture – Edge to Cloud Data Flow

How It Works

  1. A factory worker opens a mobile web application served through Amazon CloudFront and submits a natural language query such as “Where is the forklift?”
  2. The message is enqueued in an Amazon SQS queue, which triggers a configured Amazon EventBridge rule.
  3. Amazon EventBridge invokes the Amazon Bedrock AgentCore agent, which is backed by Amazon Nova Lite on Amazon Bedrock. The agent interprets the natural language question and constructs a structured JSON object query payload describing the target asset.
  4. The object query payload is published over MQTT through AWS IoT Core and distributed to each edge camera device that overlooks the warehouse floor.
  5. On each device, the Edge Impulse Runner executes the YOLO Pro model, Edge Impulse’s proprietary object detection model trained on facility-specific asset classes (forklifts, containers, tools, personnel), to detect and classify objects matching the query payload and crop the relevant image region. The agent’s query scope is bounded by this trained class list; natural language queries referencing objects outside the model’s training set return a “not found” response rather than a hallucinated match. The cropped image is then passed to the local Qwen2-VL 8B model, which generates a natural language description and contextual label for the detected object.
  6. The camera device uploads matching images to Amazon S3 and publishes a response over MQTT through AWS IoT Core containing a natural language explanation of the object’s location along with the corresponding Amazon S3 image URI.
  7. The Amazon Bedrock AgentCore agent receives the edge response, synthesizes it into a human-readable answer, and enqueues the result in Amazon SQS. The worker’s mobile application updates with the location of the forklift and a supporting image. Beyond information retrieval, the AgentCore agent can trigger downstream actions such as creating a work order in the warehouse management system or dispatching a notification to the asset’s assigned operator, closing the perception-reasoning-action loop that defines Physical AI.
  8. Periodically, Edge Impulse Studio retrains the detection model using captured images and metadata from previous object queries stored in Amazon S3. Once data reaches Amazon S3, Edge Impulse’s automated data pipeline applies a bronze-silver-gold transformation process—running custom Docker containers on AWS compute resources—to sanitize the dataset by removing low-confidence captures, duplicate frames, and mislabeled samples. This produces an updated model reinforced with curated, real-world examples.
  9. Through Edge Impulse Studio, a human operator reviews the retrained model’s performance metrics and approves it for deployment. Once approved, the updated model is distributed to each camera device as an AWS IoT Greengrass component update.

This architecture uses native integration between AWS services and Edge Impulse’s platform. Edge Impulse models deploy directly to devices through AWS IoT Greengrass component updates. AWS IoT Core provides the bidirectional MQTT communication backbone between edge and cloud, while Amazon Bedrock AgentCore exposes standardized APIs that allow the natural language interface to be embedded into existing workforce applications, mobile apps, or operational dashboards without custom development.

Industry Applications

The two-stage cascade inference pattern extends well beyond warehouse asset tracking. The same approach, lightweight, always-on edge detection triggering deeper contextual analysis, applies across a range of industrial and commercial verticals:

Figure 3: Industry Applications of Two-Stage Cascade Model

  • Production Line Monitoring: An object detection model identifies areas of interest on the production line and triggers specialized defect detection models, running different classifiers depending on the product being manufactured.
  • Vehicle Detection and License Plate Recognition: A primary detection model crops license plate regions from vehicle images and passes them to an OCR model for accurate, real-time plate reading without processing full-frame images.
  • Wildlife Monitoring: Low-power microphones perform sound classification to detect animal presence and wake a camera, which then runs an animal classification model, conserving battery life while maintaining 24/7 monitoring capability.
  • Human Safety: Fall detection algorithms trigger camera capture upon detecting a fall event, then invoke a VLM to provide contextual descriptions to first responders, improving emergency response time and situational awareness.

In each case, the underlying infrastructure remains the same: Edge Impulse models at the edge, AWS IoT Greengrass managing the runtime, and AWS IoT Core providing connectivity and fleet orchestration. Organizations can deploy this pattern once and adapt it across operations by swapping detection models and cascade triggers for different use cases.

Getting Started

To deploy edge intelligence for your manufacturing or logistics operations, start with these resources:

Conclusion

The convergence of on-device inferencing and cloud capabilities is changing how manufacturing and logistics operations interact with the physical world. By deploying Edge Impulse’s optimized models at the edge and combining them with Amazon Bedrock’s natural language capabilities in the cloud, organizations can build systems that are both performant and accessible to frontline workers.

This solution represents a shift from complex, centralized vision systems to distributed intelligence that processes data where it is generated, reducing latency, improving reliability, and lowering bandwidth requirements. The two-stage cascade inference pattern ensures that compute resources are used efficiently: lightweight detection runs continuously, while deeper contextual analysis activates only when needed. The natural language interface removes technical barriers, making AI capabilities accessible regardless of a worker’s technical background.

This architecture extends naturally toward autonomous action: triggering work orders in warehouse management systems, dispatching maintenance notifications, or commanding autonomous vehicles to return to base. These extensions close the loop from perception through reasoning to physical-world response, fully realizing the Physical AI paradigm. Edge Impulse and AWS are committed to making these systems accessible, scalable, and secure. We invite you to explore this solution and join us in building the next generation of edge-intelligent industrial systems.