How WIRobotics is teaching humanoid robots to use human tools with AWS and NVIDIA

The next frontier of automation is built for human environments

Industrial automation has transformed work that is predictable and repeatable. Robots can perform the same motion thousands of times with remarkable consistency, which makes them well suited to structured production environments. But much of the work that keeps industrial operations running is not always orderly. A technician picks up a drill, lines it up with a bolt, adjusts for the angle, and applies pressure as the tool engages. The motion feels routine to a person, but for a robot, every step requires careful coordination.

That gap helps explain why so many useful jobs remain difficult to automate. Traditional automation works best when the environment is designed around the machine. Human workplaces usually are not, leaving robots to navigate workspaces where tools, objects, and conditions continually change. For organizations already managing labor shortages and rising labor costs, those challenges have limited automation’s impact across physically demanding workflows.

WIRobotics is taking a different approach. Instead of asking customers to redesign the workplace around robots, the company is building robots that can operate in the environments people already use. As part of the Physical AI Fellowship powered by Amazon Web Services (AWS), NVIDIA, and MassRobotics, WIRobotics partnered with the AWS Generative AI Innovation Center and NVIDIA to explore one of the hardest capabilities in robotics: tool use.

The team focused on a deceptively familiar task, teaching a robot to use a power drill. To accomplish that, WIRobotics, with support from the AWS GenAIIC team, built a cloud-based training workflow for fine-tuning NVIDIA’s GR00T N1.7 robotic foundation model on real teleoperation data from its robot, ALLEX.

Meet ALLEX

WIRobotics designed ALLEX to operate in environments built for people, reflecting the company’s broader vision of robots that safely collaborate with humans.

ALLEX combines highly dexterous hands with whole-body force sensing across its 48-degree-of-freedom upper body. Hardware designed for responsive physical interaction, including low friction, low inertia, and high backdrivability, allows the robot to respond naturally to physical contact. These characteristics make ALLEX especially well-suited for manipulation tasks that depend on continuous feedback rather than rigid, pre-programmed motion.

Figure 1: Product photo or diagram of ALLEX with callouts highlighting dexterous hands, force sensing, and key manipulation capabilities

From pick-and-place to power tools

Before joining the Physical AI Fellowship, WIRobotics had already demonstrated learning-based control on ALLEX using a lightweight imitation learning model known as ACT. The robot could perform single-arm pick-and-place tasks, providing a strong foundation for future development. The next challenge was contact-rich, bimanual manipulation, the kind of work required for fastening, equipment maintenance, inspection, and other industrial workflows where robots must continuously respond to changing forces rather than simply execute a fixed sequence of motions.

A power drill offered a compelling test case because success depends on continuous physical interaction rather than simply recognizing and grasping an object. Instead of building a task-specific solution, WIRobotics used the project to determine whether a robotic foundation model could transfer that capability to a completely new robot embodiment.

The fellowship also provided WIRobotics with a structured environment to accelerate how it developed and evaluated new robotic skills. As part of the project, WIRobotics evaluated NVIDIA’s GR00T N1.7 Vision-Language-Action (VLA) foundation model with support from AWS GenAIIC. GR00T was selected because its pre-training on human egocentric video aligned well with ALLEX’s head-mounted camera, while its state representation could accommodate the robot’s proprioceptive inputs without redesigning the underlying foundation model.

Figure 2: Drill manipulation workflow showing grasping, alignment, and bolt fastening stages performed by ALLEX

Building a foundation model workflow on AWS

Working with AWS Generative AI Innovation Center, WIRobotics built its first cloud-native robotics training workflow on AWS.

The training infrastructure included:

• Compute: Amazon Elastic Compute Cloud (Amazon EC2 g7e.48xlarge instances) equipped with 16 NVIDIA RTX PRO 6000 Blackwell GPUs (16 GPUs total across the cluster), enabling large-scale, multi-GPU training and parallel experimentation.
• Storage: Amazon FSx for Lustre for high-throughput checkpointing and shared storage during training, with Amazon Simple Storage Service (Amazon S3) serving as the durable entry point for raw teleoperation data and the destination for deployment-ready model checkpoints.
• Data processing: A pipeline that synchronized 44-dimensional robot state information, including joint positions, velocities, and torques, with three camera views capturing the drill, bolt, and surrounding workspace.

Human operators teleoperated ALLEX through drill manipulation tasks while the system captured synchronized robot state data and camera views for GR00T fine-tuning.

The team also adapted GR00T to ALLEX’s unique embodiment through proprioceptive state mapping and LeRobot v2 data conversion, allowing the model to translate learned representations into actions suited to the robot’s physical design.

The following architecture diagram illustrates the full production pipeline beyond the core training loop. In production, teleoperation data flows through Amazon Kinesis Video Streams and Amazon Kinesis Data Streams for real-time ingestion, with Amazon Data Firehose delivering processed streams to Amazon S3. AWS Glue handles data cataloging and transformation, while AWS Batch manages large-scale training jobs. Amazon CloudWatch and AWS CloudTrail provide observability and audit logging across the pipeline, and AWS Identity and Access Management (IAM) governs access control throughout.

Figure 3: Architecture diagram showing teleoperation data flowing into AWS training infrastructure, GR00T fine-tuning, and deployment back to ALLEX

What the team learned

By the end of the six-week engagement, ALLEX achieved a 100% success rate on drill grasping and actuation tasks. For the more demanding drill-to-bolt alignment task, the robot successfully completed bolt tightening demonstrations and achieved a 50% alignment success rate.

The experiments yielded several insights that will help guide future development. Across more than 150 physical robot evaluations, the team identified patterns that consistently improved performance. Data quality proved more valuable than data volume, with carefully curated demonstrations consistently outperforming larger datasets containing inconsistent behavior. Performance also depended on how the model was tuned for the task. Longer action horizons improved grasping performance, while additional denoising steps sharpened precision during alignment – and the two had to be balanced together, with the best results coming from pairing a long action horizon with more denoising steps.

The project delivered a repeatable workflow, practical VLA fine-tuning guidelines, and a foundation for teaching future manipulation skills without rebuilding the underlying infrastructure.

Looking ahead

For WIRobotics, drill manipulation is one example of a broader class of skills that robots will need in order to perform useful work in real environments. The company envisions building a growing library of capabilities that can be applied to tasks such as assembly, maintenance, inspection, and machine operation. As that library expands, foundation models can help coordinate and extend those skills rather than requiring every new behavior to be developed independently.

As foundation models continue to evolve, WIRobotics expects robots to become more capable of operating in the environments people already use. The power drill demonstrates how robots can continuously expand their capabilities through learning, one task at a time.

Learn more

Visit WIRobotics to learn more about ALLEX and the company’s vision for humanoid robotics. Find additional information about NVIDIA GR00T and robotic foundation models on NVIDIA. To build and scale physical AI workloads, explore Amazon EC2 accelerated computing services or connect with your AWS account team.