AWS Insights
Better feedback for building with agents
A coding agent can build a lot of software around an incorrect assumption. The sooner it can test that assumption, the sooner it can correct its work.
As we give agents more responsibility, they need feedback throughout that work: what ran successfully, what failed, and what needs to change. Fast, accurate feedback determines how much work we can delegate and how confidently we can use the results.
Today we’re introducing AWS Emulator, a local environment for building and testing applications that use AWS. Developers and agents can use familiar AWS APIs, SDKs, and tools without provisioning cloud resources or incurring AWS service charges for the operations it emulates.
It adds another layer of checks alongside existing tests and development accounts. Catching mistakes locally helps developers and agents reach testing on AWS sooner, with more of the application already tested.
Checking assumptions while building
Suppose a coding agent builds an application that reads orders from an Amazon SQS standard queue and stores them in Amazon DynamoDB. The same order can arrive more than once, so the application needs to recognize duplicates.
The agent uses a conditional write that succeeds only if the order identifier is absent from the table. A test sends the same order twice. The first write succeeds. The second is rejected because the record already exists. The worker treats that rejection as a temporary failure, leaves the message in the queue, and receives it again when the visibility timeout expires.
An emulator that accurately implements these behaviors lets the agent encounter the failure, investigate it, and correct the application. For this duplicate order, the worker should delete the message from the queue and continue. Other write failures still need appropriate handling.
A mock can reproduce those responses if someone has anticipated them. The emulator evaluates requests against the current state, allowing developers and agents to test different sequences without implementing the service behavior themselves.
For developers, this means more opportunities to catch incorrect assumptions while making changes. For agents, it provides feedback they can act on before returning their work. The longer a mistake goes undetected, the more code an agent can build around it.
Independent environments also make it practical to test alternatives in parallel. An agent could implement two approaches, run the same tests against both, and give the developer results to review.
Evaluating and training agents
The same feedback loop matters to teams building the agents themselves. They need to test across many applications and tasks to understand whether a change improves performance.
One successful application provides limited evidence. Teams need to understand how an agent performs with unfamiliar code, different requirements, and longer sequences of actions. They also need to measure whether a new model or prompt improves those results.
Provisioning and cleaning up cloud resources for every attempt takes time and adds cost. Local, independent environments make it practical to run more tasks and evaluate more outcomes.
Those environments can also give agents more opportunities to practice. A reinforcement learning system can give an agent tasks, evaluate the outcomes, and use the results to improve the agent or underlying model. Lower costs make it practical for teams to attempt harder tasks, introduce more variation, and test whether improvements extend beyond familiar examples.
An agent learns from the environment it experiences. If the emulator accepts behavior that AWS rejects, training can reinforce a mistake. An agent might also discover a shortcut that succeeds only because of an emulator defect.
For these results to be useful, teams need to know where the emulator behaves like AWS and where it differs.
AWS Emulator spans hundreds of AWS service interfaces. Implementation completeness varies by service and operation; the published results show what has been verified.
We compare AWS Emulator’s responses with recorded AWS responses, accounting for expected differences such as timestamps. We also run sequences of operations to confirm that complete workflows behave as expected. We publish the versioned tests and success rates for both whole scenarios and individual steps.
Available today
AWS Emulator is available at no charge for Linux, as a Docker container, and with source available on GitHub. The documentation covers setup and service coverage.
Download. Fire it up. Let us know what you build.