Amazon released AWS Emulator on Nov. 11, a local tool that imitates AWS services so developers can test cloud applications on their own machines. The company says autonomous AI agents wrote and reviewed its implementation.
In Amazon’s engineering account, engineer Matthew Goldberg says his team neither wrote nor reviewed the emulator code. He also describes the substantial engineering work needed to turn an experiment into a product, including the guardrails and checks that determine which changes can ship.
Amazon describes three teams of agents: one manages fidelity tests, another finds differences between the emulator and real AWS, and a third fixes individual issues. Changes face checks for compilation, coding standards, security and regressions before merging, according to the company.
AI agents just completed a four-day cryptography test with only 20 human messages needed – DeepMind says
The test agents work from public AWS API models and verify their scenarios against the real cloud. The public evaluation suite says every fidelity test was checked against at least two sandboxed AWS accounts.
What a passing test establishes
AWS calls the degree of agreement with its cloud services “fidelity.” A test passes when the emulator matches AWS for every call in that workflow, including status codes, resulting resources, side effects and errors, according to its fidelity documentation. Returning success where AWS returns an error fails that test.
The boundary is coverage. The evaluation repository explicitly says behavior without a fidelity test is unverified. Passing a scenario establishes a match for that scenario; it does not establish that every way an application uses AWS will work.
A passing workflow is evidence for that scenario. It does not validate all AWS behavior or application security.
Diagram shows AWS’s pass rule, not a particular test result or measured coverage. Application security still requires testing on real AWS before production.
Security has a separate limit. AWS’s security guidance warns against using the emulator to determine whether an application is secure. It does not check AWS KMS key policies or Amazon S3 bucket policies, and a request denied by AWS may succeed locally.
Agent review of implementation security therefore cannot be read as validation of a user’s application permissions. The emulator is for building and testing, not production workloads. AWS instructs developers to test their whole application and security settings on real AWS before production.
For builders adopting agent-written software, the release makes the acceptance process as relevant as the code’s authorship: which behaviors are checked, what remains outside coverage, and where the developer must still test against the real service.
