Evaluate your Amazon Nova Sonic voice agent at scale, no microphone required
AWS released the open source Nova Sonic Test Harness to automatically evaluate voice agents at scale without a microphone.
“It runs complete multi-turn conversations with Amazon Nova Sonic automatically, evaluates them using LLM-as-judge techniques, and can even detect cases where the model's audio output doesn't match its text output (audio hallucinations).”
AWS open-sourced the Nova Sonic Test Harness, a framework that automates multi-turn voice-agent testing using LLM-as-judge techniques and can detect audio-text divergence (audio hallucinations) without manual microphone interaction. It addresses the difficulty of regression-testing non-deterministic, bidirectional speech-to-speech agents, which matters as voice agents move into production. The signal is a practical engineering/tooling advance rather than a major industry-shaping announcement.