Let Your Agent Cook: Using Skills to Evaluate and Improve Your App — Ankur Duggal, Arize AI
Most agent builders rely on intuition rather than systematic evaluation, risking production failures
“realistically, you can't rely on a "yes, this is right" feeling because in a production environment you won't know where errors are occurring”
Arize AI's Ankur Duggal surveyed a live audience at AI Engineer and found that while many teams are building agents, very few have evaluation systems they trust enough for confident production deployment. He argues that the non-deterministic nature of LLM-based agents makes intuition-based quality checks insufficient, and that proper tracing and structured evaluation are prerequisites for reliable production agents. The talk sets up a case for agent-driven self-evaluation as the path forward.