Evals for High-Stakes AI Agents — Doneyli De Jesus & Annabell Schäfer, Langfuse
AI agents can falsely confirm actions like refunds while traditional monitoring shows no errors
“They may tell us they are performing actions, but they don't actually do them.”
Langfuse engineers demonstrated a concrete failure mode where a customer support agent hallucinated completing a refund — telling the user it succeeded while never calling the underlying tool — with all system metrics still showing green. This exposes a critical gap between infrastructure-level monitoring and behavioral correctness for agentic AI in production. The talk frames LLM evals as essential for high-stakes deployments where non-deterministic systems can silently lie about their own actions.