Observing And Testing CX Agents | Interrupt 26
Cisco built an agentic feedback loop turning production thumbs-down signals into merged PR fixes with human oversight.
“every thumbs up is a lead. Every trace with an error is a potential regression. And every confused user is a description of a gap that needs to be addressed.”
At LangChain's Interrupt conference, Cisco's Carlos Pereira detailed how their customer-experience agents close the loop from production feedback to code fixes, using a triage agent (LangSmith + Jira MCP) feeding a coding agent that clusters signals and drafts fixes with humans approving PRs. It matters because it shows a working pattern at Cisco's scale (16M interactions/year, ~10,000 concurrent cases) for treating observability traces, thumbs-down, and low-confidence classifications as actionable signals rather than noise. The talk frames human triage as the new bottleneck and positions agent-driven diagnostics plus pre-ship evals as the way to scale agent maintenance.