How Credit Genie Debugs Thousands of Agent Traces with LangSmith
Credit Genie debugs thousands of LangGraph agent traces using LangSmith's insights segmentation feature.
“One thing we like about working with the LangChain team is that it feels more similar to working with an internal team than an external company.”
Credit Genie shares how they built AskGenie, an AI financial assistant on LangGraph, and used LangSmith's trace segmentation to debug tool-confusion issues at scale across thousands of agent runs. They designed custom evaluators to distinguish between similar tools and built test suites of 100-200 questions per target behavior. The case study is a practical template for teams scaling agent observability but carries no novel research or industry-shifting announcement.