The Hardest Part of Testing Agents: Write Access
Testing agents with write access to real systems is an unsolved, underappreciated problem
“I don't really think a lot of the teams that build agents know exactly how to build these environments properly.”
LangChain highlights a critical gap in agent development: while read-only agents are straightforward to test in branched environments, agents with write access to databases, email, or other real-world systems create unsafe evaluation conditions. The speaker argues most teams lack proper isolation strategies for testing these agents. This is a meaningful signal for the agent engineering community as write-access agents become more common in production.