Continuously Improving Agents with Langfuse — Annabell Schäfer & Lotte Verheyden, ClickHouse
Langfuse defines a 5-step AI engineering cycle for continuous agent improvement in production
“AI agents are not deterministic. Your system may appear to be working, while the responses or actions taken by the agent are incorrect.”
ClickHouse engineers presented a Langfuse workshop framing 'AI engineering' as a continuous improvement discipline distinct from traditional software engineering, centered on a 5-step cycle of tracing, monitoring, dataset creation, experimentation, and offline evaluation. The core argument is that non-determinism in agents means a system can appear functional while producing incorrect or low-quality outputs, requiring dedicated tooling to catch. Langfuse positions itself as the open-source, model- and framework-agnostic platform for implementing this cycle.