Medic for Apache Spark - First Aid for Failing Jobs - Drasko Profirovic, Pinterest
Pinterest built an agentic LLM tool using MCP and ReAct to auto-diagnose Apache Spark job failures
“Our vision for a diagnostics agent was to ask it simply, 'Why did a job fail?' and get back a deep research document which provides evidence on the root cause of the failure.”
Pinterest's Draško Profirović describes Medic, an agentic diagnostics system that uses Model Context Protocol to expose Spark job data to LLMs and a ReAct agent to autonomously investigate job failures. The tool evolved from a manual-prompting MCP prototype to a single ReAct agent deployed on Slack and Airflow UI, though single-prompt tuning proved unsustainable at scale. This represents a concrete production case study of MCP-based agentic tooling applied to distributed systems operations.