Is a Faster Model Actually Faster? Test the Whole Job — Morgan Suo, MiniMax
Raw model speed misleads; end-to-end task quality with human correction cost is the true efficiency metric
“You launch a research task and the report arrives quickly. But it misses a very important source. So someone has to find this source and correct the report.”
MiniMax engineer Morgan Suo argues that inference speed is a poor proxy for real-world model value, because a faster model that produces incomplete or unverifiable outputs still requires human correction time that erases the speed gain. Using a multi-company PCB supply chain research task as a benchmark, she defines 'completed research' as output an analyst can actually verify — not just output that arrives quickly. The framing is a practical corrective to token-speed benchmarking culture, relevant to teams evaluating model upgrades for agentic workflows.