Rich Sutton: synthetic data is "just a big mistake"
Rich Sutton calls synthetic data generation a 'big mistake,' advocating real-world experience learning instead.
“No, that's that's just a big mistake.”
Reinforcement learning pioneer Rich Sutton flatly dismisses synthetic data generation—currently a central strategy at major foundation model labs—as fundamentally misguided. He counters with the 'big world hypothesis,' which holds that the real world is infinitely large and AI systems should learn directly from experience rather than human-curated synthetic datasets. This is a significant contrarian signal from a historically influential researcher whose scaling intuitions have proven correct before, directly challenging the industry's near-term roadmap for pushing past internet data limits.