Can Frontier Models See Past a Decoy? Elorian Tests Visual Reasoning
Frontier models all scored below 50% on visual counting benchmark DecoyCount, exposing a core gap in visual reasoning.
“stronger general reasoning does not by itself generalize to aligned visual reasoning”
Elorian launched Glimpse, a visual reasoning model, alongside the DecoyCount benchmark designed to test whether models genuinely count objects or exploit visual shortcuts. Every frontier model tested scored below 50% accuracy — GPT-6 Astra led at 43.17% — while Glimpse achieved 50.48%, the only model to clear the threshold. The result highlights a structural weakness in current frontier models: general reasoning capability does not transfer reliably to aligned visual perception tasks.