Robot-Use Agents: Why General-Purpose Models May Win in Robotics
General-purpose LLM agents may outperform specialized models for robot control
“We are working on creating LLMs that drive robots”
MIT professor Philip Isola's viral essay argues we may be entering an era where general-purpose model agents make diverse robots more capable, echoing the generalization breakthroughs seen in coding agents. YC-backed startups Waddle Labs and RoboCurve are building on this thesis, using LLMs to control robots across tasks like unscrewing lids and multi-robot coordination. The RT2 paper is cited as an early foundation, showing that fine-tuned language models can output robot end-effector coordinates instead of text — a paradigm now being pushed further by agent-based approaches.