The Hallway Track
Engineering Insights

HTML Is All Agents Need — James Russo, HeyGen

Andrej Karpathy · AI Engineer · Jul 21, 2026 · Engineering Insights

HeyGen uses HTML as the native canvas for AI agents to generate full video compositions

“Uh when you try to teach a model a new DSL or even your own custom JSON structure, it's forcing it to speak another language.”— Andrej Karpathy

HeyGen's James Russo argues that HTML/CSS/JS are the native languages of LLMs — since most training data is scraped web content — making custom DSLs an unnecessary impedance mismatch. Their Hyperframes product lets agents compose full videos (A-roll, B-roll, captions, music) by writing HTML, demonstrated by an agent-generated video in a single shot. The insight aligns with emerging thinking from Karpathy and others that HTML is the new markdown for LLM output.

agentic-video html-as-llm-interface heygen multimodal video-generation

Watch / read the original source →