Cerebras CTO Sean Lie: Deploying Trillion-parameter Frontier Models
Cerebras CTO demonstrates running a 3-trillion-parameter open-source model on Wafer Scale Engine
“because we have so much SRAM on our wafer-scale chip, it actually integrates very naturally, and we can support even the largest models today”
Cerebras CTO Sean Lie walks through how the company's Wafer Scale Engine handles Chinchilla 3, an open-source 3-trillion-parameter model, using 4-bit weight quantization, 8-bit KV cache, and 256K context length across 44GB of onboard SRAM per wafer. The key claim is that Cerebras's massive wafer-level SRAM integration inverts the usual constraint — where SRAM chips struggle with large models — making frontier-scale deployment practical. This matters because it positions Cerebras as a credible inference substrate for the largest open-source models, directly competing with GPU clusters for trillion-parameter workloads.