Beyond the KV cache: Training and serving i14’s Laplace 3B linear attention model on AWS Trainium2 with native PyTorch
i14's Laplace 3B linear attention model keeps inference state fixed at 29.5 MB regardless of context length
“Its recurrent state stays at 29.5 MB per sequence, independent of context length.”
i14 trained a 3B-parameter linear attention model (Laplace 3B) on AWS Trainium2 that replaces the growing KV cache with a fixed 29.5 MB recurrent state, achieving 120,000 tokens/sec training throughput and serving 11,544 output tokens/sec across 1,280 concurrent sequences. The architecture changes training attention complexity from O(T²) to O(T), making it particularly suited for long-running agents and large-document workloads. While a meaningful alternative to transformer softmax attention, this is a single 3B-model case study from a small vendor rather than a broadly adopted industry shift.