ReplaySSM: Rethinking State Storage for Faster Hybrid-Model Decoding
ReplaySSM rethinks state storage to accelerate decoding in SSM-attention hybrid models
NVIDIA's ReplaySSM proposes a new approach to state storage for hybrid models that interleave SSM layers (Mamba-2, GDN, KDA) with attention layers. Models like NVIDIA Nemotron 3, Qwen3.5-3.8, Kimi-K3, and GLM-5.3-Flash all use this architecture. The work targets inference speed improvements by reconsidering how fixed-size SSM state is managed versus the growing KV cache used by attention layers.