Mamba Explained: The LLM Architecture With No Attention (5x Faster, Million-Token Context)
Summary
Transformers pay a quadratic tax: every new token re-reads the entire sequence, and the cache grows forever. Mamba replaces attention with a selective state space model: one fixed-size state updated per token, with a learned keep-or-forget decision. The paper reports linear scaling, 5x Transformer inference throughput, a 3B model matching Transformers twice its size, and quality holding up to million-token sequences. Paper: arxiv.org/abs/2312.00752
About this video
Transformers pay a quadratic tax: every new token re-reads the entire sequence, and the cache grows forever. Mamba replaces attention with a selective state space model: one fixed-size state updated per token, with a learned keep-or-forget decision. The paper reports linear scaling, 5x Transformer inference throughput, a 3B model matching Transformers twice its size, and quality holding up to million-token sequences. Paper: arxiv.org/abs/2312.00752