1.  [](/)
2.  [Research](/research)
3.  Mamba Explained: The LLM Architecture With No Attention (5x Faster, Million-Token Context)

[Back to Research](/research)

Carnegie Mellon & Princeton[arXiv:2312.00752](https://mxp.co/e/mamba)

# Mamba Explained: The LLM Architecture With No Attention (5x Faster, Million-Token Context)

Mamba replaces attention with a selective state space model, one fixed-size state updated per token with a learned keep-or-forget decision, reporting linear scaling and 5x Transformer inference throughput with quality holding to million-token sequences.

## Where this fits in Mixpeek

Long-form video and multi-hour audio push any sequence model against attention's quadratic cost. Linear-scaling architectures like Mamba matter to Mixpeek wherever a model has to read a very long sequence (full-length transcripts, long documents, extended video) without the memory blowup that forces aggressive chunking. It informs how Mixpeek can process long modalities end-to-end rather than in lossy windows.

## About this research

Transformers pay a quadratic tax: every new token re-reads the entire sequence, and the cache grows forever. Mamba replaces attention with a selective state space model: one fixed-size state updated per token, with a learned keep-or-forget decision. The paper reports linear scaling, 5x Transformer inference throughput, a 3B model matching Transformers twice its size, and quality holding up to million-token sequences. Paper: arxiv.org/abs/2312.00752

mambastate-space-modelsarchitecturelong-contextefficiency

## Frequently asked questions

How is Mamba different from a Transformer?+

What makes Mamba's state space model selective?+

## Go deeper

*   [Diagram: The embedding landscape →](/diagrams/embedding-landscape)

Put the research to work

Mixpeek turns video, images, audio, and documents in your object storage into searchable, timestamped results through one API: the retrieval stack these papers describe.

[Search your own data](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=research_detail&utm_campaign=mvs_onramp)

## Related research

[

![Baidu's Unlimited-OCR: The Model Whose Memory Never Grows](https://cdn.mixpeek.com/videos/thumbs/77-unlimited-ocr.jpg)

Baidu

Baidu's Unlimited-OCR: The Model Whose Memory Never Grows



](/research/unlimited-ocr)