Baidu's Unlimited-OCR: The Model Whose Memory Never Grows
Baidu's Unlimited-OCR replaces decoder attention with Reference Sliding Window Attention, turning the KV cache into a fixed-size queue so 40+ page PDFs parse in one forward pass instead of being chopped at page breaks.
Where this fits in Mixpeek
Mixpeek's document extractor turns PDFs into searchable text, tables, and layout. Long-document OCR is where naive pipelines break — they split PDFs into single pages and cut tables at page boundaries, losing cross-page structure. A fixed-cache, whole-document OCR approach like Unlimited-OCR is the direction for Mixpeek's document ingestion to preserve tables and reading order across long files, so retrieval and reranking see the whole document rather than fragments.
About this research
Long-document OCR hits a memory wall: the KV cache grows with every token written, so pipelines chop PDFs into single pages and tables get cut at page breaks. Baidu's Unlimited-OCR (MIT, June 2026) replaces decoder attention with Reference Sliding Window Attention: each token attends to the document's visual tokens plus the last 128 outputs, making the cache a fixed-size queue. 40+ page PDFs parse in one forward pass, scoring 93.23 on OmniDocBench v1.5 vs DeepSeek-OCR's 87.01, about 12% faster, from a 3B MoE with 500M active. Paper: arxiv.org/abs/2606.23050
Frequently asked questions
Put the research to work
Mixpeek turns video, images, audio, and documents in your object storage into searchable, timestamped results through one API — the retrieval stack these papers describe.
Search your own data, free
