1.  [](/)
2.  [Diagrams](/diagrams)
3.  How you split decides what you find

[Back to Diagrams](/diagrams)IR Foundations

# How you split decides what you find

How you split decides what you find

![How you split a document decides what you can find: fixed-size chunks cut mid-sentence and strand the answer across two pieces, while splitting on meaning keeps each chunk a complete thought. Chunk too small and you lose the context that makes an answer usable; chunk too large and the relevant sentence is diluted by everything around it.](/diagrams/foundations/chunking.webp)Full screen

How you split decides what you find

The retrieval decision nobody reviews in design docs: where to cut.

Documents get split into chunks before embedding, and that split decides everything downstream. Cut every 512 tokens and you slice sentences in half, so the embedding captures half a thought. Cut on topic shifts and every chunk is coherent but the sizes swing wildly. Cut on document structure (headings, tables, code blocks) and chunks match how the author actually organized meaning.

Here's the part that should bother you: a study this month found plain fixed-size chunks beating clever semantic chunking on real benchmarks. The sophisticated option isn't automatically the better one. It depends on your documents, and the only way to know is to measure.

We made chunking a configurable stage in Mixpeek rather than a buried constant, because we kept seeing the same failure: teams tune models for months while a bad splitter quietly caps their ceiling.

Your retrieval can't find what your chunker cut in half.

Run this on your own data

Mixpeek turns video, images, audio, and documents in your object storage into searchable, timestamped results through one API.

[Search your own data](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=diagram_detail&utm_campaign=mvs_onramp)

## Related diagrams

[

![The unstructured-data problem: the large majority of enterprise data is unstructured (video, images, audio, and documents) that keyword systems and relational databases cannot search by meaning until it is turned into embeddings and indexed. The raw files are stored but not answerable.](/diagrams/foundations/unstructured-data.webp)

99% of your data has no rows



](/diagrams/unstructured-data)[

![An embedding turns meaning into coordinates: a model reads 'dog', 'puppy', and 'invoice' and outputs vectors, so 'dog' and 'puppy' land near each other while 'invoice' lands far away. Text, images, audio clips, video frames, and faces all map into one shared coordinate space where distance measures difference in meaning.](/diagrams/foundations/embeddings.webp)

An embedding is meaning turned into coordinates



](/diagrams/embeddings)[

![Semantic search as nearest-neighbor lookup: embed the query into the same vector space as your content, measure distance to every indexed point, and return the closest top-K results with no keyword matching. Exact brute-force compares against everything and is accurate but slow, while approximate nearest-neighbor (ANN) indexes like HNSW and IVF check a smart fraction of the space and return 95-99% of true answers in milliseconds.](/diagrams/foundations/similarity-search.webp)

Search by meaning = nearest neighbors



](/diagrams/similarity-search)