[Back to Modules](/university)

### Module 5 of 8

0 completed · 8 remaining

0%

BeginnerModule 4·14:06·By Ethan

# Chunking Strategies: Breaking Documents into Searchable Pieces

Master the art of breaking large documents into searchable chunks. Learn fixed-size, semantic, and sentence-based strategies, chunk overlap techniques, and how to balance precision vs recall for optimal retrieval.

chunkingdecompositionretrievalmultimodalprecisionrecalloverlap

Mark as Complete

ContentTranscriptResourcesQuiz

**Chunking isn't just splitting text by word count: it's the foundation of retrieval precision. The right chunking strategy determines whether your RAG system returns exact answers or overwhelming noise.**

When RAG first emerged, everyone chunked by word, then by sentence, then by paragraph. Only recently have teams started chunking by **concept** and **semantics**. This module covers the multimodal approach: chunking by any arbitrary characteristic across videos, images, audio, and documents.

## Why Chunk?

Chunking solves two critical problems:

### 1\. Context Window Limits

Encoder models have limited context windows:

*   **Small models:** 512 tokens
*   **Standard models:** 8,000 tokens
*   **Large models:** 128,000+ tokens (but still finite)

**Problem without chunking:**

`Document: 200-page legal contract (42,000 words) Query: "indemnification clause" Result: Entire 200-page document (useless) Precision: 31%`

**Solution with chunking:**

`Document: 200-page contract → 164 chunks (512 tokens each, 128 overlap) Query: "indemnification clause" Result: Exact clause with surrounding context Precision: 94%`

### 2\. Retrieval Precision

LLMs are getting better at reasoning, but reasoning is **limited to the content you give them**. Overwhelming an LLM with irrelevant data degrades output quality.

**Key Insight:** The more precise your chunks, the more relevant your LLM's responses.

## Chunking Strategies Overview

Chunking isn't limited to text. You can chunk **any modality** by **any characteristic**:

Modality

Chunking Strategies

**Text/PDF**

Fixed interval, sentence-based, semantic, layout (headers, footers, tables, figures)

**Video**

Time intervals, scenes, shots, keyframes, faces, objects (SAM2)

**Audio**

Fixed duration, speaker diarization, silence intervals, activity detection

**Images**

Objects, regions, layout elements

Let's dive into each strategy.

## Text & PDF Chunking Strategies

### 1\. Fixed-Size Chunking

**Definition:** Split text every N tokens, regardless of content structure.

**Best for:**

*   Simple use cases with consistent content
*   When you need predictable chunk sizes

**Example:**

chunk\_size = 512  # tokens
chunk\_overlap = 0

# Document: "The quick brown fox jumps over the lazy dog. The dog..."
# Chunk 1: "The quick brown fox jumps over the lazy dog. The dog was..."
# Chunk 2: "...sleeping under a tree. The tree provided shade..."

**Pros:**

*   Simple to implement
*   Predictable chunk sizes

**Cons:**

*   Cuts mid-sentence or mid-thought
*   Loses context at boundaries
*   No semantic awareness

### 2\. Fixed-Size with Overlap

**Definition:** Split every N tokens but include overlap from previous chunk.

**Best for:**

*   Preventing information loss at boundaries
*   Maintaining context across chunks

**Example:**

chunk\_size = 512  # tokens
chunk\_overlap = 128  # tokens

# Chunk 1: "The quick brown fox \[...\]" (tokens 0-512)
# Chunk 2: "fox jumps over \[...\] sleeping" (tokens 384-896)
#          ^^^ 128 token overlap ^^^

**Pros:**

*   **Safety net** against boundary loss
*   Maintains context flow
*   67% → 94% retrieval accuracy improvement

**Cons:**

*   Slight storage overhead
*   Duplicate content across chunks

### 3\. Sentence-Based Chunking

**Definition:** Split at sentence boundaries (periods, question marks, exclamation points).

**Best for:**

*   Capturing complete thoughts
*   Natural language processing

**Example:**

\# Split on periods
chunk\_delimiter = "."

# Chunk 1: "The quick brown fox jumps over the lazy dog."
# Chunk 2: "The dog was sleeping under a tree."
# Chunk 3: "The tree provided shade for hours."

**Pros:**

*   Preserves complete thoughts
*   Natural chunk boundaries

**Cons:**

*   Variable chunk sizes
*   May create very small or very large chunks

### 4\. Semantic-Based Chunking

**Definition:** Use embedding similarity to detect conceptual boundaries.

**How it works:**

1.  Encode each sentence individually
2.  Measure cosine similarity between consecutive sentences
3.  When similarity drops below threshold → chunk boundary

**Example:**

\# Encode sentences
sentence\_1 = "Ethan likes dogs."
sentence\_2 = "Dogs like Ethan."
sentence\_3 = "The weather is sunny today."

# Measure similarity
similarity\_1\_2 = cosine\_similarity(embed(sentence\_1), embed(sentence\_2))  # 0.89
similarity\_2\_3 = cosine\_similarity(embed(sentence\_2), embed(sentence\_3))  # 0.12

# Chunk boundary at 2→3 (similarity < 0.5)
# Chunk 1: "Ethan likes dogs. Dogs like Ethan."
# Chunk 2: "The weather is sunny today."

**Best for:**

*   Capturing conceptual coherence
*   Complex documents with topic shifts

**Pros:**

*   Maintains semantic coherence
*   Topic-aligned chunks

**Cons:**

*   Computationally expensive
*   Requires encoder model

### 5\. Layout-Based Chunking (PDFs)

**Definition:** Chunk by structural elements of the document.

**Chunk by:**

*   **Headers/Footers**: Separate metadata from content
*   **Tables**: Extract structured data
*   **Figures/Charts**: Isolate visual elements
*   **Columns**: Handle multi-column layouts

**Best for:**

*   Structured documents (contracts, reports, research papers)
*   Preserving document hierarchy

**Example:**

\# PDF with headers, body, tables
# Chunk 1: Header ("Merger Agreement - Section 3")
# Chunk 2: Body paragraph (indemnification clause)
# Chunk 3: Table (liability limits)
# Chunk 4: Footer (page number, date)

## Video Chunking Strategies

### 1\. Time Interval Chunking

**Definition:** Split video every N seconds.

**Example:**

chunk\_interval = 30  # seconds

# 10-minute video → 20 chunks (30 seconds each)

**Best for:**

*   Large videos that need initial segmentation
*   Predictable processing pipelines

**Pros:**

*   Simple, fast
*   Consistent chunk durations

**Cons:**

*   No semantic awareness
*   Cuts mid-scene or mid-action

### 2\. Scene Detection Chunking

**Definition:** Detect scene transitions using pixel differences between frames.

**Tools:** PySceneDetect, AutoShot

**How it works:**

\# Measure pixel differences between consecutive frames
frame\_1 = video\[t=0\]
frame\_2 = video\[t=1\]

pixel\_diff = abs(frame\_2 - frame\_1)

if pixel\_diff > threshold:
    # New scene detected
    create\_chunk()

**Best for:**

*   Content with clear scene transitions (movies, ads, tutorials)
*   Semantic video segmentation

**Pros:**

*   Semantically meaningful chunks
*   Captures complete scenes

**Cons:**

*   Variable chunk sizes
*   May miss subtle transitions

### 3\. Shot Boundary Detection

**Definition:** Detect camera position changes (cuts, pans, zooms).

**Best for:**

*   Professional video content
*   Fine-grained segmentation

### 4\. Keyframe Sampling

**Definition:** Extract representative frames at intervals or based on content change.

**Example:**

\# Extract keyframes every 3 seconds
keyframe\_interval = 3  # seconds

# Or: Extract frames with significant visual change
visual\_change\_threshold = 0.7

**Best for:**

*   Image-based retrieval
*   Video thumbnails

### 5\. Object-Based Chunking (SAM2)

**Definition:** Use Segment Anything Model 2 (SAM2) to detect objects and chunk by object presence.

**Example:**

\# Detect all scenes containing "dog" object
object = "dog"
chunks = video.chunk\_by\_object(object)

# Result: All video segments where dog appears

**Best for:**

*   Object-centric retrieval
*   Content analysis (e.g., all scenes with a product)

## Audio Chunking Strategies

### 1\. Fixed Duration Chunking

**Definition:** Split audio every N seconds.

**Example:**

chunk\_duration = 30  # seconds

**Best for:**

*   Consistent processing
*   Simple use cases

### 2\. Speaker Diarization

**Definition:** Chunk by unique speaker voices.

**How it works:**

\# Audio: "Speaker A: Hello. Speaker B: Hi there. Speaker A: How are you?"
# Chunk 1: "Speaker A: Hello."
# Chunk 2: "Speaker B: Hi there."
# Chunk 3: "Speaker A: How are you?"

**Best for:**

*   Meeting transcriptions
*   Podcast analysis
*   Multi-speaker content

**Pros:**

*   Speaker-specific chunks
*   Preserves speaker context

**Cons:**

*   Requires diarization model
*   Variable chunk sizes

### 3\. Silence-Based Chunking

**Definition:** Detect silence intervals and chunk at natural pauses.

**Example:**

silence\_threshold = -40  # dB
min\_silence\_duration = 1.0  # seconds

# Chunk at pauses > 1 second

**Best for:**

*   Natural speech boundaries
*   Audio with clear pauses

### 4\. Activity Detection Chunking

**Definition:** Chunk based on audio activity (speech vs music vs silence).

**Best for:**

*   Mixed audio content
*   Content-aware segmentation

## Chunk Overlap: The Safety Net

Chunk overlap is **critical** for preventing information loss at chunk boundaries.

### Why Overlap Matters

**Without overlap:**

`Chunk 1: "...the indemnification clause states that" Chunk 2: "the buyer assumes all liability for..." ^^^ Missing context from Chunk 1`

**With 128-token overlap:**

`Chunk 1: "...the indemnification clause states that" Chunk 2: "clause states that the buyer assumes all..." ^^^ Context preserved from Chunk 1`

### Overlap Percentages

Overlap

Use Case

Storage Overhead

**0%**

No overlap - maximum storage efficiency

0%

**25%**

Light overlap - basic safety net

+25%

**50%**

Moderate overlap - strong context preservation

+50%

**75%**

Heavy overlap - maximum context (rarely needed)

+75%

**Recommended:** 25% overlap (128 tokens for 512-token chunks)

### Overlap Example

chunk\_size = 512
chunk\_overlap = 128  # 25%

text = "The quick brown fox jumps over the lazy dog. " \* 100

chunks = chunk\_text(text, chunk\_size, chunk\_overlap)

# Chunk 1: tokens 0-512
# Chunk 2: tokens 384-896 (128 token overlap from Chunk 1)
# Chunk 3: tokens 768-1280 (128 token overlap from Chunk 2)

## Chunk Size Trade-Offs

Chunk size directly impacts precision and recall:

### Small Chunks (256-512 tokens)

**Precision:** ⭐⭐⭐⭐⭐ High **Recall:** ⭐⭐ Low **Context:** ⭐⭐ Low

**Use when:**

*   You need exact answers
*   Content is dense with distinct concepts
*   Storage is not a concern

**Example:**

`Query: "indemnification clause" Small chunk: Returns exact 3-paragraph clause Large chunk: Returns entire 20-page section`

### Medium Chunks (512-1024 tokens)

**Precision:** ⭐⭐⭐⭐ Moderate-High **Recall:** ⭐⭐⭐⭐ Moderate-High **Context:** ⭐⭐⭐⭐ Moderate-High

**Use when:**

*   Balanced retrieval needs
*   General-purpose RAG
*   Most common choice

### Large Chunks (1024-2048 tokens)

**Precision:** ⭐⭐⭐ Moderate **Recall:** ⭐⭐⭐⭐⭐ High **Context:** ⭐⭐⭐⭐⭐ High

**Use when:**

*   Context preservation is critical
*   Complex reasoning required
*   Willing to trade precision for recall

## Real-World Example: Legal Contract Analysis

**Scenario:** 200-page merger agreement (42,000 words)

**Query:** "indemnification clause"

### Approach 1: No Chunking

\# Upload entire document as single chunk
chunk\_count = 1
chunk\_size = 42000  # words

# Query result: Entire 200-page document
precision = 31%  # Too broad
recall = 100%  # Found everything, but unusable

**Problem:** LLM receives entire document, can't focus on relevant section.

### Approach 2: Fixed 512 Tokens, No Overlap

chunk\_size = 512  # tokens
chunk\_overlap = 0

# Result: 164 chunks
# Query finds clause but cuts mid-sentence
precision = 67%  # Better
recall = 82%  # Misses boundary context

**Problem:** Clause split across chunks, missing critical context.

### Approach 3: Fixed 512 Tokens, 128 Overlap ✅

chunk\_size = 512  # tokens
chunk\_overlap = 128  # 25%

# Result: 164 chunks with overlap
# Query finds exact clause with surrounding context
precision = 94%  # Excellent
recall = 94%  # Complete information

**Success:** Perfect retrieval, exact clause with context maintained across boundaries.

### The Impact of Overlap

Strategy

Precision

Recall

Explanation

No chunking

31%

100%

Finds everything but unusable (entire doc)

No overlap

67%

82%

Finds clause but cuts mid-sentence

**With overlap**

**94%**

**94%**

**Perfect: exact clause + context**

**Key Takeaway:** Overlap is the **safety net** that prevents losing information at chunk boundaries.

## Building Chunking Pipelines in Mixpeek

Mixpeek enables multimodal chunking through **object decomposition** flows.

### Object Decomposition Architecture

Think of object decomposition as **nested chunking** across modalities:

`Video Object ├─ Time Segments (every 30s) │ ├─ Scene Segments (scene detection) │ │ └─ Face Segments (face detection) │ │ └─ Expression Features (emotion detection) │ └─ Silence Segments (silence detection) │ └─ Speaker Segments (diarization) └─ Keyframe Segments (visual sampling)`

Each level is an independent **collection** that can be queried separately or together.

### Creating a Multi-Tier Chunking Pipeline

**Step 1: Create Collection with Time Segmentation**

from mixpeek import Mixpeek

client = Mixpeek(api\_key="your-api-key")

# Tier 1: Chunk by time (every 30 seconds)
time\_collection = client.collections.create(
    collection\_name="time-segments",
    source\_bucket="raw-videos",
    feature\_extractors=\[{
        "type": "multimodal",
        "split\_method": "time",
        "split\_interval": 30  # seconds
    }\]
)

**Step 2: Add Scene Segmentation**

\# Tier 2: Chunk by scene (using time segments as input)
scene\_collection = client.collections.create(
    collection\_name="scene-segments",
    source\_collection="time-segments",  # Use previous tier
    feature\_extractors=\[{
        "type": "multimodal",
        "split\_method": "scene",
        "detection\_threshold": 0.5  # Scene change sensitivity
    }\]
)

**Step 3: Add Silence Segmentation (Parallel)**

\# Tier 2 (Parallel): Chunk by silence
silence\_collection = client.collections.create(
    collection\_name="silence-segments",
    source\_collection="time-segments",
    feature\_extractors=\[{
        "type": "multimodal",
        "split\_method": "silence",
        "silence\_threshold": -40,  # dB
        "enable\_transcription": True
    }\]
)

### Multi-Collection Retrieval

Query across **all chunk tiers** simultaneously:

\# Create retriever spanning multiple collections
retriever = client.retrievers.create(
    retriever\_name="multi-tier-search",
    collections=\[
        "time-segments",
        "scene-segments",
        "silence-segments"
    \],
    stages=\[
        {
            "type": "feature\_search",
            "feature\_uri": "multimodal-extractor-v1",
            "limit": 20
        }
    \],
    fusion={
        "strategy": "reciprocal\_rank\_fusion"
    }
)

# Execute query
results = retriever.execute(
    inputs={"text": "product demo with Sarah"}
)

# Returns:
# - Time segments containing "product demo"
# - Scene segments with "Sarah" (face detection)
# - Speaker segments with Sarah's voice

**The Power of Decomposition + Recomposition:**

*   **Decompose:** Chunk objects into granular pieces
*   **Recompose:** Query across all chunks on-demand

This is the **multimodal** way to build precise retrieval systems.

## Chunking Configuration in Mixpeek

### Text/PDF Chunking

collection = client.collections.create(
    collection\_name="documents",
    feature\_extractors=\[{
        "type": "text",
        "chunk\_size": 512,
        "chunk\_overlap": 128,
        "chunk\_method": "sentence"  # or "fixed", "semantic"
    }\]
)

### Video Chunking

collection = client.collections.create(
    collection\_name="videos",
    feature\_extractors=\[{
        "type": "multimodal",
        "split\_method": "scene",
        "detection\_threshold": 0.5,
        "enable\_transcription": True,
        "enable\_embedding": True
    }\]
)

### Audio Chunking

collection = client.collections.create(
    collection\_name="audio",
    feature\_extractors=\[{
        "type": "multimodal",
        "split\_method": "speaker",  # Diarization
        "enable\_transcription": True
    }\]
)

## Key Takeaways

1.  **Chunking is essential** for encoder context windows and retrieval precision
2.  **Chunk overlap** is the safety net: prevents information loss at boundaries
3.  **Small chunks** = high precision, low recall (exact answers)
4.  **Large chunks** = low precision, high recall (more context)
5.  **Semantic chunking** preserves conceptual coherence
6.  **Multimodal chunking** works across videos, audio, images, PDFs
7.  **Object decomposition** enables nested chunking pipelines
8.  **25% overlap (128 tokens)** is the sweet spot for 512-token chunks
9.  **67% → 94% accuracy** improvement with proper overlap
10.  **Decompose + Recompose** on query time for maximum flexibility

## What's Next?

With chunking strategies mastered, explore:

*   **Retrieval Ranking**: Scoring and reranking chunks
*   **Metadata Filtering**: Combining chunks with structured filters
*   **Cross-Modal Retrieval**: Searching video chunks with text queries

* * *

## Resources

*   [Mixpeek Object Decomposition Docs](https://mixpeek.com/docs/processing/decomposition)
*   [Video Segmentation Guide](https://mixpeek.com/docs/processing/extractors/multimodal)
*   [PySceneDetect Documentation](https://scenedetect.com/)
*   [SAM2 Model](https://github.com/facebookresearch/segment-anything-2)

## Discussion Questions

*   When would you use semantic chunking over fixed-size chunking?
*   How does chunk overlap impact storage costs vs retrieval accuracy?
*   What chunking strategy would you use for a 3-hour podcast with multiple speakers?
*   How would you chunk a research paper with figures, tables, and equations?

## Frequently Asked Questions

Why is chunking necessary for search?

What is chunk overlap and why does it matter?

How do you chunk videos and audio?

What chunk size should I use?

Previous

Hybrid Search: Best of Both Worlds

K/J

Next

Image Understanding: Vision Encoders & Multimodal Search

Ready to build?

## Try it in Mixpeek Studio

Experience the power of multimodal AI firsthand. Build, test, and deploy your application with our intuitive visual interface.

[Launch Studio](https://studio.mixpeek.com/?utm_source=homepage&utm_medium=education_cta&utm_campaign=mvs_onramp)[View Documentation](https://mixpeek.com/docs)

### Quick Start with API

Get started in seconds with a simple API call

[API Reference](https://mixpeek.com/docs/api-reference)

cURL

```bash
1curl -X POST https://api.mixpeek.com/index \
2  -H "Authorization: Bearer YOUR_API_KEY" \
3  -H "Content-Type: application/json" \
4  -d '{
5    "url": "https://example.com/video.mp4",
6    "collection": "my_collection"
7  }'
```

#### Blog & Tutorials

Read our latest articles, guides, and technical deep dives

[Read Blog →](/blog)

#### Community Hub

Connect with developers, share projects, and get help from experts

[Join Community →](/community)

#### Video Library

Watch curated playlists and step-by-step video tutorials

[Watch Videos →](/videos)