Chunking Strategies: Breaking Documents into Searchable Pieces

Chunking isn't just splitting text by word count: it's the foundation of retrieval precision. The right chunking strategy determines whether your RAG system returns exact answers or overwhelming noise.

When RAG first emerged, everyone chunked by word, then by sentence, then by paragraph. Only recently have teams started chunking by concept and semantics. This module covers the multimodal approach: chunking by any arbitrary characteristic across videos, images, audio, and documents.

Why Chunk?

Chunking solves two critical problems:

1. Context Window Limits

Encoder models have limited context windows:

• Small models: 512 tokens
• Standard models: 8,000 tokens
• Large models: 128,000+ tokens (but still finite)

Problem without chunking:

Solution with chunking:

2. Retrieval Precision

LLMs are getting better at reasoning, but reasoning is limited to the content you give them. Overwhelming an LLM with irrelevant data degrades output quality.

Key Insight: The more precise your chunks, the more relevant your LLM's responses.

Chunking Strategies Overview

Chunking isn't limited to text. You can chunk any modality by any characteristic:

Let's dive into each strategy.

Text & PDF Chunking Strategies

1. Fixed-Size Chunking

Definition: Split text every N tokens, regardless of content structure.

Best for:

• Simple use cases with consistent content
• When you need predictable chunk sizes

Example:

chunk_size = 512  # tokens
chunk_overlap = 0

# Document: "The quick brown fox jumps over the lazy dog. The dog..."
# Chunk 1: "The quick brown fox jumps over the lazy dog. The dog was..."
# Chunk 2: "...sleeping under a tree. The tree provided shade..."

Pros:

• Simple to implement
• Predictable chunk sizes

Cons:

• Cuts mid-sentence or mid-thought
• Loses context at boundaries
• No semantic awareness

2. Fixed-Size with Overlap

Definition: Split every N tokens but include overlap from previous chunk.

Best for:

• Preventing information loss at boundaries
• Maintaining context across chunks

Example:

chunk_size = 512  # tokens
chunk_overlap = 128  # tokens

# Chunk 1: "The quick brown fox [...]" (tokens 0-512)
# Chunk 2: "fox jumps over [...] sleeping" (tokens 384-896)
#          ^^^ 128 token overlap ^^^

Pros:

• Safety net against boundary loss
• Maintains context flow
• 67% → 94% retrieval accuracy improvement

Cons:

• Slight storage overhead
• Duplicate content across chunks

3. Sentence-Based Chunking

Definition: Split at sentence boundaries (periods, question marks, exclamation points).

Best for:

• Capturing complete thoughts
• Natural language processing

Example:

# Split on periods
chunk_delimiter = "."

# Chunk 1: "The quick brown fox jumps over the lazy dog."
# Chunk 2: "The dog was sleeping under a tree."
# Chunk 3: "The tree provided shade for hours."

Pros:

• Preserves complete thoughts
• Natural chunk boundaries

Cons:

• Variable chunk sizes
• May create very small or very large chunks

4. Semantic-Based Chunking

Definition: Use embedding similarity to detect conceptual boundaries.

How it works:

1. Encode each sentence individually
2. Measure cosine similarity between consecutive sentences
3. When similarity drops below threshold → chunk boundary

Example:

# Encode sentences
sentence_1 = "Ethan likes dogs."
sentence_2 = "Dogs like Ethan."
sentence_3 = "The weather is sunny today."

# Measure similarity
similarity_1_2 = cosine_similarity(embed(sentence_1), embed(sentence_2))  # 0.89
similarity_2_3 = cosine_similarity(embed(sentence_2), embed(sentence_3))  # 0.12

# Chunk boundary at 2→3 (similarity < 0.5)
# Chunk 1: "Ethan likes dogs. Dogs like Ethan."
# Chunk 2: "The weather is sunny today."

Best for:

• Capturing conceptual coherence
• Complex documents with topic shifts

Pros:

• Maintains semantic coherence
• Topic-aligned chunks

Cons:

• Computationally expensive
• Requires encoder model

5. Layout-Based Chunking (PDFs)

Definition: Chunk by structural elements of the document.

Chunk by:

• Headers/Footers: Separate metadata from content
• Tables: Extract structured data
• Figures/Charts: Isolate visual elements
• Columns: Handle multi-column layouts

Best for:

• Structured documents (contracts, reports, research papers)
• Preserving document hierarchy

Example:

# PDF with headers, body, tables
# Chunk 1: Header ("Merger Agreement - Section 3")
# Chunk 2: Body paragraph (indemnification clause)
# Chunk 3: Table (liability limits)
# Chunk 4: Footer (page number, date)

Video Chunking Strategies

1. Time Interval Chunking

Definition: Split video every N seconds.

Example:

chunk_interval = 30  # seconds

# 10-minute video → 20 chunks (30 seconds each)

Best for:

• Large videos that need initial segmentation
• Predictable processing pipelines

Pros:

• Simple, fast
• Consistent chunk durations

Cons:

• No semantic awareness
• Cuts mid-scene or mid-action

2. Scene Detection Chunking

Definition: Detect scene transitions using pixel differences between frames.

Tools: PySceneDetect, AutoShot

How it works:

# Measure pixel differences between consecutive frames
frame_1 = video[t=0]
frame_2 = video[t=1]

pixel_diff = abs(frame_2 - frame_1)

if pixel_diff > threshold:
    # New scene detected
    create_chunk()

Best for:

• Content with clear scene transitions (movies, ads, tutorials)
• Semantic video segmentation

Pros:

• Semantically meaningful chunks
• Captures complete scenes

Cons:

• Variable chunk sizes
• May miss subtle transitions

3. Shot Boundary Detection

Definition: Detect camera position changes (cuts, pans, zooms).

Best for:

• Professional video content
• Fine-grained segmentation

4. Keyframe Sampling

Definition: Extract representative frames at intervals or based on content change.

Example:

# Extract keyframes every 3 seconds
keyframe_interval = 3  # seconds

# Or: Extract frames with significant visual change
visual_change_threshold = 0.7

Best for:

• Image-based retrieval
• Video thumbnails

5. Object-Based Chunking (SAM2)

Definition: Use Segment Anything Model 2 (SAM2) to detect objects and chunk by object presence.

Example:

# Detect all scenes containing "dog" object
object = "dog"
chunks = video.chunk_by_object(object)

# Result: All video segments where dog appears

Best for:

• Object-centric retrieval
• Content analysis (e.g., all scenes with a product)

Audio Chunking Strategies

1. Fixed Duration Chunking

Definition: Split audio every N seconds.

Example:

chunk_duration = 30  # seconds

Best for:

• Consistent processing
• Simple use cases

2. Speaker Diarization

Definition: Chunk by unique speaker voices.

How it works:

# Audio: "Speaker A: Hello. Speaker B: Hi there. Speaker A: How are you?"
# Chunk 1: "Speaker A: Hello."
# Chunk 2: "Speaker B: Hi there."
# Chunk 3: "Speaker A: How are you?"

Best for:

• Meeting transcriptions
• Podcast analysis
• Multi-speaker content

Pros:

• Speaker-specific chunks
• Preserves speaker context

Cons:

• Requires diarization model
• Variable chunk sizes

3. Silence-Based Chunking

Definition: Detect silence intervals and chunk at natural pauses.

Example:

silence_threshold = -40  # dB
min_silence_duration = 1.0  # seconds

# Chunk at pauses > 1 second

Best for:

• Natural speech boundaries
• Audio with clear pauses

4. Activity Detection Chunking

Definition: Chunk based on audio activity (speech vs music vs silence).

Best for:

• Mixed audio content
• Content-aware segmentation

Chunk Overlap: The Safety Net

Chunk overlap is critical for preventing information loss at chunk boundaries.

Why Overlap Matters

Without overlap:

With 128-token overlap:

Overlap Percentages

Recommended: 25% overlap (128 tokens for 512-token chunks)

Overlap Example

chunk_size = 512
chunk_overlap = 128  # 25%

text = "The quick brown fox jumps over the lazy dog. " * 100

chunks = chunk_text(text, chunk_size, chunk_overlap)

# Chunk 1: tokens 0-512
# Chunk 2: tokens 384-896 (128 token overlap from Chunk 1)
# Chunk 3: tokens 768-1280 (128 token overlap from Chunk 2)

Chunk Size Trade-Offs

Chunk size directly impacts precision and recall:

Small Chunks (256-512 tokens)

Precision: ⭐⭐⭐⭐⭐ High Recall: ⭐⭐ Low Context: ⭐⭐ Low

Use when:

• You need exact answers
• Content is dense with distinct concepts
• Storage is not a concern

Example:

Medium Chunks (512-1024 tokens)

Precision: ⭐⭐⭐⭐ Moderate-High Recall: ⭐⭐⭐⭐ Moderate-High Context: ⭐⭐⭐⭐ Moderate-High

Use when:

• Balanced retrieval needs
• General-purpose RAG
• Most common choice

Large Chunks (1024-2048 tokens)

Precision: ⭐⭐⭐ Moderate Recall: ⭐⭐⭐⭐⭐ High Context: ⭐⭐⭐⭐⭐ High

Use when:

• Context preservation is critical
• Complex reasoning required
• Willing to trade precision for recall

Real-World Example: Legal Contract Analysis

Scenario: 200-page merger agreement (42,000 words)

Query: "indemnification clause"

Approach 1: No Chunking

# Upload entire document as single chunk
chunk_count = 1
chunk_size = 42000  # words

# Query result: Entire 200-page document
precision = 31%  # Too broad
recall = 100%  # Found everything, but unusable

Problem: LLM receives entire document, can't focus on relevant section.

Approach 2: Fixed 512 Tokens, No Overlap

chunk_size = 512  # tokens
chunk_overlap = 0

# Result: 164 chunks
# Query finds clause but cuts mid-sentence
precision = 67%  # Better
recall = 82%  # Misses boundary context

Problem: Clause split across chunks, missing critical context.

Approach 3: Fixed 512 Tokens, 128 Overlap ✅

chunk_size = 512  # tokens
chunk_overlap = 128  # 25%

# Result: 164 chunks with overlap
# Query finds exact clause with surrounding context
precision = 94%  # Excellent
recall = 94%  # Complete information

Success: Perfect retrieval, exact clause with context maintained across boundaries.

The Impact of Overlap

Key Takeaway: Overlap is the safety net that prevents losing information at chunk boundaries.

Building Chunking Pipelines in Mixpeek

Mixpeek enables multimodal chunking through object decomposition flows.

Object Decomposition Architecture

Think of object decomposition as nested chunking across modalities:

Each level is an independent collection that can be queried separately or together.

Creating a Multi-Tier Chunking Pipeline

Step 1: Create Collection with Time Segmentation

from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

# Tier 1: Chunk by time (every 30 seconds)
time_collection = client.collections.create(
    collection_name="time-segments",
    source_bucket="raw-videos",
    feature_extractors=[{
        "type": "multimodal",
        "split_method": "time",
        "split_interval": 30  # seconds
    }]
)

Step 2: Add Scene Segmentation

# Tier 2: Chunk by scene (using time segments as input)
scene_collection = client.collections.create(
    collection_name="scene-segments",
    source_collection="time-segments",  # Use previous tier
    feature_extractors=[{
        "type": "multimodal",
        "split_method": "scene",
        "detection_threshold": 0.5  # Scene change sensitivity
    }]
)

Step 3: Add Silence Segmentation (Parallel)

# Tier 2 (Parallel): Chunk by silence
silence_collection = client.collections.create(
    collection_name="silence-segments",
    source_collection="time-segments",
    feature_extractors=[{
        "type": "multimodal",
        "split_method": "silence",
        "silence_threshold": -40,  # dB
        "enable_transcription": True
    }]
)

Multi-Collection Retrieval

Query across all chunk tiers simultaneously:

# Create retriever spanning multiple collections
retriever = client.retrievers.create(
    retriever_name="multi-tier-search",
    collections=[
        "time-segments",
        "scene-segments",
        "silence-segments"
    ],
    stages=[
        {
            "type": "feature_search",
            "feature_uri": "multimodal-extractor-v1",
            "limit": 20
        }
    ],
    fusion={
        "strategy": "reciprocal_rank_fusion"
    }
)

# Execute query
results = retriever.execute(
    inputs={"text": "product demo with Sarah"}
)

# Returns:
# - Time segments containing "product demo"
# - Scene segments with "Sarah" (face detection)
# - Speaker segments with Sarah's voice

The Power of Decomposition + Recomposition:

• Decompose: Chunk objects into granular pieces
• Recompose: Query across all chunks on-demand

This is the multimodal way to build precise retrieval systems.

Chunking Configuration in Mixpeek

Text/PDF Chunking

collection = client.collections.create(
    collection_name="documents",
    feature_extractors=[{
        "type": "text",
        "chunk_size": 512,
        "chunk_overlap": 128,
        "chunk_method": "sentence"  # or "fixed", "semantic"
    }]
)

Video Chunking

collection = client.collections.create(
    collection_name="videos",
    feature_extractors=[{
        "type": "multimodal",
        "split_method": "scene",
        "detection_threshold": 0.5,
        "enable_transcription": True,
        "enable_embedding": True
    }]
)

Audio Chunking

collection = client.collections.create(
    collection_name="audio",
    feature_extractors=[{
        "type": "multimodal",
        "split_method": "speaker",  # Diarization
        "enable_transcription": True
    }]
)

Key Takeaways

1. Chunking is essential for encoder context windows and retrieval precision
2. Chunk overlap is the safety net: prevents information loss at boundaries
3. Small chunks = high precision, low recall (exact answers)
4. Large chunks = low precision, high recall (more context)
5. Semantic chunking preserves conceptual coherence
6. Multimodal chunking works across videos, audio, images, PDFs
7. Object decomposition enables nested chunking pipelines
8. 25% overlap (128 tokens) is the sweet spot for 512-token chunks
9. 67% → 94% accuracy improvement with proper overlap
10. Decompose + Recompose on query time for maximum flexibility

What's Next?

With chunking strategies mastered, explore:

• Retrieval Ranking: Scoring and reranking chunks
• Metadata Filtering: Combining chunks with structured filters
• Cross-Modal Retrieval: Searching video chunks with text queries

Resources

• Mixpeek Object Decomposition Docs
• Video Segmentation Guide
• PySceneDetect Documentation
• SAM2 Model

Discussion Questions

• When would you use semantic chunking over fixed-size chunking?
• How does chunk overlap impact storage costs vs retrieval accuracy?
• What chunking strategy would you use for a 3-hour podcast with multiple speakers?
• How would you chunk a research paper with figures, tables, and equations?