Keyword Search vs Semantic Search vs Hybrid Search: A Developer's Guide

Choosing the right search strategy is one of the most consequential technical decisions in any application that deals with content retrieval. The three main approaches — keyword search, semantic search, and hybrid search — each have distinct strengths, and understanding their tradeoffs is essential for building effective search experiences.

Keyword Search (Lexical Search)

Keyword search, also called lexical or full-text search, finds documents containing the exact terms in a query. Algorithms like BM25 and TF-IDF score documents based on term frequency, document frequency, and field length normalization.

How It Works

When a document is indexed, it is tokenized into individual terms. An inverted index maps each term to the documents containing it. At query time, the query is tokenized the same way, and the inverted index is used to quickly find matching documents. BM25 then scores each match based on how often the term appears in the document relative to how common it is across all documents.

Strengths

• Precision for exact matches — Searching for "SKU-12345" or "error code 0x80070005" returns exactly what you need
• Fast and well-understood — Inverted indices are mature technology with sub-millisecond latency at scale
• No ML infrastructure required — Works with Elasticsearch, PostgreSQL full-text, or SQLite FTS out of the box
• Transparent ranking — You can explain why a result appeared and tune the ranking directly

Weaknesses

• Vocabulary mismatch — "car" will not match "automobile" or "vehicle"
• No conceptual understanding — Cannot find documents about a topic if different words are used
• Single modality — Only works with text; cannot search images or video by description

Semantic Search (Vector Search)

Semantic search uses embedding models to convert queries and documents into vectors that capture meaning. Similar concepts produce similar vectors, enabling retrieval based on semantic similarity rather than keyword overlap.

How It Works

An embedding model (like E5, BGE, or CLIP) encodes text into high-dimensional vectors (typically 768-1536 dimensions). These vectors are stored in a vector database with an approximate nearest neighbor (ANN) index. At query time, the query is encoded into the same vector space, and the ANN index efficiently finds the closest stored vectors using cosine similarity or dot product.

Strengths

• Understands meaning — "car" matches "automobile", "vehicle", and "Tesla Model 3"
• Cross-modal capability — Text queries can find images, videos, and audio
• Handles natural language — Questions like "how do I fix a leaking faucet" work naturally
• Multilingual — Multilingual embedding models can match content across languages

Weaknesses

• Requires ML infrastructure — Embedding models need GPU compute for generation
• Less precise for exact matches — May rank conceptually similar but irrelevant results above exact matches
• Opaque ranking — Harder to explain why specific results appeared
• Higher latency — Embedding generation adds 10-50ms per query

Hybrid Search

Hybrid search combines keyword and semantic approaches, using both lexical matching and vector similarity to produce the final ranking. This is the approach most production systems should use.

How It Works

Both a keyword search and a vector search run in parallel on the same query. The results from each are normalized to a common score range, then combined using a fusion algorithm. The most common fusion methods are:

• Reciprocal Rank Fusion (RRF) — Combines rankings by summing reciprocal ranks. Simple and effective.
• Weighted linear combination — Applies configurable weights (e.g., 0.7 semantic + 0.3 keyword) to normalized scores.
• Conditional routing — Uses keyword search for queries that look like exact matches (SKUs, codes) and semantic search for natural language.

from mixpeek import Mixpeek

client = Mixpeek(api_key="your-api-key")

# Create a hybrid retriever
retriever = client.retrievers.create(
    name="hybrid_search",
    collection_id="my-collection",
    stages=[
        {
            "type": "vector_search",
            "model": "text-embedding",
            "top_k": 50,
            "weight": 0.7
        },
        {
            "type": "keyword_search",
            "fields": ["title", "content"],
            "top_k": 50,
            "weight": 0.3
        },
        {
            "type": "sort",
            "method": "rrf"
        }
    ]
)

When to Use Each Approach

Key Takeaway

Start with hybrid search. It gives you the precision of keyword matching and the recall of semantic understanding. Tune the weights based on your data — if your users search with exact terms, lean toward keyword; if they use natural language, lean toward semantic. And if you need cross-modal search (text to image, text to video), semantic search is the only option.

See our FAQ on keyword vs semantic search for a quick summary, or dive into the multimodal search glossary entry for more on cross-modal retrieval.