Enhanced

Multimodal RAG Pipeline

Build a retrieval-augmented generation system that works with text, images, and video. Feed relevant multimodal context to LLMs for grounded responses.

text

image

video

Multi-Tier

4.9K runs

Run in Builder

from mixpeek import Mixpeek
import openai

client = Mixpeek(api_key="YOUR_API_KEY")

# 1. Build knowledge base
namespace = client.namespaces.create(name="rag-kb")
collection = client.collections.create(
    namespace_id=namespace.id,
    name="docs-and-media",
    extractors=["text-embedding-v2", "image-embedding-v2"],
    chunk_strategy="semantic"
)

# 2. Ingest your content
client.buckets.upload(
    collection_id=collection.id,
    url="s3://your-bucket/knowledge-base/"
)

# 3. Retrieve + Generate
def rag_query(question: str):
    # Retrieve relevant context
    results = client.retrievers.execute(
        retriever_id=retriever.id,
        query=question,
        settings={"limit": 5}
    )

    # Build context from results
    context = "\n".join([r.content for r in results])

    # Generate answer with LLM
    response = openai.chat.completions.create(
        model="gpt-4",
        messages=[
            {"role": "system", "content": f"Answer based on this context:\n{context}"},
            {"role": "user", "content": question}
        ]
    )
    return response.choices[0].message.content

Feature Extractors

Retriever Stages

Related Recipes & Resources

Explore these related resources to deepen your understanding and discover more powerful features

Recipe

Taxonomy Enrichment Pipeline

Automatically classify and tag content using custom taxonomies. Map your content to IAB categories, custom hierarchies, or industry-specific classifications.

Learn more

Recipe

Content Clustering Pipeline

Automatically group similar content together using embedding-based clustering. Discover themes, identify duplicates, and organize large content libraries.

Learn more

Recipe

Metadata Enrichment Pipeline

Automatically enrich your data with extracted metadata: entities, topics, sentiment, language, and custom attributes. Transform raw content into structured, queryable data.

Learn more

Recipe

Multimodal Hybrid Search Pipeline

Combine vector search with keyword search (BM25) across text, images, and video for the most comprehensive multimodal retrieval system.

Learn more

Recipe

Clinical Documentation Structuring

Production-grade pipeline for ingesting clinical documents, scanned charts, EHR exports, wound photos, and therapy notes, and structuring them into coded fields aligned with MDS 3.0, PDPM, and CMS audit requirements. Combines OCR, clinical NER, taxonomy classification, and hybrid retrieval to turn unstructured bedside documentation into queryable, auditable data.

Learn more

Recipe

Multimodal Content Moderation

Automated content moderation pipeline that analyzes text, images, and video for policy violations. Uses hierarchical taxonomy classification to label content as safe, sensitive, or prohibited across multiple categories simultaneously.

Learn more