MixedEmbeddingsConverter
Generate unified vector embeddings from mixed-modality inputs -- text, images, audio, and video combined. Enables cross-modal search where any modality can query any other modality in a single vector space.
How It Works
Provide one or more inputs of any modality.
Each input is processed through its modality-specific encoder.
Modality embeddings are projected into a shared vector space.
A fused embedding is produced that represents the combined input.
The unified embedding enables cross-modal similarity search.
Code Examples
import os, requests
API = "https://api.mixpeek.com"
H = {"Authorization": f"Bearer {os.environ['MIXPEEK_API_KEY']}",
"X-Namespace": os.environ["NAMESPACE_ID"]}
# 1. a bucket, with a schema that declares the field you will send
bucket = requests.post(f"{API}/v1/buckets", headers=H, json={
"bucket_name": "multimodal-inputs",
"bucket_schema": {"properties": {"multimodal": {"type": "video"}}},
}).json()
# 2. land the file as an object. the URL goes in data, on the blob
requests.post(f"{API}/v1/buckets/{bucket['bucket_id']}/objects", headers=H, json={
"key_prefix": "run-1",
"blobs": [{"property": "multimodal", "type": "video",
"data": "https://example.com/clip.mp4"}],
})
# 3. a collection over that bucket, running the extractor
collection = requests.post(f"{API}/v1/collections", headers=H, json={
"collection_name": "multimodal-to-embeddings",
"source": {"type": "bucket", "bucket_ids": [bucket["bucket_id"]]},
"feature_extractor": {"feature_extractor_name": "universal_extractor", "version": "v1"},
}).json()
# 4. run extraction over the bucket
requests.post(f"{API}/v1/buckets/{bucket['bucket_id']}/batches", headers=H, json={
"collection_ids": [collection["collection_id"]],
"auto_submit": True,
})
# 5. read the output
docs = requests.get(
f"{API}/v1/collections/{collection['collection_id']}/documents", headers=H
).json()
print(docs)Use Cases
Supported Input Formats
Quick Info
Run it over a library
Mixpeek runs this conversion as a pipeline over a whole library in your object storage, with the output landing as queryable documents. It is not a single-file converter.
Frequently Asked Questions
Related Converters
Video to Embeddings
Generate dense vector embeddings for video content using multimodal models. Embeddings capture visual, audio, and temporal features, enabling semantic search and similarity matching across video collections.
Image to Embeddings
Convert images into dense vector representations using state-of-the-art vision models. Embeddings capture semantic visual features and can be used for similarity search, clustering, and cross-modal retrieval.
Audio to Embeddings
Convert audio files into dense vector embeddings that capture spoken content, tone, and acoustic features. Use embeddings for audio search, speaker verification, and content-based recommendation.
Text to Embeddings
Convert text strings, paragraphs, or documents into dense vector embeddings using state-of-the-art language models. Supports batching, chunking, and multiple model options for optimal retrieval performance.
Ready to convert mixed to embeddings?
Start using the Mixpeek Multimodal to Embeddings in minutes. Sign up for a free API key and follow the documentation to get started.