> ## Documentation Index
> Fetch the complete documentation index at: https://docs.mixpeek.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Agents
> Give AI agents tools to search, ingest, classify, and monitor multimodal content
Mixpeek exposes its entire platform as agent-callable tools. Connect via MCP for zero-code setup, use the built-in Agent Runtime for stateful conversations, or wire retrievers into LangChain, OpenAI, or any framework via REST.
## MCP (Model Context Protocol)
The fastest way to connect an AI agent to Mixpeek. Four hosted servers expose different tool scopes:
| Scope | URL | Tools |
| ------------- | --------------------------------------- | ----- |
| **Full** | `https://mcp.mixpeek.com/mcp` | 48 |
| **Ingestion** | `https://mcp.mixpeek.com/ingestion/mcp` | 20 |
| **Retrieval** | `https://mcp.mixpeek.com/retrieval/mcp` | 11 |
| **Admin** | `https://mcp.mixpeek.com/admin/mcp` | 17 |
```json theme={null}
{
"mcpServers": {
"mixpeek": {
"url": "https://mcp.mixpeek.com/mcp",
"headers": { "Authorization": "Bearer YOUR_API_KEY" }
}
}
}
```
```bash theme={null}
claude mcp add mixpeek \
--transport streamable-http \
--url https://mcp.mixpeek.com/mcp \
--header "Authorization: Bearer YOUR_API_KEY"
```
```json theme={null}
{
"mcpServers": {
"mixpeek": {
"url": "https://mcp.mixpeek.com/mcp",
"headers": { "Authorization": "Bearer YOUR_API_KEY" }
}
}
}
```
### Per-Retriever Server
For a focused search agent, scope the MCP server to a single retriever. It reads your retriever's `input_schema` and generates a typed `search` tool:
```bash theme={null}
pip install mixpeek-mcp-retriever
mixpeek-mcp-retriever \
--retriever-id ret_xxx \
--namespace-id ns_xxx \
--api-key YOUR_API_KEY
```
Exposes three tools: `search` (typed to your schema), `describe` (retriever metadata), and `explain` (pipeline walkthrough).
[Full MCP reference →](/agent-integrations/mcp)
## Agent Sessions
Mixpeek's built-in agent runtime gives you stateful, multi-turn conversations backed by your data. Each session runs as a dedicated process with tool access, conversation memory, and SSE streaming.
```bash theme={null}
# Create a session
curl -X POST "https://api.mixpeek.com/v1/agents/sessions" \
-H "Authorization: Bearer $MIXPEEK_API_KEY" \
-H "X-Namespace: $NAMESPACE_ID" \
-H "Content-Type: application/json" \
-d '{
"agent_config": {
"system_prompt": "You help users search and analyze video content.",
"available_tools": ["execute_retriever", "list_collections", "get_taxonomy"]
}
}'
```
```bash theme={null}
# Send a message (SSE streaming)
curl -N -X POST "https://api.mixpeek.com/v1/agents/sessions/$SESSION_ID/messages" \
-H "Authorization: Bearer $MIXPEEK_API_KEY" \
-H "X-Namespace: $NAMESPACE_ID" \
-H "Content-Type: application/json" \
-d '{ "content": "Find videos about machine learning", "stream": true }'
```
The agent reasons through a `analyze → plan → execute → synthesize` workflow, calling tools as needed and streaming events back:
| Event | Description |
| ------------- | ------------------------------ |
| `thinking` | Agent is analyzing or planning |
| `tool_call` | Agent is calling a tool |
| `tool_result` | Tool execution result |
| `message` | Response text chunk |
| `done` | Processing complete |
### Available Tools
| Tool | Description |
| ------------------- | ----------------------------------------- |
| `execute_retriever` | Search documents via a retriever pipeline |
| `search_retrievers` | Find available retrievers |
| `get_retriever` | Get retriever configuration |
| `list_collections` | List collections in the namespace |
| `get_collection` | Get collection details |
| `list_taxonomies` | List taxonomies |
| `get_taxonomy` | Get taxonomy details |
| `list_clusters` | List cluster configurations |
| `get_object` | Get object metadata |
Sessions persist for 7 days and automatically rehydrate after idle periods.
[Agent Sessions API →](/api-reference/agent-sessions/create-session)
## LangChain
The `langchain-mixpeek` package provides a retriever, individual tools, and a full toolkit:
```bash theme={null}
pip install langchain-mixpeek
```
```python theme={null}
from langchain_mixpeek import MixpeekToolkit
from langgraph.prebuilt import create_react_agent
from langchain_anthropic import ChatAnthropic
toolkit = MixpeekToolkit(
api_key="mxp_...",
namespace="my-namespace",
bucket_id="bkt_...",
collection_id="col_...",
retriever_id="ret_...",
)
agent = create_react_agent(
ChatAnthropic(model="claude-sonnet-4-20250514"),
toolkit.get_tools(),
)
result = agent.invoke({
"messages": [("user", "Find product demos and summarize what's shown")]
})
```
### Toolkit Tools
| Tool | What it does |
| ------------------ | ------------------------------------------------- |
| `mixpeek_search` | Search video, images, audio, documents |
| `mixpeek_ingest` | Upload content (text, images, video, audio, PDFs) |
| `mixpeek_process` | Trigger feature extraction |
| `mixpeek_classify` | Run taxonomy classification |
| `mixpeek_cluster` | Group similar documents |
| `mixpeek_alert` | Set up monitoring (webhook, Slack, email) |
Scope tools to what your agent needs with `toolkit.get_tools(actions=["search", "ingest"])`.
[Full LangChain guide →](/agent-integrations/langchain)
## OpenAI Function Calling
Define a Mixpeek retriever as an OpenAI function schema:
```python theme={null}
from openai import OpenAI
from mixpeek import Mixpeek
openai_client = OpenAI()
mixpeek_client = Mixpeek(api_key="mxp_...")
tools = [{
"type": "function",
"function": {
"name": "search_mixpeek",
"description": "Search video, image, and audio content",
"parameters": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "Search query"},
"limit": {"type": "integer", "description": "Max results"}
},
"required": ["query"]
}
}
}]
response = openai_client.chat.completions.create(
model="gpt-4o", messages=messages, tools=tools
)
# Handle tool_calls by calling mixpeek_client.retrievers.execute()
```
Works with both the Chat Completions API and Assistants API.
[Full OpenAI guide →](/agent-integrations/openai-function-calling)
## Any Framework (REST)
The same pattern works with CrewAI, LlamaIndex, Haystack, Autogen, or plain HTTP — wrap the retriever execute endpoint as a tool:
```python theme={null}
import requests
def search(query: str, limit: int = 10) -> list:
resp = requests.post(
f"https://api.mixpeek.com/v1/retrievers/{RETRIEVER_ID}/execute",
headers={
"Authorization": f"Bearer {API_KEY}",
"X-Namespace": NAMESPACE_ID,
"Content-Type": "application/json",
},
json={"inputs": {"query_text": query}, "limit": limit},
)
return resp.json()["results"]
```
[Retriever Execute API →](/api-reference/retrievers/execute-retriever-auto-optimized)
## Choosing an Integration
| I want to... | Use |
| ------------------------------------- | --------------------------------------------------- |
| Connect Claude or Cursor with no code | [MCP](#mcp-model-context-protocol) |
| Build a stateful conversational agent | [Agent Sessions](#agent-sessions) |
| Build a LangChain/LangGraph agent | [LangChain](#langchain) |
| Add tools to GPT models | [OpenAI Function Calling](#openai-function-calling) |
| Use any other framework | [REST](#any-framework-rest) |