Transformation
Code Execution
Execute custom Python code during retrieval for advanced transformations and logic
Why do anything?
Built-in stages may not cover all use cases. Custom logic requires code execution.
Why now?
Complex retrieval pipelines need extensibility beyond declarative stages.
Why this feature?
Execute sandboxed Python code during retrieval with access to document data and utilities.
How It Works
Code execution enables custom Python logic in retrieval pipelines.
1
Code Validation
Validate code for security
2
Sandbox Setup
Initialize isolated execution environment
3
Execution
Run code with document data
4
Result Collection
Collect outputs and apply to documents
Why This Approach
Sandboxed execution provides flexibility with security. Python is familiar to ML teams.
Where This Is Used
Integration
retriever = client.retrievers.create(stages=[{"type": "code_execution", "code": "..."}])