Rag Prepare
Prepare documents for LLM context windows with token management
Configuration
Maximum tokens for combined context output.
Tokenizer for token counting.
How to handle overflow: 'priority_truncate', 'proportional', 'drop_last'.
Output format: 'single_context' (combined) or 'formatted_list' (individual).
Template for formatting each document. Supports {{DOC.*}}, {{CONTEXT.INDEX}}, {{CONTEXT.CITATION}}.
Citation configuration for source tracking.
Output
No output yet
Configure the stage parameters and click "Run Stage" to see the simulated output.
Related to rag_prepare
Pages on the same stages, extractors and patterns.
Json Transform
Transform documents using Jinja2 templates that render to JSON
External Web Search
Enrich documents with web search results using Exa AI
Api Call
Enrich documents with external API calls
Sql Lookup
Enrich documents with SQL query results from external databases
Llm Enrich
Enrich documents with LLM-generated fields using natural language prompts
Taxonomy Enrich
Classify documents against taxonomy nodes via vector similarity