Reduce
Deduplicate
Remove duplicate documents by field match or content similarity
Note: This playground provides simulated output to showcase functionality. No data is processed on our servers. Use this demo to explore the stage's configuration options before integrating it into your retriever pipeline.
Configuration
string
Field to deduplicate on.
string
Dedup strategy: 'exact' (field match) or 'similarity' (embedding-based).
number
Similarity threshold for 'similarity' strategy.
Output
No output yet
Configure the stage parameters and click "Run Stage" to see the simulated output.