NEWVectors or files. Pick a path.Start →
    Back to DiagramsRetrieval Mechanics

    314 Timestamped Runs, Thirty Actual Sessions: What moment_group Recovers That a Count Cannot

    Mixpeek writes every retriever execution into mxp_retriever_executions in your own namespace, so a retriever can read your own telemetry. Grouped by query_class with a sixty-second tolerance, 314 executions resolve into 30 moments: ten of them a single run, and the largest 24 runs inside 165 seconds.

    A portrait diagram showing the moment_group retriever stage reducing 314 real retriever executions to 30 moments. A bar chart of the seven biggest moments is led by a keyword moment of 24 runs spanning 165 seconds on Sep 3 at 18:45, followed by filter_only moments of 19, 11, 9 and 7 runs and natural_language moments of 11 and 9. A second chart plots all 30 moments by size: ten moments hold a single run, three hold two, three hold three, and single moments hold nineteen and twenty-four. A code panel shows stage_id moment_group with parent_field query_class, time_field occurred_at_ts, merge_tolerance_ms 60000 and score_strategy max, with a comment that occurred_at_ts holds epoch seconds and the stage scales it, so 60000 means sixty seconds. A footer reads 314 documents in, 3 parents, 30 moments, 3.74 ms of stage time.
    Mixpeek writes every retriever execution into mxp_retriever_executions in your own namespace, so a retriever can read your own telemetry. Grouped by query_class with a sixty-second tolerance, 314 executions resolve into 30 moments: ten of them a single run, and the largest 24 runs inside 165 seconds.

    A log tells you when things happened. It cannot tell you which of them were the same sitting. moment_group takes documents that share a parent field, sorts them in time, and merges any two whose gap falls inside merge_tolerance_ms, so a burst of work becomes one row and an isolated run stays one row.

    This is that stage run over 314 real executions from mxp_retriever_executions, grouped by query_class. The shape of the result is the point: ten of the 30 moments hold exactly one run, and the largest holds 24 runs inside 165 seconds. A count of executions cannot tell those two apart. A count of moments can, and the difference is someone working against someone querying once.

    Tolerance is the argument the stage asks you to make, and it has no correct default, because the right value is a claim about how your users work. Widen it and bursts swallow the quiet gaps between them; narrow it and one sitting splits into several. On this corpus the curve happens to be nearly flat, because these executions sit minutes apart: tolerances of 0, 1,000 and 10,000 ms all produce 30 moments and 60,000 produces 29. A corpus with genuine sub-second bursts would show a steep curve. The flat result here is a property of this data rather than a verdict on the parameter.

    One detail on the image is worth reading closely, because it was a real bug. merge_tolerance_ms is compared against the time field's own values, and the scale used to be keyed off the field NAME: *_ms passed through, *_time and *_s and *_seconds scaled by 1000, and anything unrecognised was assumed to be milliseconds already. occurred_at_ts matched neither list, so epoch seconds were read as milliseconds, a tolerance of 60,000 meant 16.7 hours, and these same 314 runs collapsed into 9 moments. Output fields named start_ms and duration_ms carried seconds. Nothing warned. That is fixed: _ts now scales, and a magnitude check logs a warning when a value's size contradicts its name, in both directions and log-only so a false warning can never change a merge. Every number here is from the fixed build.

    Getting to this collection needed a second stage first. mxp_retriever_executions carries vector_indexes: [] and no vectors, so feature_search cannot source from it; attribute_filter can, because as a first stage it fetches directly rather than requiring a prior vector search. Sourcing returned zero documents on three of four attempts, and those runs were discarded as unmeasured rather than counted, so the figures come only from the run that delivered.

    Every number traces to one execution against col_60783f4a99 in ns_cbb394f1a5 with skip_cache true: 314 documents in, 3 parents, 30 moments, 3.74 ms of stage time.

    Where this diagram appears

    Run this on your own data

    Mixpeek turns video, images, audio, and documents in your object storage into searchable, timestamped results through one API.

    Search your own data

    Related diagrams

    A portrait diagram in two rails. The headline reads Filters exclude, what ranking would have shown, with the subline: a pre_filter drops rows before ranking is ever shown to you, useful, and it hides what it removed. A search bar holds the query yellow armchair, annotated: three of the top five are out of stock. Rail A, badged no filter, shows five armchair photos in rank order with prices and scores: 872 dollars at 0.080, 515 dollars at 0.065, 180 dollars at 0.060, 451 dollars at 0.060 and 506 dollars at 0.057; the second, third and fourth are greyed and stamped dropped, and the count chip reads 5 of 5. Rail B, badged pre_filter with in_stock equals true, shows the two survivors at 872 and 506 dollars followed by three new armchairs at 123 dollars 0.048, 446 dollars 0.047 and 606 dollars 0.046; its chip reads 2 kept plus 3 new. A bold line states that three of the top five were out of stock and all three were removed from the response entirely. A dark code window shows the execute request with the filters line in_stock true highlighted, and skip_cache true. The footer prints mxp.co/d/filters-exclude and the words real query, real ranking, mixpeek.

    Filters Exclude What Ranking Would Have Shown

    A portrait diagram in two rails. The headline reads Scope decides, what can be found at all, with the subline: the same question, asked of the whole catalog and then of one category. The search bar holds the query something to keep me warm outside, annotated: the query names no category. Rail A, badged whole catalog with the note semantic reach across every category, shows five photos in rank order: a lit tent at night for 543 dollars at 0.062, a snowy forest scene for 564 dollars at 0.060, a steaming pot on a stove for 444 dollars at 0.059, boots on a textured rug for 203 dollars at 0.052, and a tent under a starry sky for 100 dollars at 0.048; the second, third and fourth are greyed and stamped dropped, and the chip reads 5 of 5. Rail B, badged scoped with category equals tent and the note only one category can answer, shows the two surviving tents followed by three more tents at 592 dollars 0.046, 73 dollars 0.039 and 554 dollars 0.029; its chip reads 2 kept plus 3 new. A bold line states that unscoped, the index answered with tents, backpacks, rugs and pans, and that scope is a decision about what is allowed to be right. A code window shows the execute request with the filters line category tent highlighted. The footer prints mxp.co/d/scope-changes-candidates.

    Scope Decides What Can Be Found at All

    A portrait diagram in two rails. The headline reads A price ceiling, is not a price sort, with the subline: a price ceiling removes the expensive matches, it does not promote the cheap ones by price. The search bar holds the query premium floor lamp, annotated: premium is a word, not a number. Rail A, badged no ceiling with the note best matches, any price, shows five photos in rank order: floor lamps at 729 dollars 0.097, 299 dollars 0.093, 908 dollars 0.092 and 322 dollars 0.089, then an armchair at 451 dollars 0.086; four are greyed and stamped dropped, only the 299 dollar lamp survives, and the chip reads 5 of 5. Rail B, badged pre_filter with price less than 300 and the note best matches under the ceiling, shows the 299 dollar lamp first at 0.093, then lamps at 230 dollars 0.071, 104 dollars 0.068, 129 dollars 0.063 and 38 dollars 0.059; its chip reads 1 kept plus 4 new. A bold line states that rank order inside the filter is still semantic, so the cheapest item lands fourth. A code window shows the execute request with the filters line price lt 300 highlighted. The footer prints mxp.co/d/numeric-filter-reranks.

    A Price Ceiling Cuts the Set and Leaves the Order Semantic