NEWVectors or files. Pick a path.Start →
    Back to DiagramsRetrieval Mechanics

    One Index, Two Ways to Ask: Text Query and Image Query Disagree

    Text query "worn leather armchair" against image embeddings scores 0.08 to 0.11 and returns armchairs. The same index queried with a photograph scores 1.0 on itself, then 0.86, 0.83, and puts a floor lamp at rank four.

    A portrait diagram in two rails. The headline reads One index, two ways to ask, with the subline: one index, one collection, two query modalities, the answers disagree. The search bar holds worn leather armchair, annotated: same corpus, same embeddings. Rail A, badged text query with the note text query against image embeddings, shows five leather armchair photos in rank order at 496 dollars 0.114, 938 dollars 0.106, 489 dollars 0.103, 606 dollars 0.097 and 872 dollars 0.085, chip 5 of 5. Rail B, badged image query with input_mode equals content and the note an image as the query itself, shows five different photos: a white wooden armchair at 446 dollars scoring 1.000, an armchair draped with a blanket at 180 dollars 0.865, a grey chair among plants at 205 dollars 0.834, a pink sofa beside a floor lamp at 104 dollars 0.824, and a grey wooden armchair at 924 dollars 0.824; its chip reads a different five. A bold line states that the image query scores 1.0 on itself, then returns a floor lamp among the armchairs, and that visual similarity answers with pixels while your taxonomy is nowhere in the embedding. A code window shows the execute request with two highlighted input lines, query worn leather armchair for rail A and image_url pointing at a Pexels photo for rail B. The footer prints mxp.co/d/text-vs-image-query.
    Text query "worn leather armchair" against image embeddings scores 0.08 to 0.11 and returns armchairs. The same index queried with a photograph scores 1.0 on itself, then 0.86, 0.83, and puts a floor lamp at rank four.

    One collection of 118 images, one set of SigLIP embeddings, two ways to ask. The answers disagree, and the disagreement is informative.

    The text query worn leather armchair, run against the image embeddings, returns leather armchairs scored between 0.08 and 0.11. Those are typical magnitudes for a cross-modal match, where a sentence is compared to a picture.

    The image query uses a photograph from the corpus as the query itself, with input_mode set to content. The top result is that same photograph at a score of 1.0, which is how you know the visual leg ran against the right index. Then come 0.86 and 0.83, and at rank four, a floor lamp.

    The lamp is the lesson. It sits in a different category and it is there because the photograph looks like the query photograph, in composition, palette and lighting. Visual similarity is a statement about pixels. Your taxonomy is nowhere in the embedding. For a more like this feature, that is often exactly the behaviour you want. For a search box inside a department, it is a result you need a filter to remove.

    The two rails also show why scores across modalities are hard to compare. A 0.11 from a text query and a 0.82 from an image query say nothing about which result is better; they live on different scales. Fuse them with a rank-based method, or keep them as separate legs with separate thresholds.

    What is real here: the photos are Pexels images with every photographer credited, the embeddings and ranking came from the live API with the cache skipped on both calls, and the order is untouched. Titles, prices and categories are demo catalog data generated for this corpus.

    Run this on your own data

    Mixpeek turns video, images, audio, and documents in your object storage into searchable, timestamped results through one API.

    Search your own data

    Related diagrams

    A portrait diagram in two rails. The headline reads Filters exclude, what ranking would have shown, with the subline: a pre_filter drops rows before ranking is ever shown to you, useful, and it hides what it removed. A search bar holds the query yellow armchair, annotated: three of the top five are out of stock. Rail A, badged no filter, shows five armchair photos in rank order with prices and scores: 872 dollars at 0.080, 515 dollars at 0.065, 180 dollars at 0.060, 451 dollars at 0.060 and 506 dollars at 0.057; the second, third and fourth are greyed and stamped dropped, and the count chip reads 5 of 5. Rail B, badged pre_filter with in_stock equals true, shows the two survivors at 872 and 506 dollars followed by three new armchairs at 123 dollars 0.048, 446 dollars 0.047 and 606 dollars 0.046; its chip reads 2 kept plus 3 new. A bold line states that three of the top five were out of stock and all three were removed from the response entirely. A dark code window shows the execute request with the filters line in_stock true highlighted, and skip_cache true. The footer prints mxp.co/d/filters-exclude and the words real query, real ranking, mixpeek.

    Filters Exclude What Ranking Would Have Shown

    A portrait diagram in two rails. The headline reads Scope decides, what can be found at all, with the subline: the same question, asked of the whole catalog and then of one category. The search bar holds the query something to keep me warm outside, annotated: the query names no category. Rail A, badged whole catalog with the note semantic reach across every category, shows five photos in rank order: a lit tent at night for 543 dollars at 0.062, a snowy forest scene for 564 dollars at 0.060, a steaming pot on a stove for 444 dollars at 0.059, boots on a textured rug for 203 dollars at 0.052, and a tent under a starry sky for 100 dollars at 0.048; the second, third and fourth are greyed and stamped dropped, and the chip reads 5 of 5. Rail B, badged scoped with category equals tent and the note only one category can answer, shows the two surviving tents followed by three more tents at 592 dollars 0.046, 73 dollars 0.039 and 554 dollars 0.029; its chip reads 2 kept plus 3 new. A bold line states that unscoped, the index answered with tents, backpacks, rugs and pans, and that scope is a decision about what is allowed to be right. A code window shows the execute request with the filters line category tent highlighted. The footer prints mxp.co/d/scope-changes-candidates.

    Scope Decides What Can Be Found at All

    A portrait diagram in two rails. The headline reads A price ceiling, is not a price sort, with the subline: a price ceiling removes the expensive matches, it does not promote the cheap ones by price. The search bar holds the query premium floor lamp, annotated: premium is a word, not a number. Rail A, badged no ceiling with the note best matches, any price, shows five photos in rank order: floor lamps at 729 dollars 0.097, 299 dollars 0.093, 908 dollars 0.092 and 322 dollars 0.089, then an armchair at 451 dollars 0.086; four are greyed and stamped dropped, only the 299 dollar lamp survives, and the chip reads 5 of 5. Rail B, badged pre_filter with price less than 300 and the note best matches under the ceiling, shows the 299 dollar lamp first at 0.093, then lamps at 230 dollars 0.071, 104 dollars 0.068, 129 dollars 0.063 and 38 dollars 0.059; its chip reads 1 kept plus 4 new. A bold line states that rank order inside the filter is still semantic, so the cheapest item lands fourth. A code window shows the execute request with the filters line price lt 300 highlighted. The footer prints mxp.co/d/numeric-filter-reranks.

    A Price Ceiling Cuts the Set and Leaves the Order Semantic