The 3072-Dimension Problem: Reduce, Then Place

A 3072-dimensional embedding encodes everything about a video and distinguishes nothing. The lighting, the camera angle, the dominant color, who is on screen, the room they are in and the vibe of that room, all of it smeared across three thousand floats. Cosine similarity treats every dimension equally. Your application does not.

That is why a search for a person holding a coffee cup comes back with videos of people, videos of cups, and a surprising number of videos with neither. Reranking helps a little and hybrid BM25 helps a little more, and somewhere around the fourth round of tuning the suspicion sets in that the retriever was never the problem. The unit of work is wrong.

The reframe is old, and representation learning has always done it: reduce complex things to small interpretable representations. What is rare is running it as managed infrastructure, continuously, across millions of objects, with the reduced features exposed as first-class queryable surfaces.

Decomposition is where the dimension reduction actually lives. A shot detector turns a thirty-second video into eight scenes with timestamps. A face identity model turns a frame into bounding boxes and 512-dimension face embeddings. A multimodal extractor turns a scene into an embedding plus OCR output plus dominant colors. Each output becomes its own collection with its own schema and embedding space, which is what makes a feature filterable.

Reduction alone is not enough. Stop after the extractor stage and you have a smarter vector database, still a system where every query starts from scratch. The leverage is the hierarchy: once a taxonomy exists, every new object is placed against it at ingest, the fingerprint is computed once, its location is computed once, and retrieval afterwards is mostly traversal. That collapses enrichment and search into one operation run in two directions, because the taxonomy that locates a new ad is the same retriever a user invokes to ask for ads in that brand.