Retrieval

8 min read

Updated 2026-09-04

# How Do You Search a 3D World You Generated?

World models hand back a scene, not a file you can watch, and the export API gives you geometry rather than pixels. This is what World Labs' Marble API actually exports, why none of it is searchable by content, and the one artifact that is.

![Ethan Steininger](/people/ethan.webp)

By [Ethan Steininger](/blog/authors/ethan), Founder & CEO, Mixpeek

World Models

Marble

Atlas

Gaussian Splats

3D

Video Search

Retrieval

[Run this on your own data. Try it as you read.Search your own archive →](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=guide_cta&utm_campaign=mvs_onramp)

[One-click templatesStart from a templateEach template deploys a working search app on your own files.Browse templates](/templates)

## The Short Answer

  
  
You search the renders, not the world. A generated 3D scene is stored as Gaussian splats or a mesh, and no retrieval system reads those by content, so the searchable artifact is the video or images you render out of the scene. Index those, and carry the scene id and the splat URI in payload so a hit on a render hands you back the world it came from.  
  
The catch worth knowing before you plan a pipeline: World Labs' World API exports `splats` or `mesh` and nothing else. The flythrough video is produced by the Record tool in the Marble app, not by the API. So the automated path stops one step short of the thing you need to index.  
  

## Why this is different from searching generated video

  
  
A text-to-video model hands you one artifact and you search it. A world model hands you a scene, and a scene is not something anyone can watch.  
  
What you can watch is a camera path through it, chosen by whoever ran the job. So your index sees excerpts rather than content, and the excerpts are arbitrary: two paths through the same reconstruction produce two files that look unrelated while sharing every underlying particle. A courtyard nobody pointed a camera at is absent from search no matter how well the model reconstructed it.  
  
That is a coverage problem, not a relevance problem, and no amount of better embeddings fixes it. It is fixed at generation time by rendering more than one path.  
  

## What the export API actually gives you

  
  
Read from the World Labs API reference on 2026-09-02. `POST /marble/v1/worlds/{world_id}:export` takes an `ExportWorldRequest`:  
  

Field

Values

Note

`asset_type`

`splats`, `mesh`

required, and this is the whole list

`format`

`ply`, `glb`

required

`resolution`

`full_res`, `500k`, `150k`, `100k`

splat density when exporting PLY

`mesh_variant`

`textured`, `vertex_colored`

when an HQ mesh already exists

  
  
PLY splat exports convert synchronously and come back as completed operations. HQ mesh exports are async and return an in-progress operation you poll at `/marble/v1/operations/{operation_id}`.  
  
There is no video in that enum. Marble does produce flythrough video, through the Record tool in the app, and that is a UI action rather than an API call. So today the generate-to-searchable path is: generate through the API, render in the app, then index the render.  
  
Atlas, announced 2026-09-01, is a different situation again: it is in early access with select partners and its announcement documents no output endpoint at all, so its renders reach you however your partner arrangement delivers them.  
  

## What is searchable, and what is only reachable

  
  

Artifact

Searchable by content

How you reach it

Rendered video

Yes

multimodal extractor: frames, audio, on-screen text

Rendered stills, panoramas

Yes

image extractor

Depth panoramas

As images

matches silhouette and layout, never distance

PLY Gaussian splats

No

store the file, put its URI in payload

GLB meshes

No

same

  
  
`GET /v1/discovery/extractors` reports six input types: image, video, audio, pdf, text and string. No shipping extractor reads a PLY, a GLB or a point cloud. That is a statement about today, and it is the reason the pattern below puts geometry in payload rather than pretending to index it.  
  

## The pattern

\# 1. bucket: land the render, wherever it lives
POST /v1/buckets/{bucket\_identifier}/objects
{"blobs":\[{"property":"video","type":"video",
           "data":"https://your-storage/worlds/scene-0417-path-a.mp4",
           "metadata":{"world\_id":"wrl\_0417",
                       "camera\_path":"path-a",
                       "splat\_ply":"s3://worlds/wrl\_0417/full\_res.ply",
                       "mesh\_glb":"s3://worlds/wrl\_0417/textured.glb"}}\]}

# 2. collection: extraction runs on arrival
POST /v1/collections
{"collection\_name":"worlds",
 "source":{"type":"bucket","bucket\_ids":\["{bucket\_identifier}"\]},
 "feature\_extractor":{"feature\_extractor\_name":"multimodal\_extractor","version":"v1"}}

# 3. retriever: ask in plain language
POST /v1/retrievers/{retriever\_id}/execute
{"inputs":{"query":"stone archway with light coming through from the left"}}

`data` takes an HTTP, HTTPS or S3 URL and Mixpeek fetches it, so the render never passes through your machine. The metadata is promoted onto the object at ingest and is filterable afterwards, which is what makes `world_id` a join back to every other render of the same scene and to the geometry nobody can search.  
  

## Render for coverage, not for the shot

  
  
The instinct from video is to render the best take. The instinct that works for retrieval is to render several ordinary ones.  
  
Two or three paths at different heights turn a scene from findable-from-one-angle into findable. It costs render time and one extraction each, and it is the only lever that moves coverage, because everything downstream can only match what a camera saw.  
  
Deduplicate before extracting if the paths overlap heavily. Near-identical passes over the same geometry are the easiest extraction cost to avoid, and it is a cheaper problem to solve than the coverage one.  
  

## Frequently Asked Questions

  
  

### Can I search a Gaussian splat or a mesh directly?

  
  
No, and not in a way any current vector database changes. `GET /v1/discovery/extractors` lists six input types and none of them reads a PLY, a GLB or a point cloud. Retrieval over geometry is a different problem from retrieval over appearance, and the working pattern today is to index the renders and carry the geometry's URI in payload so a semantic hit hands you the file.  
  

### Does the World Labs API export video?

  
  
No. `ExportWorldRequest` takes `asset_type` of `splats` or `mesh` and `format` of `ply` or `glb`. Flythrough video comes from the Record tool inside the Marble app, which is a UI action. Plan for a manual or scripted render step between generation and indexing, and do not design a fully automated pipeline around a video export endpoint that does not exist.  
  

### How do I search a part of the world nobody rendered?

  
  
You cannot, and this is the failure people hit second. The index only ever sees what a camera path passed through, so an unrendered corner is absent regardless of reconstruction quality. Fix it at generation time with two or three paths per scene rather than hoping better retrieval compensates. Nothing downstream can recover content that was never rendered.  
  

### Are depth panoramas worth indexing?

  
  
Only for shape. A depth panorama is an image, so it ingests and its embedding matches on silhouette and layout, which is genuinely useful for "scenes arranged like this one". It cannot answer "objects within two metres of the camera", because the extractor reads brightness values and has no idea they encode distance.  
  

### What does a query match on for a rendered world?

  
  
The features extracted from the rendered frames: segment-level and frame-level representations plus speech and on-screen text. Not the world id, not the prompt, not the filename, unless you attached those as metadata, and metadata is a filter rather than a semantic match. That is why a description of an archway finds a file called scene-0417-path-a.mp4.  
  

### How much does the generation side cost?

  
  
World Labs prices the API in credits at \\$1.00 per 1,250 credits, minimum purchase 6,250 credits, and the credits do not expire. One gotcha their docs call out in a warning: API credits are bought on the World Labs Platform and credits bought inside the Marble app cannot be used with the API. The indexing side is separate and is paid once per object, so the cost that grows is the number of renders rather than their length.  
  

### Does any of this need a World Labs integration on the Mixpeek side?

  
  
No, and there is not one. Mixpeek has 20 storage connectors and none of them is World Labs. The handoff is a URL: put the render wherever you already keep files and pass that URL in `data`. That is deliberately boring, and it is why the same three calls work for Marble, Atlas, and whatever ships next.  
  

## Related

  
  
*   [Search every world you generate with Atlas](/guides/search-every-world-you-generate-with-atlas) for the Atlas-specific version of this flow
*   [Search every world you generate with Runway](/guides/search-every-world-you-generate-with-runway) for a world generated live, where only the recording survives
*   [Search everything you generate on fal](/guides/search-everything-you-generate-on-fal) for the same three calls against a text-to-video model
*   [Search every image you generate with ChatGPT](/guides/search-every-image-you-generate-with-chatgpt) for the same export-then-index pattern applied to Images 2.5
*   [Search every conversation you have with GPT Live](/guides/search-every-conversation-you-have-with-gpt-live) for the same capture-then-index pattern applied to a live full-duplex voice session
*   [Monocular depth estimation](/guides/monocular-depth-estimation-3d-from-one-image) for what a depth map does and does not encode
*   [Video scene segmentation](/guides/video-scene-segmentation) for how a render is cut into the units a query matches

## Run this on your own data

Point Mixpeek at the storage you already have and search your video, images, audio, and documents the way this guide describes. Build starts at $25/mo for up to 1M vectors.

[Search your own archive](https://studio.mixpeek.com/namespaces/create?mode=managed&utm_source=homepage&utm_medium=guide_cta&utm_campaign=mvs_onramp)[Read Docs](/docs)

[Explore MVS](/mvs)[See pricing](/pricing)

## Related guides

[

Retrieval

### How Do I Search Video or Audio for a Sound, Like Glass Breaking or Applause?

Embed the audio in short overlapping windows with a text-audio model such as CLAP and search them with a description of the sound. A runnable script found all seven sounds in a 35-second track of labeled clips from plain-text queries, with why thresholds must be per query and how window length sets timing.

Read guide →](/guides/how-to-search-video-or-audio-for-a-sound)[

Retrieval

### What Is Hybrid Search? BM25, Vector Retrieval, and How to Fuse Their Rankings

Why keyword and vector retrieval fail on opposite queries, what BM25 actually computes, and how Reciprocal Rank Fusion combines two rankings whose scores are not on the same scale. Covers the term-frequency saturation and length normalization inside BM25, why raw score addition breaks, RRF versus min-max and convex combination, how to pick the weighting, when hybrid is worse than either half, and what changes when one side of the index is video or images.

Read guide →](/guides/hybrid-search-bm25-vector-fusion)[

Retrieval

### Should a RAG Pipeline Use a Classification Model or an LLM to Filter and Rerank Results?

Use a classification model for any filter or rerank step whose answer is a choice from a list: keep or drop, a label, a relevance score. Keep a generative LLM for steps that write new text. This compares cross-encoder rerankers, decision models such as Jev and Liquid AI d1, and LLM judges on cost, speed and how far you can trust each score.

Read guide →](/guides/decision-models-in-retrieval-pipelines)