deplot
by google
Turns a chart image into a table you can actually search
google/deplotDeploy deplot
Single-tenantMixpeek has no managed extractor for this model. On a single-tenant deployment you upload the weights and a custom plugin serves them next to the rest of your pipeline.
Overview
DePlot reads a chart or plot image and emits a linearised data table. That is a narrow job and a useful one: a number that exists only as a bar in a PNG is invisible to every text index you own, so the answer sits in your archive and no search returns it. Converting the chart to a table makes the underlying values ordinary text, which can then be embedded, filtered and cited like anything else.
Architecture
Pix2Struct encoder-decoder, 768-dimensional hidden state on both the vision and text sides, 282,285,696 parameters in safetensors. Image-to-text rather than a VQA head: it emits the table, and a downstream model does the reasoning.
Mixpeek SDK Integration
# Chart pages are images. Run DePlot over them, then index the TABLE it
# returns as text, so the numbers become searchable.
from transformers import Pix2StructProcessor, Pix2StructForConditionalGeneration
from mixpeek import Mixpeek
proc = Pix2StructProcessor.from_pretrained("google/deplot")
model = Pix2StructForConditionalGeneration.from_pretrained("google/deplot")
mx = Mixpeek(api_key="API_KEY")
inputs = proc(images=chart_image, text="Generate underlying data table:", return_tensors="pt")
table = proc.decode(model.generate(**inputs, max_new_tokens=512)[0], skip_special_tokens=True)
mx.documents.upsert(
collection_id="col_your_collection",
documents=[{
"document_id": "deck-q3-slide-14",
# The table is text now, so give the collection a text vector index and
# embed it. Keeping it only in payload makes it filterable, not findable.
"payload": {"chart_table": table, "source_key": "decks/q3.pdf#page=14"},
}],
)Capabilities
- Chart or plot image in, linearised data table out
- Pairs with an LLM for question answering over the extracted numbers
- 282M parameters, small enough to run on modest hardware
- Apache-2.0
Use Cases on Mixpeek
Frequently Asked Questions
Why not just ask a vision model the question directly?
You can, and for a one-off it is simpler. The reason to extract the table first is search: a table can be embedded and indexed once, so the chart is findable by anyone later. Asking a VLM per question means re-reading the image every time and never building an index.
How accurate is chart to table?
Good enough to search on, not good enough to audit with. Values read off a rendered chart inherit whatever the rendering lost, and a busy or unlabelled plot degrades it further. Treat the output as a search surface pointing back at the source, and read the number off the source before anyone acts on it.
Does Mixpeek run DePlot?
Not as a managed extractor. Run it yourself over your chart pages and upsert the table it returns, or bring the numbers in as payload alongside a text index.
Specification
Research Paper
DePlot: One-shot visual language reasoning by plot-to-table translation
arxiv.orgBuild a pipeline with deplot
Add this model to a processing pipeline alongside other extractors. Combine with retrieval stages for end-to-end search.
Run it on your own data, free