Text
Web Scraper
Extract structured data from webpages while maintaining semantic context and relationships
Note: This playground provides simulated output to showcase functionality. No input data is processed or stored on our servers. Use this demo to explore the feature extractor's capabilities before integrating it into your application.
Input
Enter the text you want to process
How many levels of nested content to extract. Default: 3
Minimum confidence score to include a field (0.0-1.0). Default: 0.7
How strictly to enforce the target schema. Default: flexible
Whether to preserve HTML structure in output. Default: false
Whether to extract page metadata (title, description, etc.). Default: true
Output
{
"url": "https://example.com/article",
"extracted_at": "2024-01-20T10:30:00Z",
"metadata": {
"title": "Sample Article Title",
"description": "Article description from meta tags",
"author": "John Doe",
"published_date": "2024-01-15",
"language": "en"
},
"content": {
"main_heading": "Article Main Heading",
"body_text": "Full article text content...",
"sections": [
{
"heading": "Section 1",
"content": "Section 1 content...",
"confidence": 0.95
}
]
},
"structured_data": {
"products": [],
"prices": [],
"ratings": [],
"reviews": []
},
"semantic_relationships": [
{
"type": "parent-child",
"from": "main_heading",
"to": "sections",
"confidence": 0.98
}
],
"confidence_scores": {
"overall": 0.92,
"field_scores": {
"title": 0.98,
"body_text": 0.95,
"metadata": 0.88
}
}
}Ready to run Web Scraper on your data? Spin it up in Studio: no infra to host.
Run this in StudioWeb Scraper: common questions
- What is the Web Scraper + Multimodal Embeddings extractor used for?
- Documentation freshness detection, knowledge base building, job board ingestion, API example indexing, curriculum validation.
- How much does the Web Scraper + Multimodal Embeddings extractor cost?
- 14 cr/page, 1 cr/extraction, 2 cr/image. Credits are billed per unit processed; see mixpeek.com/pricing for the credit rate.
- What file types does Web Scraper + Multimodal Embeddings support?
- Text. It produces embeddings from intfloat__multilingual_e5_large_instruct, jinaai__jina_embeddings_v2_base_code, google__siglip_base_patch16_224, facebook__dinov2_base.
Already have embeddings? Skip extraction: search your own vectors with MVS, from $25/mo for up to 1M vectors.
Try MVS →Recent updates
Full changelog- Sep 20, 2026A page the model refused to read reports a typed policy blockWhen the provider declines to generate content, its SDK raises an opaque error about quick accessors and blocked prompts, and that raw string is what landed in the document's error field. For brand safety a content-policy block is the signal, so a consumer had to pattern-match a provider stack trace to learn one happened. The web scraper now reports it as a typed outcome carrying the block reason.
- Jul 30, 2026The web scraper gets through reCAPTCHA v2 gatesScrapes that hit a reCAPTCHA v2 interstitial used to stop there. The scraper now detects the gate, extracts the sitekey, solves it, injects the token and fires the widget callback. One case needed more than that. Form-style gates run a callback that clears error state without submitting anything, so the page sat on the gate even though every step had worked. When the callback leaves the page un-navigated and still thin, the widget's enclosing form is submitted directly, bypassing client validation that can swallow a programmatic submit. Pages whose callback does navigate are left alone. The JS render path was also brought to parity so the same gate handling applies whether or not the page needs rendering.
- Jul 29, 2026One blocked page no longer fails every other URL in the scrapeA failed scrape leaves its columns empty, and pandas fills those with NaN floats. The document expander then sliced and measured them as if they were strings and lists, so a single unreachable page crashed the processor for the entire Ray batch. Observed on a four-URL run where one publisher's captcha took down the three siblings that had scraped perfectly. The subtle part is why the obvious guard did not help: `x or []` does not rescue a NaN, because NaN is truthy, so it returns the NaN and the length check still raises. Every column the expander touches is now normalised once, up front, so a blocked page fails only itself.
- Jul 29, 2026Scraping JavaScript pages no longer runs out of memory mid-runThe scraper's headless Chromium was landing on a 6 GiB worker and getting OOM-killed, which showed up as a scrape that simply died rather than an error you could act on. Sizing is now explicit: one URL per actor call so a call means exactly one browser session, and a memory reservation large enough that two sessions can never share a pod. The extractor runs on a dedicated 16 GiB profile applied at the worker level, deliberately without changing its tier, so a browser-heavy extractor does not get pulled onto the shared cluster as a side effect.