Text
Web Scraper
Extract structured data from webpages while maintaining semantic context and relationships
Note: This playground provides simulated output to showcase functionality. No input data is processed or stored on our servers. Use this demo to explore the feature extractor's capabilities before integrating it into your application.
Input
Enter the text you want to process
How many levels of nested content to extract. Default: 3
Minimum confidence score to include a field (0.0-1.0). Default: 0.7
How strictly to enforce the target schema. Default: flexible
Whether to preserve HTML structure in output. Default: false
Whether to extract page metadata (title, description, etc.). Default: true
Output
{
"url": "https://example.com/article",
"extracted_at": "2024-01-20T10:30:00Z",
"metadata": {
"title": "Sample Article Title",
"description": "Article description from meta tags",
"author": "John Doe",
"published_date": "2024-01-15",
"language": "en"
},
"content": {
"main_heading": "Article Main Heading",
"body_text": "Full article text content...",
"sections": [
{
"heading": "Section 1",
"content": "Section 1 content...",
"confidence": 0.95
}
]
},
"structured_data": {
"products": [],
"prices": [],
"ratings": [],
"reviews": []
},
"semantic_relationships": [
{
"type": "parent-child",
"from": "main_heading",
"to": "sections",
"confidence": 0.98
}
],
"confidence_scores": {
"overall": 0.92,
"field_scores": {
"title": 0.98,
"body_text": 0.95,
"metadata": 0.88
}
}
}Ready to run Web Scraper on your data? Spin it up in Studio: no infra to host.
Run this in StudioWeb Scraper: common questions
- What is the Web Scraper + Multimodal Embeddings extractor used for?
- Documentation freshness detection, knowledge base building, job board ingestion, API example indexing, curriculum validation.
- How much does the Web Scraper + Multimodal Embeddings extractor cost?
- 5 cr/page, 1 cr/extraction, 2 cr/image. Credits are billed per unit processed; see mixpeek.com/pricing for the credit rate.
- What file types does Web Scraper + Multimodal Embeddings support?
- Text. It produces embeddings from intfloat__multilingual_e5_large_instruct, jinaai__jina_embeddings_v2_base_code, google__siglip_base_patch16_224, facebook__dinov2_base.
Already have embeddings? Skip extraction: search your own vectors with MVS, from $25/mo for up to 1M vectors.
Try MVS →Recent updates
Full changelog- Jul 30, 2026The web scraper gets through reCAPTCHA v2 gatesScrapes that hit a reCAPTCHA v2 interstitial used to stop there. The scraper now detects the gate, extracts the sitekey, solves it, injects the token and fires the widget callback. One case needed more than that. Form-style gates run a callback that clears error state without submitting anything, so the page sat on the gate even though every step had worked. When the callback leaves the page un-navigated and still thin, the widget's enclosing form is submitted directly, bypassing client validation that can swallow a programmatic submit. Pages whose callback does navigate are left alone. The JS render path was also brought to parity so the same gate handling applies whether or not the page needs rendering.
- Jul 29, 2026One blocked page no longer fails every other URL in the scrapeA failed scrape leaves its columns empty, and pandas fills those with NaN floats. The document expander then sliced and measured them as if they were strings and lists, so a single unreachable page crashed the processor for the entire Ray batch. Observed on a four-URL run where one publisher's captcha took down the three siblings that had scraped perfectly. The subtle part is why the obvious guard did not help: `x or []` does not rescue a NaN, because NaN is truthy, so it returns the NaN and the length check still raises. Every column the expander touches is now normalised once, up front, so a blocked page fails only itself.
- Jul 29, 2026Scraping JavaScript pages no longer runs out of memory mid-runThe scraper's headless Chromium was landing on a 6 GiB worker and getting OOM-killed, which showed up as a scrape that simply died rather than an error you could act on. Sizing is now explicit: one URL per actor call so a call means exactly one browser session, and a memory reservation large enough that two sessions can never share a pod. The extractor runs on a dedicated 16 GiB profile applied at the worker level, deliberately without changing its tier, so a browser-heavy extractor does not get pulled onto the shared cluster as a side effect.
- Jul 28, 2026The web scraper renders JavaScript again instead of quietly fetching static HTMLPlaywright and its Chromium build lived in a tenant-only dependency group, so the shared engine image never installed them. Nothing failed loudly: the browser-rendering service saw the missing package and switched itself off, and the in-process scraper fell back to a plain HTTP fetch that cannot run JavaScript or get past bot mitigation. A page that renders its content client-side therefore came back empty rather than erroring, and scrape batches ran about 38 minutes to produce zero documents. The browser and both driver revisions are now baked into the shared image at a fixed path that survives cache-volume mounts, so a scrape either renders the page or fails where you can see it.