Scraping API
Web scraping use cases
Recipes for turning web pages into structured data: CSS selector rules to JSON, product prices, tables and repeated lists, JavaScript-rendered apps, pages behind a login, links and emails, Load more buttons and your own Puppeteer code running on a managed browser. One problem per page, with the rules or function that solve it, code you can paste, and the limits to plan for.
Use cases
Pick the problem you are solving
Any website to JSON
Declare the fields you want as CSS selector rules and get typed JSON back, with null for anything the page does not have.
Product prices and stock
Read prices as numbers and stock as text from any product page, with fallback rules that survive template changes.
Tables and lists to JSON arrays
Turn table rows, product grids and search results into an array of objects, one nested rule per column.
Custom fields from JavaScript apps
Render React, Vue or Angular apps in a real browser, wait for the element you need, then run your rules.
Scraping behind a login
Forward a session cookie or bearer token as a request header and extract data from pages only your users can see.
Every link and email on a page
Get every link as an absolute URL and every email address as a bare string, scoped to the part of the page you choose.
Cached JSON endpoints
Fetch any JSON endpoint without a browser, keep its shape intact and serve repeat calls from the cache.
Pagination and Load more buttons
Scrape numbered pages in parallel, one call per page, or click Load more inside a function until the list is complete.
Puppeteer without hosting Chrome
Send a Puppeteer function with a URL and get its return value back. The browser, the sandbox and the cleanup run on Microlink.
JavaScript with npm packages
Require any npm package inside a remote function, pin its version, and skip the browser entirely when the code does not need one.
Ready to scrape any website to JSON?
Every recipe on this page runs on the same endpoint. Start on the free API key, with every Pro feature included, and pick a paid plan when you need volume.
Other use cases
Explore the other Microlink APIs
Mobile screenshots at any viewport
Emulate an iPhone, a Pixel, an iPad or any custom viewport and capture the page exactly as those visitors see it.
Screenshots without cookie banners or ads
Ads, trackers and consent popups are blocked before the page renders, and one option dismisses first-party banners.
PDF invoices from authenticated pages
Print the invoice page your app already renders: forward the session, hide the chrome, name the file.
Clean PDFs without ads or banners
Get a document, not a browser tab: ads and consent popups blocked by default, sticky chrome removed with one CSS rule.
Markdown with metadata frontmatter
Get each page as Markdown with a YAML frontmatter block: title, author, date, word count and reading time.
Clean Markdown, no boilerplate
Convert only the article body: one selector keeps navigation, ads and widgets out of the Markdown.
Custom fields alongside metadata
Get prices, ratings, headings or any CSS selector, typed and returned next to the normalized metadata.
Fix missing or wrong og:image
Override any normalized field with a rule and chain fallbacks so link previews never render empty.
Scrape Cloudflare-protected websites
Get the real page instead of “Just a moment”: with any API key, blocked requests escalate through proxy tiers up to residential IPs.
Fix 403 and 429 scraping errors
Tell an antibot 403, a throttling 429 and your own quota apart, then route, cache or wait accordingly.
Brand and media monitoring from Google News
Query Google News by brand, country and time window and get headline, publisher and ISO 8601 date for every article.
Price comparison from Google Shopping
Get every merchant Google Shopping lists for a product, with a numeric price, the merchant name and the rating, per country.