Skip to content
Metadata API · Use case

Fix link previews blocked by Cloudflare and other bot protection

A link preview blocked by Cloudflare, DataDome or Akamai shows a challenge page where the title, description and image should be. Chat apps, bookmark managers, CRMs and newsletter editors hit this on the links their users share most. With any API key the Metadata API resolves its built-in proxy automatically, so the link unfurls into the real card.

Start with the Metadata API
The problem

A Cloudflare challenge page has no metadata worth showing

Antibot services answer automated requests with a verification page. Its title is “Just a moment…”, its description is empty and it has no image, so a naive unfurler renders that for every link to the site. News outlets, marketplaces and social networks sit behind this kind of protection, and those are the links people paste most.

The usual fixes do not hold. Changing the user agent does not help when the target is judging the origin IP, and datacenter ranges are the first ones it rejects. Renting a proxy list means rotating IPs, watching ban rates and paying a second vendor, all to keep a feature working that should be one HTTP call.

With an API key, the free one included, automatic proxy resolution is on by default: when a request hits a 403, Microlink identifies the antibot provider and escalates through proxy tiers, ending on residential exits, until one gets through, then caches that tier for the domain. On the keyless endpoint the same block fails with EPROXYNEEDED instead of handing you the challenge page as metadata.

How it works

How to unfurl protected links through the built-in proxy

Send protected links with your API key, or start on the keyless endpoint and move only the EPROXYNEEDED links to the key, then cache the preview so the proxy path runs once per URL. The proxy guide covers the same pattern for every workflow.

1 · Metadata with an API key
import createClient from 'microlink.io'

const microlink = createClient({
  apiKey: process.env.MICROLINK_API_KEY
})

const { title, description, image } = await microlink.metadata(
  'https://hard-target.com/article'
)

The client carries your API key, so a blocked request escalates through the proxy tiers with no parameter to add. The response is the usual normalized object: title, description, image, logo, publisher and the rest.

2 · Keyless first, API key when blocked
import createClient from 'microlink.io'

const microlink = createClient({
  apiKey: process.env.MICROLINK_API_KEY
})

const freeTier = createClient()

const preview = async url => {
  try {
    return await freeTier.metadata(url)
  } catch (error) {
    if (error.code !== 'EPROXYNEEDED') throw error
    return microlink.metadata(url, { retry: 3, ttl: '1d' })
  }
}

Most links unfurl on the keyless endpoint. EPROXYNEEDED marks the ones behind antibot protection, and only those move to the keyed client, where retry adds server-side attempts and ttl caches the unblocked preview for a day.

3 · The same request as a URL
curl 'https://pro.microlink.io/?url=https%3A%2F%2Fhard-target.com%2Farticle&meta.title=true&meta.description=true&meta.image=true' \
  -H 'x-api-key: $MICROLINK_API_KEY'

The proxy comes with every API key, so the URL targets pro.microlink.io with your API key as a header and no proxy parameter. The meta object keeps detection to the three fields a card renders.

Parameters used
  • proxy Automatic with any API key, no value needed. Pass location to pin a country or url for your own proxy server.
  • retry Server-side retries with exponential backoff on unexpected browser errors. Default 2.
  • ttl How long the unblocked preview stays cached, from 1 minute to 31 days. Default 24 hours. Any API key.
  • meta Restrict detection to title, description and image for a lighter request.
  • ping On by default: image and logo URLs are verified as reachable before they are returned.

Confirm the route with the x-fetch-mode response header: any value ending in -proxy, such as fetch-proxy or prerender-proxy, means the request went through the proxy.

Why it works

Why link previews need the proxy inside the metadata API

Unfurling a link is one step in your product. It should stay one step even when the target pushes back.

01 · Provider-aware
The route depends on who is blocking.
Microlink identifies which antibot provider is behind a 403 and escalates through proxy tiers until one gets through. The tier that worked is cached per domain, so the next link to the same site unfurls without the climb.

The same API key unblocks screenshots of blocked websites and Markdown from bot-protected pages.

02 · Explicit signal
EPROXYNEEDED instead of a wrong preview.
On the keyless endpoint a detected block fails with a code you can catch. Your app moves that link to your API key or falls back to a plain link, and it never renders “Just a moment…” as the headline of a story.

The error surfaces on the keyless endpoint. Automatic proxy resolution comes with every API key, the free one included; see pricing for the plans.

03 · Cached unfurls
The proxy runs once per URL, the cache serves the rest.
Chat apps and feeds unfurl the same link thousands of times. With ttl and staleTtl the proxied preview is served from the cache while a fresh copy is fetched in the background, and cache hits do not count against your quota.

When not to: the proxy gets past bot blocks, not logins or paywalls. Content behind authentication needs forwarded headers, and only where you are permitted to fetch it.

FAQ

Why does my link preview show “Just a moment…” as the title?

The site sits behind antibot protection and answered the request with a verification page. An extractor that does not notice the block reads that page’s title and its empty description. Microlink flags a detected block with the EPROXYNEEDED error code, so you can send that link to your API key instead of storing a wrong card.

How do I fix a link preview blocked by Cloudflare?

Send the same metadata request with an API key, no extra parameter. Microlink identifies the protection in front of the site, escalates through proxy tiers up to residential and returns the normalized title, description and image. Cache the result with ttl so the proxy path runs once per URL.

Is the metadata API proxy included with an API key?

Yes. Automatic proxy resolution is part of every API key, the free one included, with no separate proxy subscription and no IP list to manage. If you already pay for a proxy service, you can pass it as proxy.url instead.

Can I pin the metadata proxy to a specific country?

Yes. Pass proxy.location with an ISO 3166-1 alpha-2 country code and the request exits from that country. See region-specific metadata for combining it with the Accept-Language header.

Does every link preview request need an API key?

No. Most sites answer a direct request on the keyless endpoint. Catch EPROXYNEEDED and send only those links to your API key, then let the cache serve the repeats. With a key the proxy only kicks in after a 403, so open sites stay on the direct route.
Related use cases

Solve the next problem with the same API

Link previews at scale

Unfurl links at any volume: no throttling, background refresh and cache hits that never count against your quota.
Scale your link previews 

Region-specific metadata

Fetch titles, descriptions and prices as a visitor from a given country and language sees them.
Get region-specific metadata 

Fix missing or wrong og:image

Override any normalized field with a rule and chain fallbacks so link previews never render empty.
Fix a broken og:image 

Metadata from single-page apps

Render React, Vue or Angular apps in a headless browser, wait for the tags, then read the normalized metadata.
Read metadata from single-page apps 

Screenshot blocked websites

When a site blocks headless browsers, any API key escalates the capture through proxy tiers automatically.
Screenshot blocked websites 

Markdown from bot-protected pages

Convert pages behind Cloudflare, DataDome or Akamai: with any API key the built-in proxy resolves automatically.
Convert bot-protected pages 

Ready to unfurl every link?

No option, no proxy list, real previews for the sites that block everyone else. Get your free API key and fix your first blocked preview today.

Get your free API key
Unblock link previews