Skip to content
Metadata API · Use case

Fix link previews blocked by Cloudflare and other bot protection

A link preview blocked by Cloudflare, DataDome or Akamai shows a challenge page where the title, description and image should be. Chat apps, bookmark managers, CRMs and newsletter editors hit this on the links their users share most. On Pro plans the Metadata API routes the request through its built-in proxy, so the link unfurls into the real card.

Start with the Metadata API
The problem

A Cloudflare challenge page has no metadata worth showing

Antibot services answer automated requests with a verification page. Its title is “Just a moment…”, its description is empty and it has no image, so a naive unfurler renders that for every link to the site. News outlets, marketplaces and social networks sit behind this kind of protection, and those are the links people paste most.

The usual fixes do not hold. Changing the user agent does not help when the target is judging the origin IP, and datacenter ranges are the first ones it rejects. Renting a proxy list means rotating IPs, watching ban rates and paying a second vendor, all to keep a feature working that should be one HTTP call.

On Pro plans, Microlink includes automatic proxy resolution: it identifies the antibot provider and routes the request through a dedicated resolution path over a rotating proxy pool. When it detects the block and no proxy was used, the API fails with EPROXYNEEDED instead of handing you the challenge page as metadata.

How it works

How to unfurl protected links through the built-in proxy

Try the direct request first, retry through the proxy on EPROXYNEEDED, and cache the preview so the proxy path runs once per URL. The proxy guide covers the same pattern for every workflow.

1 · Metadata through the managed proxy
import createClient from 'microlink.io'

const microlink = createClient({
  apiKey: process.env.MICROLINK_API_KEY
})

const { title, description, image } = await microlink.metadata(
  'https://hard-target.com/article',
  { proxy: true }
)

proxy: true sends the request through the managed pool with nothing else to configure. The response is the usual normalized object: title, description, image, logo, publisher and the rest.

2 · Only when the target needs it
import createClient from 'microlink.io'

const microlink = createClient({
  apiKey: process.env.MICROLINK_API_KEY
})

const preview = async url => {
  try {
    return await microlink.metadata(url)
  } catch (error) {
    if (error.code !== 'EPROXYNEEDED') throw error
    return microlink.metadata(url, { proxy: true, retry: 3, ttl: '1d' })
  }
}

Direct requests skip the extra hop, so the proxy only runs for targets that reject them. retry adds server-side attempts for intermittent challenges and ttl caches the unblocked preview for a day.

3 · The same request as a URL
curl 'https://pro.microlink.io/?url=https%3A%2F%2Fhard-target.com%2Farticle&proxy=true&meta.title=true&meta.description=true&meta.image=true' \
  -H 'x-api-key: $MICROLINK_API_KEY'

proxy is a Pro option, so the URL targets pro.microlink.io and carries your API key as a header. The meta object keeps detection to the three fields a card renders.

Parameters used
  • proxy true for automatic resolution, an object with location to pin a country, or url for your own proxy server. Pro plans.
  • retry Server-side retries with exponential backoff on unexpected browser errors. Default 2.
  • ttl How long the unblocked preview stays cached, from 1 minute to 31 days. Default 24 hours. Pro plans.
  • meta Restrict detection to title, description and image for a lighter request.
  • ping On by default: image and logo URLs are verified as reachable before they are returned.

Confirm the route with the x-fetch-mode response header: any value prefixed with proxy-, such as fetch-proxy or prerender-proxy, means the request went through the proxy.

Why it works

Why link previews need the proxy inside the metadata API

Unfurling a link is one step in your product. It should stay one step even when the target pushes back.

01 · Provider-aware
The route depends on who is blocking.
Microlink identifies which antibot provider is blocking the request and picks a dedicated resolution path for it. The pool is tested against the 500 most popular websites worldwide, which is where most shared links point.

The same option unblocks screenshots of blocked websites and Markdown from bot-protected pages.

02 · Explicit signal
EPROXYNEEDED instead of a wrong preview.
A detected block fails with a code you can catch. Your app retries through the proxy or falls back to a plain link, and it never renders “Just a moment…” as the headline of a story.

The error surfaces on every plan. Routing through the proxy needs a Pro key; see pricing for the plans.

03 · Cached unfurls
The proxy runs once per URL, the cache serves the rest.
Chat apps and feeds unfurl the same link thousands of times. With ttl and staleTtl the proxied preview is served from the cache while a fresh copy is fetched in the background, and cache hits do not count against your quota.

When not to: the proxy gets past bot blocks, not logins or paywalls. Content behind authentication needs forwarded headers, and only where you are permitted to fetch it.

FAQ

Why does my link preview show “Just a moment…” as the title?

The site sits behind antibot protection and answered the request with a verification page. An extractor that does not notice the block reads that page’s title and its empty description. Microlink flags a detected block with the EPROXYNEEDED error code, so you can retry through the proxy instead of storing a wrong card.

How do I fix a link preview blocked by Cloudflare?

Send the same metadata request with proxy: true on a Pro plan. Microlink identifies the protection in front of the site, routes the request through its rotating proxy pool and returns the normalized title, description and image. Cache the result with ttl so the proxy path runs once per URL.

Is the metadata API proxy included in Pro plans?

Yes. Automatic proxy resolution is part of every Pro plan, with no separate proxy subscription and no IP list to manage. If you already pay for a proxy service, you can pass it as proxy.url instead.

Can I pin the metadata proxy to a specific country?

Yes. Pass proxy.location with an ISO 3166-1 alpha-2 country code and the request exits from that country. See region-specific metadata for combining it with the Accept-Language header.

Should every link preview request go through the proxy?

No. Most sites answer a direct request, which skips the extra hop. Catch EPROXYNEEDED and retry with the proxy only for the targets that need it, then let the cache serve the repeats.
Related use cases

Solve the next problem with the same API

Link previews at scale

Unfurl links at any volume: no throttling, background refresh and cache hits that never count against your quota.
Scale your link previews 

Region-specific metadata

Fetch titles, descriptions and prices as a visitor from a given country and language sees them.
Get region-specific metadata 

Fix missing or wrong og:image

Override any normalized field with a rule and chain fallbacks so link previews never render empty.
Fix a broken og:image 

Metadata from single-page apps

Render React, Vue or Angular apps in a headless browser, wait for the tags, then read the normalized metadata.
Read metadata from single-page apps 

Screenshot blocked websites

When a site blocks headless browsers, one option routes the capture through a managed, rotating proxy pool.
Screenshot blocked websites 

Markdown from bot-protected pages

Convert pages behind Cloudflare, DataDome or Akamai: one option routes the request through the built-in proxy.
Convert bot-protected pages 

Ready to unfurl every link?

One option, no proxy list, real previews for the sites that block everyone else. Get a Pro key and fix your first blocked preview today.

Unblock link previews