plans come with a built-in proxy layer that does automatic proxy resolution and rotation based on the input URL.
The Internet is a wild place where every URL is different. That's an issue especially when you are doing things at scale.
facing with fetching problems.
When you are surfing the net and visit a website, there are situations you can't handle at all:
- The are geographic location restrictions (e.g., visiting a site from China).
- Your IP address is blacklisted (e.g., performing a request from a data center).
- You have to validate your identity (e.g., filling a CAPTCHA).
You can quickly face these problems when sending enough traffic to popular sites, like Instagram or Bloomberg, which, in turn, makes getting a successful response inconsistent.
Wouldn't it be nice if every time you want to retrieve data from any URL, the Microlink API just returned the data all while handling any of the problems above transparently?
No additional setup needed, just a Microlink Pro plan.
That's exactly what Microlink Proxy layer does. No failures. No maintenance. You don't need to do anything additional. It just works™.
The Microlink Proxy
is enabled for all Microlink Pro
plans and it will handle all the situations for the
most popular worldwide websites, in a transparent way, by default.
Just for informational purposes, you can check if a specific request was resolved using the proxy layer by checking
x-fetch-mode response header.
$ microlink https://www.bloomberg.com/news/articles/2016-05-24/as-zenefits-stumbles-gusto-goes-head-on-by-selling-insurance
SUCCESS 3.64 kB in 329ms
cache HIT (19h 20m 54.8s left)
mode prerender-proxy (8.5s)
If the value there is prefixed by
`proxy` then it was handled by the proxy layer.
As you can see, the proxy layer respects other query parameters, such as prerender
Also, you can always provide your own proxy
Last but not least, Microlink Proxy is a cross-feature: it's available for all the Microlink products.