prerender

Type:

Default: 'auto'
Values:

'auto' | true | false

It sets how the content over the target url should be fetched.

The following examples show how to use the Microlink API with CLI, cURL, JavaScript, Python, Ruby, PHP & Golang, targeting 'https://www.sportsnet.ca/hockey/nhl/leafs-john-tavares-return-new-york-hope-positive' URL with 'prerender' API parameter:

CLI Microlink API example

microlink https://www.sportsnet.ca/hockey/nhl/leafs-john-tavares-return-new-york-hope-positive&prerender=auto

cURL Microlink API example

curl -G "https://api.microlink.io" \
  -d "url=https://www.sportsnet.ca/hockey/nhl/leafs-john-tavares-return-new-york-hope-positive" \
  -d "prerender=auto"

JavaScript Microlink API example

import mql from '@microlink/mql'

const { data } = await mql('https://www.sportsnet.ca/hockey/nhl/leafs-john-tavares-return-new-york-hope-positive', {
  prerender: "auto"
})

Python Microlink API example

import requests

url = "https://api.microlink.io/"

querystring = {
    "url": "https://www.sportsnet.ca/hockey/nhl/leafs-john-tavares-return-new-york-hope-positive",
    "prerender": "auto"
}

response = requests.get(url, params=querystring)

print(response.json())

Ruby Microlink API example

require 'uri'
require 'net/http'

base_url = "https://api.microlink.io/"

params = {
  url: "https://www.sportsnet.ca/hockey/nhl/leafs-john-tavares-return-new-york-hope-positive",
  prerender: "auto"
}

uri = URI(base_url)
uri.query = URI.encode_www_form(params)

http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true

request = Net::HTTP::Get.new(uri)
response = http.request(request)

puts response.body

PHP Microlink API example

<?php

$baseUrl = "https://api.microlink.io/";

$params = [
    "url" => "https://www.sportsnet.ca/hockey/nhl/leafs-john-tavares-return-new-york-hope-positive",
    "prerender" => "auto"
];

$query = http_build_query($params);
$url = $baseUrl . '?' . $query;

$curl = curl_init();

curl_setopt_array($curl, [
    CURLOPT_URL => $url,
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_ENCODING => "",
    CURLOPT_MAXREDIRS => 10,
    CURLOPT_TIMEOUT => 30,
    CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
    CURLOPT_CUSTOMREQUEST => "GET"
]);

$response = curl_exec($curl);
$err = curl_error($curl);

curl_close($curl);

if ($err) {
    echo "cURL Error #: " . $err;
} else {
    echo $response;
}

Golang Microlink API example

package main

import (
    "fmt"
    "net/http"
    "net/url"
    "io"
)

func main() {
    baseURL := "https://api.microlink.io"

    u, err := url.Parse(baseURL)
    if err != nil {
        panic(err)
    }
    q := u.Query()
    q.Set("url", "https://www.sportsnet.ca/hockey/nhl/leafs-john-tavares-return-new-york-hope-positive")
    q.Set("prerender", "auto")
    u.RawQuery = q.Encode()

    req, err := http.NewRequest("GET", u.String(), nil)
    if err != nil {
        panic(err)
    }

    client := &http.Client{}
    resp, err := client.Do(req)
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()

    body, err := io.ReadAll(resp.Body)
    if err != nil {
        panic(err)
    }

    fmt.Println(string(body))
}

import mql from '@microlink/mql'

const { data } = await mql('https://www.sportsnet.ca/hockey/nhl/leafs-john-tavares-return-new-york-hope-positive', {
  prerender: "auto"
})

The prerendering is a technique that consists of preloading all the elements of the page as a previous step before a web crawler can see the page correctly. Three values be used there:

true: A headless browser will be spawned for getting the content of the site. It could be slower since it will use waitUntil to determinte what browser event to wait until consider content is ready.
false: A simple HTTP GET will be performed for getting the content of the site. It's faster, since it doesn't evaluate scripts inside the HTML markup.
auto: The service will determine if a site needs to have prerendering enabled to retrieve the content or not.

You are going to be interested in to use prerendering for a bunch of reasons; one of them is because a high percentage of internet websites are Single Page Application (SPA).

That means that a major part of the content is built on the client when the user enters in the page, like React, Ember, Angular, etc.

In practice, there is no difference between doing that and being a normal user since both interact using a browser, so prerendering is a better way to simulate a normal user and see the content exactly as he would see it.

However, prerendering have a little trade-off: the cloud-based browser needs to wait until DOM events are done, taking extra time for that. But if you do not do this you will not get the data in any way.

We provided two extra headers for reflecting the decision taken by the service. They are:

x-fetch-mode: It determines which fetch technique has been used (being possible 'prerender' or 'fetch' as values).
x-fetch-time: It represents the total amount of time spent into the fetch step in a human readable format.

We just recommend to disable prerendering explicitly if you know the target url doesn't need it.