Skip to content
Scraping API · Use case

Run JavaScript remotely with any npm package and no deploy step

A run JavaScript remotely API is useful when the code is small and the setup around it is not. Parsing HTML with cheerio, normalizing data with lodash or computing a value next to the target site all need a runtime, dependencies and somewhere to host them. With Browser Functions you send the function, require what you need, and get the return value back.

The problem

A ten-line scraper should not need a deployment pipeline

The logic is short: fetch a page, load it into cheerio, pick a few values, return them. Around it you need a function host, a package.json, a bundle, a deploy, logs and a way to update the dependency when it ships a fix.

Serverless platforms solve hosting but not the ceremony. Every new script is a new function to configure and deploy, packages have to be bundled ahead of time, and if the code sometimes needs a browser you are back to shipping Chromium alongside it.

The function runtime installs dependencies for you. Any require() in your code is detected, installed in a sandbox and cached, and require('[email protected]') pins a version. When the function never references page, no browser starts at all, so plain JavaScript runs faster. The writing functions guide covers each step.

How it works

How to run serverless JavaScript scraping with npm packages

Start without a browser. Add page only when the content you need is rendered by JavaScript on the target.

1 · Cheerio in the cloud, no browser
import createClient from 'microlink.io'

const microlink = createClient({
  apiKey: process.env.MICROLINK_API_KEY
})

const headings = async ({ url }) => {
  const cheerio = require('[email protected]')
  const response = await fetch(url)
  if (!response.ok) throw new Error(`HTTP ${response.status}`)
  const $ = cheerio.load(await response.text())
  return $('h2').map((_, el) => $(el).text().trim()).get()
}

const { isFulfilled, value } = await microlink.function(
  'https://example.com/docs',
  headings
)

The function never touches page, so no browser starts. It fetches the target itself, which counts as a same-origin request and is allowed on every plan, and returns the headings as an array.

2 · Cheerio over the rendered page
import createClient from 'microlink.io'

const microlink = createClient({
  apiKey: process.env.MICROLINK_API_KEY
})

const rendered = async ({ page }) => {
  const cheerio = require('[email protected]')
  const $ = cheerio.load(await page.content())
  return $('[data-price]').map((_, el) => $(el).attr('data-price')).get()
}

const { value } = await microlink.function('https://app.example.com', rendered, {
  waitForSelector: '[data-price]'
})

Referencing page starts a browser and navigates first, so page.content() is the HTML after JavaScript ran. waitForSelector holds the call until the elements exist.

3 · Reusable functions with parameters
import createClient from 'microlink.io'

const microlink = createClient({
  apiKey: process.env.MICROLINK_API_KEY
})

const pick = ({ url, fields }) => {
  const { pick: select } = require('lodash')
  return fetch(url)
    .then(response => response.json())
    .then(items => items.map(item => select(item, fields.split(','))))
}

const { value } = await microlink.function(
  'https://api.example.com/products.json',
  pick,
  { fields: 'id,name,price' }
)

fields is not an API parameter, so it reaches the function as a named argument. One function serves many requests without changing its code.

Parameters used
  • function The code to run. require() calls are detected and installed automatically.
  • waitForSelector Waits for an element before a function that uses page is called.
  • meta Skipped by default from the SDK, so the request only pays for the function.
  • ttl Caches the function result like any other response, from 1 minute to 31 days. Pro plans.

The first run of a new dependency set shows a high install time in profiling.phases; later runs with the same dependencies skip installation and it drops to zero. The function reference documents the install, build, spawn and run phases.

Why it works

Why a remote function beats deploying a script per job

The unit you write is a function. The runtime, the dependencies and the browser are resolved per request, so there is nothing else to maintain.

01 · Dependencies on demand
require() is the whole install step.
Packages are parsed from your code, installed into an isolated sandbox, bundled and cached. Pin a version with the package@version form, or leave it off to get the latest release.

The Browser Functions feature summarizes the runtime, and the sitemap tool is built on this same function method.

02 · Browser optional
Plain JavaScript runs without Chrome.
A function that does not reference page skips the browser entirely and is faster for it. The same call gains a full Puppeteer page the moment your code asks for one.

For click loops, waits and interaction, see run Puppeteer without hosting Chrome.

03 · Sandboxed
Clear limits, clear errors.
Free plans get 15 seconds, 64 MB, 1024 bytes of compressed code and same-origin requests only; Pro gets up to 60 seconds, 128 MB, unlimited code and any outgoing request. Each limit has its own named error.

When not to: packages that spawn child processes or write to the filesystem outside the sandbox fail with ERR_ACCESS_DENIED, and heavy CPU work can hit CpuTimeError. If a declarative rule can read the value, scrape it to JSON instead.

FAQ

Can I use cheerio in the cloud without deploying a server?

Yes. require('cheerio') inside a function sent with microlink.function. The package is installed on the fly and cached, and the function can load HTML it fetched from the target or the rendered page.content() when it uses page.

Which npm packages can I require in a remote function?

Any package on npm, as long as it does not need restricted system capabilities. Spawning child processes and writing to the filesystem outside the sandbox are blocked, and a package that tries returns an ERR_ACCESS_DENIED error.

Why is the first run of my remote JavaScript function slower?

The first run installs and bundles the dependencies, which shows as install time in profiling.phases. The result is cached, so later runs with the same dependencies skip installation. Fewer packages mean shorter install and build phases.

Can serverless JavaScript scraping call other domains?

On Pro, yes: outgoing requests are unrestricted. On the free plan a function can only make same-origin requests to the target URL’s host, and a cross-origin call returns OutgoingRequestError. See pricing for plan details.

How do I pin an npm package version in a remote function?

Append the version to the package name inside require, for example require('[email protected]'). Without a version the latest release is installed.
Related use cases

Solve the next problem with the same API

Puppeteer without hosting Chrome

Send a Puppeteer function with a URL and get its return value back. The browser, the sandbox and the cleanup run on Microlink.

Every link and email on a page

Get every link as an absolute URL and every email address as a bare string, scoped to the part of the page you choose.

Cached JSON endpoints

Fetch any JSON endpoint without a browser, keep its shape intact and serve repeat calls from the cache.

Any website to JSON

Declare the fields you want as CSS selector rules and get typed JSON back, with null for anything the page does not have.

Pagination and Load more buttons

Scrape numbered pages in parallel, one call per page, or click Load more inside a function until the list is complete.

LLM context from any URL

Compose Markdown, links, emails, metadata and tech stack from one URL into a context object for your agent.

Ready to run JavaScript remotely?

Any npm package, no deploy step, a browser only when you ask for one. Start on the free tier and send your first function today.