# extract

## Table of Contents

- [A rule](#a-rule)
- [selector](#selector)
- [Fallback selectors](#fallback-selectors)
- [selectorAll](#selectorall)
- [attr](#attr)
- [Whole-page serialization](#whole-page-serialization)
- [Nested rules](#nested-rules)
- [Fallback values](#fallback-values)
- [type](#type)
- [evaluate](#evaluate)
- [Options](#options)
- [Result](#result)
- [Rules alongside metadata](#rules-alongside-metadata)
- [Fallback rules](#fallback-rules)
- [Everywhere else](#everywhere-else)

---

Typed values pulled with your own rules. Declare the data you want from a page — a CSS selector, the attribute to read, the type to validate it as — and get it back normalized. It takes the rules as its second argument and resolves to an object with one key per rule:

```js
importcreateClientfrom'microlink.io'

constmicrolink=createClient()

const{image}=awaitmicrolink.extract('https://microlink.io',{

image:{

selector:'meta[property="og:image"]',

attr:'content',

type:'image'

}

})

console.log(image.url,image.width,image.height)
```

For any [url](https://microlink.io/docs/api/parameters/url), [Microlink API](https://microlink.io/docs/api/getting-started/overview) already returns normalized [data fields](https://microlink.io/docs/api/getting-started/data-fields) extracted from Open Graph, JSON-LD, and a series of DOM fallbacks. Rules let you go further: extract any value present on any website, overwrite a normalized field whose source is wrong, and build your own API on top of any site. Every example on this page assumes the `microlink` client above.

## A rule

A rule is defined by a handful of primitives. Each one answers a single question about the value you want:

| Primitive                                                                  | Question it answers                                                            | Example                       |
| -------------------------------------------------------------------------- | ------------------------------------------------------------------------------ | ----------------------------- |
| [selector ✓](https://microlink.io/docs/sdk/methods/extract#selector)       | Which element? The first match of a CSS selector                               | `'meta[property="og:image"]'` |
| [selectorAll ✓](https://microlink.io/docs/sdk/methods/extract#selectorall) | Which elements? Every match, returning a collection                            | `'.athing'`                   |
| [attr ✓](https://microlink.io/docs/sdk/methods/extract#attr)               | What to read from it: an HTML attribute, or `text`, `html`, `markdown`, `json` | `'content'`                   |
| [type ✓](https://microlink.io/docs/sdk/methods/extract#type)               | How to validate and normalize the value                                        | `'image'`                     |
| [evaluate ✓](https://microlink.io/docs/sdk/methods/extract#evaluate)       | Run JavaScript in the page instead of querying the DOM                         | `'window.next.version'`       |

A rule needs at least a query — `selector`, `selectorAll`, or `evaluate` — or an `attr` alone to serialize the whole page. Omitted primitives fall back to `attr: 'html'` and `type: 'auto'`.

Rules compose in two ways: an object under `attr` builds [nested](https://microlink.io/docs/sdk/methods/extract#nested-rules) structures, and an array of rules defines [fallbacks](https://microlink.io/docs/sdk/methods/extract#fallback-rules) evaluated in order until one yields a value.

## selector

Type:

\<string\> \| \<string\[\]\>

\
Values: [CSS selector](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors)

It defines the [HTML element](https://developer.mozilla.org/en-US/docs/Web/HTML/Element) you want to pick from the HTML markup over the [url](https://microlink.io/docs/api/parameters/url):

```js
importcreateClientfrom'microlink.io'

constmicrolink=createClient()

constgithub=username=>

microlink.extract(`https://github.com/${username}`,{

avatar:{

selector:'meta[property="og:image"]:not([content=""])',

attr:'content',

type:'image'

}

})

constusername='kikobeats'

const{avatar}=awaitgithub(username)

console.log(

`GitHub avatar for @${username}: ${avatar.url} (${avatar.size_pretty})`

)
```

It's equivalent to [Document.querySelector()](https://developer.mozilla.org/en-US/docs/Web/API/Document/querySelector) and any [CSS selector](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors) can be specified, such as:

- An HTML tag (e.g., 'img').
- A CSS class or pseudo-class, id or data-attribute (e.g., '#avatar').
- A combination of both (e.g., 'img:first').

When `selector` is omitted, [attr](https://microlink.io/docs/sdk/methods/extract/attr) operates on the entire page — see [whole-page serialization](https://microlink.io/docs/sdk/methods/extract/attr#whole-page-serialization). The same `selector` is what the [markdown](https://microlink.io/docs/sdk/methods/markdown), [html](https://microlink.io/docs/sdk/methods/html), [text](https://microlink.io/docs/sdk/methods/text), and [links](https://microlink.io/docs/sdk/methods/links) methods accept as an option to scope their extraction.

## Fallback selectors

A collection of selectors is an array of [fallback rules](https://microlink.io/docs/sdk/methods/extract#fallback-rules): the first selector that yields a typed value wins.

## selectorAll

Type:

\<string\> \| \<string\[\]\>

\
Values: [CSS selector](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Selectors)

It's the same as [selector](https://microlink.io/docs/sdk/methods/extract/selector) but it returns a collection of results, being equivalent to [Document.querySelectorAll()](https://developer.mozilla.org/en-US/docs/Web/API/Document/querySelectorAll):

```js
importcreateClientfrom'microlink.io'

constmicrolink=createClient()

consthackerNews=()=>

microlink.extract('https://news.ycombinator.com/',{

posts:{

selectorAll:'.athing',

attr:{

title:{

type:'title',

selector:'.titleline > a',

attr:'text'

},

url:{

type:'url',

selector:'.titleline > a',

attr:'href'

}

}

}

})

const{posts}=awaithackerNews()

console.log('latest hacker news posts:',posts)
```

Without a nested `attr`, each match contributes one plain value, which is how [links](https://microlink.io/docs/sdk/methods/links) and the other sweep methods work:

```js
const{links}=awaitmicrolink.extract('https://news.ycombinator.com/',{

links:{

selectorAll:'.titleline > a',

attr:'href',

type:'url'

}

})

console.log(links)// => ['https://…', 'https://…', …]
```

## attr

Type:

\<string\> \| \<string\[\]\>

\
Default: 'html'\
Values:

[tagName](https://developer.mozilla.org/en-US/docs/Web/API/Element/tagName) \| [nodeName](https://developer.mozilla.org/en-US/docs/Web/API/Node/nodeName) \| 'html' \| 'outerHTML' \| 'text' \| 'markdown' \| 'json' \| 'val'

It specifies how the value should be extracted from the matched [selector](https://microlink.io/docs/sdk/methods/extract/selector):

```js
importcreateClientfrom'microlink.io'

constmicrolink=createClient()

constgithub=username=>

microlink.extract(`https://github.com/${username}`,{

avatar:{

selector:'meta[property="og:image"]:not([content=""])',

attr:'content',

type:'image'

}

})

constusername='kikobeats'

const{avatar}=awaitgithub(username)

console.log(

`GitHub avatar for @${username}: ${avatar.url} (${avatar.size_pretty})`

)
```

Any [HTML attribute](https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes) is supported, plus the following special cases:

- 'html': Get the inner HTML content of the matched selector.
- 'outerHTML': Get the outer HTML of the matched selector, including the element itself.
- 'text': Returns the combined text content, including its descendants, by removing leading, trailing, and repeated whitespace.
- 'markdown': Converts the HTML content into Markdown, preserving headings, links, and formatting.
- 'json': Parses the page body as JSON and returns structured data. Whole-page only — do not combine with `selector`. See [Extract JSON](https://microlink.io/docs/guides/data-extraction/defining-rules#extract-json) in the Data extraction guide.
- 'val': Get the current value of the matched selector, oriented for select or input fields.

## Whole-page serialization

When [selector](https://microlink.io/docs/sdk/methods/extract/selector) is omitted, `attr` operates on the entire page. This is useful for serializing a full page into a new output format:

```js
const{content}=awaitmicrolink.extract('https://example.com',{

content:{

attr:'markdown'

}

})

console.log(content)

// => '# Example Domain\n\nThis domain is for use in illustrative examples…'
```

The [markdown](https://microlink.io/docs/sdk/methods/markdown), [html](https://microlink.io/docs/sdk/methods/html), and [text](https://microlink.io/docs/sdk/methods/text) methods are shortcuts over exactly this rule.

You can also scope the conversion to a specific element by combining `selector` with `attr`:

```js
const{article}=awaitmicrolink.extract('https://example.com',{

article:{

selector:'article',

attr:'markdown'

}

})

console.log(article)

// => '# Article Title\n\nArticle content as markdown…'
```

For JSON endpoints, use `attr: 'json'` to parse the response body as structured data:

```js
const{content}=awaitmicrolink.extract(

'https://pokeapi.co/api/v2/pokemon',

{

content:{

attr:'json'

}

}

)

console.log(content)

// => { count: 1302, next: '…', results: [ … ] }
```

`json` is whole-page only — it cannot be combined with `selector`. See [Extract JSON](https://microlink.io/docs/guides/data-extraction/defining-rules#extract-json) for the full walkthrough.

## Nested rules

An object under `attr` maps a data structure over the same property key. Each nested rule is evaluated relative to the element matched by the parent `selector`, and the result is an object with one key per nested rule:

```js
constgithub=username=>

microlink.extract(`https://github.com/${username}`,{

stats:{

selector:'.application-main',

attr:{

followers:{

selector:'.js-profile-editable-area a[href*="tab=followers"] span',

type:'number'

},

following:{

selector:'.js-profile-editable-area a[href*="tab=following"] span',

type:'number'

},

stars:{

selector:'.js-responsive-underlinenav a[data-tab-item="stars"] span',

type:'number'

}

}

}

})

constusername='kikobeats'

const{stats}=awaitgithub(username)

console.log(`GitHub stats for @${username}:`,stats)

// => { followers: 1234, following: 56, stars: 789 }
```

The same structure applies to every item of a list when the parent uses [selectorAll](https://microlink.io/docs/sdk/methods/extract/selectorAll), which is how you turn repeated markup into an array of objects:

```js
const{stories}=awaitmicrolink.extract('https://news.ycombinator.com',{

stories:{

selectorAll:'.athing',

attr:{

title:{selector:'.titleline > a',attr:'text'},

url:{selector:'.titleline > a',attr:'href',type:'url'}

}

}

})

console.log(stories[0])// => { title: '…', url: 'https://…' }
```

Nested rules can nest again, so a parent rule can describe a whole section of a page as one JSON document.

## Fallback values

Multiple `attr` values use the same [fallback](https://microlink.io/docs/sdk/methods/extract#fallback-rules) form: the first attribute that resolves a value is used.

## type

Type:

\<string\> \| \<string\[\]\>

\
Default: 'auto'\
Values:

'audio' \| 'author' \| 'auto' \| 'boolean' \| 'date' \| 'description' \| 'email' \| 'image' \| 'ip' \| 'lang' \| 'logo' \| 'number' \| 'object' \| 'publisher' \| 'regexp' \| 'string' \| 'title' \| 'url' \| 'video'

It defines how the value extracted should be considered.

```js
importcreateClientfrom'microlink.io'

constmicrolink=createClient()

constproductHunt=id=>

microlink.extract(`https://www.producthunt.com/posts/${id}`,{

name:{

selector:'h1 a',

attr:'text',

type:'string'

},

upvotes:{

selector:'.bigButtonCount_10448',

attr:'text',

type:'number'

}

})

constproductSlug='microlink-2-0'

const{name,upvotes}=awaitproductHunt(productSlug)

console.log(`'${name}' has ${upvotes} upvotes`)
```

The data shape ensures that the extracted value will only be considered as valid when it's of the declared shape: a rule whose value doesn't match its `type` resolves to `null`, which is what lets [fallback rules](https://microlink.io/docs/sdk/methods/extract#fallback-rules) move on to the next candidate.

Media types do more than validate. 'image', 'video', 'audio', and 'logo' resolve the value to an absolute URL and expand it into an asset object with `url`, `type`, `width`, `height`, `size`, and `size_pretty`, the same shape the normalized [data fields](https://microlink.io/docs/api/getting-started/data-fields) use:

```js
const{cover}=awaitmicrolink.extract(

'https://www.youtube.com/watch?v=9P6rdqiybaw',

{

cover:{

selector:'meta[property="og:image"]',

attr:'content',

type:'image'

}

}

)

console.log(cover.width,cover.height,cover.size_pretty)
```

## evaluate

Type:

\<string\> \| \<function\>

It evaluates the JavaScript provided inside the browser context over the target URL, returning the result.

It's quite similar to [selector](https://microlink.io/docs/sdk/methods/extract/selector), but designed to specify the value to be obtained in a JavaScript-like way.

```js
importcreateClientfrom'microlink.io'

constmicrolink=createClient()

constgetNextVersion=url=>

microlink.extract(url,{

version:{

evaluate:'window.next.version',

type:'string'

}

})

const{version}=awaitgetNextVersion('https://vercel.com')

console.log(`Next.js version is: ${version}`)
```

You can combine evaluate with types for data correctness.

It can evaluate anything browser compatible in the JavaScript context. A function is serialized to its source before being sent, so it can be as long as you need — but it runs in the page, not in your process, so it can only reach what the page can:

```js
constgetExcerpt=url=>

microlink.extract(url,{

excerpt:{

evaluate:async()=>{

constresponse=awaitwindow.fetch(

'https://cdn.jsdelivr.net/npm/@mozilla/readability/Readability.js'

)

constscript=awaitresponse.text()

window.eval(script)

constreader=newwindow.Readability(window.document)

returnreader.parse().excerpt

},

type:'string'

}

})

const{excerpt}=awaitgetExcerpt(

'https://levelup.gitconnected.com/how-to-load-external-javascript-files-from-the-browser-console-8eb97f7db778'

)

console.log(excerpt)
```

When the logic outgrows a single expression — clicks, waits, npm packages — reach for [function](https://microlink.io/docs/sdk/methods/function), which gives the function full Puppeteer access instead of a page-side evaluation.

## Options

A third argument takes the [shared options](https://microlink.io/docs/sdk/getting-started/options), useful for pairing rules with [prerender](https://microlink.io/docs/api/parameters/prerender), [waitForSelector](https://microlink.io/docs/api/parameters/waitForSelector), [ttl](https://microlink.io/docs/api/parameters/ttl), or [headers](https://microlink.io/docs/api/parameters/headers) forwarded to the target page:

```js
const{price}=awaitmicrolink.extract(

'https://example.com/product',

{price:{selector:'.price',attr:'text',type:'number'}},

{prerender:true,waitForSelector:'.price',ttl:'1h'}

)
```

## Result

It resolves to an object with one key per rule. Values are normalized by their [type](https://microlink.io/docs/sdk/methods/extract#type): a `'string'` is a string, a `'number'` is a number, and `'image'`, `'video'`, `'audio'`, and `'logo'` become asset objects carrying `url`, `type`, `width`, `height`, `size`, and `size_pretty`. A rule that matches nothing, or whose value fails its type, resolves to `null`, so destructuring is always safe.

The complete API response — its [status](https://microlink.io/docs/api/basics/format#status), the payload, and the HTTP response with its headers — stays available on `microlink.last` after every call; see [inspect the last request](https://microlink.io/docs/sdk/getting-started/errors#inspect-the-last-request). Failures throw a [`MicrolinkError`](https://microlink.io/docs/sdk/getting-started/errors) like every other method.

## Rules alongside metadata

`extract` returns only your fields. To evaluate rules next to the normalized [data fields](https://microlink.io/docs/api/getting-started/data-fields) in one request, pass them as the `data` option of [metadata](https://microlink.io/docs/sdk/methods/metadata):

```js
const{title,description,price}=awaitmicrolink.metadata(

'https://example.com/product',

{

data:{

price:{selector:'.price',attr:'text',type:'number'}

}

}

)
```

A rule named after a normalized field — `title`, `image`, `author` — overrides that field, which is how you fix a page whose metadata is wrong or missing.

## Fallback rules

A field can be defined by more than one rule. Pass an array and the rules are evaluated respecting the order: if the first one fails, the second is tried, then the third, and so on. The value is the one obtained by the first rule that succeeds:

```js
constgithub=username=>

microlink.extract(`https://github.com/${username}`,{

avatar:[

{

selector:'meta[name="twitter:image:src"]:not([content=""])',

attr:'content',

type:'image'

},

{

selector:'meta[property="og:image"]:not([content=""])',

attr:'content',

type:'image'

}

]

})

constusername='kikobeats'

const{avatar}=awaitgithub(username)

console.log(

`GitHub avatar for @${username}: ${avatar.url} (${avatar.size_pretty})`

)
```

A rule fails when its query matches nothing or when the value doesn't pass its [type](https://microlink.io/docs/sdk/methods/extract#type), so a fallback chain is also how you make a rule resilient to pages whose markup varies. The same array form works for [selector](https://microlink.io/docs/sdk/methods/extract#fallback-selectors) and [attr](https://microlink.io/docs/sdk/methods/extract#fallback-values) on their own.

## Everywhere else

The [markdown](https://microlink.io/docs/sdk/methods/markdown), [html](https://microlink.io/docs/sdk/methods/html), [text](https://microlink.io/docs/sdk/methods/text), and [links](https://microlink.io/docs/sdk/methods/links) methods are shortcuts over rules like these, and accept the same primitives as options to scope their extraction. The same grammar — historically known as MQL, the Microlink Query Language — is what the [data](https://microlink.io/docs/api/parameters/data) query parameter takes when you call the API directly, and what the [CLI](https://microlink.io/docs/sdk/getting-started/cli) accepts as JSON through `extract --data`.

Underneath, the SDK talks to the API through [@microlink/mql](https://github.com/microlinkhq/mql), the low-level client it ships as a dependency; you never need to install or import it yourself. Reading a response body as a stream or a buffer is the one thing that still lives there rather than in the SDK.

See the [data extraction guide](https://microlink.io/docs/guides/data-extraction) for defining rules, page preparation, response shaping, and troubleshooting.