Skip to content
Metadata API · Use case

Extract custom fields alongside the normalized metadata

Extract custom fields from the same metadata API request that returns the title, description and image. Open Graph tags cover the basics, but the value your product needs, a price, a rating, a stock status or a list of headings, usually lives somewhere else on the page. Declare it as a rule and it comes back typed, next to the normalized fields.

Start with the Metadata API
The problem

Open Graph metadata gets you close, not all the way

A link preview needs title, description and image, and the Metadata API normalizes those from Open Graph, Twitter Cards, JSON-LD and the HTML itself. A product card, a job listing, a recipe or a directory entry needs one or two more values that no standard tag carries.

The common workaround is a second scraper: fetch the page again, parse it with your own selectors, and keep two caches and two failure modes in sync. If you want to scrape a price with the Open Graph data of the same product, you end up loading the page twice.

The data parameter takes extraction rules: a CSS selector, the attribute to read and the type to validate as. Passed with a metadata request, the rules ride along and the response carries both the normalized fields and yours. The defining rules guide covers the full grammar.

Open the live JSON with a custom headings field
How it works

How to add a CSS selector field to a metadata API request

A rule answers three questions: which element, what to read from it, and how to validate it. Add as many rules as you need to one request; each becomes a key in the response.

1 · Metadata plus a price
import createClient from 'microlink.io'

const microlink = createClient({
  apiKey: process.env.MICROLINK_API_KEY
})

const { title, image, price } = await microlink.metadata(
  'https://example.com/product',
  {
    data: {
      price: { selector: '.price', attr: 'text', type: 'number' }
    }
  }
)

The normalized fields come back as usual. price is read from the first element matching .price and parsed as a number.

2 · Several fields, including a list
import createClient from 'microlink.io'

const microlink = createClient({
  apiKey: process.env.MICROLINK_API_KEY
})

const { title, rating, headings } = await microlink.metadata(
  'https://example.com/product',
  {
    data: {
      rating: { selector: '[itemprop=ratingValue]', attr: 'content', type: 'number' },
      headings: { selectorAll: 'h2', attr: 'text' }
    }
  }
)

selectorAll returns every match as an array. Each rule validates independently, so a missing rating does not affect the headings.

3 · The same request as a URL
curl 'https://api.microlink.io/?url=https%3A%2F%2Fexample.com%2Fproduct&data.price.selector=.price&data.price.attr=text&data.price.type=number'

Rules flatten to data.price.selector, data.price.attr and data.price.type query parameters, so the request works from any language or from the CLI.

Parameters used
  • data Object of custom extraction rules, evaluated next to the normalized metadata.
  • selector Which element: the first match of a CSS selector.
  • selectorAll Every match of a CSS selector, returned as an array.
  • attr What to read: any HTML attribute, or text, html, outerHTML, markdown, json or val. Default html.
  • type How to validate: number, url, date, image, email, boolean and more. Default auto.
  • meta Pick which normalized fields to keep alongside your rules.

A rule that matches nothing, or whose value fails its type, resolves to null, so destructuring is always safe. When you want only your fields and none of the normalized ones, call extract instead.

Why it works

Why extracting custom fields with rules beats a second scraper

The normalized fields and your fields come from the same fetch, the same cache entry and the same request.

01 · One request
Standard and custom fields from a single fetch.
There is no second HTTP call, no second parse and no second cache to keep consistent. The page is fetched once and every rule reads from that document.

The same rules can override a normalized field when a page ships a wrong title or no og:image.

02 · Typed output
Values are validated before they reach you.
A number rule returns a number, a url rule an absolute URL, an image rule an asset object with url, type, width, height and size. Values that fail their type become null instead of surprising strings.

An array of rules is a fallback chain: the first rule that yields a valid value wins, so a layout change degrades gracefully.

03 · Same options
Rules run after waits, clicks and the proxy.
Client-rendered prices need prerender and waitForSelector. Blocked sites need the proxy. Every shared option applies before the rules evaluate, so a rule reads the page a visitor would see.

When not to: if the page already exposes the value in JSON-LD or Open Graph, the normalized metadata probably has it, so check the default response before adding a rule. When you need the whole article rather than one field, use Markdown with metadata frontmatter.

FAQ

Can I get metadata and custom fields in one request?

Yes. Pass the rules as the data option of a metadata request and the response contains the normalized fields plus one key per rule. It is a single request and a single cache entry.

What types can a custom metadata field have?

string, number, boolean, date, url, email, ip, lang and regexp, among others, plus the media types image, video, audio and logo. Media types resolve to asset objects with url, type, width, height, size and size_pretty. A value that fails its type becomes null.

How do I extract a list of values next to the metadata?

Use selectorAll instead of selector. The rule returns an array with one value per matching element, validated by the same type. Put an object of rules under attr to turn each match into a structured item.

How do I scrape a price with the Open Graph metadata of a product page?

Add a data rule such as price, with selector .price, attr text and type number, to the metadata request. The response carries title, description and image from the page’s tags and price from your selector. For client-rendered stores, add prerender: true and waitForSelector for the price element.

Can a custom metadata rule read content that only appears after JavaScript runs?

Yes. Add prerender: true and waitForSelector for the element, and the rules evaluate against the rendered DOM. See metadata from single-page apps for the wait options.
Related use cases

Solve the next problem with the same API

Fix missing or wrong og:image

Override any normalized field with a rule and chain fallbacks so link previews never render empty.
Fix a broken og:image 

Only the fields you need

Include or exclude normalized fields per request and trim the JSON for faster, lighter metadata calls.
Request only the fields you need 

Metadata from single-page apps

Render React, Vue or Angular apps in a headless browser, wait for the tags, then read the normalized metadata.
Read metadata from single-page apps 

Brand colors from images

Get the dominant palette and an accessible text and background pair from a site’s logo and preview image.
Extract brand colors 

Markdown with metadata frontmatter

Get each page as Markdown with a YAML frontmatter block: title, author, date, word count and reading time.
Add metadata frontmatter 

LLM context from any URL

Compose Markdown, links, emails, metadata and tech stack from one URL into a context object for your agent.
Build LLM context 

Ready to extract the fields you need?

Normalized metadata plus your own rules in one call. Start on the free tier and add your first custom field today.

Add a custom field