Proxy · Use case

Scrape the price each country sees, as structured data

To scrape prices by country, every request has to look local to the market you are measuring, because stores set currency, tax display, discounts and availability from the visitor’s IP. Pricing teams, marketplace sellers, travel aggregators and analysts comparing regional pricing all need the same thing: one row per country, with numbers they can compare. Extraction rules plus a country pin turn each storefront into that row.

See how the proxy works

The problem

Your scraper records one country’s price and calls it the price

A scraper running in one region sees one version of the store. The German shopper sees euros with VAT included, the US shopper sees dollars before tax, the UK page shows a promotion nobody else gets, and some products are simply not sold everywhere. Monitoring from a single exit gives you a clean, confident dataset of the wrong prices for every other market.

Getting the other versions the hard way means an exit per market and a scraper that knows which to use. You buy country proxies, map each one to a job, keep a browser running for stores that render prices in JavaScript, and parse a different currency format per locale. When a number looks off, there is no quick way to tell whether the store changed its price or a proxy landed in the wrong country.

Microlink combines both halves in one request. proxy.location routes the call through an IP in the country you name, one of 181, and data rules read the price, currency and availability from the rendered page with a type that casts the price to a number. One request per country gives you one comparable row per market.

How it works

How to scrape prices by country with extraction rules

Define the rules once, run them once per country, and merge the rows. The extract method documents selectors, attributes and types.

1 · One extraction per country

import createClient from 'microlink.io'

const microlink = createClient({
  apiKey: process.env.MICROLINK_API_KEY
})

const url = 'https://shop.example.com/product/42'
const countries = ['us', 'gb', 'de', 'jp']

const rules = {
  price: { selector: '[itemprop="price"]', attr: 'content', type: 'number' },
  currency: { selector: '[itemprop="priceCurrency"]', attr: 'content' },
  availability: { selector: '[itemprop="availability"]', attr: 'href' }
}

const rows = await Promise.all(
  countries.map(async country => {
    const row = await microlink.extract(url, rules, {
      proxy: { location: country },
      ttl: '1h'
    })
    return Object.assign({ country }, row)
  })
)

Four requests run in parallel, each from a different country, and rows resolves to one object per market with a numeric price, its currency and the availability link. ttl keeps each country’s answer for an hour.

2 · Match the language to the market

import createClient from 'microlink.io'

const microlink = createClient({
  apiKey: process.env.MICROLINK_API_KEY
})

const row = await microlink.extract(url, rules, {
  proxy: { location: 'de' },
  headers: { 'x-api-header-accept-language': 'de-DE' }
})

Some stores pick the currency from the IP but the language, and occasionally the price format, from Accept-Language. The x-api-header- prefix forwards it to the store so both signals point at the same market.

3 · The same request as a URL

curl 'https://pro.microlink.io/?url=https%3A%2F%2Fshop.example.com%2Fproduct%2F42&data.price.selector=%5Bitemprop%3D%22price%22%5D&data.price.attr=content&data.price.type=number&data.currency.selector=%5Bitemprop%3D%22priceCurrency%22%5D&data.currency.attr=content&proxy.location=de&meta=false' \
  -H 'x-api-key: $MICROLINK_API_KEY'

One URL per country: change proxy.location and keep the rest. The Pro endpoint with your x-api-key header returns the price and currency under data.

Parameters used

proxy.location ISO 3166-1 alpha-2 country code for the market, case-insensitive. Default us. Pro plans.
data Extraction rules: selector, attr and a type such as number for the price.
headers Forwards Accept-Language for stores that localize by language too. Pro plans.
ttl Cache lifetime per country, from 1 minute to 31 days. Pro plans.

Microlink has no scheduler, so run the job from your own cron or queue and store the rows with a timestamp. If the price only renders after JavaScript, add waitForSelector with the price selector.

Why it works

Why per-country extraction beats one global price scraper

A price comparison is only as good as its weakest market. Treating the country as a request option makes every market a first-class row.

01 · Comparable numbers

Typed values, not strings to clean.

The number type returns the price as a number and the currency comes back as its own field, so the table you build compares like with like instead of parsing “1.299,00 €” against “$1,299.00”.

Structured rules are covered end to end in the data extraction guide.

02 · Cached per market

Each country is its own cache entry.

The cache key includes every recognized query parameter, proxy.location among them, so the German and Japanese answers never overwrite each other. Cache hits never count toward your quota.

Each country is one request on a cold cache, so four markets are four requests. Plans and quotas are on the pricing page.

03 · Rendered, not raw

Prices injected by JavaScript still count.

The rules run against the page after a real browser renders it, so a price that a storefront fills in client-side is read like any other element. The country pin and the rendering happen in the same request.

When not to: if you only need a localized title, description or preview image rather than a price table, localized metadata does it without writing rules.

FAQ

How do I scrape prices by country?

Write extraction rules for the price and currency, then send one request per country with proxy.location set to that country’s ISO code. Each response is the price a local visitor sees, and merging them gives you one row per market.

Why does a scraped price differ from the one in my browser?

The store localizes by IP. Your browser and your scraper leave from different countries or networks, so they are shown different currencies, taxes or promotions. Pin the scraper to the market you are checking with proxy.location.

How many countries can I monitor prices in?

181. proxy.location accepts any ISO 3166-1 alpha-2 code in the supported list; unknown codes fail with EINVALQUERY rather than falling back to a default.

Does each country count as a separate price request?

Yes. Each country is its own request and its own cache entry. Repeat reads within the cache lifetime are cache hits, which never count toward your quota, so a dashboard that re-reads the same markets costs nothing extra until the ttl expires.

Should I use price extraction or localized metadata for regional pages?

Use extraction rules when you need numbers to compare, such as price, currency and stock. Use localized metadata when you need the regional title, description and image for a link preview. Both rely on the same proxy.location option.

Related use cases

Solve the next problem with the same API

Fetch geo-blocked websites

Route any request through one of 181 countries with a two-letter code, and verify the exit before you trust the page.

Reach geo-blocked pages

Scrape Cloudflare-protected websites

Get the real page instead of “Just a moment”: on Pro plans, blocked requests escalate through proxy tiers up to residential IPs.

Scrape Cloudflare-protected sites

Region-specific metadata

Fetch titles, descriptions and prices as a visitor from a given country and language sees them.

Get region-specific metadata

Ready to compare prices across markets?

One set of rules, one request per country, 181 markets available. Get a Pro key and build the table.

Compare prices by country