Sitemap URLs

Paste a site URL. We read its sitemap (and robots.txt) and list every page location.

Respects robots.txt
Finds all sitemap files
Fast and free

Try an example: microlink.io·vercel.com·stripe.com

How do I list every URL in a website sitemap?

Paste any site URL into this sitemap tool. It reads Sitemap: from that origin’s robots.txt, walks nested sitemap indexes, and lists every page loc.

Copy the list, download it as a text file, or reuse the same function from your own code with the Microlink SDK.

How does this find the sitemap?

Discovery is robots-only. The function fetches /robots.txt on the origin you pasted and expands each Sitemap: URL. If robots.txt has no sitemap lines, the result is empty.

Nested sitemap indexes are followed. Page URLs are the non-XML loc values.

Can I run this from my own code?

The tool calls microlink.function(). The function does not start a browser — it reads Sitemap: from robots.txt, then xml-urls walks nested indexes with isolate fetch.

import createClient from 'microlink.io'

const microlink = createClient()

const { value } = await microlink.function(
  'https://microlink.io',
  async ({ site }) => {
    // Discover sitemap URLs from this origin's robots.txt
    const { href: robotsUrl } = new URL('/robots.txt', site)
    const res = await fetch(robotsUrl)
    const body = res.ok ? await res.text() : ''
    const robotsParser = require('robots-parser')
    const sitemaps = robotsParser(robotsUrl, body).getSitemaps()
  
    // Walk nested sitemap indexes. `fetcher` is how each document is loaded.
    return require('xml-urls')(sitemaps, { fetcher: fetch })
  },
  {
    site: "https://microlink.io"
  }
)

Why did the list time out or come back empty?

The list is empty when robots.txt has no Sitemap: lines. A Function failure (timeout, blocked fetch, or budget) shows the error panel instead. Large or deeply nested sitemaps can exceed the free 5-second limit.

See the function troubleshooting guide for timeout, memory, and outgoing-request limits.