Automate literature reviews with a Google Scholar API
A Google Scholar API turns a literature search into data: each paper comes back with its title, publication year, citation count and a direct PDF link when one exists. Research teams, R&D groups, grant writers and research agents all start a review the same way, and all lose hours copying results by hand. The Search API returns Scholar results as JSON you can sort, filter and read.
A literature review starts with hours of copying Scholar results
Scholar is where many literature searches begin, and it is built for reading one page at a time. Collecting the top papers across a dozen queries, noting citation counts and years, and finding which ones have a free PDF is manual work that starts over every time the research question shifts.
Scraping the results page yourself means parsing a byline that mixes authors, venue and year into one string, pulling citation counts out of link text, and keeping the requests from being blocked. Copying into a spreadsheet does not scale past a handful of queries, and it goes stale the week after.
type: scholar returns each paper with title, url, description, the byline as publisher, year and citations as numbers, a Scholar id and pdf.url when a direct PDF exists. Sort by citations, filter by year, and pass the PDFs or result.markdown() to whatever reads the papers.
How to collect papers and citation counts with the Google Scholar API
Run the queries that define the review, merge and rank the papers, then fetch the full text of the shortlist. The Scholar guide lists every field.
import createClient from 'microlink.io'
const microlink = createClient({
apiKey: process.env.MICROLINK_API_KEY
})
const { results } = await microlink.search('retrieval augmented generation', {
type: 'scholar'
})
const papers = results
.sort((a, b) => b.citations - a.citations)
.map(({ id, title, year, citations, publisher, pdf, url }) => ({
id,
title,
year,
citations,
byline: publisher,
pdf: pdf?.url,
url
}))citations and year are numbers, so ranking needs no parsing. publisher is the Scholar byline, authors plus venue in one string, and pdf is present only when a direct PDF link exists.
import createClient from 'microlink.io'
const microlink = createClient({
apiKey: process.env.MICROLINK_API_KEY
})
const queries = [
'retrieval augmented generation',
'dense passage retrieval',
'long context language models'
]
const pages = await Promise.all(
queries.map(query => microlink.search(query, { type: 'scholar' }))
)
const corpus = new Map()
for (const { results } of pages) {
for (const paper of results) {
if (paper.year >= 2020) corpus.set(paper.id, paper)
}
}The Scholar id deduplicates papers that match several queries, and the year cut runs in your code. Three queries are three requests; call next() on any page for more depth.
import createClient from 'microlink.io'
const microlink = createClient({
apiKey: process.env.MICROLINK_API_KEY
})
const shortlist = Array.from(corpus.values())
.sort((a, b) => b.citations - a.citations)
.slice(0, 5)
const sources = await Promise.all(
shortlist.map(async paper =>
paper.pdf
? { title: paper.title, pdf: paper.pdf.url }
: { title: paper.title, markdown: await paper.markdown() }
)
)Papers with a PDF link go to your PDF pipeline. For the rest, markdown() fetches the paper page as Markdown, one request per paper.
- type 'scholar' returns title, url, description, publisher, year, citations, id and an optional pdf.
- page Later results pages for broad topics, one request each.
- limit Maximum number of papers per page.
- markdown() Reads one paper page as Markdown, one request per call.
Scholar results are what Google Scholar ranks for a query, not a complete bibliography, and the citation count is the one Scholar shows. Treat the corpus as a starting point and record the queries and date with every review. For filings rather than papers, search prior art in Google Patents with the same client.
Why a Scholar search API speeds up literature review automation
The slow part of a review is not reading, it is finding and triage. Structured results turn triage into a sort.
The academic research pattern sorts by citations and routes papers to PDF or Markdown the same way.
Turn the PDFs themselves into text with PDF and document to Markdown conversion, which takes the pdf.url as input.
When not to: Scholar results do not include reference lists, author profiles or h-index metrics. Each result is a paper with its citation count, and a systematic review still needs the databases its protocol names.
FAQ
How do I get citation counts from a Google Scholar API?
Can I download the papers found through the Scholar API?
How do I automate a literature review with Scholar results?
How many requests does a Google Scholar search cost?
Is the Microlink Scholar API affiliated with Google?
Solve the next problem with the same API
Prior art search in Google Patents
Live search to ground LLM answers
Keyword research with Google Autocomplete
PDF and office documents to Markdown
LLM context from any URL
Archive web articles as PDF
Ready to automate your literature review?
Papers, citation counts and PDF links from Google Scholar as JSON. Get a Pro key and build your first corpus today.