How Many AI Papers Appear Each Year, and the Query That Counts Them

#ai #discuss #llm #machinelearning

Paper counts are one of the few AI statistics anyone can compute from scratch, because the bibliographic databases are free and have public APIs. This page gives the queries. It does not give the answer, because the answer changes weekly and because five counting decisions move it by a factor of three.

Why there is no single number

“AI papers” has no definition that survives contact with a database. A paper can be counted by preprint server category, by keyword search, by venue, by a machine-learned topic classifier, or by author affiliation, and those five methods select overlapping but substantially different sets. Every published figure is one of them, and almost none of them says which.

What you can do is compute a number whose definition you control, state the definition alongside it, and re-run it later on the same definition. That is a defensible statistic. A figure lifted from an article is not.

Counting arXiv, exactly

arXiv is where most machine learning research appears first, its API is free and unauthenticated, and the query below returns a total count without downloading any records.

# Papers with cs.LG as primary or cross-listed category, submitted in 2025.
# max_results=0 returns metadata only; read <opensearch:totalResults>.

curl -s 'http://export.arxiv.org/api/query?search_query=cat:cs.LG+AND+submittedDate:[202501010000+TO+202512312359]&max_results=0'

# The count is in this element of the returned Atom feed:
#   <opensearch:totalResults>NNNNN</opensearch:totalResults>

# The categories that matter for this question:
#   cs.LG   machine learning
#   cs.CL   computation and language (NLP, and where most LLM work goes)
#   cs.AI   artificial intelligence
#   cs.CV   computer vision
#   cs.NE   neural and evolutionary computing
#   stat.ML statistics: machine learning

# Extract the number:
curl -s 'http://export.arxiv.org/api/query?search_query=cat:cs.CL+AND+submittedDate:[202501010000+TO+202512312359]&max_results=0' \
  | grep -o '<opensearch:totalResults[^>]*>[0-9]*' \
  | grep -o '[0-9]*$'

Two things about that query will bite you. It matches papers where the category appears at all, primary or cross-listed, so summing across categories double-counts heavily — a typical LLM paper is listed under cs.CL, cs.LG and cs.AI simultaneously. And submittedDate refers to the original submission, so a paper revised in 2026 still counts under its first year, which is the behaviour you usually want and not the behaviour every tool has.

arXiv also publishes its own aggregate submission statistics by month across all subject areas, which is the right source for the denominator: AI papers as a share of all preprints is a far more meaningful series than the absolute count, because it controls for the growth of preprinting generally.

Counting the published literature

arXiv is preprints, skewed towards Western machine learning and away from applied AI in other fields. For the published record, three open databases have usable APIs.

# OpenAlex — free, no key required. meta.count is the answer.
curl -s 'https://api.openalex.org/works?filter=publication_year:2025,title_and_abstract.search:large%20language%20model&per-page=1'
# ->  {"meta":{"count":NNNNN, ...}, "results":[...]}

# Crossref — free, no key. message.total-results is the answer.
curl -s 'https://api.crossref.org/works?query=machine+learning&filter=from-pub-date:2025-01-01,until-pub-date:2025-12-31&rows=0'

# Semantic Scholar Graph API — free, key recommended for rate limits.
curl -s 'https://api.semanticscholar.org/graph/v1/paper/search?query=large+language+model&year=2025&limit=1'
# ->  {"total": NNNNN, ...}

# Always send a contact address; these services ask for it and give
# better rate limits in return:
curl -s -H 'User-Agent: my-analysis/1.0 (mailto:[email protected])' ...

Use more than one and compare. If OpenAlex and Crossref disagree by less than about ten per cent your definition is probably robust; if they disagree by a factor of two, your query is matching different things in each and the number is not usable.

OpenAlex (OurResearch) is the most complete open index and carries topic classifications, institution and funder metadata, and citation counts. It is the one to start with.
Crossref covers registered DOIs, so it is publisher record rather than discovery: good for journals and conferences with DOIs, poor for anything without.
DBLP is computer-science specific, curated by hand, and therefore cleaner and narrower than either. Best when you want conference proceedings counted properly.
The Stanford HAI AI Index publishes annual publication counts assembled from these kinds of sources with its method documented, which makes it a useful cross-check on whatever you compute.

Five decisions that change the answer

Preprints, publications, or both. Most machine learning papers exist as both a preprint and a proceedings entry. Counting both double-counts; counting only published work misses most of the field’s recent output, because the preprint arrives a year earlier.
Primary category or any category. On arXiv this alone can change a count by more than half. Decide, and say which you used.
Keyword or classifier. A title-and-abstract keyword search finds papers that use the phrase; a topic classifier finds papers about the topic. The overlap is far from complete in both directions, and keyword searches are biased towards whatever terminology was fashionable that year.
Submission date or publication date. They can differ by more than a year, so a series built on one is shifted relative to a series built on the other.
Version handling. A paper with six arXiv versions is one paper. Some naive scrapers count versions, which inflates recent years because recent papers are still being revised.

Turning counts into a growth rate

The count is less interesting than its rate of change, and the conversion is worth doing properly because the headline claims in this area are usually about doubling times.

Compound annual growth rate from two counts, y years apart:

  CAGR  =  (end / start) ^ (1 / years)  -  1

Doubling time in years, from a growth rate g:

  doubling_time  =  ln(2) / ln(1 + g)

Worked at three plausible growth rates:

  g = 0.20  ->  ln(2)/ln(1.20)  =  0.693/0.182  =  3.8 years
  g = 0.35  ->  ln(2)/ln(1.35)  =  0.693/0.300  =  2.3 years
  g = 0.50  ->  ln(2)/ln(1.50)  =  0.693/0.405  =  1.7 years

The sensitivity is the point. A growth-rate estimate that is 15
percentage points off produces a doubling-time estimate that is off by
more than two years, so quote the growth rate with its interval or do
not quote a doubling time at all.

Always compute the share as well as the count. If AI papers grew 40 per cent and all preprints grew 30 per cent, the interesting quantity moved by about eight per cent, not forty.

What a paper count does not measure

It does not measure progress. Volume and advance are different quantities, and the correlation between them is weak enough that treating one as evidence of the other is a category error.
It is inflated by incentives. Publication counts drive hiring, funding and rankings, which produces salami-slicing and minimal-increment papers. The counts are real; what they count has changed.
Coverage is uneven by geography and language. The open indexes cover English-language and Western-indexed venues best. A count of “world AI research” from these sources is a count of the part of it these sources see.
The most consequential work is often not a paper. A model card, a system card, a technical report without peer review, or a released set of weights can matter more than any indexed publication, and none of them appears in these counts. This is a genuine and growing gap, discussed in arXiv culture and open science in AI.
Citation counts inherit every problem above and add more. They lag by years, favour survey papers, and are gamed. Use them for finding things to read, not for ranking anything.

Any number you compute from these APIs is correct as of the day you ran it and will differ next month, both because new papers arrive and because the indexes backfill. Record the query, the database and the date you ran it beside the number; without those three it is not reproducible and therefore not a citation.