Methodology

How the data is collected, classified, and presented.

1. Data sources

Papers come from the official proceedings of each tracked venue:

• OpenReview — ICLR (2018-)
• PMLR (proceedings.mlr.press) — ICML, AISTATS, COLT, CoRL, UAI, ACML, L4DC, MIDL, MLHC, PGM, CLeaR, ALT, AutoML
• ACL Anthology — ACL, EMNLP, NAACL, COLING, EACL, AACL, IJCNLP, CoNLL, SemEval
• CVF Open Access — CVPR, ICCV, WACV
• ECVA — ECCV
• NeurIPS Proceedings — NeurIPS (2006-)
• plus AAAI, IJCAI, JMLR, MICCAI, Interspeech, USENIX, RSS

Scrapers live in the researchpooler Python project. They fetch title, authors, year, venue, and PDF URL. Abstracts are extracted from proceedings pages or, where a venue offers one, its API (ICLR abstracts come from OpenReview) (88% of papers).

2. Topic classification

Papers with an abstract are eligible for classification into a 3-level taxonomy by an LLM working through a fixed set of steps stated in the prompt; the share actually classified is given under Limitations. The pipeline runs in two stages:

1. Pre-classification (PG, zero LLM cost): match title + abstract against 215k known keywords. The matched keywords vote for candidate top-level domains. This narrows the taxonomy from 752 nodes down to 2-3 relevant branches.
2. LLM classification: the model sees only the pruned sub-taxonomy and the full abstract, and returns selected topic paths and normalised keywords - one paper per request, never a batch. It runs on a local GPU machine rather than a hosted API, currently on granite4.1:8b. A topic path the model returns that is not in the candidate list is discarded rather than added to the taxonomy. Which model labelled a given paper is recorded on that paper - the split is under Limitations.

This two-stage design reduces tokens by 67-83% (measured on a 50-paper validation harness, RE-29) and eliminates off-list topic invention (122 of 323 used topics were off-list under the old single-pass design, 0% since) by avoiding the cognitive load of choosing from all 752 options at once.

3. Taxonomy evolution

The taxonomy is not static. When the LLM consistently proposes the same new topic that doesn't yet exist (e.g. "ML > Learning Types > In-Context Learning"), the topic is added automatically once it covers ≥5 papers (rake classify:evolve_taxonomy).

Most of the taxonomy came from that process, not from hand curation. The curated config defines 752 nodes across 15 top-level domains; the live taxonomy holds 752. Treat the deeper branches as machine-proposed structure that happened to recur, not as an editorially designed hierarchy. A leaf name that already exists under a different domain is no longer re-created as a separate node - but that guard only stops new splits, and 171 leaf names already sit under more than one top-level domain ("Anomaly Detection" is under five). So one concept is in places split across branches, and a per-topic count is a count of that branch, not of the concept.

The top-level domains also overlap by design of the source config - "Machine Learning", "Artificial Intelligence" and "Deep Learning" are siblings even though the third is a subset of the second and first. A paper therefore commonly sits in 2-3 domains, and "spans N domains" should be read as breadth of labelling, not evidence of interdisciplinary work.

4. Achievements and trends

Author achievements (28 badges, see /achievements) are computed by batch SQL over the papers/authors/topics/keywords graph (one badge, Dynamic Duo, reads a materialized view of co-author pairs). Rarity tiers are computed from real distribution (legendary < 0.1%, epic < 1%, rare < 5%, uncommon < 10%, common > 10%).

Emerging terms (/trends) come from n-grams in paper titles. We tokenise all titles, count year-over-year occurrences, and surface terms with min 5 occurrences over min 2 years and positive growth rate.

Two further rules keep the ranked lists meaningful. A term needs at least 10 occurrences in the previous year to be ranked by growth: a jump from 1 paper to 56 is "+5500%" and says nothing, and before this floor such terms owned the top of the list. Terms with no previous-year occurrences have no ratio at all, so they are listed separately by volume rather than given a percentage - a missing prior year is usually a gap rather than a debut, since every stored term appears in at least two years.

Growth is computed on raw counts and is not normalised by how much the corpus grew that year. Normalising was measured and rejected: dividing every term by the same year-over-year ratio is monotonic, so it changes the numbers without changing the order.

5. Co-occurrence graph (Neo4j)

215k keywords, 752 topics and the papers that carry keywords form a knowledge graph in Neo4j (papers without keywords are not exported, so the graph is smaller than the corpus). Edges:

• CO_OCCURS — two keywords share at least 2 papers (configurable minimum weight)
• SPECIALIZATION_OF — a keyword that contains another keyword as a substring points at the broader one (e.g. "convolutional neural network" → "neural network")
• ABBREVIATION_OF — abbreviation → full form (e.g. "NER" → "named entity recognition")

The /explore graph view runs Cypher queries against this graph for 8 different visualisation presets.

6. Limitations

• Sample bias: only 36 top venues. We miss preprints, regional conferences, and almost all journals and workshops - the two exceptions inside the set are JMLR (a journal) and SemEval (a workshop), both kept because they sit in the same citation neighbourhood as the conferences around them.
• Co-located events: the ACL Anthology publishes joint conferences (ACL-IJCNLP, AACL-IJCNLP) and co-located workshops in one volume that belongs to two venues' proceedings, so the scraper files the same PDF under both codes. Those rows are detected by identical PDF URL and counted once, under the venue named in the Anthology's own volume ID; the second filing stays reachable and redirects to it. This removes about 6,700 rows from every corpus-wide figure - and from the venue totals, which is why IJCNLP and AACL are much smaller here than a proceedings-count would suggest.
• Year coverage: each venue is indexed over a scrape window, not its full history - see the year range on every card in /conferences. Some venues start decades after their first edition (MICCAI, for example, begins in 2024 here), and early years inside a window can be thin.
• Author identity: authors are keyed by normalised name, so researchers who share a name are collapsed into one profile. Common names ("Yang Liu", "Wei Wang") therefore overstate paper counts, and every achievement and "top authors" list inherits that. Deduplication merges spelling variants only - it cannot split a shared name.
  
  What we can do instead: split by co-authorship. There is no ORCID, affiliation or email anywhere in this data, so nothing identifies a person directly - but the co-author graph carries a signal. Papers are joined when they share any other author, so one researcher's output chains together through recurring collaborators while several people under one name fall into disconnected groups. Where the largest group holds under half of a profile's papers, the profile is marked as several researchers, its papers are presented per group, and its achievement badges are withheld - a badge on a profile that is forty people describes nobody.
  
  The threshold is deliberately far from the range real researchers occupy, measured 2026-08-28: Sergey Levine 98% of papers in one group, Dacheng Tao 97%, Yoshua Bengio 94%, against Wei Wang 33% and Yang Li 11%. That buys precision at the cost of recall - Yang Liu sits at 76% and is not marked, because that profile is one very prolific researcher plus a tail of others, and splitting a real career is a worse error than declaring a merge. It is a heuristic over co-authorship, not an identity resolution: two researchers who never share a collaborator will be separated correctly, and one who changed field and team entirely may be separated wrongly.
• Abstract coverage: 12% of papers (mostly pre-2015) have no abstract. They are stored and browsable - hidden from the default listing and the search index, one click away in /papers?all=1 - but a title-only search cannot match text they do not have.
• Classification on title-only: not attempted. The classifier only runs on papers with an abstract; the rest are marked but never sent to the LLM. A small leftover set from earlier pipeline versions may still carry topics assigned before this policy, but going forward title-only papers stay unclassified rather than get a lower-confidence guess.
• Accuracy: 96% of papers with an abstract have been classified so far. There is no rigorous accuracy validation against a human-labeled benchmark yet - treat topic/keyword assignments as LLM-generated best guesses.
• LLM dependence: the prompts are version-controlled, but the model's own sampling is not pinned per run, so a re-run is not guaranteed to reproduce an earlier labelling. What IS recorded per paper is which model produced its labels:
  • not recorded (labelled before this was tracked) — 190,953 papers (96.1%)
  • granite4.1:8b — 7,616 papers (3.8%)
  • darwine/qwen3.8-27b — 94 papers (0.0%)
  Rows marked "not recorded" are not attributed to a model after the fact: naming one would be a guess, and a guess about provenance is worse than saying it is unknown.

7. Open source

The data pipeline is on GitHub: researchpooler (Python scrapers + LLM classification), a fork of Karpathy's original. The Rails application that serves this site is not published, so the pipeline is what you can inspect and reproduce.