Counting "repeated questions" in exam PDFs: the rule that keeps the numbers honest

When I tallied GTU (Gujarat Technological University) Physics papers for juniors, the hardest part was not the code. It was deciding what "repeated" means, so the numbers do not lie to you.

The setup

22 official GTU Physics question papers, Summer 2022 to Summer 2026, across three subject codes (3110011, 3110018 and the new BE01000021), downloaded from GTU's own repository. Text came out with pdftotext, then every sub-question, including the OR options, got a topic tag.

Rule 1: count a topic once per paper

If Ruby laser shows up in Q.3(c) and again in Q.5(a) OR, that is still one paper. Otherwise a single paper that loves lasers inflates the count.

from collections import defaultdict

def papers_per_topic(tagged):
    # tagged: list of (paper_id, topic) for every sub-question
    seen = defaultdict(set)
    for paper_id, topic in tagged:
        seen[topic].add(paper_id)
    return {t: len(p) for t, p in seen.items()}

The set does all the work. Topic count = number of distinct papers it appeared in.

Rule 2: deduplicate identical files

Two BE01R00021 PDFs turned out to be byte-for-byte the same files as BE01000021 papers (the header literally says "BE01000021/BE01R00021"). Counting them twice would have doubled the evidence for free. A hash check catches this:

import hashlib, pathlib

def unique_pdfs(folder):
    seen = {}
    for f in sorted(pathlib.Path(folder).glob("*.pdf")):
        h = hashlib.sha256(f.read_bytes()).hexdigest()
        seen.setdefault(h, f)
    return list(seen.values())

So BE01000021 ended up with 4 papers, not 6.

Rule 3: say when extraction failed

A few lines in two 3110018 papers (Summer 2025 and Summer 2026) did not come out as text. Those two may be slightly under-counted, and the write-up says so instead of pretending.

What the honest numbers showed

For 3110018 (Group 2 Physics), solar cell or photovoltaic effect appeared in 9 of 9 papers. Kronig-Penney, hot point probe and four probe method each hit 7 of 9. Some questions repeated word for word, like "Kronig and Penney model explains origin of band gap" in W2022, S2024 and W2025.

Past repetition is a pattern, not a promise. But with these three rules, at least the pattern is real.

I put the papers on GTU Hero if you want to check the Winter 2025 one yourself.