ReproAgent: Contract-Guided Paper-to-Code Reproduction (arXiv 2608.24291, Findings of EMNLP 2026) argues that scientific agents fail at paper-to-code because the spec is split: explicit content (algorithms, metrics, artifacts) gets lost across long trajectories, while implicit details (framework defaults, conventions from related work) aren't in the paper at all.
Their fix is a persistent implementation contract with two channels — an implementation-requirement channel that turns paper snippets into code obligations, and a reference-evidence channel that pulls structure/interface evidence from related repos — bound to work packages, projected into file-level contracts, and consumed across generation and repair. It tops PaperBench Code-Dev among same-backbone scaffolds.
So I built paper-to-repo, a daily-build-grade Python CLI implementing that exact pipeline: prepare → plan → generate → verify → repair, plus a FastAPI + React web UI that drives the same stages in the browser (Agno orchestrates the stage agents when a DeepSeek key is present).
The heart is a real, inspectable contract.json: work packages, obligations with machine-checkable predicates (file exists / function present / signature matches / test passes), reference evidence from a checked-in bundle (FAISS, Annoy, CommonCrawl conventions), and a live contract score.
The honest part is the repair loop. On the bundled sample paper (an LSH index with two algorithms, a Recall@K metric, and a data loader), generation emits the algorithms and metric correctly but the data loader as a placeholder — so the contract-verifier scores 3/4 (75%).
Repair regenerates only the failing file, re-verifies, and the contract ends at 4/4 (100%). The before/after is recorded in score.history in the contract JSON itself, so it's reproducible, not hand-waved.
What I like most: the contract makes the pipeline legible end to end. repro status prints a table of work package / file / obligation / status, so you can see exactly which promise is broken and when repair fixed it.
It's a focused primitive (~700 LOC of core), not a framework — the paper lives in the repo (PAPER.md + paper/), inference is just an OpenAI-compatible endpoint from .env, and the whole demo runs offline in deterministic mode if you haven't dropped in an API key yet.
Try it: pip install -e ".[dev]", then bash demo/run_demo.sh — you'll watch the contract score climb from 75% to 100% in one repair iteration.
Code & more: https://www.dailybuild.xyz/project/234-paper-to-repo
Top comments (0)