Before an AI agent touches a repo, give it a verifiable context pack

I published one Doramagic internal tool as a public repository:

https://github.com/tangweigang-jpg/doramagic-ai-context-pack-benchmark

The project is not trying to make an AI read more code. It is trying to solve a narrower problem: before an AI agent takes over a repository, compile the facts, boundaries, evidence links, risk cards, and continuation checks into assets that can be inspected.

That boundary matters.

This benchmark does not install the target project. It does not simulate runtime behavior. It does not claim that the target project works in production. Repomix is only used as the source packaging layer. Doramagic owns the schema, rendering, validation, and consumer-facing contract above it.

I think of it as a pre-install handoff pack for AI agents, not a universal repo summarizer.

For each project, the compiler can produce:

• CLAIM_GRAPH.json: separates claims from evidence instead of treating README language as fact.
• EVIDENCE_INDEX.json: keeps evidence snippets and source paths available for review.
• CAPABILITY_CONTRACT.json: describes capabilities under evidence constraints.
• AI_CONTEXT_PACK.md: host-AI readable context.
• CONTINUE_CHECK.md: a decision card for “should I continue now?”
• PROMPT_PREVIEW.md: a consumer prompt to try once before installation.
• HOST_CONSUMPTION_REPORT.json: checks whether the host AI received enough structure.
• validation_report.json and human_ai_consistency_report.json: schema, evidence, positioning, and Human Manual consistency checks.

The design choice is simple: compress noise, not judgment-critical context.

Many repo summary tools optimize for shorter descriptions. In practice, agents usually fail because the context is under-structured rather than merely too long. Boundaries are vague. Evidence is detached from claims. Risk cards are missing. The entry prompt is not reusable. The agent moves quickly, but it crosses lines it should have been forced to see.

So the benchmark keeps task routes, role and Skill indexes, prompt recipes, evidence snippets, boundaries, risk cards, and Continue Check decisions. The goal is not to look complete. The goal is to make downstream agents do fewer plausible-but-unsupported jumps.

For this public release I kept the checks explicit:

• The target GitHub repository was checked before creation.
• The staging copy was scanned for real API keys, PATs, PEM private keys, and .env files.
• Runtime artifacts, private operator state, caches, and local environment files were excluded.
• Python unit and syntax checks passed; the AI Context Pack suite passed 41 tests.

My current view: context engineering should not mean dumping larger folders into larger windows. The useful layer is a compiler that decides which facts can be consumed, which claims need evidence, and which operations should stop until a human or stronger check intervenes.

Repository:
https://github.com/tangweigang-jpg/doramagic-ai-context-pack-benchmark