Build a Citation-Checked Research Report in Python with Evidence Graph Studio

Polished technical writing can still hide a basic failure: a sentence has no source, a citation points to a missing record, or two sources disagree and the draft does not show it.

If you are reviewing AI-assisted research, a pile of links is not enough. You need to see which source supports which claim, which claims need review, and what a writer can cite in the final report.

This tutorial uses Evidence Graph Studio, a small MIT-licensed Python project that turns a JSON research dossier into a claim-to-source graph, a Markdown report, a citation pack, JSON graph data, and Mermaid source. It is deterministic and uses only the Python standard library.

TL;DR

Create a dossier containing sources, claims, and evidence links. Run the validator and builder. A successful build exits with status 0 when every claim is supported. A dossier that needs review exits with status 2, so a CI job can stop before an unsupported claim reaches publication.

Prerequisites

You need Python 3.10 or newer, Git, and a shell. The project does not require a database, API key, model download, or network access at runtime.

Clone the repository and enter it:

git clone https://github.com/paladini/evidence-graph-studio.git
cd evidence-graph-studio

The repository currently documents version 0.1.0 in pyproject.toml, requires Python 3.10 or newer, and is released under the MIT license. The commands below target the current default branch because the project has no GitHub release.

Run the bundled example

The repository includes a dossier about AI agent reliability. Set PYTHONPATH to the source directory, then validate the input before generating output:

$env:PYTHONPATH = "$PWD\src"
python -m evidence_graph_studio validate `
  --dossier examples\ai-agent-reliability-dossier.json

The validator checks the dossier structure without writing reports. The bundled example contains three claims and three sources.

Now build the complete evidence pack:

python -m evidence_graph_studio build `
  --dossier examples\ai-agent-reliability-dossier.json `
  --out build\evidence-report.md `
  --graph-out build\evidence-graph.json `
  --citations-out build\citation-pack.md `
  --mermaid-out build\evidence-graph.mmd

The command writes four artifacts:

• build/evidence-report.md contains summary counts and per-claim findings.
• build/evidence-graph.json preserves the graph for downstream tools.
• build/citation-pack.md groups citations by source and claim.
• build/evidence-graph.mmd can be rendered in documentation or a pull request.

The documented example produces 3 supported, 0 need review and exits successfully.

Understand the dossier shape

The input is deliberately explicit. A source has an ID, URL, summary, tags, and optional quotes. A claim has an ID, text, required tags, and evidence entries that point back to source IDs.

Here is a minimal dossier:

{
  "project": "Release review",
  "sources": [
    {
      "id": "source.release-notes",
      "title": "Release notes",
      "url": "https://example.com/release-notes",
      "summary": "The release adds a validation command.",
      "source_type": "documentation",
      "tags": ["release"],
      "quotes": ["The validation command checks the input dossier."]
    }
  ],
  "claims": [
    {
      "id": "claim.validation",
      "text": "The tool can validate a dossier before building reports.",
      "required_tags": ["release"],
      "evidence": [
        {
          "source_id": "source.release-notes",
          "stance": "supports",
          "note": "The release notes describe the validation command.",
          "quote": "The validation command checks the input dossier."
        }
      ]
    }
  ]
}

Evidence uses one of three stances: supports, contradicts, or context. That distinction matters. A source that mentions a topic is not automatically support for a claim, and a contradiction should remain visible for review rather than being silently discarded.

Optional links can express relationships between claims, such as depends_on. The output graph therefore captures more than a flat bibliography: it records the reasoning structure that a reviewer needs to inspect.

Verify the result in a repeatable way

The project includes seven unit tests. Run them from the repository root:

python -m unittest discover -s tests

For a practical check, assert that the generated report and citation pack exist, inspect the summary, and review the Mermaid graph. A useful CI policy is to fail when the build exits with status 2. That preserves the distinction between a valid report and a report that was generated but still needs human review.

You can also inspect the JSON graph in a later step without reparsing Markdown. That makes the output suitable for a documentation pipeline, a pull request check, or a custom review interface.

Why this works

The design makes support a data relationship instead of a formatting convention. Each evidence item names its source and stance. The builder can then detect missing source IDs, unsupported claims, contradictory evidence, and missing required topic tags consistently.

This is especially useful for AI-assisted drafting. A language model can help propose claims or source notes, but the dossier gives a reviewer a concrete object to validate. The generated Markdown is an output of that review model, not the model's memory of where a statement came from.

Failure modes and boundaries

An exit status of 2 is not a crash. It means the report contains claims that need review. Treat it as a gate in automation, not as proof that the sources are false.

The tool checks the structure and relationships you provide. It does not fetch URLs, verify that a quote is authentic, measure source quality, or decide whether a claim is true. Those remain research and editorial responsibilities.

The path is local and deterministic, but the input can still contain sensitive material. Do not place credentials, private customer data, or confidential research in a dossier that will be committed or uploaded. The MIT license permits reuse, but it does not change your obligations for the sources and data you process.

FAQ

Does it use an AI model?

No. Version 0.1 is documented as deterministic and uses the Python standard library.

Does it need internet access?

No runtime network access is required. The dossier stores source metadata and URLs, while the tool checks the relationships locally.

Can it replace fact checking?

No. It makes missing, conflicting, or incomplete support visible. A person still needs to inspect the source and decide whether the claim is accurate.

Can I delete old files automatically?

The current project is an evidence graph builder and report exporter. It does not describe a content deletion workflow. Keep cleanup and retention policy outside the build step until you have reviewed your own requirements.

Takeaway

If a research workflow ends with a Markdown draft and a loose list of URLs, the review boundary is hard to see. Evidence Graph Studio gives each claim an explicit relationship to its sources and turns unresolved support into a machine-checkable result.

Start with the bundled example, then adapt the dossier format to one small article or research brief. Keep the validator in CI and require human review whenever the build reports claims that need review.

I used AI assistance to organize and edit this tutorial. The commands, project behavior, version details, and limitations were checked against the public repository and a local run of its documented example.

What evidence relationship would be most valuable in your own review workflow: missing support, contradictions, required topic coverage, or claim dependencies?