From Twelve Sources to One Decision Record: A Reproducible Research Workflow

Technical research often produces an impressive pile of inputs and a surprisingly fragile conclusion.

You have RFCs, vendor docs, benchmark posts, meeting recordings, issue threads, and notes from a proof of concept. Then someone asks why the team chose option B, and the answer is buried in a chat transcript.

The fix is not another summary. It is a workflow that preserves the path from source to claim to decision.

Define the decision boundary

Start with a small manifest:

decision: "choose an event delivery approach for internal services"
owner: "platform team"
review_date: "YYYY-MM-DD"
criteria:
  - operational complexity
  - delivery guarantees
  - debugging experience
  - migration cost
out_of_scope:
  - external customer events
  - analytics pipelines

The out_of_scope section matters. Without it, every source expands the problem and the research never converges.

Checkpoint: a teammate should be able to tell whether a new source belongs in this inquiry without asking you.

Build a source registry

Do not paste everything directly into a narrative. Register each source first.

id,type,title,owner_or_publisher,date,question,limitation
S01,rfc,Current delivery architecture,Platform Team,YYYY-MM-DD,current state,may be outdated
S02,video,Incident review 42,SRE,YYYY-MM-DD,failure modes,partial attendance
S03,web,Vendor retry semantics,Vendor,YYYY-MM-DD,delivery guarantees,vendor-authored
S04,notes,Prototype observations,Developer A,YYYY-MM-DD,migration cost,small test

This registry does three useful things:

1. It gives every claim a stable source ID.
2. It makes source limitations visible before synthesis.
3. It prevents five pages repeating one vendor claim from looking like five independent pieces of evidence.

Extract claims, not highlights

A highlight depends on its original page. A claim should be understandable on its own.

Use a structure like this:

claim_id: C07
text: "The prototype required manual replay handling for one tested failure path."
source_ids: [S04]
kind: observation
scope: "single prototype; not a production benchmark"
confidence: low

Keep kind explicit. Useful values include:

• documented_fact
• observation
• interpretation
• vendor_claim
• open_question

This is intentionally less elegant than a generated paragraph. The structure makes overstatement harder.

Run a contradiction pass

Most synthesis prompts ask for themes or summaries. Add a separate contradiction pass:

Using only the registered sources:
1. Group claims that agree.
2. Identify claims that appear to conflict.
3. Check whether each conflict is explained by different scope, date, or definitions.
4. List missing evidence that could change the decision.
5. Do not resolve disagreement unless the sources support the resolution.

This prevents “helpful” prose from smoothing away the most important part of the research.

For example, one benchmark may show higher throughput while an incident review shows worse recovery behavior. Those findings are not necessarily contradictory. They may measure different operational qualities. The decision record should preserve both.

Create a decision matrix with evidence IDs

Criterion	Option A	Option B	Evidence	Confidence
Operational complexity	Lower initial setup	More managed components	S01, S04	Medium
Delivery behavior	Requires application handling	Built-in mechanism documented	S03, S04	Low
Debugging	Existing team familiarity	Better tooling claimed	S02, S03	Low

Do not convert confidence into a fake numeric score unless your team has defined what the numbers mean. “Low” plus a reason is usually more honest than 7.4/10.

Produce an architecture decision record

The final artifact can be short:

# ADR: Event delivery approach

## Status
Proposed

## Context
What problem and constraints led to this decision?

## Decision
What are we choosing?

## Evidence
- C02 (S01, S02): ...
- C07 (S04): ...

## Consequences
What becomes easier, harder, or newly required?

## Uncertainty
Which claims are weak or disputed?

## Validation plan
What test could change this decision?

## Review date
When should the team revisit it?

The source registry can remain detailed. The ADR should expose only the evidence necessary to understand and challenge the choice.

Where an AI knowledge workspace helps

The mechanical work increases when inputs span PDFs, webpages, images, audio, video, and notes. An AI workspace can help extract, organize, compare, and retrieve the material—but only if the workflow keeps source context visible.

iWeaver is designed for multi-format knowledge work, AI chat over source material, summaries, structured knowledge, and mind maps. It can be useful for bringing heterogeneous research into one inquiry and turning it into a reusable output. Important technical claims should still be checked against the original documentation, code, or experiment before they enter an ADR.

Verification checklist

Before merging the decision record, confirm:

• every material claim has at least one source ID;
• vendor statements are labeled as vendor statements;
• dates and version-specific behavior are recorded;
• conflicting evidence remains visible;
• generated suggestions are not presented as observations;
• the validation plan could genuinely overturn the decision;
• the review date has an owner.

The aim is not to automate judgment. It is to make judgment inspectable. A good research workflow leaves behind more than an answer: it leaves enough evidence for the next engineer to understand, challenge, and update it.

---

Disclosure: This article was created with AI assistance for the iWeaver content team, then reviewed and edited for factual accuracy and practical usefulness.