DEV Community

Multigrid
Multigrid

Posted on • Originally published at multigrid.ai

Extracting Deposition Testimony Into a Structured Fact Table

A deposition summary is prose somebody has to trust. A fact table is different: every row carries a page and line reference, so anybody can open the transcript at that spot and see whether the row is right. The citation is not decoration on this document. It is the entire reason the output is worth building.

The citation is the unit of work

Deposition transcripts are laid out to a fixed convention: numbered pages, and on each page a fixed number of numbered lines — twenty-five in the most common format. Testimony is cited as page and line, written 112:4 for a single line, 112:4-14 for a span within a page, and 112:22-113:6 for a span that crosses a page boundary. That convention exists so that two people reading different copies can point at the same words.

It also gives you something rare in document extraction: an output field that can be validated against the input mechanically. If a row claims that the witness said something at 112:4-9, you can go back to lines 4 through 9 of page 112 and check that the quoted words are there. A row that fails that check is discarded before a human ever sees it. Build the pipeline so that check is possible and the quality problem largely takes care of itself; skip it and you have written a summariser with extra steps.

Line numbers live in the gutter, not the text

The line number is printed in a left-hand gutter, in its own column, visually separate from the testimony. Most text extraction flattens that into the line, producing strings like 12 Q. Did you review the report? — and the obvious fix, stripping a leading number with a regular expression, is wrong. A line of testimony can legitimately begin with a number, and 1985 was the year we moved becomes was the year we moved under that rule, silently.

Separate the gutter by horizontal position instead. Every line number on a page sits in the same narrow band of x-coordinates; the testimony starts to the right of it. A word-level extraction with coordinates gives you the split reliably. It is the same coordinate-clustering problem as a two-column PDF whose reading order is not visual order, and it belongs to the ingestion layer alongside PDF parsing.

Two more layout facts about transcripts that break naive pipelines:

  • Condensed transcripts. A four-up or two-up condensed printing puts several transcript pages on one sheet. The PDF page index is then not the transcript page number, and every citation your pipeline emits is wrong by a factor. Always read the printed page number from the page header; never use the file’s page index as a citation.
  • The word index. Transcripts often end with a concordance: an alphabetical list of every word with page and line references. It is dense, it is full of page:line pairs, and fed to a model it produces confident nonsense. Detect and exclude it, along with the front matter, the appearances page and the reporter’s certificate, before extraction begins.

A fact in the question is not testimony

Transcripts alternate Q. and A., with named speakers in capitals for colloquy between counsel. That structure carries meaning your table has to respect: an assertion made in a question is counsel’s assertion, not the witness’s evidence.

The common pattern is that the fact lives across the pair. A question states a proposition and the answer is “Yes”. The row you want records the proposition, attributes it to the witness by adoption, and cites the whole exchange rather than just the one-word answer. So the fact record needs a citation span that can cover both lines and a field distinguishing an assertion the witness made unprompted from one they adopted.

Colloquy is the other exclusion. Lines beginning MR. ALVAREZ: Objection, form. are not testimony, nor are the reporter’s parentheticals, nor the deposition officer’s statements. Classify each line by speaker role first — witness, examining counsel, other counsel, reporter — and only pass witness-adjacent content to the extraction step.

Building the table

  1. Normalise to line records. Produce one record per transcript line with the printed page number, the line number from the gutter band, the text, and a speaker role. This is the only step that touches coordinates, and everything downstream depends on it being right.
  2. Drop everything that is not the examination. Front matter, appearances, exhibit index, the certificate page and the word index. Keep the dropped ranges in a manifest so a reviewer can confirm nothing real was cut.
  3. Group into exchanges. One question and its answer, including follow-on lines. An exchange is the smallest unit that carries a complete fact, and it is the right size to send to a model.
  4. Extract per window, with the citations in the input. Send a window of a few exchanges with every line prefixed by its own page and line, and require the model to return the span it used. It cannot invent a citation it was not given if the validator checks the span exists in the window.
  5. Validate the quote against the source. For each returned row, reassemble the text of the cited lines and check that the quoted words appear in it. Reject rows that fail, rather than correcting them — a row whose citation does not match is a row whose content you have no reason to trust.
  6. Sort, de-duplicate and store. The same fact is often given twice under different questioning; keep both citations against one fact rather than two rows, and sort by citation so the table reads in transcript order.
{
  "fact": "The witness signed the inspection report on 4 May 2021.",
  "cite": { "page_start": 112, "line_start": 4, "page_end": 112, "line_end": 9 },
  "attribution": "adopted",          // "stated" | "adopted"
  "quote": "Q. And you signed it on the fourth of May?  A. Yes, that is my signature.",
  "topic": "inspection_report",
  "exhibit_refs": ["P-14"]
}

// validator: the quote must be recoverable from the cited lines
const source = lines
  .filter((l) => within(l, row.cite))
  .map((l) => l.text)
  .join(" ");
if (!normalise(source).includes(normalise(row.quote))) reject(row);
Enter fullscreen mode Exit fullscreen mode

The topic field is what makes the table sortable and is the one field with no ground truth in the document, so treat it as a classification with its own per-field confidence rather than as an extracted value. Constraining it to a fixed list beats free text, for the reason set out in structured output support.

A single day of testimony is two to three hundred transcript pages, which becomes a few hundred windowed requests, and a matter has several depositions in it. That is a long, resumable batch where the things that decide whether it finishes are per-request cost visibility, a spend cap, and somewhere to fail over when one provider starts returning rate-limit errors halfway through. Multigrid gives you those across providers behind one API and one key, so re-running the tail of a failed batch on a different model does not mean a second integration.

Errata, and what invalidates a table

A witness may submit changes to the transcript on an errata sheet after reading it, listing page, line, the original text and the change. A fact table built from the transcript alone is therefore provisional until the errata period has run.

Model errata as amendments keyed to page and line, not as edits to the transcript text. Keeping both means a row can display the original testimony and the change together, which is what a reader actually wants, and it means your validator still passes against the original source file. Overwriting the transcript instead breaks every citation check you built the table around.

Two other events invalidate rows. A corrected or replacement transcript issued by the reporter can shift pagination, which is why the transcript file’s identity must be stored on every row. And a rough or realtime draft has no stable pagination at all, so rows built from one should be marked as such and rebuilt against the certified transcript when it arrives.

Related

Top comments (0)