What 612 Diagram-Reconstruction Runs Taught Us About OCR and Connectors

Turning a diagram image into editable objects sounds like one vision-model call: ask for boxes, text, and arrows, then write a file. That prototype works on a clean flowchart. It breaks on the diagrams people actually keep in architecture decks: small labels, repeated icons, nested containers, parallel edges, and lines routed through crowded corridors.

To find where quality was being lost, we ran a controlled benchmark for LayerBack's image-to-editable-diagram pipeline: 336 recognition runs plus 276 topology runs, 612 model runs in total, across 12 diagrams. Ten fixtures had exact ground truth and two were real-world diagrams.

The goal was not to declare one universal “best model.” It was to answer narrower engineering questions:

• Does an OCR inventory help a multimodal model find every node?
• Does more reasoning effort improve structured extraction?
• Which model should observe topology?
• Is a second model needed to judge candidate edges?
• Can computer vision safely skip topology analysis on diagrams with no lines?

The pipeline we tested

The baseline separated the task into stages:

1. OCR extracts text and coordinates.
2. A multimodal model recognizes nodes, types, positions, and styles.
3. Deterministic code merges OCR text into the recognized structure.
4. A topology observer traces visible connectors against a catalog of known nodes.
5. A judge optionally accepts or rejects candidate edges.
6. Segmentation isolates icons or custom graphics.
7. Deterministic serializers produce draw.io, VSDX, PPTX, SVG, and a preview.

This separation made failures measurable. A missing label is not the same problem as a hallucinated connector, and neither should be “fixed” by asking one prompt to do more.

Lesson 1: OCR is not a fallback; it is grounding

The strongest recognition configuration paired a vision model with an explicit OCR inventory. In the benchmark, terra-low-ocr reached average and minimum node recall of 1.00 / 1.00 across its 33 measured runs. The active fast configuration, luna-low, averaged 0.93 but had a minimum of 0.32 on a difficult case.

The important detail is not the model name. It is that visible text is a useful checklist. A vision model can understand a diagram globally and still omit a small database label or a row of repeated cards. Supplying OCR text and coordinates constrains the extraction: every readable label must be explained by a node, edge label, title, or note.

OCR should not dictate topology. It should ground coverage.

Lesson 2: more reasoning was sometimes worse

We expected medium reasoning effort to beat low effort on dense diagrams. It did not. The low-effort, OCR-grounded configuration was the only zero-miss setup in the tested recognition matrix. Higher reasoning sometimes gave the model more freedom to reorganize or summarize the image instead of transcribing its structure faithfully.

This is a recurring structured-vision pattern: the desired output is not the most elegant interpretation. It is the least surprising inventory that matches the pixels.

Prompting should therefore reward coverage, stable identifiers, and coordinate fidelity—not abstraction.

Lesson 3: topology observers tied, but cost differed by 20×

Five topology-observer configurations were effectively tied on quality. Representative precision/recall results were around 0.95–0.97 precision and 0.97–0.98 recall.

Cost, however, varied sharply:

• Gemini 3.6 Flash observer: about $0.0147 per run in the benchmark.
• Gemini 3.1 Flash-Lite: about $0.0028.
• Luna observer: about $0.0007.

When quality differences are within noise, architecture should choose the cheaper observer and keep a different provider as a fallback. Model prestige is not a performance metric.

Lesson 4: the topology judge added almost no value

We used a second model to judge every candidate connector from the observer. Across the tested observers, precision changed by no more than about 0.02, and sometimes became worse. The observer's hallucination rate was already low.

Removing the judge from the proposed architecture saved one model call and roughly 8–10 seconds without a measured quality loss.

The general rule: do not add an “LLM verifier” because verification sounds safe. Measure whether it changes the error distribution. A verifier that agrees with the first model is latency, not redundancy.

Lesson 5: a cheap CV precheck can skip expensive work safely

A line-detection precheck correctly classified 11 of 12 fixtures. More importantly, it had zero errors in the dangerous direction: whenever it predicted “no connectors,” the diagram truly had no connectors.

That asymmetry matters. A false “has lines” prediction only loses a speed optimization; a false “no lines” prediction would skip necessary topology work and corrupt the output.

Safe early exits should be optimized for the cost of each error direction, not overall accuracy alone.

Lesson 6: some topology errors are in the source pixels

Two generated fixtures originally routed elbow connectors through sibling nodes while the ground truth described a fan-out. Every observer read the visible pixels as a chain. The models were consistent—the fixture was wrong.

After rerouting those lines through clear corridors, the benchmark aligned with the intended topology.

This exposed a product truth: a diagram can be logically intended one way and visually drawn another way. If a connector crosses a node border or shares a narrow corridor with several edges, even a human may infer the wrong endpoint.

For reconstruction systems, source ambiguity must be surfaced for review rather than hidden behind a confidence score.

What we changed in the architecture proposal

The experiment suggested a faster staged design:

• run OCR and a fast recognizer in parallel;
• use an OCR-grounded high-recall hedge only for dense or low-quality cases;
• run a cheap CV precheck before topology;
• use one low-cost topology observer, with provider fallback;
• remove the separate topology judge;
• overlap segmentation and topology work;
• keep serialization deterministic and validate output packages.

The projected change in the benchmark report was roughly 47 seconds to about 30 seconds for a typical path, with estimated per-conversion model cost moving from roughly $0.03 to $0.015. These were architecture estimates from the experiment, not a guarantee for every production image.

Reproducibility and caveats

The dataset was small: 12 diagrams, with generated fixtures overrepresented because exact truth was needed. The benchmark is strong enough to make pipeline decisions for this product, but not to rank general-purpose vision models across every diagram domain.

Useful next steps include:

• more real-world, multilingual, and low-quality screenshots;
• stratified scores for text density, edge crossings, and icon count;
• package-level tests in Visio, PowerPoint, LibreOffice, and draw.io;
• human review time as a metric, not only node/edge precision and recall;
• failure queues that feed ambiguous samples back into the benchmark.

The broader takeaway

Editable-diagram reconstruction is not “OCR plus SVG tracing.” It is a structured perception problem with at least three separate truths:

• semantic truth: what objects the diagram represents;
• geometric truth: where those objects appear;
• topological truth: how they connect.

The best pipeline we tested did not ask one expensive model to solve all three. It grounded coverage with OCR, specialized topology analysis, removed an unhelpful judge, and left file generation to deterministic code.

That architecture powers the practical product problem: turning a diagram screenshot into editable VSDX, PPTX, draw.io, and SVG. The benchmark matters because a pretty preview is easy; a file whose labels, shapes, and connectors survive the next edit is much harder.