Five Integration Failures I Hit While Building on SuperDocs

Kalpesh Sharma — Tue, 25 Aug 2026 08:01:05 +0000

I built two document workflows on top of SuperDocs: a closing-document reconciliation workflow and PaperForge, a multi-document research/reporting system with a human-reviewed editing step.

The happy path was straightforward:

document
→ SuperDocs edit
→ proposed change
→ human review
→ approval
→ export

The interesting engineering work showed up around that path.

The five problems below were not hypothetical edge cases. They came from real implementation and live integration work. None of them required replacing SuperDocs. They required being much more precise about the boundary between an external document API and the application built around it.

The biggest lesson was this:

A document-editing API can give you the right primitives, but the surrounding application still owns state, contracts, retry behavior, review UX, and the definition of “done.”

Scope note: I use “integration failure” deliberately. Some problems below exposed API-contract details I had misunderstood; others were defects in my own application assumptions around SuperDocs. I am not treating every item as a SuperDocs product bug.

Build context: Closing Document Reconciliation

The reconciliation build where I first exercised SuperDocs document upload, asynchronous editing, explicit approval, and export in a real workflow.

1. Additive provider metadata broke my strict pending-change model

What I was trying to do

PaperForge stores SuperDocs proposals in a strict immutable model.

The application only needs a small set of fields for each proposed change:

change_id
operation
chunk_id
document_id
old_html
new_html
ai_explanation
insertion anchors

I deliberately configured that internal Pydantic model with:

ConfigDict(
    extra="forbid",
    strict=True,
    frozen=True,
)

The intent was good: malformed persisted review state should fail loudly instead of quietly entering the application.

What broke

A live pending_changes response contained every field PaperForge expected plus additional provider-side diagnostics.

One response included a machine_checks object containing values such as:

claims
resolved
words_new
words_old
blocks_new
blocks_old
kept_chars_ratio

Those fields were useful metadata, but they were not part of PaperForge's domain model.

Because I was validating the entire provider object directly against a model with extra="forbid", an otherwise valid proposed change could fail validation simply because the provider had added more information.

The bad assumption

My mental model was effectively:

A SuperDocs pending-change object and my application's PendingChange object are the same contract.

They were not.

The external response belongs to the provider. The persisted domain object belongs to PaperForge.

Root cause

I put strictness at the wrong boundary.

The real shape is:

SuperDocs response
        ↓
integration boundary
        ↓
PaperForge-owned model

The provider should be allowed to add fields that PaperForge does not use.

PaperForge should still be strict about the fields it does depend on.

The fix

I kept the internal model strict and added an explicit projection at the integration boundary:

_PENDING_CHANGE_FIELDS = frozenset(
    {
        "change_id",
        "operation",
        "chunk_id",
        "document_id",
        "old_html",
        "new_html",
        "ai_explanation",
        "insert_after_chunk_id",
        "insert_before_chunk_id",
    }
)

Before validating the object, PaperForge now selects only the fields it owns:

def _normalize_pending_change(value):
    if not isinstance(value, Mapping):
        raise SuperDocsProtocolError(
            "SuperDocs returned invalid pending changes."
        )

    return {
        key: value[key]
        for key in _PENDING_CHANGE_FIELDS
        if key in value
    }

The projected value is then validated by the strict model.

So the boundary is tolerant of additive provider metadata, while the persisted contract remains strict.

How I proved it

I added a regression test using the same additive machine_checks shape from the live response.

The test verifies that:

the change still reaches awaiting_approval;
the expected review fields survive;
machine_checks does not leak into persisted review.json;
the internal model remains strict.

The useful pattern is:

accept provider evolution at the edge
→ project the fields you own
→ validate those fields strictly
→ persist the smaller stable contract

Code:

PaperForge review model

PaperForge review normalization

Regression test

2. A retry could have started the same SuperDocs workflow twice

What I was trying to do

Starting a review is an external side effect.

PaperForge sends a request to SuperDocs, receives a job_id, and polls that asynchronous job until it becomes actionable.

I wanted this endpoint to be safe to retry.

What broke

There is a dangerous failure window:

PaperForge starts review
        ↓
SuperDocs accepts it and returns job-1
        ↓
local process/request is interrupted
        ↓
client retries POST /review

If the retry simply starts again from the beginning, it can create a second provider job for the same report.

That can mean duplicated work, duplicated usage, two independent change sets, and uncertainty over which job local state should trust.

The bad assumption

The incorrect assumption was:

If my HTTP request did not finish, the external operation did not finish either.

A local timeout or exception tells me almost nothing about whether an external side effect already occurred.

Root cause

I had been thinking of review creation as a normal request-response operation.

It is actually a small distributed state machine:

A. no provider job exists

B. provider job exists and is processing

C. provider job reached an actionable or terminal state

A retry in state B must not behave like a request in state A.

The job_id is therefore not disposable response metadata. It is the recovery handle for the external side effect.

The fix

PaperForge persists review state and checks it before starting anything new.

If the state is already processing and has a job_id, the service polls the existing job:

existing = self._read_optional_state(report_id)

if existing is not None:
    if existing.status == "processing" and existing.job_id:
        return self._poll(existing)

    return existing

Only a report with no recoverable review state is allowed to start a new SuperDocs job.

The session ID is also deterministic:

session_id = f"paperforge-review-{report_id}"

The rule became:

Persist external identity before treating a retry as new work.

How I proved it

The regression test preloads review.json with:

ReviewState(
    report_id=report_id,
    session_id="paperforge-review-existing",
    job_id="job-1",
    status="processing",
)

Then it calls start_review() again.

The critical assertion is:

assert not client.start_calls

The retry polls job-1. It does not create another one.

This changed how I think about idempotency.

It is not just “support the same request twice.” It is:

Know exactly when the external side effect happened, persist its identity, and recover that same work on retry.

Code:

Review recovery logic

Recovery regression test

3. I inferred the approval write contract from the read contract

This was a smaller bug, but a useful integration lesson.

What I was trying to do

The review job returned a list of proposed changes, each with a change_id.

So the natural approval payload looked like this:

{
  "job_id": "job-...",
  "changes": [
    {
      "change_id": "change-...",
      "approved": true
    }
  ]
}

Each change already had an explicit decision. That looked semantically complete.

What broke

The approval endpoint also required a top-level approved value.

The working request therefore needed both:

payload = {
    "job_id": job_id,
    "approved": approved,
    "changes": changes,
}

The nested decisions described the human intent, but the endpoint still had its own required envelope.

The bad assumption

I assumed the write shape would mirror the read shape:

pending_changes[]
        ↓
changes[{ change_id, approved }]

But read and write contracts are not automatically symmetrical.

The actual request had another required field:

approval request
├── job_id
├── approved
└── changes[]
    ├── change_id
    └── approved

Root cause

I inferred the provider's transport schema from the application data I wanted to send.

That is fragile because action endpoints often contain workflow-level fields that appear redundant from the domain model's point of view.

The fix

I moved provider-specific request construction into the SuperDocs client.

The workflow decides what the human chose.

The client decides how that decision must be encoded on the wire.

In the Closing Document Reconciliation build:

payload = {
    "job_id": job_id,

    # Required even when every individual change
    # below has its own approval decision.
    "approved": approved,

    "changes": changes,
}

Later, PaperForge used a narrower one-change-at-a-time approval request:

payload = {
    "job_id": job_id,
    "change_id": change_id,
    "approved": approved,
}

In both cases, the provider adapter owns the exact request contract.

What I took away

Do not infer a write schema from a read schema, especially for action endpoints such as:

approve
reject
continue
commit
cancel

A field that looks redundant may still be part of the endpoint's actual workflow contract.

Code:

Closing-document SuperDocs client

Human-review demo flow

PaperForge SuperDocs client

4. A targeted edit produced too much HTML to be a useful human diff

What I was trying to do

PaperForge uses SuperDocs for a deliberately narrow editing step:

SuperDocs proposes one Executive Summary revision
        ↓
human sees BEFORE / AFTER
        ↓
human approves or rejects

I initially expected a targeted edit to naturally produce a small review payload.

What broke

The semantic change was small, but the old_html and new_html could include a large amount of surrounding document structure.

Dumping that raw content into the UI technically showed the provider response, but it made the human-review step worse.

A reviewer should not have to scan a document-sized block to discover what changed.

The product requirement became:

Show the smallest useful human-readable window around the changed text, while keeping the underlying provider proposal untouched.

The bad assumption

I had treated the provider's edit representation as if it were also the correct presentation model.

Those are different responsibilities.

The provider representation answers:

What document content changed?

The review UI must answer:

What does the human need to read to decide?

The fix

I added a dedicated presentation transform.

First, the frontend parses the HTML and extracts normalized text:

const plainText = (html: string | null) => {
  const document = new DOMParser().parseFromString(
    html ?? "",
    "text/html"
  );

  return (document.body.textContent ?? "")
    .replace(/\s+/g, " ")
    .trim();
};

Then it finds the common prefix and common suffix between before and after.

Finally, it keeps only a bounded amount of context around the changed region:

const radius = 260;

The UI renders the result as normal React text under explicit Before and After labels.

It does not render provider HTML with dangerouslySetInnerHTML.

Why I did not modify the provider change

The original PendingChange still contains the complete old_html and new_html.

The bounded diff is only a presentation.

That separation matters:

provider representation
        ↓
domain representation
        ↓
human presentation

The approval workflow keeps the original change intact while giving the reviewer a much smaller and safer view of it.

What the review UI looked like

The live PaperForge review surface: the human sees the proposed before/after change and must explicitly approve or reject it before the reviewed DOCX becomes available.

How I proved it

The frontend test provides:

old_html: "<p>Before sentence.</p>",
new_html: "<p>After sentence.</p>"

and checks that the interface renders:

Before sentence.
After sentence.

It then approves only the selected change_id.

The real end-to-end run used the same presentation path against a much larger change block and produced a readable review card rather than a raw HTML dump.

What I took away

A provider's wire representation is not automatically a good human interface.

For review systems, I now think in three layers:

1. provider representation
   enough structure to describe/apply the edit

2. application representation
   stable fields needed to track the decision

3. review presentation
   the minimum readable context needed by a human

Human approval is only meaningful if the change is easy enough to inspect.

Code:

Bounded diff logic

Diff UI

Frontend regression test

5. `completed` did not prove the exported DOCX contained the approved edit

This was the most important correctness lesson.

What I was trying to do

The end of the workflow looked reassuring:

human approves
        ↓
SuperDocs job reaches completed
        ↓
export endpoint returns DOCX bytes

It would be natural to display:

Reviewed document ready

at that point.

I decided that PaperForge needed one more check.

What could go wrong

A provider can report a workflow as completed and still leave room for artifact-level failure:

stale session state;
wrong export target;
change not reflected in the file;
structurally valid DOCX with old text;
delete operation whose text remains.

A successful HTTP response proves that bytes came back.

A provider status proves that the provider workflow finished.

Neither directly proves:

The human-approved edit is present in the file PaperForge is about to offer for download.

Root cause

I was defining success too early.

There are three different levels:

1. transport success
   the API call worked

2. workflow success
   the provider job completed

3. artifact success
   the exported file reflects the approved decision

PaperForge's user-facing promise depends on level 3.

The fix

PaperForge independently verifies the exported DOCX before setting:

final_docx_available = true

A DOCX is a ZIP container, so the verifier opens it and reads:

word/document.xml

It parses the WordprocessingML text nodes and normalizes the resulting document text.

Then it checks the approved operation.

For edit and create:

if not new_text or new_text not in document_text:
    raise ReviewVerificationError(
        "The approved change was not found "
        "in the DOCX export."
    )

For delete:

if old_text and old_text in document_text:
    raise ReviewVerificationError(
        "The deleted change remains "
        "in the DOCX export."
    )

If the DOCX is not a usable ZIP, the XML is missing, or the document cannot be parsed, verification fails closed.

How I proved it

I added a deliberately bad export test.

The approved change is:

Original executive summary.
        ↓
Clear revised executive summary.

The fake provider reports:

completed

but exports a DOCX containing:

The old text remains.

The test requires:

with pytest.raises(ReviewVerificationError):
    service.approve(report_id, "ch-1")

and then verifies:

assert service.get_review(report_id).status == "failed"

So PaperForge refuses to claim success even though the simulated provider workflow itself reached completed.

Important limitation

This verifier does not prove complete DOCX fidelity.

It does not prove every style, page break, list, table, or layout detail survived.

It proves the narrower property PaperForge promises at this boundary:

approved replacement/create text must be present
approved deleted text must be absent

That postcondition is deterministic and testable.

What I took away

External status should not automatically become application-level truth.

For consequential artifacts, define a postcondition your own system can verify:

provider says completed
        ↓
export artifact
        ↓
inspect artifact independently
        ↓
verify the promised property
        ↓
only then expose "completed"

Code:

DOCX verification

Fail-safe regression test

What changed in my integration design

These bugs look unrelated at first:

extra JSON fields
duplicate jobs
approval payload shape
oversized diffs
DOCX verification

But they all came from the same mistake: letting the external API boundary leak too far into the rest of the application.

The more robust architecture became:

                SuperDocs
                   │
                   ▼
          narrow provider adapter
                   │
         normalize + validate
                   │
                   ▼
        PaperForge-owned state
                   │
          explicit state machine
                   │
                   ▼
            human review UI
                   │
                   ▼
          explicit human decision
                   │
                   ▼
              DOCX export
                   │
          independent verification
                   ▼
               completed

The important word there is owned.

SuperDocs owns the document-editing primitives.

PaperForge owns:

what provider data becomes persistent application state;
how retries recover external jobs;
how a human sees a proposed change;
when a decision is allowed;
and what evidence is required before the application says the workflow is complete.

That separation made the integration much easier to reason about.

The five lessons I would carry into the next external API integration

Be tolerant at the provider boundary, strict inside your own domain.

Additive upstream metadata should not break a stable application model.
Persist external operation identity before relying on retries.

A failed local request does not prove an external side effect did not happen.
Do not infer write contracts from read contracts.

Action endpoints often have envelope-level semantics that are easy to miss.
Separate machine representations from human review representations.

Correct payloads can still create unusable approval UX.
Verify the product-level postcondition yourself when you can.

200 OK, completed, and “file returned” are different claims.

The integration became much stronger once I stopped treating SuperDocs as a black-box call inside a happy-path script and started treating it as one component in a stateful, human-reviewed system.

That is also the main thing I would do differently if I started again: define the boundary contracts and state transitions first, before wiring the first live provider call.

Repositories

PaperForge: https://github.com/Kalpesh1Sharma/paperforge
Closing Document Reconciliation: https://github.com/Kalpesh1Sharma/superdocs-builds/tree/closing-document-reconciliation/use-cases/Kalpesh1Sharma/closing-document-reconciliation

DEV Community: Kalpesh Sharma

Five Integration Failures I Hit While Building on SuperDocs

Build context: Closing Document Reconciliation

1. Additive provider metadata broke my strict pending-change model

What I was trying to do

What broke

The bad assumption

Root cause

The fix

How I proved it

2. A retry could have started the same SuperDocs workflow twice

What I was trying to do

What broke

The bad assumption

Root cause

The fix

How I proved it

3. I inferred the approval write contract from the read contract

What I was trying to do

What broke

The bad assumption

Root cause

The fix

What I took away

4. A targeted edit produced too much HTML to be a useful human diff

What I was trying to do

What broke

The bad assumption

The fix

Why I did not modify the provider change

What the review UI looked like

How I proved it

What I took away

5. completed did not prove the exported DOCX contained the approved edit

What I was trying to do

What could go wrong

Root cause

The fix

How I proved it

Important limitation

What I took away

What changed in my integration design

The five lessons I would carry into the next external API integration

Repositories

5. `completed` did not prove the exported DOCX contained the approved edit