DEV Community: Mykola Melnyk

Reading the results: text, boxes, entities and how they line up

Mykola Melnyk — Sat, 05 Sep 2026 18:12:17 +0000

Written 2026-09-05. Targets @stabrise/scaledp@0.1.1. Assumes you have run a pipeline; assumes nothing about the schemas.

TL;DR: transform() returns plain objects. Four schemas cover everything — ScaleDpImage, Document, DetectorOutput, NerOutput — every one carries an exception field, and every entity carries the boxes its characters fell inside. That last property is what lets you draw a redaction rectangle over a name the model found in a string.

A row is just an object

const rows = await pipeline.transform(file)
console.log(Object.keys(rows[0]))
// [ 'path', 'page', 'image', 'text', 'ner', 'row_time', 'execution_time' ]

There is no result class, no accessor API, no .get(). A stage reads a named field and writes another one, and what you get back is whatever the stages wrote. path and page come along from the reader; row_time and execution_time are added by the runner.

That means destructuring works, JSON.stringify works, and passing a row to another function does not require importing anything.

The four schemas

`ScaleDpImage` — a rendered page

interface ScaleDpImage {
  path: string
  resolution: number      // DPI it was rendered at; 0 when unknown
  data: Uint8Array        // encoded bytes, PNG unless imageType says otherwise
  imageType: 'png' | 'webp' | 'jpeg'
  height: number
  width: number
  exception: string
}

Encoded bytes, not raw pixels — so row.image.data is a PNG you can hand to createImageBitmap or a Blob.

`Document` — text plus the boxes it came from

interface Document {
  path: string
  text: string
  type: string            // 'ocr' | 'pdf' | an engine name
  bboxes: Box[]
  exception: string
}

text is the whole page as one string. bboxes is every word (or region — see boxLevel) with its geometry and its own text.

`DetectorOutput` — boxes with no text

interface DetectorOutput {
  path: string
  type: string            // 'dbnet-onnx' | 'paddle' | 'yolo' | …
  bboxes: Box[]
  exception: string
}

The difference from Document is exactly the absence of text. A detector found regions; nobody has read them yet.

`NerOutput` — entities

interface Entity {
  entity_group: string
  score: number
  word: string
  start: number           // character offset into the document text
  end: number
  boxes: Box[]            // the boxes those characters fall inside
  source?: 'model' | 'propagated'
}

The `Box` convention, which is not what you expect

This one catches people, so it is worth stating precisely:

interface Box {
  text: string
  score: number
  x: number       // top-left of the AXIS-ALIGNED box of the same size,
  y: number       // centred on the ROTATED rect's centre
  width: number   // always the LONGER side
  height: number
  angle: number   // degrees about that same centre, normalised to (-90, 270]
}

A Box is not xyxy and it is not a polygon. For an unrotated box, x, y, width, height behave exactly as you would hope and ctx.strokeRect(box.x, box.y, box.width, box.height) is correct. For a rotated one it is not — you need the centre and the angle:

function strokeBox(ctx: CanvasRenderingContext2D, box: Box) {
  const cx = box.x + box.width / 2
  const cy = box.y + box.height / 2

  ctx.save()
  ctx.translate(cx, cy)
  ctx.rotate((box.angle * Math.PI) / 180)
  ctx.strokeRect(-box.width / 2, -box.height / 2, box.width, box.height)
  ctx.restore()
}

Two consequences of "width is always the longer side" worth knowing: a genuinely tall, narrow region is reported as a wide box rotated 90°, and that is correct rather than a bug. And coordinates are in the rendered page's pixel space — the same space PdfToImage produced — so they line up with row.image without conversion. That is a deliberate divergence from the Python library, which leaves text-layer boxes in PDF points.

Entities carry their boxes, and that is the useful part

An NER model operates on a string. It returns character offsets. On its own that gives you a table, not a redaction.

for (const entity of row.ner.entities) {
  console.log(entity.entity_group, entity.word, entity.score)
  for (const box of entity.boxes) {
    strokeBox(ctx, box)   // draw it on the page
  }
}

person       Marta Feldmann        0.99
email        accounts@acme.de      0.97
iban         DE89 3704 0044 0532…  0.94

Getting from start/end to boxes is not trivial and is worth knowing about, because it was a real bug. The text was reconstructed in reading order — boxes clustered by y, then sorted by x — while bboxes keeps the detector's order. A forward-only cursor walking the boxes therefore found that boxes routinely sat behind it, gave up, and left the entity with no boxes at all: listed in the table, invisible on the page. For a redaction tool that is the worst possible failure, since nothing looks wrong.

If you need this machinery yourself, it is exported:

import { buildCharToBoxMap, boxesForRange } from '@stabrise/scaledp/ner'

The builder renders bboxes directly, which is the fastest way to see what the schema actually contains:

One row per Box. The text it carries, the score the model gave it, and the geometry — x, y, w, h, angle. Note info@stabrise.com at 1.000 and the ScaleDP wordmark at 0.948: confidence is per box, so you can threshold on it per box.

Displaying it

Four helpers, all returning a DOM element rather than a string, all mirroring ScaleDP's notebook helpers:

import {
  renderInto, showImage, showText, showNer, visualizeNer, showBoxes,
} from '@stabrise/scaledp/display'

renderInto('#page',     showImage(row.image))
renderInto('#text',     showText(row.text))              // layout-preserving
renderInto('#entities', showNer(row.ner))                // a table
renderInto('#inline',   visualizeNer(row.text, row.ner)) // highlighted in the text
renderInto('#boxes',    showBoxes(row.text))             // geometry, first 20

Three things they do that are easy to miss:

showText renders monospace, and that is load-bearing. Layout-preserving text encodes the page's layout in spaces and blank lines. In a proportional font the columns do not line up and it looks like broken output rather than preserved layout.

Failures render as failures. Every helper checks exception first and returns a red error block. A stage that failed shows up as a visible error rather than an empty panel, which is the difference between "something went wrong" and "there was nothing on the page".

visualizeNer drops overlapping entities rather than nesting them — highest score wins — because two spans cannot occupy the same characters in a flat text run.

Annotating the page is a stage, not a helper

Drawing boxes onto the image is a pipeline stage, as in ScaleDP. The annotated page is just another image column:

import { ImageDrawBoxes } from '@stabrise/scaledp'

pipeline.stages.push(
  new ImageDrawBoxes({
    inputCols: ['image', 'text', 'ner'],
    outputCol: 'annotated',
  })
)

One caveat: image_with_boxes is an Image, so it typechecks everywhere an image does — including as the input to a detector. Wiring a detector after an annotation pass means it will faithfully detect the rectangles and label text you just drew. The stage catalogue marks ImageDrawBoxes as terminal for this reason, rather than forbidding it, because chaining two annotation passes for two colours is a genuinely useful idiom.

Trade-offs

Plain objects mean no runtime validation. You get TypeScript types and nothing else. A typo in a column name produces undefined, not an error.
bboxes can be large. A dense page at word level is hundreds of boxes per page; forty pages is tens of thousands of objects. Drop columns you have consumed.
The display helpers are deliberately minimal. They exist so you can see a result in three lines, not so you can ship a viewer. For anything real, read the schemas and render them yourself.

Try it

npm install @stabrise/scaledp pdfjs-dist onnxruntime-web ppu-paddle-ocr

Open this pipeline in the builder and switch the result panel between Text and Boxes. Those two views are row.text.text and row.text.bboxes — the same two fields this post has been describing, rendered.

The same pipeline, minus the cluster

Mykola Melnyk — Sat, 05 Sep 2026 18:11:36 +0000

Written 2026-09-05. Compares @stabrise/scaledp@0.1.1 with ScaleDP on Python/Spark. Assumes you can read both languages; assumes no Spark expertise.

TL;DR: The browser library mirrors the Python one stage for stage — same names, same parameters, same schemas — so porting a pipeline is mechanical rather than a rewrite. What does not carry over is Spark, and the surprising part of the port was how little of Spark was actually load-bearing.

The same pipeline, twice

Python, on Spark:

pipeline = PipelineModel(stages=[
    PdfDataToImage(resolution=300),
    TesseractOcr(inputCol="image", outputCol="text", keepFormatting=True),
    Ner(inputCol="text", outputCol="ner"),
])
result = pipeline.transform(df)

TypeScript, in a tab:

const pipeline = new Pipeline([
  new PdfToImage({ resolution: 300 }),
  new TesseractOcr({ inputCol: 'image', outputCol: 'text', keepFormatting: true }),
  new GlinerNer({ inputCol: 'text', outputCol: 'ner' }),
])
const rows = await pipeline.transform(file)

The parameter names match because they are the same parameters. resolution, inputCol, outputCol, keepFormatting — none of these were renamed to feel more idiomatic in TypeScript, and that was a deliberate choice. A pipeline should read the same in both, so that porting a notebook is a transcription job and not a design job.

What is actually different

Options objects instead of positional column arguments. The column names are still options with the same defaults.

Rows, not a DataFrame. transform returns Row[] — plain objects. A stage that produces several rows per input (a PDF becoming one row per page) does exactly what posexplode does.

Everything is async. Model loading, decoding and inference are all promises. In Python they are blocking calls.

Different NER architecture. Python's Ner is BERT token classification with a fixed tag set. GlinerNer is GLiNER — zero-shot, where the entity labels are the prompt. That is a genuine capability difference, not a naming one: adding a new entity type is an edit rather than a training run.

The part that surprised us

Going in, "port a Spark library to the browser" sounds like it means replacing a distributed execution engine. It did not, and the reason is worth writing down because it generalises.

Reading ScaleDP's stages, two things stood out:

Every stage is a pure Transformer. There are no Estimators anywhere in the library, so there is no fit()/transform() duality to model. A stage is a function from a row to a row.
The only DataFrame surface stages use is withColumn, drop and select. No joins, no groupBy, no window functions, no UDF registration beyond the obvious.

Those two facts together mean the Spark coupling is one abstraction thick. Strip it and what remains is: an ordered array of stages, each reading a named field and writing a named field, over an array of plain objects.

That is the entire runner, and it is 215 lines:

export abstract class Stage<P extends BaseStageParams = BaseStageParams> {
  abstract readonly name: string

  /** Transform one row's input value into this stage's output value. */
  protected abstract apply(input: unknown, row: Row, ctx: StageContext): Promise<unknown>

  /** Value written to `outputCol` when `apply` throws. */
  protected abstract onError(message: string, row: Row): unknown

  async transform(rows: Row[], ctx: StageContext): Promise<Row[]> {
    // column wiring, error capture, keepInputData and timing, once, for everyone
  }
}

The generalisable lesson: audit the surface a framework's users actually touch before assuming you need the framework. ScaleDP looked Spark-shaped from the outside. From the inside, Spark was providing partitioning and shuffles — neither of which a single-document browser pipeline needs — plus a withColumn API that is three lines of object spread.

What Spark was genuinely providing

Not nothing. Being honest about this matters, because it is exactly the list of things the browser version cannot do:

Horizontal scale. Forty executors reading forty PDFs. One tab is one machine.
Spill to disk. A dataset larger than memory. The browser pipeline holds its rows in memory and that is that.
Fault tolerance across a cluster. A lost executor is retried. A crashed tab is a crashed tab.

If your workload needs any of those, the Python library is the answer and the pipeline you wrote for the browser will port back almost unchanged. That symmetry is the point of matching the names.

What carries over unchanged

The non-throwing error contract. Failures land in the output schema's exception field and the pipeline completes. propagateError opts into throwing. Identical in both.
The Box convention. x/y is the top-left of the axis-aligned box of the same size centred on the rotated rect's centre, angle is degrees about that centre, and width is always the longer side. Getting this wrong shifts every downstream consumer silently, so it is reproduced exactly.
Layout-preserving text reconstruction under keepFormatting, including the per-line indent and blank-line rules.
Parameter names and defaults, wherever a Python equivalent exists.

The last two are not asserted, they are tested. test/fixtures/*.py run the real ScaleDP code and write JSON goldens; the parity suites diff against them. Verifying a port against its own expectations just encodes your own misunderstanding twice.

The builder will generate the TypeScript for any pipeline you assemble, which makes the correspondence easy to check against a notebook you already have:

PdfToImage, DbnetOnnxDetector, TesseractRecognizer — the same stage names, the same parameter names, the same order. The imports are the only genuinely new thing to learn.

What has no browser equivalent

Python stage	Status
`CraftTextDetector`	PyTorch only
`LayoutDetector`	Needs the PaddleOCR Python runtime
`EasyOcr`, `SuryaOcr`, `DocTROcr`	No browser builds — use `PaddleTextRecognizer`
`LLMOcr`, `LLMNer`, `LLMExtractor`	Any fetch-based OpenAI client works
`TextSplitter`, `TextEmbeddings`	Not yet ported

And one stage that goes the other way: PaddleRecognizer has no Python equivalent. It is PP-OCR's recognition model reading another detector's boxes, which the Python API cannot express.

Trade-offs

Matching names locks you to someone else's API decisions. Some of ScaleDP's parameter names are not what we would pick starting fresh. Portability was worth more than taste.
Matching behaviour means reproducing quirks. The DBNet path feeds the model BGR channels normalised against RGB ImageNet statistics, because the reference does and the model was trained that way. "Fixing" it shifts every box.
It also means deciding, bug by bug, when not to match. Five Python bugs are deliberately not reproduced. Each one is listed publicly, because a silent divergence is worse than either choice.

Try it

npm install @stabrise/scaledp pdfjs-dist onnxruntime-web ppu-paddle-ocr

Open the ported pipeline in the builder and compare the generated TypeScript against the Python you already have. If porting a notebook is not close to mechanical, that is a bug worth reporting.

onError must return the right shape, or the crash just moves

Mykola Melnyk — Sat, 05 Sep 2026 18:10:56 +0000

Written 2026-09-05, about @stabrise/scaledp@0.1.1. Follows on from the non-throwing error contract; assumes TypeScript.

TL;DR: A pipeline where stages never throw needs each failed stage to still produce a value. Return null and you have not removed the crash — you have moved it three stages downstream, to a place with no idea what went wrong. So onError returns a well-formed empty instance of the stage's output schema, every schema ships a create* factory to make that trivial, and the payoff is that no downstream stage needs an error branch at all.

The half of the contract nobody designs

The interesting half of "stages never throw" is easy to state: catch the exception, record the message. Most people get there.

The half that decides whether the design works is: what value goes in the output column?

There is an obvious answer, and it is wrong.

The wrong answer

// Tempting. Do not do this.
protected onError(message: string): DetectorOutput | null {
  this.lastError = message
  return null
}

It reads as honest — the stage produced nothing, so it returns nothing. What happens next:

// Three stages later, in a recognizer that knows nothing about the detector
const boxes = row.boxes as DetectorOutput
for (const box of boxes.bboxes) {    // TypeError: Cannot read properties of null

You have converted one caught, attributed, per-row failure into an uncaught TypeError in a different stage. Worse, that TypeError is caught by that stage's error handling, so the message the user finally sees is:

PaddleRecognizer: TypeError: Cannot read properties of null (reading 'bboxes')

The detector's actual failure — a model that would not load, a 404 on the weights — is gone. The reported stage is not the failed stage. This is precisely the outcome the contract exists to prevent, reintroduced by its own error handler.

undefined is the same bug with a different message. So is throwing from onError, which takes down the runner's catch block.

The right answer

protected onError(message: string): DetectorOutput {
  return createDetectorOutput({ exception: message })
}

createDetectorOutput is the factory that ships with the schema:

export function createDetectorOutput(init: Partial<DetectorOutput> = {}): DetectorOutput {
  return {
    path: init.path ?? 'memory',
    type: init.type ?? 'detector',
    bboxes: init.bboxes ?? [],
    exception: init.exception ?? '',
  }
}

Correct fields, empty values, one non-empty string. Every schema has one — createImage, createDocument, createDetectorOutput, createNerOutput — and they exist so that "an empty but valid instance of this schema" is never something a stage author has to get right by hand.

The base class makes the requirement unavoidable by typing it as abstract:

/**
 * Value written to `outputCol` when `apply` throws. Subclasses return an
 * empty instance of their output schema carrying the message, so downstream
 * stages see a well-formed value rather than `undefined`.
 */
protected abstract onError(message: string, row: Row): unknown

You cannot write a stage without answering the question.

What this buys downstream

Here is the payoff, and it is bigger than it looks. Consider a recognizer consuming a detector's boxes:

protected async apply(input: unknown, row: Row): Promise<Document> {
  const image = row[this.params.inputCols[0]] as ScaleDpImage
  const boxes = row[this.params.inputCols[1]] as DetectorOutput

  // Check exception first -- a failed upstream stage returns a well-formed
  // but empty value, so testing the payload first buries the real cause.
  if (image.exception) return this.onError(image.exception, row)
  if (boxes.exception) return this.onError(boxes.exception, row)

  for (const box of boxes.bboxes) {
    // ...
  }
}

Note what is not there. No null check. No ?.. No if (!boxes) return. The bboxes array is guaranteed to exist because the schema guarantees it, so the loop is over an empty array and does nothing.

Multiply that across eighteen stages and every display helper. The invariant is worth more than any individual null check, because null checks are things you forget and invariants are things you establish once.

The two exception lines are doing something different from a null check: they are propagating attribution. Without them the recognizer would happily return an empty document with an empty exception, and a genuinely failed pipeline would look like a blank page.

Empty is not the same as failed

A deliberate consequence, worth being explicit about because it is a design decision rather than an accident.

After a failed detector and after a detector that found nothing, boxes.bboxes.length === 0. Downstream code cannot distinguish them, and that is intentional — it is exactly why downstream code needs no error branch.

The distinction lives in one place, and one place only:

if (row.boxes.exception) {
  // failed
} else if (row.boxes.bboxes.length === 0) {
  // genuinely nothing on this page
}

Some stages lean on this deliberately. TesseractScriptDetector returns an empty script with an empty exception when it cannot identify a script, because a blank page is a legitimate answer rather than a failure. Encoding "I looked and there was nothing" as an error would make blank pages noisy for no benefit.

Here is what that looks like when it actually happens. A PaddleRecognizer pointed at a column no stage writes, run in the builder:

The recognizer failed, and the result is still a Document. It has a text (empty), it has bboxes (empty), and it has an exception naming the stage and the cause. Nothing downstream of it has to know that anything went wrong — which is the entire argument for returning a shape instead of a null.

The display helpers cooperate

The invariant extends past the pipeline. Every display helper checks exception first:

export function showText(document_: Document, options: ShowTextOptions = {}): HTMLElement {
  if (document_.exception) return errorBlock(document_.exception)
  // ...
}

So a failed stage renders as a red error block, not an empty panel. That matters more than it sounds: the failure mode of a quiet error contract is that failures become invisible, and an empty <pre> looks exactly like a page with no text on it. Making the UI layer aware of the same field is what keeps a silent contract from producing a silent product.

The rule, generalised

This is not really about OCR. Any pipeline that converts exceptions into values faces the same question, and the same answer applies:

If a failure produces a value, that value must satisfy every invariant a success would have satisfied. Otherwise you have not handled the error; you have deferred it to code that cannot attribute it.

Practically, that means three things:

Give every result type a factory that produces a valid empty instance. If constructing "empty but valid" is fiddly, it will be got wrong.
Put the failure marker inside the type, not beside it. A Result<T, E> wrapper also works, and is arguably cleaner — but it does not survive postMessage as nicely, and it forces every consumer to unwrap, which is exactly the per-consumer branch this design is avoiding.
Make the error path type-checked, not documented. An abstract onError returning the schema type is enforced by the compiler. A comment saying "return an empty instance" is not.

Trade-offs

Result<T, E> is the better-typed alternative and we did not choose it. It makes the failure impossible to ignore, which is a genuine advantage over a string field you can forget to read. We took the schema-with-exception route for Python parity — the field exists in ScaleDP and has the same meaning — and because it clones across a worker boundary for free. That is a defensible trade, not an obviously correct one.
A well-formed empty value is easy to ignore. It is the same criticism as the contract itself: an empty Document flows into your index without complaint.
Factories add a small amount of ceremony. Every schema needs one, and a new field means updating it. In exchange, no stage ever hand-rolls an empty instance and gets a field wrong.
onError receives a string, not the original error. You lose the ability to branch on the error type inside onError. In practice the stage that threw already knows what it threw; the ones that do not are just passing an upstream message through.

Try it

npm install @stabrise/scaledp pdfjs-dist onnxruntime-web ppu-paddle-ocr

Open a pipeline in the builder and break a stage on purpose — point its inputCols at a column no stage writes. What you get back is a well-formed empty Document with a populated exception, which is exactly what this post is about.

Privacy is an architecture, not a checkbox

Mykola Melnyk — Fri, 04 Sep 2026 08:17:53 +0000

Written 2026-09-04, about @stabrise/scaledp@0.1.1. No OCR background assumed.

TL;DR: "We delete it after processing" is a promise — it depends on a vendor's behaviour, and you verify it by reading a document. "It never left the tab" is a property — it depends on the data flow, and you verify it by opening the network panel. If you are building anything that handles documents people care about, the difference is worth designing for.

Promises versus properties

Here is a claim from a typical OCR vendor's security page:

Documents are processed in memory and deleted immediately after the response is returned. We do not use customer data for training.

I have no reason to doubt it. But look at what it takes to believe it: a policy document, an audit report, a contract with penalties, and trust in an organisation's future behaviour — including after an acquisition, a breach, or a change in leadership. Those are all real mitigations, and they are all promises.

Now the same claim for a client-side pipeline:

The document is never transmitted.

The verification is different in kind. You open DevTools, filter the network panel to XHR and fetch, drop a file in, and watch nothing happen. You can do that yourself, in thirty seconds, without asking anyone.

A promise constrains behaviour. A property constrains what is possible. Both are worth having; only one of them survives an organisation you cannot audit.

What "never left the tab" actually means here

Being precise, because overclaiming this is worse than not claiming it.

What genuinely never leaves: the document bytes, the rendered page images, the recognised text, the detected boxes, the extracted entities. All of it lives in JavaScript memory and OffscreenCanvas buffers inside one tab, is passed between stages as plain objects, and is garbage-collected when you drop the reference.

const pipeline = new Pipeline([
  new PdfToImage({ resolution: 300 }),
  new PaddleTextRecognizer({ keepFormatting: true }),
  new GlinerNer({ labels: ['person', 'email', 'iban'] }),
])

// Every byte of this stays in the tab.
const rows = await pipeline.transform(file)

What does leave, once: the model weights, downloaded from a model host. That is an outbound request for a public artefact — it says "this origin fetched PaddleOCR v6-small", not "this user uploaded a contract". If even that is too much signal, you self-host:

configure({
  modelHost: 'https://models.internal.example.com',
  cache: 'indexeddb',
})

After which there is no third-party request at all. Which brings us to the demonstration.

The demo states its own runtime in a strip across the top, which is the same information your app should surface:

Two honest things are visible there. The inference is running on WebAssembly in this tab. And multithreading is unavailable, because the page is not cross-origin isolated — GitHub Pages cannot send the headers that would allow it. That is a real performance cost of the deployment, printed on screen rather than hidden.

How to prove it to a reviewer

This is the part that actually matters in a procurement conversation, and it takes about a minute.

Load the app once with a network connection, and let the models cache. They go to IndexedDB, scoped to your origin.
Turn the network off. Airplane mode, or DevTools' offline throttling.
Drop a document in and run the pipeline.
It works.

A system that produces correct output with no network cannot be exfiltrating anything, and that argument needs no policy document. It is the same reasoning a reviewer applies to an air-gapped machine, just cheaper.

For the version that goes in a security questionnaire, the three bullets are:

Document bytes are processed in-page by WebAssembly; there is no server-side component and no endpoint to send them to.
Model weights are static files, served from <your origin>; they are inputs to the computation, not a channel out of it.
Verification: the pipeline runs to completion with networking disabled after first load.

What this does not buy you

Every honest privacy claim comes with a boundary, and here is this one's.

An untrusted device is still untrusted. If the user's machine has malware, or a hostile browser extension with page access, running locally does not save you. It removes your server from the threat model, not theirs.

Your own app is still in the path. The library does not upload. Your application code can. Nothing stops a well-meaning analytics call from including a filename, or an error reporter from capturing a page image in a breadcrumb. The property holds for the pipeline; it holds for your product only if you keep it.

That is worth an explicit rule in review: anything that touches a row is in scope. Error reporting, session replay, analytics, and log shipping all deserve a second look in a codebase that makes this claim.

Persistence is a decision you now own. Cached models live in IndexedDB on the user's disk. That is weights, not documents — but on a shared machine it is still a fact about what the user did, and evict() exists for the flows where that matters.

Compliance is not automatic. Not sending data to a processor removes one category of obligation. It does not remove your obligations as a controller for anything you do store server-side afterwards.

The pattern this enables

The strongest version of this is not "build a fully local product" — that is a big commitment. It is a much smaller change to an ordinary upload flow:

drop file → OCR + NER locally → show the user what's in it
          → they redact → upload the redacted version

The backend does not change at all. The document that reaches your server has already had its PII blacked out, by code running on the user's machine, before anything was transmitted. You have improved your data-handling posture without a migration.

Trade-offs

You inherit the operational failure. No server means no server to blame, and no retry policy. A model that fails to load on someone's laptop is a UI state you have to design.
First load is expensive. ~6 MB for OCR is nothing; ~580 MB for the default NER model is not. The privacy property costs bytes.
"Verify it yourself" only works if it stays true. This is a property of a build. It deserves a test, not just a paragraph — a check that no network request is made during a pipeline run is cheap to write and cheap to keep.
It does not scale. One tab, one machine. At volume, a server with a proper DPA is both cheaper and, done well, perfectly respectable.

Try it

npm install @stabrise/scaledp pdfjs-dist onnxruntime-web ppu-paddle-ocr

Open the pipeline in the builder, run it once so the weights are cached, then turn off your network and run it again. It works. That is the demonstration, and it takes about a minute.

Running OCR entirely in the browser meant decoupling detection from recognition

Mykola Melnyk — Fri, 04 Sep 2026 08:17:21 +0000

Written 2026-09-04. Code examples target @stabrise/scaledp@0.1.1 and onnxruntime-web@1.20.x. This post assumes basic familiarity with OCR (text detection vs. text recognition) and TypeScript; it does not assume you know onnxruntime-web.

TL;DR: @stabrise/scaledp is a document-AI pipeline — PDF rendering, OCR, NER — that runs entirely client-side on onnxruntime-web, no server involved. The recognizer that ships with it, PaddleOCR, detects and reads text in one pass, which meant no other detector could ever feed it a box. We built PaddleRecognizer, a second stage with the contract [image, boxes] -> Document, so DBNet or a YOLO signature detector can hand PaddleOCR's recognition model boxes it never found itself. Along the way we hit a WebGPU kernel gap, a batching constraint in the underlying library, and a word-splitting bug that was quietly dropping characters.

The problem: an OCR stage that couldn't be composed

@stabrise/scaledp is a TypeScript port of ScaleDP, a Python/Spark document-processing library. The pipeline model is the same in both: a list of stages, each stage a pure function over a row, [image, text, boxes, ...] in, [image, text, boxes, ...] out. The point of the TypeScript version is that it never leaves the browser — no upload, which matters a lot once you're running OCR over things like IDs, contracts, or medical forms.

For text recognition we use ppu-paddle-ocr, a WASM/ONNX port of PP-OCR. Our PaddleTextRecognizer stage wraps it directly: hand it a page image, get back text and boxes. Convenient, but it hides a detail that matters once you have more than one detector — PP-OCR's run() call does detection and recognition together. There was no way to say "read this specific set of boxes"; the boxes always came from PP-OCR's own detector.

That's a real limitation, because the library also ships DbnetOnnxDetector (the same DBNet ONNX model ScaleDP uses server-side) and a YOLO-based signature/face detector. If your pipeline picks DBNet for detection — maybe because it's faster, maybe because you're comparing detectors, maybe because you need YOLO to find signature regions specifically — there was no path from those boxes into PP-OCR's recognizer. You'd fall back to Tesseract, which does support recognize-only via TesseractRecognizer, even if PaddleOCR would read your script better.

The fix: split detection out of the recognizer

PaddleRecognizer is the missing half. Same contract as TesseractRecognizer:

[imageColumn, boxColumn] -> Document

so any stage that writes a DetectorOutput column — DBNet, YOLO, PaddleOCR's own detector — can feed it.

import { Pipeline, configure } from '@stabrise/scaledp'
import { PdfToImage } from '@stabrise/scaledp/pdf'
import { DbnetOnnxDetector, PaddleRecognizer } from '@stabrise/scaledp/ocr'

configure({ cache: 'indexeddb', pdf: { workerSrc: '/pdf.worker.min.mjs' } })

const pipeline = new Pipeline([
  new PdfToImage({ resolution: 300 }),
  new DbnetOnnxDetector({ inputCol: 'image', outputCol: 'boxes' }),
  new PaddleRecognizer({ inputCols: ['image', 'boxes'], preset: 'v6-small' }),
])

const rows = await pipeline.transform(file)
console.log(rows[0].text.text)

Detection and recognition are now two stages you can mix independently: swap DbnetOnnxDetector for the YOLO signature detector to recognize only signature regions, or run DBNet once and try several recognizer presets against the same boxes without re-detecting.

Run in the builder, that is DBNet's boxes with PaddleOCR's text in them:

Worth noticing the box around the man's face: DBNet found structure there and PaddleOCR dutifully tried to read it. Detection errors do not disappear because you changed which model reads them — they just become somebody else's boxes.

Three things made this harder than "call the recognizer per box," and each is the kind of detail that only shows up once you've profiled or diffed real output.

1. Straighten before you crop

ppu's own cropping is axis-aligned. Hand it a rotated box and it crops that box's bounding rectangle, not the box itself — fine for horizontal text, wrong for anything at an angle (a rotated scan, a stamped/rotated field). PaddleRecognizer straightens each box with a perspective warp before cropping, so a 15° rotated line is read upright instead of read with its neighbors bleeding into the crop.

2. Batch by canvas, not by call

The natural implementation is "for each box, call run()." That's also the slow one: ppu only batches crops within a single run() call, and it cuts them from one canvas. Calling it per box costs one inference and one main-thread yield per line — on a page with 80 lines, that's 80 round trips.

PaddleRecognizer stacks the straightened crops onto a single sheet canvas first, then reads them together in one run(). Results come back sorted into reading order rather than array order, so we match them back to boxes by the slot each crop occupied on the sheet, not by index.

// Simplified shape of what happens internally
const crops = boxes.map((box) => straightenAndCrop(image, box))
const sheet = stackOntoSheet(crops)          // one canvas, N slots
const results = await recognizer.run(sheet)  // one inference call
const text = matchBySlotOffset(results, crops)  // not by array position

3. WebGPU can't run this graph

onnxruntime-web's WebGPU backend rewrites convolutions into an internal com.ms.internal.nhwc op set. PP-OCR's recognition graph hits a kernel that isn't implemented for that rewrite, and session creation fails outright — not a slowdown, a hard error. ppu-paddle-ocr already retries on WASM internally when this happens; we exposed the same behavior in our own createSession() as an opt-in fallbackToWasm flag. It's off by default: if you asked for WebGPU and it's genuinely misconfigured for some other reason, we want that to fail loudly, not silently degrade to WASM and leave you wondering why inference is slower than expected.

import { createSession } from '@stabrise/scaledp/ocr'

const session = await createSession(modelBuffer, {
  executionProviders: ['webgpu'],
  fallbackToWasm: true, // PP-OCR recognition specifically needs this
})

A word-splitting bug we found while wiring this up

Building PaddleRecognizer meant looking hard at how word-level boxes get derived from a line the model read as a whole — and that surfaced a pre-existing bug in PaddleTextRecognizer too. The old code cut each line at its ink gaps first and recognized every word crop independently. PP-OCR's recognizer is a CTC model trained on full lines; feeding it a three-character crop stretched to a fixed input height is nothing like its training distribution, and it also throws away the sentence-level context the model's accuracy depends on.

The fix reverses the order: read the line whole, then split the words out of the result, reconciling the model's own word boundaries against a vertical-projection ink-gap scan. Equal counts zip together directly. Where they disagree, the ink decides how many boxes exist and the text decides what's in them — more ink-spans than words merges adjacent spans smallest-gap-first, more words than spans joins the extras back onto the span they overlap. On a real page this measurably fixed dropped characters — https:/stabrise.com/scaledp/ (missing a slash) became https://stabrise.com/scaledp/ — and it's cheaper too: one inference per line instead of one per word.

It also fixed something subtler: on a signature, the old per-word recognition emitted a scatter of single letters for one continuous stroke, and cutting the line up to match produced a row of boxes with identical width and height — the character count rendered as geometry, not anything measured from the pixels. Reading whole-line-first, that signature went from 15 boxes (10 of them these fake uniform ones) down to accurate stroke-level geometry.

Why this shape, not a bigger abstraction

We didn't build a generic "pluggable detector/recognizer interface" with a registry and adapters. PaddleRecognizer and TesseractRecognizer just happen to share the same input contract — [image, boxes] -> Document — because that's the natural shape of "read boxes someone else found," not because we designed an interface for it. Every stage in the pipeline is engine-specific and takes exactly the parameters that engine needs; composability comes from stages agreeing on column names and schemas, the same way Python ScaleDP's Spark stages do. Adding a third recognizer later means writing a third stage with that same two-column contract, not implementing an interface.

That also matters for a rule this library holds hard: stages never throw by default. A batch job over forty PDF pages can't lose the other thirty-nine because page twelve's crop was degenerate. PaddleRecognizer records a failure in the output Document's exception field and returns a well-formed empty document for that row; you opt into throwing with propagateError: true if you'd rather fail fast during development.

Trade-offs

Sheet-batching costs memory, not just time. Stacking every crop on a page onto one canvas means the whole page's worth of crops lives in memory at once. For most pages that's trivial; for a page with hundreds of tiny regions (a densely annotated form, say) you may want to chunk boxes into multiple sheets — recBatchSize exists for exactly this, and defaults conservatively.
fallbackToWasm trades silent correctness for silent slowness. We chose to make WebGPU failures loud by default specifically so a genuinely broken provider doesn't quietly become "OCR is fine but 3x slower than it should be" with no error to grep for. Turn the flag on once you've confirmed WebGPU works for your other stages and just isn't implemented for this one op set.
Straightening every box costs a warp per crop. For an unrotated page (the common case) this is wasted work; we don't currently skip it for axis-aligned boxes, because detecting "close enough to axis-aligned" reliably is its own source of edge cases. If your pipeline is always horizontal-only and profiling shows this mattering, that's a reasonable place to special-case.

Try it

npm install @stabrise/scaledp onnxruntime-web ppu-paddle-ocr pdfjs-dist

import { Pipeline, configure } from '@stabrise/scaledp'
import { PdfToImage } from '@stabrise/scaledp/pdf'
import { DbnetOnnxDetector, PaddleRecognizer } from '@stabrise/scaledp/ocr'

configure({ cache: 'indexeddb', pdf: { workerSrc: '/pdf.worker.min.mjs' } })

const rows = await new Pipeline([
  new PdfToImage({ resolution: 300 }),
  new DbnetOnnxDetector(),
  new PaddleRecognizer({ preset: 'v6-small' }),
]).transform(file)

The live demo (drop a PDF or image, pick a detector and recognizer, run it) is at scaledp-ts.stabrise.com/demo; docs at scaledp-ts.stabrise.com/docs.

One bad page must not lose the other forty

Mykola Melnyk — Fri, 04 Sep 2026 08:16:37 +0000

Written 2026-09-04, about @stabrise/scaledp@0.1.1. Assumes TypeScript and async/await.

TL;DR: In a document pipeline, an exception is almost never worth the batch it destroys. So stages here never throw by default: every output schema carries an exception: string, a failed stage writes a well-formed empty value with the message in that field, and the pipeline completes. propagateError: true opts back into throwing. One sentence of requirement generates the whole design — and it has a real cost, which is the second half of the post.

The requirement

You have a 40-page PDF. Page 12 has a degenerate detection — a box of zero width, a crop that lands entirely outside the page, a region the recognizer chokes on. The natural implementation throws, the await rejects, and you have zero pages of output from a run that successfully processed thirty-nine.

That is the wrong outcome, and it is wrong in a way that gets worse with scale. The larger the batch, the higher the chance that something in it is malformed, and the more expensive it is to have lost the rest.

State it as a requirement and the design follows: one bad page must not lose the other forty.

The contract

Every output schema carries an exception field:

interface Document {
  path: string
  text: string
  type: string
  bboxes: Box[]
  exception: string     // <-- here
}

Same for ScaleDpImage, DetectorOutput and NerOutput. It is not an optional field and it is not Error | null — it is a string, empty when nothing went wrong, because it crosses postMessage to a worker and has to survive structured cloning.

The runner does the catching, once, for every stage:

try {
  const expanded = await this.expand(input, row, ctx)
  next = expanded ?? [{ ...row, [outputCol]: await this.apply(input, row, ctx) }]
} catch (error) {
  if (propagateError) throw error
  next = [{ ...row, [outputCol]: this.onError(formatException(this.name, error), row) }]
}

Which means a stage author writes ordinary code that throws when something is wrong. They do not write try/catch. They write one extra method — onError — that returns an empty instance of their output schema.

Reading it

const rows = await pipeline.transform(file)   // never rejects for a page failure

for (const row of rows) {
  if (row.text.exception) {
    console.warn(`page ${row.page}:`, row.text.exception)
    continue
  }
  index(row.text.text)
}

Here is that actually happening, in the live builder. The pipeline is PdfToImage → PaddleRecognizer, with the recognizer deliberately pointed at a column no stage writes:

Three things in that screenshot are the whole contract. The run completed — there is a result to look at. The page still rendered, because PdfToImage succeeded and its failure-free output is untouched by what happened downstream. And the failure is in the column, tagged on the tab with a red dot, carrying the message the stage wrote:

PaddleRecognizer: OcrError: No boxes in column "nosuchcolumn".
This stage reads a detector's output; run a text detector before it.

The stage name is prepended deliberately. With two recognizers in a pipeline, the column tells you which output failed and the message tells you which stage wrote it — and those are different questions once a pipeline has ten stages in it.

Check `exception` before you check the payload

This is a one-line ordering rule that appears verbatim in seven files, and it is the difference between an error message that names the cause and one that buries it.

A failed upstream stage does not return undefined — it returns a well-formed but empty value. So this looks reasonable and is wrong:

// WRONG: reports the symptom, hides the cause
if (image.data.length === 0) {
  throw new OcrError('no decoded image bytes', this.name)
}

PdfToImage failed three stages ago because the pdf.js worker was not configured. What the user sees is "no decoded image bytes" from the recognizer, and they spend an afternoon debugging the wrong stage.

// RIGHT: the real cause propagates
if (image.exception) return this.onError(image.exception, row)
if (image.data.length === 0) {
  throw new OcrError('no decoded image bytes', this.name)
}

The comment explaining this is copy-pasted into every stage that consumes another stage's output rather than abstracted into a helper. Seven small correct things beat one clever thing that a future stage forgets to call.

Opting into throwing

new PaddleTextRecognizer({ propagateError: true })

Per stage, not global, and useful in two opposite situations.

While developing. A mis-wired column name produces an empty result and no error, which is a genuinely annoying thing to debug. With propagateError it is a stack trace.

In a batch job where silence is worse than a crash. If you are OCRing a legal archive and a page fails, an empty string that flows into your search index is worse than a failed job, because nobody will ever notice it.

The error types all extend ScaleDpError and carry the stage they came from, so instanceof works when you have opted in:

import { OcrError, DetectionError, ConfigError } from '@stabrise/scaledp'

try {
  await pipeline.transform(file)
} catch (error) {
  if (error instanceof ConfigError) showSetupHelp(error)
  else if (error instanceof OcrError) reportPage(error.stage)
  else throw error
}

Error	Raised by
`ImageError`	decoding, encoding, cropping, empty data
`OcrError`	recognition, missing input columns on OCR stages
`DetectionError`	detectors
`NerError`	GLiNER
`ConfigError`	a missing peer dependency, a bad `configure()` value

What is deliberately not covered

Two failures happen outside the contract, on purpose.

Input normalisation. transform('https://example.com/doc.pdf') fetches, and a non-ok response throws before any stage runs. There is no row yet to write an exception onto.

Constructor validation. An unknown OCR preset or an out-of-range threshold throws RangeError when the stage is constructed:

new PaddleTextRecognizer({ preset: 'v6-smal' })
// RangeError: preset: unknown preset 'v6-smal'

That is deliberate. A parameter mistake is a programming error, not a data error. Reporting it at construction names the field; reporting it at row three names nothing useful, and reporting it as an empty column names nothing at all.

The distinction generalises: the non-throwing contract is for data failures, which are expected and per-row. Programming failures should still be loud.

Failures are timed

A small detail with a real diagnostic payoff:

// Failures are timed too: a stage that spent nine seconds before
// throwing still cost nine seconds.
const elapsed = performance.now() - started

Timing is recorded outside the try/catch. A stage that downloads a 580 MB model, times out and fails still shows 40 seconds in row_time. If failures were untimed, the pipeline that took a minute would report ten milliseconds of work, and the profile would be a lie exactly when you most need it.

Trade-offs

This is the part that matters, because the contract is not free.

Nothing forces you to check. row.text.text on a failed row is '', and '' flows happily into a search index, a diff, or a summary. The contract converts a loud failure into a quiet one, and the discipline of checking exception moves onto the caller. That is a real cost and I would not pretend otherwise.
An empty result and a failed result look identical downstream. A detector that found nothing and a detector that crashed both hand on zero boxes. That is intentional — it is why downstream stages need no error branch — but it means "no entities on this page" is ambiguous unless you look at exception.
You lose the stack at the throw site unless you set propagateError. The message includes the stack as a string, which is not the same as a debugger paused at the frame.
onError is one more method per stage, and a stage that returns the wrong shape from it converts one failure into a worse one — which is its own post.

The mitigation for the first two is not cleverness, it is a habit: treat exception as part of the result you destructure, not as an edge case. The display helpers cooperate here — anything with a non-empty exception renders as a red error block rather than an empty panel, so a failure is visible rather than merely absent.

Try it

npm install @stabrise/scaledp pdfjs-dist onnxruntime-web ppu-paddle-ocr

The screenshot above is reproducible in one click: open the mis-wired pipeline in the builder, then point the recognizer's inputCols at a column nothing writes and run it. The page still renders, the run still finishes, and the failure shows up where the contract says it will.

A document pipeline in 30 lines, running in a browser tab

Mykola Melnyk — Thu, 03 Sep 2026 13:21:20 +0000

Written 2026-09-03. Targets @stabrise/scaledp@0.1.1, pdfjs-dist@6, onnxruntime-web@1.29. Assumes TypeScript and npm; assumes nothing about OCR or ONNX.

TL;DR: Install one package plus the engines you actually use, call configure() once, and build a Pipeline out of stages. Every line of the thirty is explained below, including the three that people get wrong on the first try.

What we are building

Drop a PDF in, get back one row per page, each carrying the recognised text and a word box for every word on the page. No server, no upload, no API key.

Prerequisites

Node 20+ and a bundler that handles ESM (Vite, Next, Rspack — anything modern)
A browser with WebAssembly, which is all of them
Somewhere to serve static assets from your own origin

Step 1: install

npm install @stabrise/scaledp
npm install pdfjs-dist onnxruntime-web ppu-paddle-ocr

Only @stabrise/scaledp is a hard dependency. The engines are optional peer dependencies, reached through a dynamic import() inside the stage that needs them — so a project that only reads PDF text layers never pulls an ML runtime into its bundle. Install what your pipeline uses and nothing else.

If you skip one you need, you get a message naming the package rather than a bare Cannot find module.

Step 2: serve the assets

This is the step that fails first for almost everyone. Two runtimes need files you have to host yourself:

cp node_modules/pdfjs-dist/build/pdf.worker.min.mjs public/

onnxruntime-web's .wasm binaries are resolved automatically from the resolved package version, so you usually do not have to touch those — but if you serve them yourself, they must match the loader's build variant and version exactly.

Step 3: configure, once

import { configure } from '@stabrise/scaledp'

configure({
  cache: 'indexeddb',
  pdf: {
    workerSrc: '/pdf.worker.min.mjs',
    cMapUrl: '/cmaps/',
    standardFontDataUrl: '/standard_fonts/',
  },
  onProgress: ({ file, loaded, total }) => {
    console.log(`${file}: ${Math.round((loaded / total) * 100)}%`)
  },
})

Three things worth knowing here:

There is no default workerSrc. The library never guesses where your app puts things — asset URLs, model hosts and auth all come from configure(). A hardcoded /pdf.worker.min.mjs inside a library would break every app that serves from a sub-path, so there isn't one. If you forget this, pdf.js fails with Setting up fake worker failed, and the library rewrites that into a message containing the two-line fix.

cache: 'indexeddb' is what makes the second visit instant. Weights are stored per origin and reused. It is opt-in because writing hundreds of megabytes to a user's disk should be a decision, not a default side effect.

onProgress fires during model downloads, not during inference. You want it wired before the first run, because the first run is where the bytes are.

Step 4: the pipeline

import { Pipeline } from '@stabrise/scaledp'
import { PdfToImage } from '@stabrise/scaledp/pdf'
import { PaddleTextRecognizer } from '@stabrise/scaledp/ocr'

const pipeline = new Pipeline([
  new PdfToImage({ resolution: 300 }),
  new PaddleTextRecognizer(),
])

const rows = await pipeline.transform(file)
console.log(rows[0].text.text)

That is the whole thing. transform accepts a File, a Blob, a Uint8Array, an ArrayBuffer, a URL string, or rows you built yourself.

The composition model is worth a paragraph, because it is unusual and it is the reason the rest of the library is small. Stages are not connected to each other. There is no .pipe(), no interface, no adapter registry. PdfToImage writes a field called image on the row; PaddleTextRecognizer reads a field called image. They compose because they agree on a name, and the array order is the execution order.

Which means re-wiring is a string change:

new PdfToImage({ outputCol: 'page' }),
new PaddleTextRecognizer({ inputCol: 'page' }),

Step 5: read the results

for (const row of rows) {
  if (row.text.exception) {
    console.warn(`page ${row.page} failed:`, row.text.exception)
    continue
  }

  console.log(row.page, row.text.text)

  for (const box of row.text.bboxes) {
    // x, y, width, height are in the rendered page's pixel space --
    // the same space PdfToImage produced, so they line up with the image.
    ctx.strokeRect(box.x, box.y, box.width, box.height)
  }
}

That bboxes loop is worth seeing rather than describing. Here is the same pipeline run in the builder, with the result panel switched to Boxes:

One row per entry in row.text.bboxes — the word, the model's confidence in it, and the rectangle it occupies in the rendered page's pixel space. Those x/y/w/h numbers are what the strokeRect call above draws.

That exception check is not defensive habit, it is the contract. Stages in this library never throw by default. A failure is recorded in the output schema's exception field and the pipeline completes, so one bad page does not lose the other forty. If you would rather fail fast while developing, pass propagateError: true to a stage and it throws instead.

Step 6: what the first run costs

console.log(rows[0].execution_time)
// { stages: { PdfToImage: 412, PaddleTextRecognizer: 1830 }, total: 2244 }

Those milliseconds exclude the model download, which on the first run dominates everything else. PaddleOCR's default v6-small preset is about 6 MB — small enough that most users will not notice. Adding NER changes that picture completely; the default GLiNER model is around 580 MB, and you should check the cache and warn before starting that.

The complete 30 lines

import { Pipeline, configure } from '@stabrise/scaledp'
import { PdfToImage } from '@stabrise/scaledp/pdf'
import { PaddleTextRecognizer } from '@stabrise/scaledp/ocr'

configure({
  cache: 'indexeddb',
  pdf: { workerSrc: '/pdf.worker.min.mjs' },
  onProgress: (p) => console.log(`${p.file}: ${p.loaded}/${p.total}`),
})

const pipeline = new Pipeline([
  new PdfToImage({ resolution: 300 }),
  new PaddleTextRecognizer({ keepFormatting: true }),
])

export async function readDocument(file: File) {
  const rows = await pipeline.transform(file)

  return rows.map((row) => {
    if (row.text.exception) {
      return { page: row.page, error: row.text.exception, text: '', words: [] }
    }
    return {
      page: row.page,
      error: null,
      text: row.text.text,
      words: row.text.bboxes,
    }
  })
}

Trade-offs

resolution: 300 is a real memory decision. A 300 DPI A4 page is roughly 35 MB of raw pixels, and a 40-page PDF becomes 40 rows that all exist at once. Drop to 200 for speed and memory, go to 400 only for small print, and use pageLimit on long documents.
keepFormatting: true costs you a clean string. It preserves layout by inserting spaces and newlines, which is what you want for reading and for NER context — and what you do not want if you are about to run an exact-match search.
Reusing one Pipeline across files is fine and intended, but call dispose() when you are done to release ONNX sessions.

Next steps

Add NER, run it in a worker so the UI never blocks, or skip OCR entirely on PDFs that already have a text layer — that last one is the highest-leverage change in most real pipelines, because most PDFs are not scans.

Try it

npm install @stabrise/scaledp pdfjs-dist onnxruntime-web ppu-paddle-ocr

Open exactly this pipeline in the builder — PdfToImage({ resolution: 300 }) then PaddleTextRecognizer({ keepFormatting: true }), the same two stages as the thirty lines above. Drop your own PDF on it, or use one of the samples. Nothing you drop there is uploaded anywhere.

Spark's coupling was thinner than it looked

Mykola Melnyk — Thu, 03 Sep 2026 10:28:05 +0000

Written 2026-09-03, about porting ScaleDP (Python/Spark) to @stabrise/scaledp@0.1.1 (TypeScript/browser). Assumes you have seen a Spark DataFrame; assumes no deep Spark knowledge.

TL;DR: Porting a Spark-based document library to the browser sounded like it meant replacing a distributed execution engine. It did not. Every stage was a pure Transformer, and the only DataFrame surface stages used was withColumn / drop / select. That reduces to an array of plain objects and a 215-line runner. The generalisable move: audit the surface a framework's users actually touch before assuming you need the framework.

The assumption going in

ScaleDP is a document-processing library built on Apache Spark. The pipelines look like this:

pipeline = PipelineModel(stages=[
    PdfDataToImage(resolution=300),
    TesseractOcr(inputCol="image", outputCol="text", keepFormatting=True),
    Ner(inputCol="text", outputCol="ner"),
])
result = pipeline.transform(df)

The plan was to run the same pipelines in a browser. The obvious reading of "built on Spark" is that Spark is doing something load-bearing — partitioning, shuffles, a query planner, lazy evaluation, a distributed scheduler — and that replacing it means either reimplementing a scheduler or accepting a much smaller feature set.

So before writing anything, we read every stage and made two lists: which Spark APIs are actually called, and which Spark semantics are actually depended on.

What the audit found

Finding one: there are no Estimators. Spark ML's core abstraction is the Estimator/Transformer split — fit() produces a model from data, transform() applies it. Half the complexity of a pipeline framework exists to model that duality: fitting in dependency order, caching fitted models, distinguishing a Pipeline from a PipelineModel.

ScaleDP has none. Every stage is a pure Transformer. Nothing is fitted, because OCR models and NER models arrive pre-trained. The entire fit half of the framework was unused.

Finding two: the DataFrame surface is three methods. Grepping every stage for DataFrame operations turned up withColumn, drop and select. No joins. No groupBy. No window functions. No aggregations. No SQL.

posexplode appears, for turning a PDF into one row per page — which is the one genuinely interesting operation, and it is "return several rows instead of one".

Finding three: no stage depends on laziness. Nothing relied on the query planner reordering work, pushing down a predicate, or fusing stages. The pipeline is executed in the order written.

Three findings, one conclusion: the Spark coupling is one abstraction thick. Underneath is an ordered list of pure functions over records with named fields.

What was left after removing it

/** One record flowing through the pipeline. Stages read and write named fields. */
export type Row = Record<string, unknown>

That is the data model. withColumn becomes { ...row, [outputCol]: value }. drop becomes delete row[inputCol]. select is not needed at all, because a plain object already lets you read whatever you want.

And the stage:

export abstract class Stage<P extends BaseStageParams = BaseStageParams> {
  abstract readonly name: string

  constructor(readonly params: P) {}

  /**
   * Transform one row's input value into this stage's output value.
   * Throwing is fine and expected -- `transform` converts it into the output
   * schema's `exception` field unless `propagateError` is set.
   */
  protected abstract apply(input: unknown, row: Row, ctx: StageContext): Promise<unknown>

  /** Value written to `outputCol` when `apply` throws. */
  protected abstract onError(message: string, row: Row): unknown

  /** A stage may emit several rows per input row -- PDF page explosion. */
  protected async expand(input: unknown, row: Row, ctx: StageContext): Promise<Row[] | null> {
    return null
  }
}

Two abstract methods and one optional override. apply for the single-value case, expand for posexplode, onError for the error contract. The base class handles column wiring, error capture, keepInputData and timing — so those behave identically across all eighteen stages instead of being reimplemented eighteen times.

The runner is a nested loop:

for (const [index, stage] of this.stages.entries()) {
  options.signal?.throwIfAborted()
  await stage.init()
  rows = await stage.transform(rows, { index, signal: options.signal })
}

215 lines including the timing columns, the input normalisation and the docs comments.

What Spark was genuinely providing

This is the honest other half, and skipping it would make the post a sales pitch.

Horizontal scale. Forty executors, forty PDFs, wall-clock time divided by forty. One tab is one machine, and there is no story here beyond "open more tabs", which is not a story.

Spilling to disk. A dataset larger than memory. The browser runner holds its rows in memory. A 300 DPI A4 page is roughly 35 MB of raw pixels, so a long document is a real constraint rather than a theoretical one.

Fault tolerance across a cluster. A lost executor is retried elsewhere. A crashed tab is a crashed tab.

Being the thing your data platform already runs. Not a technical property, and often the deciding one.

If your workload needs any of those, use the Python library. The pipeline you wrote in TypeScript ports back nearly line for line, which is exactly why the stage and parameter names were kept identical.

The generalisable part

The framework question is usually asked as "can we replace X?" — which invites an answer scoped to everything X can do. The more useful question is "which of X's capabilities does our code actually call, and which of its semantics do we actually depend on?"

Those are two different lists and the second is the harder one. Method calls you can grep for. Semantic dependencies — laziness, ordering guarantees, transactionality, at-least-once delivery — hide in the absence of code. The way to find them is to ask, per stage, "what breaks if this ran eagerly / in a different order / twice?"

In this case both lists came back nearly empty, and the port became a transcription rather than a rewrite. That is not the usual outcome, and the audit is what told us which one we were in. It cost about a day and would have been worth it either way: had the answer come back the other way, we would have known before writing code instead of three weeks in.

Trade-offs

We gave up the ability to say "just add executors". The ceiling is one machine, permanently.
215 lines of runner is 215 lines we now maintain. Spark's scheduler is battle-tested; ours is not. It is small enough to read in one sitting, which is the mitigation.
Matching Spark-era names locks in API decisions we did not make. inputCol/outputCol is not what you would design fresh for TypeScript. Portability was worth more than taste — a pipeline should read the same in both.
The audit is only as good as the reading. We could have missed a semantic dependency. The parity test suites, which diff against goldens generated by running the real Python, are what would catch it.

Your OCR pipeline probably uploads the document. It doesn't have to

Mykola Melnyk — Thu, 03 Sep 2026 10:27:36 +0000

Written 2026-09-03. Code examples target @stabrise/scaledp@0.1.1. No prior OCR or ONNX knowledge assumed.

TL;DR: Every mainstream OCR API has the same architecture — your document goes over the wire to someone else's disk. Browser-native inference removes that box entirely: @stabrise/scaledp runs PDF rendering, text detection, OCR and entity recognition on onnxruntime-web, in the tab, with no upload. That is a real win for sensitive documents and a real cost in first-load bytes. This post is about both.

The architecture every OCR service has

Draw the data flow of any hosted document API and you get three boxes:

[your app] --HTTPS--> [their API] --> [their storage + GPUs]

The third box is the one that generates paperwork. It is why OCR procurement involves a data processing agreement, a retention policy, a sub-processor list, and a conversation with whoever owns compliance. The engineering is five lines; the approval is five weeks.

That box also has a price. Hosted OCR is billed per page — typically somewhere between $0.50 and $1.50 per thousand pages, plus egress. For a product that OCRs every uploaded file, that is a line item that grows exactly as fast as your usage does.

None of this is a criticism of hosted APIs. For 10,000 pages a night they are the correct answer, and I say so in the trade-offs below. The point is narrower: the third box is not a law of physics. The models are small enough to run in a browser, and onnxruntime-web is good enough to run them.

The same pipeline, without the third box

[your app] --> [WASM/WebGPU in the tab]

Here is a complete, runnable pipeline. It renders a PDF, reads it with PaddleOCR, and pulls out entities — and it makes no request to any server of ours, because there isn't one.

import { Pipeline, configure } from '@stabrise/scaledp'
import { PdfToImage } from '@stabrise/scaledp/pdf'
import { PaddleTextRecognizer } from '@stabrise/scaledp/ocr'
import { GlinerNer } from '@stabrise/scaledp/ner'

configure({
  cache: 'indexeddb',
  pdf: { workerSrc: '/pdf.worker.min.mjs' },
  onProgress: ({ file, loaded, total }) => {
    console.log(`${file}: ${Math.round((loaded / total) * 100)}%`)
  },
})

const pipeline = new Pipeline([
  new PdfToImage({ resolution: 300 }),
  new PaddleTextRecognizer({ keepFormatting: true }),
  new GlinerNer({ labels: ['person', 'organization', 'email', 'phone'] }),
])

const rows = await pipeline.transform(file)

for (const row of rows) {
  console.log(`page ${row.page}:`, row.text.text.slice(0, 80))
  for (const entity of row.ner.entities) {
    console.log(' ', entity.entity_group, '=', entity.word)
  }
}

page 1: INVOICE 2024-0417  Acme Industries GmbH  Rechnungsdatum 14.03.2024 …
  organization = Acme Industries GmbH
  person = Marta Feldmann
  email = accounts@acme-industries.de

One row per page. Each stage reads a named field on the row and writes another one — PdfToImage writes image, PaddleTextRecognizer reads it and writes text, GlinerNer reads that and writes ner. That is the whole composition model, and it is the same one the Python ScaleDP library uses on Spark.

What actually disappears

The DPA. Not "is easier to get" — is not needed for this data flow. There is no processor, because there is no third party in the path.

The retention question. "How long do you keep it?" has an answer that requires no policy document: the bytes were in a tab, and the tab closed.

Egress and per-page cost. After the first load, the marginal cost of a page is CPU time on a machine you are not paying for.

Round-trip latency on re-runs. This is the underrated one. Changing a threshold and re-running is 200 ms, not a network round trip, which changes what kind of UI you can build. An interactive parameter panel over a hosted API is a rate-limiting problem; over a local pipeline it is just a re-render.

What appears instead

Bytes on first load. PaddleOCR's default preset (v6-small) is about 6 MB of weights — genuinely small. The default GLiNER NER model is ~580 MB. That is not a rounding error, and it is the single biggest objection to this whole approach. Mitigations exist — smaller models (gliner-pii-edge is ~181 MB), IndexedDB caching so it downloads once, self-hosting the weights from your own origin — but the first visit pays.

A device you do not control. Your server has known cores and known memory. Your user's machine has neither. WebGPU may be available or may not; threads need response headers your host may not send. The library reports what it actually got rather than assuming:

import { isWebGpuAvailable, isCrossOriginIsolated } from '@stabrise/scaledp/ocr'

console.log({
  webgpu: await isWebGpuAvailable(),   // false on plenty of real machines
  threads: isCrossOriginIsolated(),    // false unless COOP/COEP are set
})

Throughput. One tab is one machine. There is no horizontal scaling story here beyond "open more tabs", and that is not a story.

Trade-offs

At volume, a server wins and it is not close. If you are processing 10,000 pages a night, the per-page cost of a GPU you already own beats the per-page cost of downloading 580 MB to ten thousand browsers. Use the Python sibling; the pipeline code ports almost mechanically.
The 580 MB is real and unavoidable if you want good multilingual NER. You can drop to a 181 MB model, drop NER entirely and keep OCR at 6 MB, or self-host from a fast origin — but you cannot make a good zero-shot NER model small.
First run is much slower than the tenth. Model download dominates the first pipeline run completely. Design the UI around that (onProgress, a cache check before you start) rather than hoping users are patient.
AGPL-3.0-or-later. Same licence as the Python library. If that does not work for a closed-source product, StabRise does commercial licensing. Worth knowing before you build on it, not after.

Try it

npm install @stabrise/scaledp onnxruntime-web ppu-paddle-ocr pdfjs-dist

The engines are optional peer dependencies — install only the ones your pipeline uses. Importing the core pulls in no ML runtime at all.

Drop your own PDF into the builder at scaledp-ts.stabrise.com/demo, pick a detector and a recognizer, and watch the network panel while it runs. After the models are cached you can turn the wifi off and run it again.

[Boost]

Mykola Melnyk — Tue, 25 Aug 2026 07:55:12 +0000

Mykola Melnyk

Aug 25

I Built a Windows App That Actually Removes PII From PDFs (Not Just Covers It)

#windows #privacy #ai #productivity

3 min read

I Built a Windows App That Actually Removes PII From PDFs (Not Just Covers It)

Mykola Melnyk — Tue, 25 Aug 2026 06:59:57 +0000

Redacting a PDF sounds simple until you actually have to do it properly. Drawing a black rectangle over a name or an SSN doesn't remove it — the text is still sitting underneath, extractable with a copy-paste or a basic PDF parser. Doing it right means the sensitive content is gone from the file, not hidden under a layer.

I've been working on PDF Redaction Studio Desktop, a native Windows (and Linux) app that handles this end to end: AI-based PII detection, a review step so you stay in control of what gets flagged, and an export that actually strips the matched content out of the PDF.

It just landed on the Microsoft Store, so here's a walkthrough of what it does and how it's put together.

Why a desktop app instead of just a web app

A web version already existed, but a chunk of users — legal, HR, healthcare, anyone handling regulated documents — didn't want to upload files anywhere to get them redacted. The desktop app runs the detection stack (OCR, PII/NER models, face detection) locally, so documents stay on the machine by default. No Docker, no backend to stand up, no account required to just open a PDF and start reviewing it.

The dashboard

Once installed, you land on a dashboard that tracks documents, completed redactions, and page usage — useful if you're processing a batch rather than a one-off file.

Opening a document

Drag a PDF in, or click to browse. Nothing happens with the file until you tell it to.

Detection runs locally

This is the part I care about most. Detection can run against a cloud API or entirely in-browser/on-device using a local model — no network round-trip for the document contents. You pick the model, tune the detection threshold (higher = fewer but more confident matches), and set how many CPU threads to throw at it.

Reviewing what got flagged

After detection, matches are grouped by category — names, SSNs, emails, phone numbers, driver's license numbers, addresses, credit cards, faces — and highlighted directly on the page. You can filter by category, add anything the model missed, and remove false positives before committing to anything.

Only after you're happy with the review does export actually strip the matched content from the PDF — text, embedded objects, and image regions covering faces or signatures all get removed rather than painted over.

What's under the hood, roughly

OCR and layout parsing for scanned documents, not just digital-text PDFs
A multilingual PII/NER model bundled for local, offline detection
Face and signature detection for scanned forms and ID pages
Custom rule support for anything the built-in categories don't cover — internal codenames, account numbers, whatever your compliance checklist needs

Where it stands today

Windows and Linux builds are out now; macOS is in progress. If you're already comfortable with command-line tools there's also a Snap package for Linux, and the desktop app shares the same detection engine as the web version, so behavior is consistent across both.

The free tier covers 100 pages a month, which is enough to actually try it against your own documents before deciding whether it fits your workflow.

If you try it and hit a document type it doesn't handle well — dense forms, unusual layouts, non-Latin scripts — I'd genuinely like to hear about it. That's the kind of edge case that's easy to miss without real-world documents to test against.

Automating PDF Redaction in n8n (No Custom API Code Required)

Mykola Melnyk — Tue, 18 Aug 2026 09:55:15 +0000

If you run n8n workflows that touch PDFs — invoices, contracts, HR forms, scanned intake documents — sooner or later you hit the same wall: someone downstream doesn't need to see the names, emails, or ID numbers in that file. Maybe it's going to a shared drive, a support ticket, an analytics pipeline, or a third-party API you'd rather not hand raw PII to.

The usual fix is writing a custom HTTP Request call to some redaction API, wiring up auth headers, and hand-rolling the JSON body every time. There's now a community node that skips that step: @stabrise/n8n-nodes-pdf-redaction, which drops PDF redaction straight into the node panel.

This post walks through installing it and building a small end-to-end workflow.

What the node actually does

It wraps the PDF Redaction API in three operations:

Anonymize — pick PII categories (names, emails, addresses, credit cards, faces, signatures, and more) and get back a redacted PDF where the matched content is genuinely removed, not just covered with a black box.
Anonymize with Custom Prompt — skip the tag list and describe what to redact in plain English, e.g. "redact anything that looks like an internal project codename".
Detect PII — scan a document and get back the entities found (with bounding boxes), without touching the file. Useful for an audit step or for branching a workflow before you commit to redacting anything.

Both digital and scanned/OCR'd PDFs are supported, across multiple languages.

Install

In n8n: Settings → Community Nodes → Install, then enter:

@stabrise/n8n-nodes-pdf-redaction

That requires n8n 1.60.0+. Once it's installed, "PDF Redaction" shows up as a node, and "PDF Redaction API" shows up as a credential type.

Get an API key

Grab a free key at pdf-redaction.com/apikeys — the free tier gives you 100 pages/month, 10 pages per request, 5 requests/minute, which is plenty for testing a workflow. In n8n, create a new PDF Redaction API credential and paste the key in.

Build a minimal workflow

Here's a three-node workflow that fetches a public sample PDF and redacts a face in it — you can reproduce this exactly, no files of your own needed:

1. Manual Trigger → 2. HTTP Request → 3. PDF Redaction

HTTP Request node config:

Method: GET
URL: https://raw.githubusercontent.com/StabRise/pdf-redaction-api/main/examples/pdfs/SampleWithFace.pdf
Options → Response → Response Format: File

That writes the PDF as binary data into a field called data.

PDF Redaction node config:

Credential: your PDF Redaction API credential
Operation: Anonymize
Additional Fields → Tags: Face
Input/Output Binary Field: data (the defaults line up with what HTTP Request produced, no renaming needed)

Run it, and the output panel gives you two things:

A binary redacted.pdf with the face blacked out and the underlying pixel data actually removed.
A JSON payload with detected_pii (each match plus bounding box coordinates) and a processing_time breakdown per pipeline stage.

The original 823 kB file comes back at 377 kB — the API flattens the processed page(s) into a compact PDF rather than just painting over them.

Swap the tag to something like Person Name, Email, Address and you've got a general-purpose PII stripper for whatever document type flows into that branch of your workflow.

Detect-then-branch pattern

If you don't want to redact unconditionally, put a Detect PII node in front of an IF node:

HTTP Request → PDF Redaction (Detect PII) → IF (detected_pii is non-empty) → PDF Redaction (Anonymize)
                                                                            → (pass through unchanged)

Detect PII has no output binary field — it only returns the detected_pii array — so it's cheap to run as a gate before you decide whether the full anonymize step is even necessary. This is the pattern I'd reach for in something like a document-intake pipeline where most files are already clean and you don't want to touch every single one.

One gotcha worth calling out: if you leave the Tags field empty on Detect PII, you get back whatever the API detects by default, not everything it's capable of finding. Set the tags you actually care about explicitly if you want predictable results.

Where this fits

The node reads/writes n8n's standard binary data fields, so anything that can already put a PDF into a binary property works upstream — a webhook payload, an email trigger with an attachment, Read/Write File from Disk, or an S3/Google Drive node. Nothing about the redaction step cares where the PDF came from.

Full API reference, the complete tag list, and more tutorials (including custom-prompt redaction) are in the n8n integration docs. Source for the node itself is on GitHub.

DEV Community: Mykola Melnyk

Reading the results: text, boxes, entities and how they line up

A row is just an object

The four schemas

ScaleDpImage — a rendered page

Document — text plus the boxes it came from

DetectorOutput — boxes with no text

NerOutput — entities

The Box convention, which is not what you expect

Entities carry their boxes, and that is the useful part

Displaying it

Annotating the page is a stage, not a helper

Trade-offs

Try it

Further reading

The same pipeline, minus the cluster

The same pipeline, twice

What is actually different

The part that surprised us

What Spark was genuinely providing

What carries over unchanged

What has no browser equivalent

Trade-offs

Try it

Further reading

onError must return the right shape, or the crash just moves

The half of the contract nobody designs

The wrong answer

The right answer

What this buys downstream

Empty is not the same as failed

The display helpers cooperate

The rule, generalised

Trade-offs

Try it

Further reading

Privacy is an architecture, not a checkbox

Promises versus properties

What "never left the tab" actually means here

How to prove it to a reviewer

What this does not buy you

The pattern this enables

Trade-offs

Try it

Further reading

Running OCR entirely in the browser meant decoupling detection from recognition

The problem: an OCR stage that couldn't be composed

The fix: split detection out of the recognizer

1. Straighten before you crop

2. Batch by canvas, not by call

3. WebGPU can't run this graph

A word-splitting bug we found while wiring this up

Why this shape, not a bigger abstraction

Trade-offs

Try it

Further reading

One bad page must not lose the other forty

The requirement

The contract

Reading it

Check exception before you check the payload

Opting into throwing

What is deliberately not covered

Failures are timed

Trade-offs

Try it

Further reading

A document pipeline in 30 lines, running in a browser tab

What we are building

Prerequisites

Step 1: install

Step 2: serve the assets

Step 3: configure, once

Step 4: the pipeline

Step 5: read the results

Step 6: what the first run costs

The complete 30 lines

Trade-offs

Next steps

Try it

`ScaleDpImage` — a rendered page

`Document` — text plus the boxes it came from

`DetectorOutput` — boxes with no text

`NerOutput` — entities

The `Box` convention, which is not what you expect

Check `exception` before you check the payload