DEV Community: Hideki Mori

Dynamic isn't enough. Operations is the other half.

Hideki Mori — Mon, 18 May 2026 13:00:00 +0000

If you've been writing software for a few years, you eventually start designing for change.

A content delivery system: a new format will appear, a new player will appear, so let's not hard-code formats. An e-commerce site: a new payment method will be requested, so let's not hard-code billing logic. A document pipeline: a new processing engine will appear, so let's not hard-code engines.

This is good instinct. You read books like Analysis Patterns, you internalize the idea that the model should be flexible, you push concrete decisions out of code and into data. New format? Insert a row. New payment? Insert a row. New engine? Insert a row.

It's a beautiful pattern. It really does extend the lifespan of a system.

But after enough years of running systems built this way, here's the part that doesn't show up in the books:

Dynamic isn't enough. Operations is the other half — and that other half is usually paid by someone else.

The "wait, maybe..." moment

Reality always overshoots the model. Always.

You build a content system that handles formats dynamically, and then someone asks for a format whose delivery rules don't fit the existing shape. You build a payment system that takes new methods through configuration, and then someone wants a billing model where the "amount" isn't even computed at the same time as the "charge." You build an engine registry, and then a new engine doesn't conform to the lifecycle every other engine has assumed.

Each time, the moment looks the same. You stare at the request, and your first thought is: that's outside the model. And your second thought, a few minutes later, is:

Wait, maybe... if I interpret this field a little differently, and if I add one column here, and if I let this branch handle a special case... it fits. Sort of. It doesn't break anything.

So you do that.

The system keeps running. No downtime. No coordinated migration. No painful conversation with the integration partners. From your seat as the designer, the dynamic structure absorbed the change. The pattern worked.

That's the part designers love. That's the part that gets written about.

Who pays for the other half

Here's what's harder to see from the designer's seat: every "wait, maybe..." moment leaves a small residue somewhere in operations.

The operator now has to remember that for this particular case, the workflow is slightly different. The reporting tool needs a footnote. The on-call playbook gets one more conditional. The next person to onboard needs another paragraph of context. The integrating system has to send a value in a slightly unexpected place.

None of these are catastrophic. None of them break the system. Each one, on its own, is a small inconvenience.

But the small inconveniences accumulate. And the people accumulating them are usually not the people who designed the dynamic structure in the first place.

The designer experiences each "wait, maybe..." as a successful absorption. The operator experiences it as one more thing to remember. Same event, two ledgers. Only the designer's ledger gets reviewed.

This is where engineering satisfaction can become quietly self-serving. The system didn't break. The pattern held. Therefore the design is good — except good for whom, exactly?

The road I didn't take

There is, of course, an alternative. You can stop the system, ship version 2, ask everyone to migrate, and start clean.

A scheduled 24-hour outage. A v2 documentation packet sent to every integrating partner. A coordination meeting. A cutover.

I have, for twenty-four years, mostly chosen not to do this. I have preferred quiet absorption — interpretation, gentle reshaping, "wait, maybe..." — over coordinated rebuild.

Whether that was right, I genuinely don't know. There were probably operators along the way who would have been happier with one painful migration than with years of small accommodations. There were probably integrating systems whose engineers would have preferred a clean v2 over a v1 that kept quietly bending.

I chose to keep the shape. That choice has a cost. The cost is paid in operator-hours, in playbook footnotes, in the small extra cognitive load of working with a system that has absorbed more than its original model anticipated.

I'm not saying I was wrong. I'm saying: I'm still not sure.

Generic underneath, specific on top

There's one thing I do believe, with more confidence than the rest:

When the model gets very generic, the UI usually has to get very specific to compensate.

A maximally flexible data model — the kind that absorbs new formats, new payment methods, new engines — is, almost by definition, awkward to interact with directly. Direct interaction with a generic model means the human has to supply all the missing context. That's exhausting.

The good outcome is: the model stays generic underneath, and the UI is built specific to each use case on top. Each surface is custom. Each surface is rigid in exactly the way that makes it usable.

This is a hard decision to make, because every specific UI you build feels temporary. You're going to throw it away when the use case shifts. You build it knowing it has a shorter half-life than the model below it. That asymmetry is uncomfortable.

But I think the discomfort is the right discomfort. A long-lived generic core, with shorter-lived specific surfaces on top, is — based on what I've watched survive over twenty-four years — closer to how systems actually age well.

Three observations

None of these are conclusions:

Dynamic structure extends a system's life. It does not, on its own, make operating that system pleasant.
Every "wait, maybe..." absorption is a real engineering achievement and a small tax on someone downstream. Both of these are true.
If you make the model very generic, give the operators a UI that isn't generic.

Dynamic isn't enough. Operations is the other half — and that other half is usually paid by someone else.

After twenty-four years of this, that's the only thing I feel sure of.

Earlier in this series:

The Accordion Pattern: Why I stopped writing one fat LLM prompt
Nobody knows when a job will finish. I'd still like to report it accurately.
What survives when you build alone for 24 years.

What survives when you build alone for 24 years

Hideki Mori — Mon, 11 May 2026 13:00:00 +0000

In December 2002 I joined a company that was nearly bankrupt.

I was twenty-five. A software engineer with no title beyond that. The company was burning cash, and the next product launch had to happen in January — about a month after I started.

By January 2003, the demo shipped.

It was an HTML browser for mobile phones — with packet reduction built in. It rendered HTML, played back MIDI files linked from <a> tags, ran simple background animations. I wrote it in Java.

The server was PHP. There was no engineer for the server-side compression part, so I did that too — also in Java, called from PHP over localhost.

A strange shape, but the deadline was real.

I remember being a little angry. I also remember being much happier than angry. I had shipped something, in a month.

That's how I started designing things alone. Not as a plan. Just: someone had to, and nobody else moved.

The first thing that wasn't a plan

Looking back twenty-three years later, here's something I didn't notice at the time:

I had no responsibility. I was a junior engineer with a deadline. If the demo had failed, the company would have collapsed faster — but the failure wouldn't have been mine to own.

That made it easy to ship.

After that first month, I never really had another hard deadline. Not on the projects I owned. I'd just answer "as soon as I can" and try to deliver on that.

By 2005 I was a CTO. The work didn't get lighter; the responsibility arrived. And here's the thing I didn't expect: the shape of my code didn't change.

I kept writing things by myself. I kept ending up alone on the deep parts. The org chart rearranged around me, but at the bottom — when something had to actually work — I was still just there, writing.

Productivity and accountability don't mix easily. The shape I learned at twenty-five — write it because nobody else will — was the shape that kept showing up after the title changed.

A shape I didn't notice was a shape (2003–2005)

After the packet-reduction tool, I built a music distribution platform for mobile phones. This was the carrier-billed era. Feature phones. Dropped connections in elevators and tunnels.

The carrier had a billing rule that taught me something I wouldn't articulate for twenty years.

The rule was: don't bill until the user's device confirms the download finished.

So the flow looked like this:

User taps "buy"
The phone starts downloading (with HTTP Range, because connections drop)
The server delivers in chunks
When the device finishes, it sends a hidden completion signal
Then the server bills

If the train went into a tunnel, the user wasn't charged. The server kept holding the work for as long as the device was alive. Eventually either the download completed and we billed, or it didn't and we ate the cost.

Twenty years later I'd phrase the rule like this:

Nobody knows when a job will finish. But when it does, I want to report it accurately. So I make users wait — and in exchange, I retry as hard as I can until I have a result.

That was already the rule for the music platform in 2003. I just didn't know it was a rule yet.

The shape kept showing up (2006–2018)

In 2006 I started on an e-book distribution platform. Four months from blank page to production. I maintained it for eight years, alone.

The shape stayed the same:

Direct delivery, no intermediary platforms in the way
Real-time settlement back to publishers
HTTP Range support, because phones still dropped connections
Tar archives generated per request — buttons, branding, rights metadata stitched in at the moment of delivery, not pre-built and stored

I stepped off in 2014. The platform kept running.

By 2018, with someone else maintaining it, it was distributing tens of billions of yen in transactions per year.

It ran for seven years after I left, and then, in 2021, it was discontinued.

Eight years on my watch. Seven more without me. Then it stopped.

Not a tragedy. Not a triumph. Just what happened.

Same shape, bigger numbers (2021–)

In 2021 I started on what's now called LDX hub — a document processing platform for translation, OCR, structured extraction, and AI-assisted refinement.

Different domain. Same shape.

Year   Documents processed   Characters processed
2021   2,200                 ~20M
2022   48,500                ~810M
2023   85,000                ~2.7B
2024   163,000               ~3.5B
2025   351,000               ~8.3B

The numbers grew. I didn't change anything fundamental about how the system is shaped. Jobs are submitted, the API holds the work, the API retries through whatever needs retrying, the API reports when there's a real result to report.

Same as the carrier-billed music platform. Same as the e-book distribution. Same as that strange first month with two languages talking through localhost.

I didn't decide to keep this shape. I just never found a better one.

Productivity and accountability, again

Looking back at twenty-four years, here's the only honest thing I can say:

The shape I learned with no responsibility — the shape that lets one person ship something in a month — turned out to be the shape that kept showing up after responsibility arrived.

Not because I planned for that. Because it was the only shape I knew how to write, and I kept writing it, and nothing about the larger numbers ever required me to write differently.

Whether that means anything for your system, I genuinely don't know.

I haven't found a different shape worth writing.

For twenty-four years, I keep ending up here.

Earlier in this series:

The Accordion Pattern: Why I stopped writing one fat LLM prompt
Nobody knows when a job will finish. I'd still like to report it accurately. (scheduled for May 4)

Nobody knows when a job will finish. I'd still like to report it accurately.

Hideki Mori — Mon, 04 May 2026 13:00:00 +0000

Most async APIs commit to one thing: starting your job. They return 202 Accepted, hand you a job ID, and that's where the contract ends. The rest is your problem.

I do something different. I make one promise:

When your job is done, I'll tell you accurately. Until then, I'll keep retrying.

That's the entire contract for everything I've ever shipped. It sounds small. In practice, it's the only thing I actually do.

The shape every job in my system shares

You hand me work.

You wait.

I retry as hard as I can.

I report when it's done.

That's it. Whether the job is OCR on a scanned PDF, structured extraction from a long document, or refining the translation of an XLIFF file — the shape is identical. You give me an input. You don't watch the screen. I come back when I have something honest to report.

This sounds obvious until you try to actually deliver it.

Why "started" is easier than "finished"

Returning 202 Accepted is easy. The hard part starts right after that.

Real jobs hit things like:

Vendor APIs that occasionally throw 503. No reason. Just sometimes.
Native binaries that core dump. Twice in a row, then fine for a week.
Subprocesses that go zombie. Not crashed. Not finished. Just defunct. The OS still holds them.
Disks that fill up with stale debug files because something somewhere wrote them and forgot.

If you ship "started, here's a job ID, good luck" and call that an API, you're outsourcing all of the above to your user.

I'm not willing to do that. So I take the work back inside.

What that looks like in code

I'm not going to name any vendor. They don't matter. What matters is the shape. The code below is a simplified sketch — the production version handles a lot more (PDF library version quirks, fallback engines when the first one rejects the input, demo-mode page limits, and a long list of vendor-specific error codes that mean "retry," "skip," or "stop"). The shape is what survives.

Here's a sketch of the inside of one of my conversion services:

public JobResult runJob(Input input) throws Exception {
    for (int attempt = 0; attempt < MAX_RETRIES; attempt++) {
        Process child = new ProcessBuilder(
                "java", "-cp", classpath, EngineMain.class.getName())
            .redirectErrorStream(true)
            .start();
        passInputToStdin(child, input);

        long started = System.currentTimeMillis();
        while (child.isAlive()) {
            if (System.currentTimeMillis() - started > MAX_RUNTIME_MS) {
                child.destroyForcibly();
            }
            if (isDefunct(child)) {
                reap(child);
                break;
            }
            sweepStaleCoreFiles(workDir, MAX_CORE_AGE_MS);
            Thread.sleep(POLL_INTERVAL_MS);
        }

        ChildOutcome outcome = readOutcome(child);
        if (outcome.isTransientError()) continue; // retry
        if (outcome.isIrrelevantError()) {
            log.info("irrelevant error, treating as success: {}", outcome);
            return outcome.toSuccessResult();
        }
        if (outcome.hasResult()) return outcome.toResult();
    }
    return JobResult.failedAfterRetries(MAX_RETRIES);
}

A few things in there are worth pointing at.

new ProcessBuilder("java", ..., EngineMain.class.getName()). Not "call a library function." Not "use the SDK." I literally re-enter main from another process. The reason is that the underlying engine, in its native form, is unreliable enough that I want process-level isolation. If it dies, only the child dies.

if (isDefunct(child)) { reap(child); break; }. Native binaries don't always exit cleanly. Sometimes they're not crashed and not running — they're stuck. The parent has to notice, decide, and clean up.

sweepStaleCoreFiles(workDir, MAX_CORE_AGE_MS). When a child crashes hard, the OS dumps a core file. That file is huge. If you don't sweep it, the disk fills up. There is no clever solution here. You sweep.

outcome.isTransientError() → continue. Some vendor errors come and go. The fix is to wait and try again. If you don't try again, your user sees failed. If you do try again, your user sees "took a bit longer." I pick the second one.

outcome.isIrrelevantError() → log and return success. This is the part that surprises people. Some errors aren't actually errors for the use case. They're noise the engine emits. Knowing which is which takes years, and is most of the actual product.

None of this is elegant. None of it shows up in an architecture diagram. It all lives in the gap between "the job was submitted" and "the job is done, here's the result."

That gap is what I do.

What I gave up

I don't promise low latency. I can't. The thing I'm waiting on isn't predictable.

I don't promise the job will always succeed. Sometimes the input is genuinely broken. Then I report that, accurately, instead of pretending.

I don't promise streaming partial results. I keep the user out of the loop until I have something stable to hand back. The cost is they wait. The benefit is they don't see noise.

These trade-offs aren't sophisticated. They're just consistent.

I didn't design this. It survived.

Looking back, this is how every job-shaped API I've ever built has worked. I didn't sit down one day and decide on a contract. I kept ending up here.

Each time I tried to ship something where the API said started and stopped caring, the user came back asking what happened. So I started caring. Each time I tried to surface every transient error to the user, the user got scared. So I started absorbing them. Each time I tried to make jobs faster by skipping the cleanup, the disks filled up. So I started sweeping.

After enough years of this, what's left is a single rule:

When the job is done, I'll tell you accurately. Until then, I'll keep retrying.

Whether that's the right contract for your system, I genuinely don't know. It's just the only one I've found that survives.

Earlier in this series: The Accordion Pattern: Why I stopped writing one fat LLM prompt

The Accordion Pattern: Why I stopped writing one fat LLM prompt

Hideki Mori — Wed, 29 Apr 2026 07:51:20 +0000

Most structured-extraction tutorials look the same. Take a document, write one big prompt that says "extract A, B, C, D, E, F", get JSON back. Done.

This works on short inputs.

It quietly breaks on long ones.

After running this in production for a while, I stopped doing it. Here's what I switched to and why.

The fat prompt problem

Say you have a 50-page report and you want a structured summary out of it. The natural first move is something like:

Extract:
- title
- sections (with headings)
- purpose
- mentioned services
- acceptance criteria
- ...
Return JSON in this shape: { ... }

You hand the whole document to the model. It returns JSON. It looks fine on the first try.

Then you scale it up and three things happen:

Quality drifts. The model "forgets" mid-document. Later sections are summarized worse than earlier ones, or fields go missing.
One bad field poisons the whole call. If "acceptance criteria" hallucinates, you don't just lose that field — the whole record gets quarantined for review.
Latency goes up, parallelism goes down. A single 30k-token call takes what it takes. You can't shard it.

You can fight this with longer prompts, more examples, stricter formatting rules. I did. It buys you maybe 10% more reliability and costs you a lot of prompt-engineering time.

The structural problem doesn't go away.

What I do now: split it

The pattern I use looks like an accordion that expands:

[ document ]
     │
     ▼
[ Stage 1: segment ]   ← one prompt, one job: produce a list
     │
     ▼
[ array of segments ]
     │
     ▼ (fan out)
[ Stage 2: extract ]   ← one prompt, runs per segment
     │
     ▼
[ structured records ]

Stage 1 reads the whole document and returns a clean array of segments — sections, paragraphs, line items, whatever the right unit is for the task.

Stage 2 takes one segment at a time and extracts the structured fields you actually want.

Two prompts, each doing one thing.

Why this works better

Each prompt has a single job.
Stage 1 is "find the boundaries". Stage 2 is "extract the schema". Neither prompt has to hold both ideas at once. You can write each one tightly. Examples are shorter and more on-point.

Errors localize.
If Stage 2 fails on segment 7, you re-run segment 7. You don't redo the whole document. Bad fields get isolated to one record instead of contaminating the whole batch.

Stage 2 parallelizes naturally.
The output of Stage 1 is an array. Fan it out. Run 50 small extractions in parallel instead of one big one. Total wall-clock time drops, and so does the variance.

Cache hits go up.
If the same segment shows up twice (templates, standard headers, repeated forms), Stage 2 sees the same input and you can cache. The fat-prompt version sees the entire document as one unique input every time.

Long documents stop being scary.
The hard limit on a fat prompt is the model's context window. The accordion pattern doesn't have that ceiling. Stage 1 still has to read the whole document, but its output is small. Stage 2 only ever sees one segment.

What it costs

It's not free.

You're making more LLM calls — one for Stage 1 plus N for Stage 2 instead of one. On short inputs that's wasteful. The accordion pattern is for documents long enough that fat prompts start failing, not for two-paragraph emails.

You also need to think a little harder about what a "segment" is for your task. Sometimes it's a section heading. Sometimes it's a row in a table. Sometimes it's a logical unit that doesn't map to any visible boundary. That's a design decision and it matters.

When to use it

Reach for the accordion when:

The document is long enough that you've seen the model lose the thread mid-way.
The output schema has more than ~5 fields and they don't all care about the same context.
You need to retry failed records without redoing successful ones.
You want parallelism.

Stick with one fat prompt when:

The input is short and the schema is small.
The fields are tightly coupled (extracting one needs context from another).
You're prototyping and don't care yet.

A small concrete example

I run this on a service called StructFlow. The shape of the calls is roughly:

# Stage 1: segment
curl -X POST https://gw.ldxhub.io/structflow/jobs \
  -H "Authorization: Bearer $KEY" \
  -d '{
    "model": "google/gemini-3-flash-preview",
    "system_prompt": "Split this document into logical sections. Return one JSON record per section.",
    "example_output": { "section_title": "...", "section_text": "..." },
    "inputs": [{ "id": "doc1", "data": { "text": "..." } }]
  }'

The response gives you back an array. Then Stage 2:

# Stage 2: extract (one call per segment, run in parallel)
curl -X POST https://gw.ldxhub.io/structflow/jobs \
  -H "Authorization: Bearer $KEY" \
  -d '{
    "model": "google/gemini-3-flash-preview",
    "system_prompt": "From this section, extract: purpose, mentioned services, acceptance criteria.",
    "example_output": { "purpose": "...", "mentioned_services": [], "acceptance_criteria": [] },
    "inputs": [{ "id": "sec1", "data": { "section_text": "..." } }]
  }'

Two calls, each focused. One returns segments. The other turns each segment into structured fields.

That's the whole pattern.

Why I'm posting this

I built LDX hub partly to make this pattern easy to run — one API, async jobs, file-based input/output so Stage 1's output is directly usable as Stage 2's input. But the pattern itself doesn't depend on any specific tool. You can do it with raw OpenAI calls, Anthropic calls, anything that takes a prompt and returns text.

The takeaway isn't "use my API". It's: if your structured extraction is getting flaky on long inputs, the answer probably isn't a longer prompt. It's two prompts.

If you've tried something similar — or if you've got a case where this falls apart — I'd be curious to hear it.