DEV Community: Hideki Mori

Abstractions are fine. Starting on them isn't.

Hideki Mori — Mon, 08 Jun 2026 13:00:00 +0000

I've been writing software alone for 24 years. The shape of how I do that hasn't changed much — but I've watched the environment around me change a lot, and one shift has been quietly bothering me.

It's the shift in what application engineers are given on day one.

Pain used to teach you when something was wrong

In the old days, the lesson was simple. Your hardware stopped responding. That was the lesson.

A box couldn't keep up — you saw it on the dashboard, you felt it in the user complaints. You learned, often the hard way, that you needed an LB and two machines instead of one. The pain told you the system had outgrown its current shape.

EC2 didn't change this much. An instance could still get pinned at 100% CPU. The application could still hang. The pain was still there, just on rented hardware instead of bought hardware. The lesson — squeeze whatever you can out of the application layer first — was still delivered in the same form.

The pain was the cheapest monitoring tool any of us ever had.

Serverless took three pains away

Then serverless arrived. Auto-scaling. Pay-per-use. Managed everything. And without anyone really announcing it, three things that used to teach engineers stopped teaching them.

Pain 1: cost spikes that show up too late

In a per-use billing model, your application can be running badly for weeks before the financial signal arrives. You only notice when the invoice shows up at the end of the month — by which point, the failure is already paid for.

The "fix" most people reach for is a budget alert. But a budget alert is a smoke detector after the fire is already in the wall. You needed to know in week one, not in week four.

Modern anomaly-detection tools can shorten the gap from weeks to hours. They help. But notice the framing: each new tool exists because something underneath was hidden in the first place. The tool is a patch on a layer of invisibility, not a substitute for not having that layer.

Pain 2: performance degradation that auto-scaling hides

You're looking at a database load graph that's been climbing for three months. Why?

Is it because users are growing? Healthy. Or is it because your aggregation table is missing partitions, so query cost grows superlinearly with data volume? Not healthy.

When the database has fixed limits, this question gets forced on you. The database starts complaining. You have to look.

When the database auto-scales, the second condition is much easier to overlook. The infrastructure absorbs the inefficiency, and nothing forces you to look. The graph keeps climbing. The bill keeps climbing. Nobody traces it back to the actual code that's doing the wrong thing.

Pain 3: where the work actually happens

This is the one I think about most.

Someone I work with recently described a problem to me like this: the managed GraphQL layer is firing way too many SQL queries, and I need to do something about it.

I kept turning that statement over in my head. The managed GraphQL layer is firing too many SQL queries. What does that actually mean?

It means: queries are being issued, against a relational database, on behalf of an application this engineer is responsible for. And the engineer doesn't fully know how many, when, or with what transaction boundaries.

That last part is the one that matters. I asked them: if you're aggregating into yearly, monthly, and daily summary tables, and then setting a "processed" flag on the source records — and the very last UPDATE fails — can you roll all of that back? Is the whole sequence under your control?

I'm not sure they fully understood the question. To be fair, in the world they were handed on day one, the question doesn't naturally come up. The way the queries were resolved broke the unit of work into independent pieces, and each piece looked atomic on its own.

That isn't an SQL problem. It's a control problem.

A web design analogy

If I had to put this another way:

You can build a beautiful website using a no-code tool today. The output looks great. It deploys fine. It probably even works in production for a while.

But when something needs adjusting at the level of the generated code, you're stuck. The tool has handed you a polished surface, and not the means to repair it. For prototyping, this is excellent. For long-term operation, it's a problem you'll only discover the day you need to fix something the tool didn't anticipate.

Serverless plus heavy abstraction layers, given to an engineer on day one, is a similar situation — except the things they can't repair include cost, performance, and transactional integrity, all at once.

The application engineer is the right judge — they just need to be able to see

Here's the part that often gets reversed in these discussions.

The right person to judge whether an application is well-optimized is the application engineer who built it. They know the business requirements. They know which endpoint is hit by 1M requests a day, and which one is hit by 10. They know which tables grow linearly with users and which grow with the square of users. Infrastructure people, by their position, often don't have this context — not because they lack ability, but because the role doesn't naturally include the business reasoning behind every endpoint. They can't make this judgment from the outside.

The application engineer is the right judge.

But to judge, they need to be able to see. They need to know how many SQL queries their endpoint issues. They need to know what transactions they own. They need to know what happens when the third write succeeds and the fourth fails. None of this is obscure or fancy — it's the basic literacy of running a relational database under load.

The problem isn't that abstractions exist. Abstractions are fine. The problem is that abstractions are now the first thing an engineer encounters, and the things underneath them are made deliberately invisible.

My setup, for what it's worth

I run something called LDX hub. It's a public API platform on top of an internal hub I've been growing for years. Five services, document processing, the usual.

For the public-facing edge, I use a managed gateway on a flat-rate monthly subscription. It's "serverless" in shape, but the billing is fixed. There is no cost-spike risk to monitor for. If the bill could ever scale with traffic — even gracefully — I would seriously reconsider. The flat rate is the entire reason this part of the stack is comfortable to me.

For the compute layer, I run Java on EC2. Java's cold-start cost makes a long-running process the natural fit, but that's only half the reason. The other half is that EC2 has fixed limits. When something is wrong with the application, the fixed ceiling forces me to see it. I keep some old-fashioned things — server-side metrics, per-segment timing logs that record how long each part of a request took, alerts when something runs slower than it should — and these tell me, fairly directly, whether the application is doing its job efficiently or not.

For the database, I run my own RDB on EC2 too. Managed databases would handle backups and patches for me, but they would also schedule downtime I can't always control. I prefer the option of doing maintenance myself, on my schedule, with the techniques I've used for two decades — including running two databases in parallel through an application-layer two-phase commit, then cutting over, all without taking the system down. None of that works if the database is something I can't touch.

The whole stack is in AWS. None of it is "old school" in the sense of being on physical hardware. But each layer was chosen so that the things I should be able to see remain visible, and the things I should be able to control remain in my control.

This is not what you should do. This is what 24 years has taught one specific person to do.

Closing

I'm not against abstractions. I use them. The managed gateway in front of LDX hub is one. Cloud is one. Even my IDE is one.

What I'm against is starting there.

If you learn to write applications in an environment where you can't see what you're consuming, you don't learn the relationship between what you wrote and what your system did. You also don't learn how to cover the gap when something goes wrong. Twenty-four years from now, you'll still be guessing at it.

The engineers I see escape this trap usually do it because someone — often by accident — exposed them to a system where the lights were on.

I don't know if that's a fixable situation at the level of an industry. I work around it in my own setup by keeping the lights on — not because visibility is a virtue, but because the business logic is the one thing in my stack that no one else will get right for me. Everything else exists to give me time to get that part right.

Experience widens the set of choices that work for that. I'm not arguing against any of those choices. I'm arguing against the day one where there isn't a choice yet.

Built with Claude (Opus).

Earlier in this series:

Build a multi-step n8n Form with dynamic dropdowns (no plugin needed)

Hideki Mori — Wed, 03 Jun 2026 13:00:00 +0000

You want a multi-step form in n8n. Each step's dropdown depends on what the user picked or uploaded in the previous step. You search for "n8n dynamic dropdown" and find static fieldOptions examples and the occasional "use a Code node and hope."

I had the same problem building the distribution workflow for LDX hub, an AI document processing API. I wanted one workflow where a user picks one of five services, then walks through engine selection, file upload, and output format — with every dropdown computed from the previous step.

It turns out n8n already supports this. The feature is just hidden behind one toggle on the Form node.

This post walks through the pattern, with working code, three things to watch out for, and a link to a complete reference workflow you can import.

The one toggle that changes everything

The n8n Form node has a setting called Define Form. The default is Using Fields Below — you add fields in the UI like a typical form builder. There's a second option: Using JSON.

In JSON mode, the entire field array becomes an n8n expression. You return a JavaScript array where each element is a field definition, and you can reference any previous step's data inside that expression.

That's the whole technique. Everything below is just showing what you can do once you flip that switch.

Here's the chain we're going to build, for one branch of the workflow:

Form Trigger collects the API key and the service the user wants to use. The HTTP Request fetches the available engines from LDX hub. Then a sequence of Form / Set / Form / Set / Form steps progressively narrows the user's choices based on what came before. The LDX hub node runs the job. A final Form node displays the result.

The minimal example: dropdown from an API call

The simplest dynamic dropdown is: call an endpoint that returns a list, render that list as options.

LDX hub's /extractdoc/engines endpoint returns the available extraction engines without authentication. Calling it from n8n:

HTTP Request node
  URL: https://gw.ldxhub.io/extractdoc/engines

The actual response right now:

{
  "data": [
    {
      "id": "ki/extract",
      "display_name": "KI Extract",
      "provider": "ki",
      "description": "Extracts plain text from documents in reading order...",
      "supported_conversions": [
        { "from": "pdf",  "to": "text"  },
        { "from": "pdf",  "to": "jsonl" },
        { "from": "docx", "to": "text"  },
        { "from": "docx", "to": "jsonl" },
        { "from": "xlsx", "to": "text"  },
        { "from": "xlsx", "to": "jsonl" },
        { "from": "pptx", "to": "text"  },
        { "from": "pptx", "to": "jsonl" }
      ]
    }
  ]
}

One engine today. If the provider adds more tomorrow, they show up in the response automatically — and, as you're about to see, in the dropdown.

The Form node that follows the HTTP Request:

Form node
  Define Form: Using JSON
  JSON Output:
    {{ [
      {
        fieldLabel: "Engine",
        fieldName: "engine",
        fieldType: "dropdown",
        fieldOptions: {
          values: $json.data.map(e => ({ option: e.id }))
        },
        requiredField: true
      }
    ] }}

The expression returns an array with one field. The field is a dropdown. Its options are computed by mapping the HTTP response's data array into the shape n8n expects ({ option: "ki/extract" }).

When the form renders, the user sees ki/extract as the only choice. The list reflects whatever the API returned just now, not what you hardcoded last week. That's the whole future-proofing argument for doing it this way.

Going one level deeper: filter by user's choice

Static dropdowns are easy. The interesting case is when one dropdown's options depend on what the user chose in a previous dropdown.

After the user picks an engine, I want to compute which input file formats it supports. That information is in the engine's supported_conversions array. A Set node does the derivation:

Set node
  Assignments:
    name:  from_options
    type:  array
    value: {{ $('ExtractDoc: Get Engines').item.json.data
              .find(e => e.id === $json.engine)
              .supported_conversions
              .map(c => c.from)
              .filter((v, i, a) => a.indexOf(v) === i) }}

For ki/extract, this produces ["pdf", "docx", "xlsx", "pptx"].

Three things to notice in that expression:

$('ExtractDoc: Get Engines').item.json — the long form for cross-step reference. The short form $json only sees the immediately previous step's data. To reach back past the Form node that ran in between, you need the long form. This is the single biggest gotcha. More on this below.
$json.engine — the immediately previous step's data, which is the engine the user just selected.
.filter((v, i, a) => a.indexOf(v) === i) — uniquify. Engines advertise the same input format across multiple conversion pairs (pdf → text, pdf → jsonl), and you don't want the dropdown to repeat "pdf" four times.

The next Form node renders the file upload field, restricting accepted file types to what the chosen engine actually supports:

Form node
  Define Form: Using JSON
  JSON Output:
    {{ [
      {
        fieldLabel: "Source File",
        fieldName: "file",
        fieldType: "file",
        acceptFileTypes: "." + $json.from_options.join(",."),
        requiredField: true
      }
    ] }}

acceptFileTypes becomes the browser's <input accept="..."> attribute. With from_options = ["pdf", "docx", "xlsx", "pptx"], the resulting value is .pdf,.docx,.xlsx,.pptx — the user's file picker shows only those four types.

Even one more level: filter output by the uploaded file

The output format dropdown depends on what file the user actually uploaded — not what the engine generally supports, but what it can convert this specific input into.

This is where you reach into the uploaded file's metadata via $binary:

Set node
  Assignments:
    name:  to_options
    type:  array
    value: {{ (() => {
      const map = { jpg: 'jpeg', tif: 'tiff' };
      const raw = ($binary.file.fileExtension || '').toLowerCase();
      const from = map[raw] || raw;
      return $('ExtractDoc: Get Engines').item.json.data
        .find(e => e.id === $('ExtractDoc: Select Engine').item.json.engine)
        .supported_conversions
        .filter(c => c.from === from)
        .map(c => c.to)
        .filter((v, i, a) => a.indexOf(v) === i);
    })() }}

A few notes on this expression:

IIFE wrapper (() => { ... })() — to use local variables (map, raw, from) and an explicit return. n8n expressions are JavaScript; any valid JS works inside {{ }}.
Extension mapping — $binary.file.fileExtension returns whatever the user's file is named with. APIs sometimes use a canonical form (e.g., jpeg rather than jpg); a small map normalizes the boundary so the filter doesn't miss matches.
Two long-form references — $('ExtractDoc: Get Engines') for the engine list, $('ExtractDoc: Select Engine') for the user's selection. Long form everywhere.

For a user uploading a PDF against ki/extract, this resolves to ["text", "jsonl"]. The output dropdown is then trivial:

Form node
  Define Form: Using JSON
  JSON Output:
    {{ [
      {
        fieldLabel: "Output Format",
        fieldName: "output_format",
        fieldType: "dropdown",
        fieldOptions: {
          values: $json.to_options.map(t => ({ option: t }))
        },
        requiredField: true
      }
    ] }}

Here's what the user sees on the entry page of the rendered form, before any of the dynamic dropdowns appear:

Three things to watch out for

1. The short form `$json` does not reach across forms

$json refers to the immediately previous node's output. The moment a Form node sits between two nodes, $json can't see the data from before that Form.

Fix: the long form, $('Node Name').item.json.field. Works from anywhere in the workflow, regardless of how many nodes intervene.

Make it your default. Use the long form even when the short form would work — it'll save you when you later insert a Form node and don't want to rewrite three expressions.

2. Binary data needs an explicit pass-through

Form nodes return data on the json channel. The file the user uploaded sits on a separate binary channel that doesn't automatically propagate through Set nodes, Switch nodes, or subsequent Form nodes. To carry the binary forward to the node that actually consumes it, insert a Code node:

return $input.all().map(item => ({
  json: item.json,
  binary: $('ExtractDoc: Upload File').item.binary
}));

This is the smallest Code node that does anything useful. Steal it.

3. When exporting as a template, blank out the `webhookId` fields

The Form Trigger and Form nodes each carry a webhookId UUID in the exported JSON. n8n preserves these on import — it doesn't regenerate them — so your IDs travel with the workflow into the importer's instance.

This is a known sharp edge in n8n's import path (issue #18683). The convention for shareable templates is to strip the IDs before publishing: set every webhookId value to "". The importing instance allocates fresh ones on first save.

If you forget, the most common failure mode is silent webhook hijacking — calls intended for your original workflow get routed to the imported one, or vice versa.

The reference workflow below ships with every webhookId blanked.

Why this matters

The default n8n template is a one-configuration workflow. You build it for your exact use case, save it, and it works for you. Anyone else who imports it either matches your configuration exactly or edits the JSON by hand.

Dynamic dropdowns change that. The same workflow now adapts to the user — their file, their choice of engine, their available output formats. The template becomes a small application instead of a personal artifact.

For LDX hub, this meant one workflow could expose five different AI services with all their valid configurations, without me hardcoding any of them. When the provider adds a sixth engine tomorrow, the workflow doesn't change. The Get Engines call returns one more entry, the dropdown shows one more option, the rest of the flow handles it.

That's the reason this technique is worth knowing. Not because dynamic dropdowns are interesting in themselves, but because they're the part of the n8n stack that lets you ship a configurable workflow.

The complete reference workflow

The full demo — five services, every dropdown driven by this pattern, plus dynamic credentials (a separate post) — ships with the n8n-nodes-ldxhub npm package as examples/all-services-demo.json. Import it into your n8n instance and inspect any of the five service paths to see the pattern in production form.

If you want to run it against LDX hub, get a free API key at gw.portal.ldxhub.io — 25,000 credits per month, no card. If you just want the pattern, the JSON is permissively licensed and the dropdown logic is portable to any API.

Closing

The dropdown pattern collapses what looks like a hard problem ("how do I make my n8n template work for arbitrary users") into something almost boring ("call an endpoint, render its response as options"). Most of the configurability complexity I've fought over the years dissolves the same way, once I find the right boundary to draw.

The interesting part wasn't writing the expressions. It was discovering that the toggle had been sitting there in the Form node settings the whole time. The technique looks novel only because it's underdocumented.

If you've been building n8n templates that only work for your exact setup, try moving one configuration knob to a dropdown driven by an API call. That's where this starts paying off.

The loop I didn't notice closing

Hideki Mori — Mon, 01 Jun 2026 13:00:00 +0000

The loop I didn't notice closing

Seven weeks ago I started using AI for work. Two weeks after that, I published an article. Seven weeks after that — today — the article is one of sixteen, and they are all in a memory file that the AI reads at the start of every new conversation.

I didn't notice the loop until I named it.

This is a note about that loop, what it is, what it isn't, and why I keep publishing even though the loop doesn't strictly need me to.

The shape

It runs like this:

I decide what to do.
I work it out with the AI — usually in dialogue, sometimes by pasting raw code or data.
The dialogue becomes a record. Sometimes a memory entry. Sometimes a published article.
The record becomes context for the next conversation, which informs the next decision.

It didn't look this clean while it was happening. The numbering is hindsight. From inside, the steps overlap.

The first step is the one I keep. Direction is mine: what to build, what to write, what to negotiate. The history that shapes those decisions — twenty-four years of solo work, my company, my family, my health — is also mine. The AI is not setting direction.

The second step is where most of the leverage is. I describe what I want to do as completely as I can, sometimes by handing over source code. Then I ask: does this look right? Is there a path I'm missing? Where would this break? I'm opening drawers — possibilities I half-saw in my own head — and checking which ones open cleanly. When one opens cleanly, that is the GO signal. Not "will this succeed" but "this is doable, so do it."

The third step happens almost without effort. The conversation already exists as text. Some of it becomes a memory entry I add deliberately. Some of it becomes raw material for an article. The article writes itself partly because I have already explained the thing to the AI.

The fourth step is the one that took longest to arrive — and the one I want to be most careful about describing.

Three phases, not one

The loop didn't close in a moment. It accumulated.

In April, when I started using AI, it was just an easier way to write about things I already knew. My first published article was about a technique I had been using for years — a way of decomposing one long LLM prompt into two stages. I had named the technique privately. Explaining it to the AI turned the explanation into a draft. The article went out two weeks after I started. The AI didn't discover the technique; I had it already. The AI made it writable.

That was phase one. AI as a transcriber for things I already had.

A few weeks later, something shifted. The dialogue started shaping execution. When I redesigned the pricing model, the rate formula went through several iterations in conversation. When I built the chunked upload API, the chunk size and the format were chosen in the back-and-forth, not before. The decisions were still mine — I confirmed each one — but the candidate options came from the dialogue. I would not have arrived at exactly the same designs alone. Probably similar. Not the same.

That was phase two. AI as a tactical partner inside the execution.

The third phase took longer. It needed a critical mass of accumulated record. By the time I had a dozen published articles, a dense memory file, and weeks of transcripts, something else became possible: the AI could read across all of that in a single conversation. Now when I plan the next thing — which article to publish next, how to structure the next negotiation, whether to take a particular technical bet — I am not feeding context from scratch. The context is loaded. The conversation starts from where the last conversation left off, three weeks ago, with full continuity.

That was phase three. The accumulated record informing the next execution.

What it feels like

Two things stand out.

First, there is no shame and no time pressure. Explaining your full background to a human advisor is sometimes painful. You skip parts because they are embarrassing, or because you don't want to bore them, or because the context would take an hour to set up. The failures, the half-finished detours, the impulses you talked yourself out of — those usually get edited out. With an AI you don't edit them out. You hand over everything you have, including raw code, and ask what it sees. The friction of being honest is much lower.

Second, the bottleneck is mine. The AI doesn't tire. I do. When the loop slows down, it slows down because I am tired, distracted, or running into something I haven't thought through yet. The tool itself is patient in a way humans cannot be. This shifts the constraint structure of the work. The limit is my own attention, not someone else's calendar.

I open every drawer I might be able to pull out. Some open cleanly. Some don't. The cleanly-opening ones are the GO signal. Success or failure is not the test — the test is whether the thing looks doable from where I am standing. If it does, I go.

Tactical, not strategic

I want to be careful with the word "strategy."

The high-level direction is mine. The decisions about what to build, what to refuse, what to invest in, where my career goes — those come from my own history, my values, the people in my life, my health. The AI is not setting that direction.

The execution is collaborative. The drawer-opening is collaborative. The articulation is collaborative.

So when I describe the loop, I would call it a tactical loop, not a strategic one. The intent stays mine. The execution finds its shape in the dialogue.

This distinction matters because the situation is easy to misread. "Solo developer outsources thinking to AI" is wrong. "Solo developer uses AI as a tactical tool with persistent memory" is closer. The agency is intact.

The recursive part

This article is in the loop.

Earlier today I asked the AI what I should write about next. It read the memory and the recent transcripts and proposed this — the loop itself. We discussed it. I gave my felt-sense answers about what it is like from the inside. The conversation by then already contained most of the structure. The AI wrote a draft from it. I refined. The article will be published, become part of the memory, and inform the next decision about what to write.

The article describes the phenomenon it is also an instance of.

This is not theater. It is just what the loop looks like when it is also reflective.

Why publish, then

If the loop runs on memory and transcripts, what is the public article for?

If I am honest, the loop doesn't strictly need it. The closed loop runs fine on internal artifacts.

But two things hold.

First, writing for an audience tightens the thought differently than writing for an AI. The forcing function is different. The fear of being misread, of saying something stupid in public, of being quoted out of context — these tighten the prose in ways talking to an AI does not. I get a cleaner record by publishing than by just transcribing.

Second, and this is the one that matters more: not everyone can run this loop. I see colleagues with the same tools, the same access, the same time, and the gap is large. Some people have the sense for it, and others don't. The tool does not close that gap. If it could, the tool would already be replacing judgment — and it isn't.

This is the structural point. AI cannot teach humans to use AI well. If it could, AI would not need humans at all. The persistence of the skill gap is the same fact as the persistence of human judgment. The two cannot be separated.

So I publish. Not because the loop needs it. Because the sense — the thing that lets the loop run at all — isn't transferable through the tool. It transfers, if at all, by watching someone else use it.

Closing

I started seven weeks ago without intending to start a loop. The first article was a side effect of being asked to explain something. Then the dialogue started shaping the execution. Then the record started shaping what came next. The loop closed without my noticing.

When I named it, I could see it.

The next conversation has already started.

Built with Claude (Opus).

Earlier in this series:

72% -> 75% -> 92%: a reproducible RAG validation

Hideki Mori — Fri, 29 May 2026 13:00:00 +0000

I expected converting docs into Q&A pairs to improve retrieval. It mostly didn't.

I built three knowledge bases from the same source document and ran the same twelve questions against each, three times.

Raw markdown chunks: 72%
Q&A facts from a generic prompt: 75%
Q&A facts from a retrieval-aware prompt: 92%

The 75% caught me. Naive Q&A conversion lifts retrieval by three points — a wash, not a gain. The +20 over raw markdown comes from something else entirely: five prompt design rules applied at fact-generation time. The rules cluster around two ideas — each fact should answer fully on its own, and the tokens retrieval needs (service names, parameter names, identifiers) should survive verbatim from source to indexed fact.

This is a reproducible record of where the gains actually came from.

The full repo, including every prompt, every fact, every chatbot response: github.com/HidekiMori/rag-accordion-demo (MIT).

1. The setup

The source document was the LDX hub developer portal — an 821-line Markdown file describing five API services (StructFlow, RefineLoop, RenderOCR, CastDoc, and ExtractDoc). It mixes prose, parameter tables, JSON examples, and a cross-cutting "Errors" section. Long enough to test retrieval at scale, structured enough to test what each piece of the pipeline does.

From this document, three knowledge bases were built:

mdx_direct — the raw Markdown, chunked on \n\n, 1024 tokens per chunk, 50 token overlap.
naive_facts — Q&A facts produced from the document by a generic "extract Q&A pairs from this" prompt. 78 facts, one per JSONL line.
best_facts — Q&A facts produced by a Stage 2 prompt with five explicit rules. 82 facts.

All three are loaded into Dify Cloud as Knowledge Bases. Same embedding model (OpenAI text-embedding-3-large, default dimensions). Same retrieval (Hybrid Search, Weighted Score, 0.7 semantic / 0.3 keyword, Top K=3). Same chatbot LLM (OpenAI GPT-5.5). Same system prompt. The only variable across runs is which KB is attached.

Dify Cloud's vector storage runs on TiDB Cloud Starter — see PingCAP's case study on Dify's consolidation. I didn't write SQL against TiDB directly; what matters here is that Hybrid Search on top of TiDB is what made the keyword side of Rule 5 (below) visible in the results.

The Q&A facts were produced by a two-stage StructFlow pipeline. StructFlow is a generalized "input + instructions → structured output" function, not a structuring tool per se — the instructions decide what the output looks like, so the same function produces 53 self-contained sections in Stage 1 and Q&A facts in Stage 2. Both stages run on Google Gemini 3.5 Flash at temperature 0. Stage 1 outputs a sections array; Stage 2 outputs a facts array. Each array gets flattened into line-per-record JSONL between stages — the shape §5 below calls the "accordion." The Stage 1 prompt is identical for both naive_facts and best_facts — only the Stage 2 prompt differs.

A note on naming. StructFlow appears in this article in two roles: as a service documented in the test corpus (the LDX hub developer portal), and as the engine that built the Q&A facts from that corpus. Same tool, two contexts. When "StructFlow" shows up in the example facts and test questions, it is the service-in-corpus role; when it shows up in pipeline descriptions and the accordion diagram, it is the engine role.

Twelve test questions, each designed to probe a specific retrieval challenge (direct lookup, default value buried in a parameter list, cross-service comparison, hidden entity inside a code block, casual phrasing, cross-cutting concern scoped to a service, end-to-end workflow synthesis). Three independent runs per KB. Twelve questions × three KBs × three runs = 108 graded chatbot responses, scored as ✅ / ⚠️ / ❌.

Grading was manual, against a per-question rubric prepared before runs: ✅ for a complete answer that included every expected piece of information, ⚠️ for a partial or incomplete answer (e.g., missing the partial-failure warning in Q12), and ❌ for a wrong answer or a "not in the documentation" refusal when the answer was actually present in the source. Aggregate percentages treat ✅ as correct and both ⚠️ and ❌ as not correct — no partial credit. Per-question patterns and verbatim chatbot responses for all 108 runs are committed under results/ in the repo, so anyone can re-grade if they disagree with the rubric.

2. The headline numbers

KB	Accuracy	vs `mdx_direct`
`mdx_direct`	72%	— (baseline)
`naive_facts`	75%	+3 pt (operationally negligible)
`best_facts`	92%	+20 pt (and +17 over naive)

The +3 pt for naive Q&A conversion is a wash, not a gain. Looking question by question reveals why:

Question	`mdx_direct`	`naive_facts`	Instance change
Q2 (`max_revisions` default)	0/3	3/3	+3 (genuine fix)
Q6 (is LDX hub free?)	0/3	3/3	+3 (genuine fix)
Q4 (RenderOCR vs CastDoc)	3/3	0/3	−3 (genuine break)
Q12 (`completed` for StructFlow)	3/3	0/3	−3 (genuine break)
Q10 (scanned-contracts workflow)	2/3	3/3	+1 (mdx run variance)

The four genuine fix/break pairs cancel exactly at the instance level: +3 +3 −3 −3 = 0. The entire +3 pt difference (naive 27/36 vs mdx 26/36) comes from Q10 alone — where mdx_direct dropped to ⚠️ on a single run (the ExtractDoc→StructFlow chain collapsed that run) while naive_facts stayed ✅ across all three. That +1 instance is run variance on the raw-markdown side, not a retrieval gain from Q&A conversion. Naive Q&A shuffles where the failures land (Q2/Q6 fixed, Q4/Q12 broken) and nets out at zero; the headline +3 pt is noise on Q10.

best_facts keeps all four ✅, and brings six more questions from partial to full credit. Same Stage 1 sections, same LLM, same retrieval — only the Stage 2 prompt differs between naive_facts and best_facts.

The structural gap reproduced across all three runs of all three KBs without exception. It is not statistical noise.

That said, this is not a benchmark paper. It is a controlled engineering validation on one realistic corpus, with a deliberately small question battery designed to probe specific retrieval failure modes. Whether the five rules below hold under different document structures, retrieval backends, or chunking strategies is something the repo is designed to let you check on your own corpus — not something this single validation establishes.

3. The five rules

These are the rules encoded in the retrieval-aware Stage 2 prompt. Each one prevents a specific failure mode.

Rule 1 — Self-contained answers

Every fact must answer its question fully on its own, without needing to read other facts.

Hybrid Search returns the top three chunks. If the answer is split across multiple facts — "this is the default" + "the parameter is max_revisions" + "RefineLoop accepts these settings" — retrieval has to pull all three. That does not always happen. When each fact carries the full picture in one place, retrieval only needs to land on that one entry.

In practice: the best_facts entry for "what are the possible job statuses?" names every value (queued, processing, completed, failed) and what each means, in one fact.

Rule 2 — Developer-friendly question phrasing

Use the way real users would actually type the query: "How do I...?", "What is the default value of...?", "Which formats does X support?"

Embeddings reward semantic similarity. If the generated question is phrased like a real user query, the embedding lands close in vector space. Declarative summaries ("Describe the configuration of...") sit further away from how queries actually arrive at retrieval time.

This rule is easy to underestimate. It costs nothing at generation time and pays back on every retrieval.

A note on evidence: this validation does not isolate Rule 2's effect. Both naive_facts and best_facts already produce question-shaped questions, so the rule is not directly tested here. It is included because the failure mode (declarative summaries instead of questions) is common in prompts that ask for "summaries" or "key points," and once it occurs, retrieval degrades.

Rule 3 — Exact preservation of technical identifiers

API names, endpoint paths, parameter names, enum values, code snippets — keep them verbatim. Do not rephrase.

Hybrid Search has two channels: vector and keyword. The keyword channel is a literal-token match. max_revisions, results[].status, ki/ocr — these strings only contribute if they appear verbatim. Paraphrased ("the revision count parameter") loses the keyword channel entirely.

The vector channel can sometimes carry it. Often it cannot. The combined score drops below the cutoff and the fact never enters the top three.

Rule 4 — Service-specific facts for cross-cutting information

When a section describes errors, statuses, or behaviors tied to a specific service, generate a fact with the service name in both the question and the answer.

Source documents often place cross-cutting information in standalone sections away from the services they affect. Our source has a ## Errors section that includes:

StructFlow jobs may also have status: completed with some individual records marked failed — always check summary.failed_count and each results[].status to detect partial failures.

The section header is just "Errors", not "StructFlow errors". A generic Q&A extractor handles this content inconsistently. In our naive_facts run, the partial-failure content was skipped entirely — grep "partial failure" data/facts_naive.txt returns nothing.

The retrieval pipeline then cannot answer What does completed mean for a StructFlow job? because no indexed fact connects completed, StructFlow, and partial failures in the same line. This is Q12, and naive achieves zero correct answers across three runs.

The retrieval-aware version forces Stage 2 to generate a service-scoped variant: a fact whose question is How should I check for partial failures in a completed StructFlow job?, with answer text that mentions StructFlow explicitly and keeps every identifier verbatim. Q12 then resolves cleanly on every run.

Rule 5 — Deliberate keyword design

Each fact carries a keywords field with three to seven short tokens — service names, parameters, concepts. These tokens are direct ammunition for the keyword channel of Hybrid Search.

Without explicit guidance, the LLM tends toward loose, descriptive keywords. A naive fact about the wait parameter ends up with ["wait", "connection", "timeout", "polling"] — only wait is LDX hub-specific. The retrieval-aware version keeps ["StructFlow", "curl", "job_id", "wait"] — service name and resource type included.

This is the rule that resolves Q4 in the validation. The naive prompt did generate a RenderOCR primary-role fact, but its keywords were ["OCR", "scanned PDF", "Office files", "layout preservation", "languages"] — no literal RenderOCR token. When the user types "RenderOCR vs CastDoc?", the keyword channel contributes far less than it could, and the vector channel alone does not consistently lift this fact into the top three. The retrieval-aware version puts RenderOCR in every RenderOCR-scoped fact's keywords. The fact gets retrieved.

The full prompts, with examples and failure modes for each rule, are in prompt_engineering.md.

4. The one question that defeats every prompt

Q8 ("what engines are available for OCR?") is the only question that fails in all three KBs. The source mentions "KI OCR" once in a prose bullet at line 157 — Powered by KI OCR, a battle-tested enterprise OCR engine — and the engine ID ki/ocr lives inside the JSON response example of GET /renderocr/engines at line 644. Neither location surfaces consistently for the query. The prose bullet gets dominated by other RenderOCR feature points. The JSON code block is hard for both raw chunking and Q&A generation to digest.

Neither prompt rescued this. Q&A conversion in this run did not promote either form into a retrievable "OCR engine name" fact. The Stage 2 best prompt knows about identifier preservation (Rule 3), but it was not given explicit guidance to lift entities out of code blocks into prose-style facts.

This is the honest limit of what the five rules can do here. Missing data is missing data. Future work: explicit code-block entity extraction, or a follow-up Stage 2 pass that lifts JSON-embedded identifiers into their own facts.

5. The accordion pattern

The mechanism that produces one JSONL line per Q&A fact, from any structured source:

document
  → Stage 1 (segmenter): { sections: [...] }
  → flatten: one section per JSONL line
  → Stage 2 (extractor): { facts: [...] } per section
  → flatten: one fact per JSONL line

The same shape implemented in Dify. Stage 1 (S1) and Stage 2 (S2) StructFlow nodes both run on Google Gemini 3.5 Flash. Flatten nodes turn each stage's array output into line-per-record JSONL. Save nodes write the intermediate files for inspection. Full YAML: workflows/dify-accordion.yml in the repo.

The name "accordion" comes from the shape: 1 doc → N sections → M facts, with a flatten step after each stage to expand array outputs into line-per-record JSONL. Both stages are StructFlow jobs — the LDX hub structured-extraction service. The same mechanism works on HTML, PDF chapters, DOCX heading styles, or any text where a section boundary can be defined.

What this demo shows is that the mechanism itself is neutral. Paired with a generic Stage 2 prompt, the accordion produces facts that perform at roughly raw-chunking parity. The +17 pt over naive Q&A is not in the mechanism. It is in the Stage 2 prompt.

This is the part that surprised me most. I had expected the mechanism — the two-stage segment-then-extract design — to carry more of the weight. It does not. The Stage 2 prompt is the lever.

6. What this means in practice

A lot of RAG advice focuses on the retrieval side: which embedding model, which similarity function, which chunk size, how big the top-K window. Those decisions matter, but they all act on whatever facts have already been indexed. The shape of those facts — what each fact says, what tokens it carries, how it phrases its question — is decided at fact-generation time. And that decision is what this validation isolates.

Three takeaways for anyone building Q&A-style RAG over their own documents:

Do not assume Q&A conversion is automatically better than raw chunks. It can be, but only when the conversion preserves what retrieval actually needs. A generic "extract Q&A pairs" prompt is roughly neutral against well-chunked Markdown.
Keyword design carries the keyword-channel lift. Rule 5 above — putting service names in the keyword field is what fixed Q4. Identifier preservation (Rule 3) targets the same channel and becomes load-bearing when the model paraphrases, but at temperature 0 here the model preserved identifiers on its own, so it was insurance rather than the driver. Both cost nothing at generation time and are easy to encode in the Stage 2 prompt.
Cross-cutting sections need to be re-anchored to their services. Rule 4. A ## Errors section that mentions four products should produce four product-scoped facts, not one neutral fact. Otherwise no indexed fact connects the service name and the error in the same line, and retrieval cannot bridge the gap at query time.

These rules generalize to any domain with characteristic identifiers — drug names, statute codes, model numbers, CSS class libraries. The retrieval system already knows how to match strings. The prompt's job at generation time is to make sure the strings stay strings.

7. Open question — the prompt is the next bottleneck

The five rules describe what works. They do not describe how to apply them.

The Stage 2 best prompt used in this validation is heavily LDX hub-specific. It encodes that max_revisions is a parameter, that RenderOCR and CastDoc are distinct services, that ## Errors is a cross-cutting section, that StructFlow jobs can have completed status with failed records inside. Every one of those decisions was hand-written into the prompt by someone who already understood the domain.

This is fine for one domain. It does not generalize.

A team adopting this pattern for, say, drug labeling, statute codes, or CSS class libraries would have to write their own version of the Stage 2 prompt — with their own service names, their own cross-cutting concerns, their own identifiers to preserve. The know-how is not in the LLM. It is in the prompt author's head.

There are two paths out, and I have not yet validated which is correct.

The first is to codify the methodology. Build a checklist (the repo already has one in prompt_engineering.md) and trust that engineers using the pattern will fill it in for their domain. This is the documentation-as-scaffolding approach. It works for teams with strong technical writers. It does less for teams without them.

The second is to make the prompt itself a StructFlow output. Add a Stage 0 that ingests the source document and produces the Stage 2 prompt for that domain. The accordion stretches one more time: 1 doc → 1 prompt (Stage 0) → N sections (Stage 1) → M facts (Stage 2). The Stage 0 prompt would be the only domain-agnostic prompt in the system, and it would be reusable across any source.

I lean toward Stage 0 being a real possibility, but this part is still hypothetical — I have not validated it. The accordion mechanism already proves that one StructFlow step can turn one input into many structured outputs. There is no reason in principle why "the prompt for the next StructFlow step" cannot be one of those outputs. Whether that actually beats a hand-written domain prompt for retrieval quality is an open question. That is the next thing I want to test.

8. Closing

The repo is MIT-licensed and the Dify Workflow YAML is included — import it into Dify, drop in your own document, and rerun. The shortest possible summary of this whole note is to diff the two Stage 2 prompts: stage2_best.md vs stage2_naive.md. Everything else in this article unpacks what that diff is doing.

TiDB Cloud, accessed through Dify's Knowledge feature, is the silent infrastructure that made this validation reproducible without standing up a vector database from scratch.

Built with Claude (Opus).

Live report: at this speed, you don't theorize. You eliminate.

Hideki Mori — Mon, 25 May 2026 13:00:00 +0000

From April 15 to April 30. Two weeks. Solo. Plus an AI.

This is a live report — not a retrospective.

The earlier posts in this series have been about shape: the contract, the pattern, the twenty-four years of building alone. This one is about what happens when an AI is added to that shape and the speed changes.

I'm writing this while still inside it. The shape held. The speed didn't.

What I built

Two weeks. One person. Here's the list, flat and unromantic:

API gateway: Zuplo, five services unified behind one public API
Custom domains: gw.ldxhub.io for production, gw.portal.ldxhub.io for the developer portal
DevPortal: API reference auto-generated, plus introduction, per-service guides, pricing, MCP setup, "use anywhere"
Auth: Auth0 with Google OAuth, GitHub OAuth, and email — sign-up to first call in roughly thirty seconds
Free tier: 25,000 credits per month, no credit card required
Billing: a unified credit system, $0.0001 per credit, four plans (Free, Starter $15, Standard $50, Pro $150), overage billed at the same rate
Stripe integration: connected for paid plan subscriptions and overage settlement
Async metering pipeline: a separate path that fires credit consumption after a job finishes (minutes or longer after 202 Accepted), batched to the gateway's billing API
Five services published: StructFlow, RefineLoop, RenderOCR, CastDoc, ExtractDoc
Four channels integrated: n8n nodes (verified, on the marketplace), Dify plugin (verified, multilingual), MCP for AI assistants like Claude Desktop, plus the direct API
Marketing pages: four product pages on the corporate LP (ldxlab.io/ldxhub, plus per-service pages), each linking to the developer portal
Eight scheduled articles on dev.to and Zenn, covering the design philosophy. This one is article five.

That's the surface. None of it is glamorous. All of it had to be done.

The shape held

The shape from the earlier posts didn't change.

Jobs go in. The API holds the work. The API retries through whatever needs retrying. The API reports when there's a real result to report. The same shape that ran the music platform in 2003 and the e-book platform from 2006 to 2014, now wrapping a structured-extraction service and four others.

What changed was the speed within it.

What sped up, what didn't

Sped up:

API design iteration — every interface lived through three or four shapes before settling
Documentation in four languages (English, Japanese, Chinese, Portuguese) — because the marketplace plugins required it and there was no time to do it sequentially
Edge-case verbalization — "what happens if the input is missing both inputs and file_id?" — answered in writing the same hour the question came up
Marketplace plugin onboarding (n8n, Dify) — submission, review, version bumps
Migrations, fixtures, sanity checks

Didn't change:

Decisions about the billing structure
Where to draw the integration boundary (which engines, which surfaces)
What to ship and what to skip
What stays in the legacy internal API and what becomes a public service

The AI didn't replace judgment. It replaced most of the typing.

(Most. The wrappers that bridge the public services to the legacy internal API — those I typed myself. They were small, they were boring, and they were exactly the shape I had in my head from twenty-four years of doing this. Faster to type than to explain.)

What was hardest

The piece I almost gave up on was the credit metering for async jobs.

Most API gateways meter on the request — easy. What I needed was: meter when the job finishes, which is minutes (or much longer) after the request returns 202 Accepted.

I evaluated two gateways. Both could do async reporting on paper. The first one I took all the way through — the API was there, the docs were there, but the UI and the operational behavior had a rough edge to it that I couldn't quite ignore. I wanted the second one. So I went back.

The second gateway was where I wanted to land. But when I got there, I wasn't sure whether the async metering I needed was actually possible. The documentation was thin in exactly that spot. I tried their support bot. The bot didn't know either. Then a human came on, and the human walked me through how to fire metering events from the job-completion side, batched.

It worked.

That's the only piece of this build where I didn't know whether it was possible until someone on the other side confirmed it was. I want to remember that. Twenty-four years of building alone, and the fastest path through this particular unknown wasn't to figure it out alone — it was to ask, and to have the other side answer well.

(There was also a smaller stumble with Stripe. I assumed I'd need to configure something on the Stripe side to make the gateway's billing flow work. I spent a while looking for that something. There wasn't one. The gateway handled it end-to-end and Stripe just received what it received. Sometimes the answer to "how do I integrate this?" is "you already did.")

The decision volume

This is the part that surprised me.

The output increased dramatically. I expected that. What I didn't expect: the volume of decisions and branches multiplied with it. Each one had to be closed individually — there's no way around that. Closing each decision was also faster, so none of them became the bottleneck.

But the fatigue is different.

I've been building things alone for twenty-four years. I know what tired feels like. This is something else. The speed compresses everything — the questions, the answers, the small course corrections — into a window where nothing has time to settle. You're never in a phase. You're always at the seam.

Here's something I didn't expect from twenty-four years of solo work: I never used to worry about missing a branch. The volume of decisions was always within what one head could hold. Now, with the output multiplied, there's a new fear — that something I should have considered slipped past while I was closing five other things. The path keeps moving and the adjacent branches keep widening, and I want to see them all before I commit. That fear didn't exist before. It's not crippling. It's just present.

I don't have a clean answer for what to do about that. I'm reporting it because it seems important and it isn't in the AI-tooling articles I've read.

Elimination, not theory

Here's the part I want to keep.

At this speed, you don't theorize. You eliminate.

There isn't time to model out every option, weigh tradeoffs in a meeting, or build a deck. You hold a thing in your hand for an hour, see whether it survives contact with the next thing, and either keep it or remove it. What survives is what you didn't have time to argue with.

This isn't a productivity claim. It's a description.

When the friction is low enough — when typing is no longer the rate-limiter — the way decisions get made shifts. They get made by elimination, by what a thing fails to support, rather than by what a thing might enable. The negative case becomes louder than the positive case. You stop asking "what if we add this?" and start asking "what does this make it harder to remove later?"

I don't know if this generalizes. I know it's what happened in these two weeks.

Closing

Speed isn't a virtue. It's just what happens when the friction is gone. After twenty-four years, the friction I had left was mostly typing.

I removed the typing. The shape was already there.

The fatigue is part of the report.

Built with Claude (Opus).

Resources

LDX hub developer portal: gw.portal.ldxhub.io
n8n integration: n8n-nodes-ldxhub on npm
Dify plugin: LDX hub on Dify Marketplace
MCP endpoint: gw.ldxhub.io/mcp

Earlier in this series:

Dynamic isn't enough. Operations is the other half.

Hideki Mori — Mon, 18 May 2026 13:00:00 +0000

If you've been writing software for a few years, you eventually start designing for change.

A content delivery system: a new format will appear, a new player will appear, so let's not hard-code formats. An e-commerce site: a new payment method will be requested, so let's not hard-code billing logic. A document pipeline: a new processing engine will appear, so let's not hard-code engines.

This is good instinct. You read books like Analysis Patterns, you internalize the idea that the model should be flexible, you push concrete decisions out of code and into data. New format? Insert a row. New payment? Insert a row. New engine? Insert a row.

It's a beautiful pattern. It really does extend the lifespan of a system.

But after enough years of running systems built this way, here's the part that doesn't show up in the books:

Dynamic isn't enough. Operations is the other half — and that other half is usually paid by someone else.

The "wait, maybe..." moment

Reality always overshoots the model. Always.

You build a content system that handles formats dynamically, and then someone asks for a format whose delivery rules don't fit the existing shape. You build a payment system that takes new methods through configuration, and then someone wants a billing model where the "amount" isn't even computed at the same time as the "charge." You build an engine registry, and then a new engine doesn't conform to the lifecycle every other engine has assumed.

Each time, the moment looks the same. You stare at the request, and your first thought is: that's outside the model. And your second thought, a few minutes later, is:

Wait, maybe... if I interpret this field a little differently, and if I add one column here, and if I let this branch handle a special case... it fits. Sort of. It doesn't break anything.

So you do that.

The system keeps running. No downtime. No coordinated migration. No painful conversation with the integration partners. From your seat as the designer, the dynamic structure absorbed the change. The pattern worked.

That's the part designers love. That's the part that gets written about.

Who pays for the other half

Here's what's harder to see from the designer's seat: every "wait, maybe..." moment leaves a small residue somewhere in operations.

The operator now has to remember that for this particular case, the workflow is slightly different. The reporting tool needs a footnote. The on-call playbook gets one more conditional. The next person to onboard needs another paragraph of context. The integrating system has to send a value in a slightly unexpected place.

None of these are catastrophic. None of them break the system. Each one, on its own, is a small inconvenience.

But the small inconveniences accumulate. And the people accumulating them are usually not the people who designed the dynamic structure in the first place.

The designer experiences each "wait, maybe..." as a successful absorption. The operator experiences it as one more thing to remember. Same event, two ledgers. Only the designer's ledger gets reviewed.

This is where engineering satisfaction can become quietly self-serving. The system didn't break. The pattern held. Therefore the design is good — except good for whom, exactly?

The road I didn't take

There is, of course, an alternative. You can stop the system, ship version 2, ask everyone to migrate, and start clean.

A scheduled 24-hour outage. A v2 documentation packet sent to every integrating partner. A coordination meeting. A cutover.

I have, for twenty-four years, mostly chosen not to do this. I have preferred quiet absorption — interpretation, gentle reshaping, "wait, maybe..." — over coordinated rebuild.

Whether that was right, I genuinely don't know. There were probably operators along the way who would have been happier with one painful migration than with years of small accommodations. There were probably integrating systems whose engineers would have preferred a clean v2 over a v1 that kept quietly bending.

I chose to keep the shape. That choice has a cost. The cost is paid in operator-hours, in playbook footnotes, in the small extra cognitive load of working with a system that has absorbed more than its original model anticipated.

I'm not saying I was wrong. I'm saying: I'm still not sure.

Generic underneath, specific on top

There's one thing I do believe, with more confidence than the rest:

When the model gets very generic, the UI usually has to get very specific to compensate.

A maximally flexible data model — the kind that absorbs new formats, new payment methods, new engines — is, almost by definition, awkward to interact with directly. Direct interaction with a generic model means the human has to supply all the missing context. That's exhausting.

The good outcome is: the model stays generic underneath, and the UI is built specific to each use case on top. Each surface is custom. Each surface is rigid in exactly the way that makes it usable.

This is a hard decision to make, because every specific UI you build feels temporary. You're going to throw it away when the use case shifts. You build it knowing it has a shorter half-life than the model below it. That asymmetry is uncomfortable.

But I think the discomfort is the right discomfort. A long-lived generic core, with shorter-lived specific surfaces on top, is — based on what I've watched survive over twenty-four years — closer to how systems actually age well.

Three observations

None of these are conclusions:

Dynamic structure extends a system's life. It does not, on its own, make operating that system pleasant.
Every "wait, maybe..." absorption is a real engineering achievement and a small tax on someone downstream. Both of these are true.
If you make the model very generic, give the operators a UI that isn't generic.

Dynamic isn't enough. Operations is the other half — and that other half is usually paid by someone else.

After twenty-four years of this, that's the only thing I feel sure of.

Earlier in this series:

The Accordion Pattern: Why I stopped writing one fat LLM prompt
Nobody knows when a job will finish. I'd still like to report it accurately.
What survives when you build alone for 24 years.

What survives when you build alone for 24 years

Hideki Mori — Mon, 11 May 2026 13:00:00 +0000

In December 2002 I joined a company that was nearly bankrupt.

I was twenty-five. A software engineer with no title beyond that. The company was burning cash, and the next product launch had to happen in January — about a month after I started.

By January 2003, the demo shipped.

It was an HTML browser for mobile phones — with packet reduction built in. It rendered HTML, played back MIDI files linked from <a> tags, ran simple background animations. I wrote it in Java.

The server was PHP. There was no engineer for the server-side compression part, so I did that too — also in Java, called from PHP over localhost.

A strange shape, but the deadline was real.

I remember being a little angry. I also remember being much happier than angry. I had shipped something, in a month.

That's how I started designing things alone. Not as a plan. Just: someone had to, and nobody else moved.

The first thing that wasn't a plan

Looking back twenty-three years later, here's something I didn't notice at the time:

I had no responsibility. I was a junior engineer with a deadline. If the demo had failed, the company would have collapsed faster — but the failure wouldn't have been mine to own.

That made it easy to ship.

After that first month, I never really had another hard deadline. Not on the projects I owned. I'd just answer "as soon as I can" and try to deliver on that.

By 2005 I was a CTO. The work didn't get lighter; the responsibility arrived. And here's the thing I didn't expect: the shape of my code didn't change.

I kept writing things by myself. I kept ending up alone on the deep parts. The org chart rearranged around me, but at the bottom — when something had to actually work — I was still just there, writing.

Productivity and accountability don't mix easily. The shape I learned at twenty-five — write it because nobody else will — was the shape that kept showing up after the title changed.

A shape I didn't notice was a shape (2003–2005)

After the packet-reduction tool, I built a music distribution platform for mobile phones. This was the carrier-billed era. Feature phones. Dropped connections in elevators and tunnels.

The carrier had a billing rule that taught me something I wouldn't articulate for twenty years.

The rule was: don't bill until the user's device confirms the download finished.

So the flow looked like this:

User taps "buy"
The phone starts downloading (with HTTP Range, because connections drop)
The server delivers in chunks
When the device finishes, it sends a hidden completion signal
Then the server bills

If the train went into a tunnel, the user wasn't charged. The server kept holding the work for as long as the device was alive. Eventually either the download completed and we billed, or it didn't and we ate the cost.

Twenty years later I'd phrase the rule like this:

Nobody knows when a job will finish. But when it does, I want to report it accurately. So I make users wait — and in exchange, I retry as hard as I can until I have a result.

That was already the rule for the music platform in 2003. I just didn't know it was a rule yet.

The shape kept showing up (2006–2018)

In 2006 I started on an e-book distribution platform. Four months from blank page to production. I maintained it for eight years, alone.

The shape stayed the same:

Direct delivery, no intermediary platforms in the way
Real-time settlement back to publishers
HTTP Range support, because phones still dropped connections
Tar archives generated per request — buttons, branding, rights metadata stitched in at the moment of delivery, not pre-built and stored

I stepped off in 2014. The platform kept running.

By 2018, with someone else maintaining it, it was distributing tens of billions of yen in transactions per year.

It ran for seven years after I left, and then, in 2021, it was discontinued.

Eight years on my watch. Seven more without me. Then it stopped.

Not a tragedy. Not a triumph. Just what happened.

Same shape, bigger numbers (2021–)

In 2021 I started on what's now called LDX hub — a document processing platform for translation, OCR, structured extraction, and AI-assisted refinement.

Different domain. Same shape.

Year   Documents processed   Characters processed
2021   2,200                 ~20M
2022   48,500                ~810M
2023   85,000                ~2.7B
2024   163,000               ~3.5B
2025   351,000               ~8.3B

The numbers grew. I didn't change anything fundamental about how the system is shaped. Jobs are submitted, the API holds the work, the API retries through whatever needs retrying, the API reports when there's a real result to report.

Same as the carrier-billed music platform. Same as the e-book distribution. Same as that strange first month with two languages talking through localhost.

I didn't decide to keep this shape. I just never found a better one.

Productivity and accountability, again

Looking back at twenty-four years, here's the only honest thing I can say:

The shape I learned with no responsibility — the shape that lets one person ship something in a month — turned out to be the shape that kept showing up after responsibility arrived.

Not because I planned for that. Because it was the only shape I knew how to write, and I kept writing it, and nothing about the larger numbers ever required me to write differently.

Whether that means anything for your system, I genuinely don't know.

I haven't found a different shape worth writing.

For twenty-four years, I keep ending up here.

Earlier in this series:

The Accordion Pattern: Why I stopped writing one fat LLM prompt
Nobody knows when a job will finish. I'd still like to report it accurately. (scheduled for May 4)

Nobody knows when a job will finish. I'd still like to report it accurately.

Hideki Mori — Mon, 04 May 2026 13:00:00 +0000

Most async APIs commit to one thing: starting your job. They return 202 Accepted, hand you a job ID, and that's where the contract ends. The rest is your problem.

I do something different. I make one promise:

When your job is done, I'll tell you accurately. Until then, I'll keep retrying.

That's the entire contract for everything I've ever shipped. It sounds small. In practice, it's the only thing I actually do.

The shape every job in my system shares

You hand me work.

You wait.

I retry as hard as I can.

I report when it's done.

That's it. Whether the job is OCR on a scanned PDF, structured extraction from a long document, or refining the translation of an XLIFF file — the shape is identical. You give me an input. You don't watch the screen. I come back when I have something honest to report.

This sounds obvious until you try to actually deliver it.

Why "started" is easier than "finished"

Returning 202 Accepted is easy. The hard part starts right after that.

Real jobs hit things like:

Vendor APIs that occasionally throw 503. No reason. Just sometimes.
Native binaries that core dump. Twice in a row, then fine for a week.
Subprocesses that go zombie. Not crashed. Not finished. Just defunct. The OS still holds them.
Disks that fill up with stale debug files because something somewhere wrote them and forgot.

If you ship "started, here's a job ID, good luck" and call that an API, you're outsourcing all of the above to your user.

I'm not willing to do that. So I take the work back inside.

What that looks like in code

I'm not going to name any vendor. They don't matter. What matters is the shape. The code below is a simplified sketch — the production version handles a lot more (PDF library version quirks, fallback engines when the first one rejects the input, demo-mode page limits, and a long list of vendor-specific error codes that mean "retry," "skip," or "stop"). The shape is what survives.

Here's a sketch of the inside of one of my conversion services:

public JobResult runJob(Input input) throws Exception {
    for (int attempt = 0; attempt < MAX_RETRIES; attempt++) {
        Process child = new ProcessBuilder(
                "java", "-cp", classpath, EngineMain.class.getName())
            .redirectErrorStream(true)
            .start();
        passInputToStdin(child, input);

        long started = System.currentTimeMillis();
        while (child.isAlive()) {
            if (System.currentTimeMillis() - started > MAX_RUNTIME_MS) {
                child.destroyForcibly();
            }
            if (isDefunct(child)) {
                reap(child);
                break;
            }
            sweepStaleCoreFiles(workDir, MAX_CORE_AGE_MS);
            Thread.sleep(POLL_INTERVAL_MS);
        }

        ChildOutcome outcome = readOutcome(child);
        if (outcome.isTransientError()) continue; // retry
        if (outcome.isIrrelevantError()) {
            log.info("irrelevant error, treating as success: {}", outcome);
            return outcome.toSuccessResult();
        }
        if (outcome.hasResult()) return outcome.toResult();
    }
    return JobResult.failedAfterRetries(MAX_RETRIES);
}

A few things in there are worth pointing at.

new ProcessBuilder("java", ..., EngineMain.class.getName()). Not "call a library function." Not "use the SDK." I literally re-enter main from another process. The reason is that the underlying engine, in its native form, is unreliable enough that I want process-level isolation. If it dies, only the child dies.

if (isDefunct(child)) { reap(child); break; }. Native binaries don't always exit cleanly. Sometimes they're not crashed and not running — they're stuck. The parent has to notice, decide, and clean up.

sweepStaleCoreFiles(workDir, MAX_CORE_AGE_MS). When a child crashes hard, the OS dumps a core file. That file is huge. If you don't sweep it, the disk fills up. There is no clever solution here. You sweep.

outcome.isTransientError() → continue. Some vendor errors come and go. The fix is to wait and try again. If you don't try again, your user sees failed. If you do try again, your user sees "took a bit longer." I pick the second one.

outcome.isIrrelevantError() → log and return success. This is the part that surprises people. Some errors aren't actually errors for the use case. They're noise the engine emits. Knowing which is which takes years, and is most of the actual product.

None of this is elegant. None of it shows up in an architecture diagram. It all lives in the gap between "the job was submitted" and "the job is done, here's the result."

That gap is what I do.

What I gave up

I don't promise low latency. I can't. The thing I'm waiting on isn't predictable.

I don't promise the job will always succeed. Sometimes the input is genuinely broken. Then I report that, accurately, instead of pretending.

I don't promise streaming partial results. I keep the user out of the loop until I have something stable to hand back. The cost is they wait. The benefit is they don't see noise.

These trade-offs aren't sophisticated. They're just consistent.

I didn't design this. It survived.

Looking back, this is how every job-shaped API I've ever built has worked. I didn't sit down one day and decide on a contract. I kept ending up here.

Each time I tried to ship something where the API said started and stopped caring, the user came back asking what happened. So I started caring. Each time I tried to surface every transient error to the user, the user got scared. So I started absorbing them. Each time I tried to make jobs faster by skipping the cleanup, the disks filled up. So I started sweeping.

After enough years of this, what's left is a single rule:

When the job is done, I'll tell you accurately. Until then, I'll keep retrying.

Whether that's the right contract for your system, I genuinely don't know. It's just the only one I've found that survives.

Earlier in this series: The Accordion Pattern: Why I stopped writing one fat LLM prompt

The Accordion Pattern: Why I stopped writing one fat LLM prompt

Hideki Mori — Wed, 29 Apr 2026 07:51:20 +0000

Most structured-extraction tutorials look the same. Take a document, write one big prompt that says "extract A, B, C, D, E, F", get JSON back. Done.

This works on short inputs.

It quietly breaks on long ones.

After running this in production for a while, I stopped doing it. Here's what I switched to and why.

The fat prompt problem

Say you have a 50-page report and you want a structured summary out of it. The natural first move is something like:

Extract:
- title
- sections (with headings)
- purpose
- mentioned services
- acceptance criteria
- ...
Return JSON in this shape: { ... }

You hand the whole document to the model. It returns JSON. It looks fine on the first try.

Then you scale it up and three things happen:

Quality drifts. The model "forgets" mid-document. Later sections are summarized worse than earlier ones, or fields go missing.
One bad field poisons the whole call. If "acceptance criteria" hallucinates, you don't just lose that field — the whole record gets quarantined for review.
Latency goes up, parallelism goes down. A single 30k-token call takes what it takes. You can't shard it.

You can fight this with longer prompts, more examples, stricter formatting rules. I did. It buys you maybe 10% more reliability and costs you a lot of prompt-engineering time.

The structural problem doesn't go away.

What I do now: split it

The pattern I use looks like an accordion that expands:

[ document ]
     │
     ▼
[ Stage 1: segment ]   ← one prompt, one job: produce a list
     │
     ▼
[ array of segments ]
     │
     ▼ (fan out)
[ Stage 2: extract ]   ← one prompt, runs per segment
     │
     ▼
[ structured records ]

Stage 1 reads the whole document and returns a clean array of segments — sections, paragraphs, line items, whatever the right unit is for the task.

Stage 2 takes one segment at a time and extracts the structured fields you actually want.

Two prompts, each doing one thing.

Why this works better

Each prompt has a single job.
Stage 1 is "find the boundaries". Stage 2 is "extract the schema". Neither prompt has to hold both ideas at once. You can write each one tightly. Examples are shorter and more on-point.

Errors localize.
If Stage 2 fails on segment 7, you re-run segment 7. You don't redo the whole document. Bad fields get isolated to one record instead of contaminating the whole batch.

Stage 2 parallelizes naturally.
The output of Stage 1 is an array. Fan it out. Run 50 small extractions in parallel instead of one big one. Total wall-clock time drops, and so does the variance.

Cache hits go up.
If the same segment shows up twice (templates, standard headers, repeated forms), Stage 2 sees the same input and you can cache. The fat-prompt version sees the entire document as one unique input every time.

Long documents stop being scary.
The hard limit on a fat prompt is the model's context window. The accordion pattern doesn't have that ceiling. Stage 1 still has to read the whole document, but its output is small. Stage 2 only ever sees one segment.

What it costs

It's not free.

You're making more LLM calls — one for Stage 1 plus N for Stage 2 instead of one. On short inputs that's wasteful. The accordion pattern is for documents long enough that fat prompts start failing, not for two-paragraph emails.

You also need to think a little harder about what a "segment" is for your task. Sometimes it's a section heading. Sometimes it's a row in a table. Sometimes it's a logical unit that doesn't map to any visible boundary. That's a design decision and it matters.

When to use it

Reach for the accordion when:

The document is long enough that you've seen the model lose the thread mid-way.
The output schema has more than ~5 fields and they don't all care about the same context.
You need to retry failed records without redoing successful ones.
You want parallelism.

Stick with one fat prompt when:

The input is short and the schema is small.
The fields are tightly coupled (extracting one needs context from another).
You're prototyping and don't care yet.

A small concrete example

I run this on a service called StructFlow. The shape of the calls is roughly:

# Stage 1: segment
curl -X POST https://gw.ldxhub.io/structflow/jobs \
  -H "Authorization: Bearer $KEY" \
  -d '{
    "model": "google/gemini-3-flash-preview",
    "system_prompt": "Split this document into logical sections. Return one JSON record per section.",
    "example_output": { "section_title": "...", "section_text": "..." },
    "inputs": [{ "id": "doc1", "data": { "text": "..." } }]
  }'

The response gives you back an array. Then Stage 2:

# Stage 2: extract (one call per segment, run in parallel)
curl -X POST https://gw.ldxhub.io/structflow/jobs \
  -H "Authorization: Bearer $KEY" \
  -d '{
    "model": "google/gemini-3-flash-preview",
    "system_prompt": "From this section, extract: purpose, mentioned services, acceptance criteria.",
    "example_output": { "purpose": "...", "mentioned_services": [], "acceptance_criteria": [] },
    "inputs": [{ "id": "sec1", "data": { "section_text": "..." } }]
  }'

Two calls, each focused. One returns segments. The other turns each segment into structured fields.

That's the whole pattern.

Why I'm posting this

I built LDX hub partly to make this pattern easy to run — one API, async jobs, file-based input/output so Stage 1's output is directly usable as Stage 2's input. But the pattern itself doesn't depend on any specific tool. You can do it with raw OpenAI calls, Anthropic calls, anything that takes a prompt and returns text.

The takeaway isn't "use my API". It's: if your structured extraction is getting flaky on long inputs, the answer probably isn't a longer prompt. It's two prompts.

If you've tried something similar — or if you've got a case where this falls apart — I'd be curious to hear it.

DEV Community: Hideki Mori

Abstractions are fine. Starting on them isn't.

Pain used to teach you when something was wrong

Serverless took three pains away

Pain 1: cost spikes that show up too late

Pain 2: performance degradation that auto-scaling hides

Pain 3: where the work actually happens

A web design analogy

The application engineer is the right judge — they just need to be able to see

My setup, for what it's worth

Closing

Build a multi-step n8n Form with dynamic dropdowns (no plugin needed)

The one toggle that changes everything

The minimal example: dropdown from an API call

Going one level deeper: filter by user's choice

Even one more level: filter output by the uploaded file

Three things to watch out for

1. The short form $json does not reach across forms

2. Binary data needs an explicit pass-through

3. When exporting as a template, blank out the webhookId fields

Why this matters

The complete reference workflow

Closing

The loop I didn't notice closing

The loop I didn't notice closing

The shape

Three phases, not one

What it feels like

Tactical, not strategic

The recursive part

Why publish, then

Closing

72% -> 75% -> 92%: a reproducible RAG validation

1. The setup

2. The headline numbers

3. The five rules

Rule 1 — Self-contained answers

Rule 2 — Developer-friendly question phrasing

Rule 3 — Exact preservation of technical identifiers

Rule 4 — Service-specific facts for cross-cutting information

Rule 5 — Deliberate keyword design

4. The one question that defeats every prompt

5. The accordion pattern

6. What this means in practice

7. Open question — the prompt is the next bottleneck

8. Closing

Live report: at this speed, you don't theorize. You eliminate.

What I built

The shape held

What sped up, what didn't

What was hardest

The decision volume

Elimination, not theory

Closing

Resources

Dynamic isn't enough. Operations is the other half.

The "wait, maybe..." moment

Who pays for the other half

The road I didn't take

Generic underneath, specific on top

Three observations

What survives when you build alone for 24 years

The first thing that wasn't a plan

A shape I didn't notice was a shape (2003–2005)

The shape kept showing up (2006–2018)

Same shape, bigger numbers (2021–)

Productivity and accountability, again

Nobody knows when a job will finish. I'd still like to report it accurately.

The shape every job in my system shares

Why "started" is easier than "finished"

What that looks like in code

What I gave up

I didn't design this. It survived.

The Accordion Pattern: Why I stopped writing one fat LLM prompt

The fat prompt problem

What I do now: split it

Why this works better

What it costs

When to use it

A small concrete example

1. The short form `$json` does not reach across forms

3. When exporting as a template, blank out the `webhookId` fields