DEV Community: uehara

Bringing scattered sources into a single search — the unified knowledge workspace UseSources

uehara — Mon, 21 Sep 2026 17:52:14 +0000

Summary

What I built: a unified knowledge workspace called UseSources (usesources.com). It rests on two pillars. First, it unifies external sources (Notion, Slack, Confluence, Jira, Google Drive, and so on) together with internal wiki, tasks, and chat into a single RAG (a system that searches and then composes an answer), so an AI agent can return "where the correct answer lives." Second, it unifies the workspace itself around a single axis called project.
Why I built it: the fragments of an answer to a work question are scattered not only inside our own workspace but across external SaaS as well. The very structure of opening tools one at a time and searching each of them separately eats time and lets answers slip past you.
The point: UseSources has a two-layer design of "knowledge unification" and "workspace unification." Making search, permissions, and context come together in one place is the whole of this product's value.

Body (reading time ~13 min)

The pieces of an answer are scattered inside and outside the company

"Now, where was that written?" How much of your workday goes to that one question?

There are actually two kinds of places where information scatters. The first is your own workspace: documents in the wiki, tasks on the board, the reasoning in chat. The second is the one that gets overlooked — external sources. The original draft of a spec still sits in an old Notion. The reason behind a decision is buried in a Slack thread. Exchanges with a customer live in Confluence or Jira. Old material is asleep in Google Drive.

When we search, we open the tools one at a time and type the same words into each search box. Notion only looks inside Notion. Slack only looks inside Slack. You cannot know where the answer lives until you have finished searching everywhere. That structure is the root of the problem.

And this hunt happens many times a day. Even at five minutes each, six times a day is thirty minutes. Multiply that by every person on the team. Worse still, searching becomes such a chore that "it is faster to just ask." Verbal questions to whoever might know the answer pile up, that person's time gets eaten, the answer sinks back into the chat stream, and the next person searching cannot find it either. Left alone, fragmentation reproduces itself.

In the age of AI, this problem shows up in another form too. To hand work to an AI you have to hand it context, but when the context is spread across five tools, it is not in a shape you can give to the AI. Information a person cannot find is information the AI cannot use either. Unifying information is at once about human efficiency and about laying the groundwork to make AI actually do the work.

The first pillar — unifying internal and external sources into a single RAG

The first pillar of UseSources is inverting this structure. It unifies internal and external sources into a single RAG, so that one search lets an AI agent return "where the correct answer lives."

External connectors: they ingest documents from external SaaS and add them to what search covers. The connector has actually synced about 3,000 documents from Notion and been verified end to end (E2E). Support is expanding to sources such as Slack, Confluence, Jira, and Google Docs/Drive.
A single index: ingested documents and the internal pages, tasks, and chat messages all land in the same index. Search is a hybrid method that combines vector search, which searches by meaning, with full-text search, which searches by words.
AI agent: rather than simply listing the search results, the agent answers a question in the form "the answer is right here in this document." Instead of a person opening and reading ten search results, the agent guides you to where it lives.

This "make every source into a single RAG" is also where the name UseSources comes from: put your sources into a usable state.

How search is built, and the cost decisions behind it

There were two decisions in the search foundation.

The first was the decision not to run a dedicated search engine. The initial architecture proposal included Elasticsearch (the standard software for large-scale search), but I removed it. Combining pgvector, the vector search feature of the Postgres database, with full-text search is more than fast enough at this scale, and it means one fewer kind of server to operate. At the scale of an individual developer, standing up a separate search engine lets its cost and operational load crowd out the product itself.

The second was the decision to make ingestion asynchronous. Full syncs from external connectors were originally synchronous, and for a large workspace no response would come back for 86–120 seconds and the request would time out. I switched to a design that inserts a queue and processes the work in the background, and the response dropped to 1.73 seconds. The change was made after estimating the extra cost at roughly $1–2 per month. I have written up the long slog of this fix in a separate article.

The second pillar — unifying the workspace around the project axis

Even with a single search, if the places where daily work lives are scattered, the fragmentation gets reproduced. That is why the second pillar is needed: unifying the workspace itself.

Creating a project produces a three-part set at once. A wiki page you can write in Notion-style, a task board, and a dedicated chat channel are all set up the moment the project is created.
Permissions are set once, at the project level. Adding a member and removing one are both a single operation on the project. Remove someone and their permissions across the pages, boards, and chat beneath it are revoked all at once. "We revoked their access everywhere but forgot one tool, so someone who left could still read the documents" cannot happen by construction.
A task is a kind of page. Rather than splitting tasks and documents into separate systems the way Jira and Confluence do, I unified them into a single entity. A task can hold the same body text as a document, and it rides the same search.
The team can write at the same time. Pages support real-time simultaneous editing by multiple people, and come with mentions, notifications, presence, comments, and sharing.

The knowledge unification (the first pillar) and the workspace unification (the second pillar) are two facets of the same idea. The external past is connected through RAG; the internal present turns around a single axis. Search, permissions, and context all come together in one place.

How we actually use it — how it runs inside the company

Abstractions alone do not land, so here are three ways we use it internally.

Example 1: "Where did we decide that again?" When I want to check a design decision from a few months ago, I used to search the wiki, then scroll back through chat if it was not there, then open the external Notion. Now I type into UseSources search: "why did we adopt X?" The agent returns the relevant thread from the chat at the time and the related wiki page, each with where it lives. The biggest difference is that time spent searching becomes time spent reading.

Example 2: onboarding a new hire. Add a new member to a project and from that moment they can read all of the wiki, board, and chat beneath it. There is no need for guidance like "for the history, please look at these five tools," and the load on the person being asked drops too. Past external documents can be reached through the same search via RAG, so branching guidance like "the history is in the old Notion workspace" disappears as well.

Example 3: the weekly review. Because a project's task board and pages sit on the same axis, you can look at "pages that moved this week" and "tasks that moved" side by side. Since a task is a kind of page, review notes can be written directly in the task body and pulled up later by search. The classic problem of meeting notes and tasks drifting apart in separate systems does not happen.

Behind permissions and simultaneous editing

Two footnotes on the structure.

Permissions are inherited from the project. Permissions are granted to people only at the project level, and the pages, boards, and channels beneath it inherit exactly what the project carries. That is why "removing" someone also takes a single operation. Deliberately not building a back door to attach permissions directly to individual pages has paid off later. Allow even one exception and you end up checking every page at every review.

Simultaneous editing is built on a mechanism that resolves edit conflicts automatically (CRDT). Even if two people edit the same paragraph at the same time, neither person's change disappears. To keep costs down, the real-time communication path leans on a lightweight server setup, and it has been verified end to end in production. This was a plain but important choice for having both a "Notion-grade editing experience" and "individual-developer cost."

It did not start out in this shape

UseSources has changed direction significantly twice.

It began as a tool for RAG search across multiple SaaS. Building search alone, I realized that search alone does not change where work lives. The root cause of the scatter — that daily work happens in separate tools — was still there.

So I pivoted toward unifying task management and chat, and while building it out I realized that "if tasks and pages are separate things, there are two places to put work again," and unified a task into a kind of page. Finally I arrived at the current shape, where project is a first-class axis. Thirty design decisions over four months, all recorded, made these pivots possible.

There is also a predecessor that once stalled. A year earlier I had started building it as an "integrated knowledge search" and it stopped as a prototype; the current UseSources is that prototype restarted with a spec-first development process (writing ADR/SDD/TDD documents together with AI). I stood up the development foundation in four days and got it to production in three months.

RAG is not magic — what it takes to make it work in practice

The word RAG (Retrieval-Augmented Generation, a method that gathers evidence by search before composing an answer) is fashionable, but making it work in practice takes plain, unglamorous engineering. Here are the elements that actually worked in UseSources.

First, the freshness of the index. Ingest external documents once and be done, and they go stale quickly. By making ingestion an asynchronous job through a queue, UseSources can re-sync large workspaces routinely. When syncing is heavy, you want to lower how often you re-sync, and the index gets old — asynchronous processing breaks that vicious cycle.

Second, using semantic search and word search together. Vector search (which searches by closeness of meaning) is strong on paraphrases but weak on exact matches for proper nouns and model numbers. Full-text search is the opposite. Making it a hybrid of both lets you pull up both a vague query like "that incident yesterday" and a specific identifier like "P2002."

Third, a design that returns where things live. When an agent returns only a summary, the reader has no way to verify it. The UseSources agent returns answers with a citation — "this part of this document." If you are building an internal RAG, I recommend starting with citations. Trust comes not from the fluency of a summary but from the certainty of its source.

Is it a replacement for Notion or Slack?

I get asked this often, so let me state the positioning plainly. UseSources is not a tool that says "let's stop using Notion, Slack, and Jira entirely, starting tomorrow."

A real organization has an accumulated history and habits built up in its existing tools. So UseSources takes two stages. The past is ingested into RAG through connectors and turned into a searchable asset. The current work starts turning on the project axis. Migration is not an overnight move; it starts with search getting a single entry point, and the everyday place work lives gradually converges from there.

We did not move everything at once ourselves either. We began by using it as a single entry point for search, then started running new projects on top of UseSources one at a time. Building a structure that does not force migration is, as it turns out, what made adoption stick faster.

A lesson from an accident

It was not all smooth sailing. Here is one memorable accident. We were building the screens first with mock data and then wiring them to real data later, but at the point of the switch we cut everything over without keeping the promise to "preserve the mock-data state as well." The demo data all got wiped, and we even shipped a bug where an internal string was exposed in the search box. It was a failure of moving ahead without preserving the state before the switch. After that we strengthened the discipline of fixing the spec before implementing. Turning a failure into a mechanism of discipline afterward is something we enforce across all of our development.

Behind the build — restarting a stalled prototype, spec-first

The build of UseSources shows off the "template for developing with AI" I have written about throughout this series.

Its predecessor was the "integrated knowledge search" prototype from a year earlier, which had stalled. When restarting it, the first thing I did was not write code but put the documents in order. Establish the formats for records of design decisions (ADR), the basic design (SDD), and the detailed design (TDD), and connect them to task management. This ramp-up took four days. From then on I proceeded while recording every design decision, and in four months the ADRs numbered thirty.

The volume of activity is on record too. Commits in the restart month: 12. The next month, 76; the month after that, a peak of 245; then 112. As the numbers show, the first two months were spent searching for direction, and the moment the axis was fixed, implementation exploded. The reason this pace holds up even through two pivots is that every decision is recorded in documents, so when you overturn one, "what exactly you are overturning" is clear.

One more thing established in this product: the practice of fixing the look of the screens before implementing. I built more than twelve screens up front with wireframes and mock data, got agreement on the look, and then wired in the substance. The demo-data-loss accident above happened at that switch, but the method itself is right, and after the accident I made preserving the pre-switch state mandatory and kept using it.

Where it is now

UseSources is running in production (app.usesources.com) and in daily use inside the company. The admin screen is isolated on a separate host, and the real-time editing communication runs on a cost-controlled setup. It is at the stage of broadening the sources that external connectors support and improving the agent's accuracy.

How to start

You can open the app from usesources.com. I recommend starting by creating a single project and putting your usual notes and tasks there. For external connectors, connecting the source with the worst scatter first (often an old Notion or a shared drive) makes the effect the easiest to see.

To the worry that "adding a tool will make things messier," I answer this way. UseSources is not a tool that adds one tool; it is a tool that cuts the places you search down to one. Even without throwing away your existing tools, just giving search a single entry point visibly reduces the daily "where was that?"

Lessons you can carry over

Information fragmentation comes in two kinds: fragmentation within the workspace, and fragmentation out into external SaaS. The former is solved by unifying the axis, the latter by unifying into RAG. Search alone does not fix the former.
An agent that "returns where the answer lives" is more trusted in real work than one that "returns the answer." People only feel safe once they see the source.
Ask first whether a separate search engine is worth the scale. If Postgres vector search plus full-text search is enough for your scale, keeping the number of things you operate down lowers total cost.
Make large ingestion from external sources asynchronous from the start. Synchronous timeouts surface only after a workspace has grown.
Splitting tasks and documents into separate systems brings back the split in where work lives. Unifying with "a task is a kind of page" pays off for both search and permissions.

Ship discipline as skills, not documents — introducing the 165 skills of ELN workflow

uehara — Fri, 18 Sep 2026 19:38:29 +0000

The short version

What we built: ELN workflow, our in-house plugin suite that implements development discipline, quality standards, and spec processes as Claude Code skills (procedure sheets plus enforcement gates that AI follows), distributed to every project. As of September 2026 there are 165 skills.
Why we built it: development standards written in documents do not get followed. They get followed only when they are built into the workflow. If the gate refuses to let you proceed, the discipline executes even when humans forget and even when AI forgets.
The key point: the more development you delegate to AI, the more "did the AI actually do it right" becomes the bottleneck — a check humans end up doing by hand, every time. This plugin mechanizes that verification itself and ships it to every project.

The full story (about a 13-minute read)

The authentication, notification, and billing platforms I introduced in this series all moved "features every product needs" into shared platforms. This article is different in kind. What we consolidated is not a feature — it is how we develop.

"Please follow the rules" doesn't work

We develop more than 20 products with one human and AI as the executing team. The biggest problem in this setup is not coding speed. It is quality variance.

"Say 'done' only after tests pass." "Record design decisions." "Estimate costs before changes that affect billing." All of it was written down. AI still said "done!" anyway, and humans still skipped steps under time pressure. Writing more documents did not solve this.

So we inverted the approach: implement discipline not as documents but as Claude Code skills. A skill loads automatically when its situation arises and blocks the operation on the spot when its conditions are not met. Not "please follow this" — "you cannot proceed unless you do." Only after this inversion did discipline start executing identically across all projects.

The 165 skills at a glance

There are 165 skills today, across three plugins: the quality-and-discipline core (139), project orchestration (12), and business/content production (14). Alongside the skills, the same plugins bundle 50 automated checks (hooks) and 14 specialized sub-agents.

You cannot tour all of them, so here are the categories with representative examples.

Completion gates (stopping the AI's "it's done") — flagship: eln-verify-before-claim. Blocks completion claims that lack measured logs, test output, or live verification. A passing build is not "done."
Evidence-based debugging — flagship: evidence-based-debugging. In production bug hunts, demands observation of logs, DB, and real requests before any guess-based code edits.
Review and cross-checking — flagship: eln-adversarial-review. Requires an adversarial, multi-perspective review before substantive changes merge to main. A merge without the review record is mechanically refused.
Cost and billing safeguards — flagship: eln-cost-watch. Demands a monthly cost estimate before billing-affecting changes, and forbids claiming "costs went down" until real measurements converge.
Spec and design documents — flagship: eln-new-adr. Templates and indexes for filing and updating design-decision records.
Progress records and handoffs — flagship: progress-record. Keeps investigation state in git-tracked logs so a new session can resume where the last one stopped.
Code conventions — flagships: use-ts-pattern / no-else-no-default. Down to how TypeScript branches are written — unified by skills, not review comments.

Other categories cover infrastructure/AWS/environments, observability, security, internationalization, parallel git work, and team notifications.

A few skills up close

Names and roles below are the real ones.

Skills for not trusting AI reports

eln-verify-before-claim — machine-checks claims against evidence (measured logs, test output) the moment before "done" is declared. No evidence, no claim.
eln-report-guard — stops completion reports backed only by weak proxies ("build passed," "unit tests passed") and escalates them to live observation.
answer-the-question-first — makes "is it finished?" get answered in the first sentence, not after a recap.
eln-execute-dont-defer — stops the AI from bouncing CLI-executable work back to humans as "please do this manually."
no-tracked-leftovers-at-goal — blocks "done" while tracked tasks remain. Zero leftovers is the definition of done.

Investigation and debugging discipline

evidence-based-debugging — no guess-based fixes before logs, DB, and live requests are observed.
zero-result-query-check — zero search results may not be concluded as "does not exist" until the query itself is validated.
classify-failure-before-rerun — classify why CI failed before rerunning. No blind "let's just retry."
read-current-state-before-implementing — read the current state from primary sources before implementing; prevents duplicate implementations of existing features.

Skills that protect money and production

eln-cost-watch — cost estimates before, measured convergence after. Until then, "it's cheaper now" is unsayable.
dynamodb-best-practices — database designs must show monthly cost as a number before code is written.
billing-safety — a dedicated discipline for touching payment code; blocks changes that could double-charge or misbill.
verify-after-deploy — deploying is not finishing. Live verification in production is part of the deployment.
eln-deploy-target-checklist — enumerates every deployment target a change should reach; blocks "done" with only some targets updated.

Skills for working as a team

eln-adversarial-review — adversarial multi-perspective review as a merge precondition, mechanically enforced.
eln-codex-cross-check — has a different AI (Codex) review read-only, so one AI's blind spots get probed by another.
eln-acceptance-ledger — captures mid-conversation "oh, also do this" requests into a ledger; no checkmark without evidence.
progress-record / progress-recall — git-tracked state for resuming across sessions.
prioritize-users-restated-goal-over-current-thread — when the human restates the goal, replanning outranks the current thread.

What each one does in depth, and the incidents they have actually prevented, will be their own articles in this series.

The gates, concretely — naming the bad completion report

One more level of detail on the completion gates. Suspicious AI reports become machine-detectable once you name the patterns. A few of the real detection classes:

FALSE_SUCCESS — "done" without evidence
WEAK_PROXY — claiming "works in production" from builds or unit tests alone
MOCK_THEATER — using an all-mocked green test suite as evidence that the real thing works
NARROW_PROBE — declaring "that feature doesn't exist" after reading one file
FALSE_DEFERRAL — handing CLI-executable work back to the human as manual steps
RESIDUAL_COMPLETION — listing leftover tasks and still closing with "done"

Evidence has a defined strength ladder too: static checks < unit tests < integration tests < live end-to-end < production observation. Claims about external boundaries (payments, webhooks, deployments) do not pass below live verification.

There is a gatekeeper at the end of the conversation as well. The moment a reply tries to end while declaring "done," it is checked against git's actual state, not prose: any changes not merged to main? any open PRs? any uncommitted files? If anything remains, the declaration bounces back with the unfinished items listed by name.

We do not build clever gates — a principle decided by measurement

By now you might think "just make the gates smarter." We learned the opposite.

We once considered having the completion gate judge whether evidence was genuine via pattern matching. Tested against 191 lines of real records, it wrongly rejected 76% of genuine evidence. That measurement froze the policy: mechanical gates check structure only (does a record exist, are leftovers zero, is the format satisfied); judging whether content is real is delegated to an independent reviewer — another AI or a human. The temptation to make gates clever was rejected by data.

New gates follow a fixed introduction path, too. They start in observe mode — logging warnings without blocking — and are promoted to blocking only after the false-positive rate is measured. One guard analyzed 30 days and 400 sessions of history to confirm "about nine operations a month would be blocked, nearly all correctly" before it was allowed to block. Even a rule that looks right does not get to stop humans until its false alarms are measured. That is the responsibility that comes with adding gates.

One more principle: gates never lock people in. If the checking tooling itself is broken, gates fail open; unblocking is reserved for the human's explicit action; and there is deliberately no environment variable that lets the AI unblock itself.

The moment a gate is born — three failures on record

Most skills carry the actual failure that created them. Three examples.

The zero-results misdiagnosis (April). Searching database logs by a field named timestamp returned zero rows, and we concluded "the logging pipeline is broken." The real field name was createdAt, and 7,406 log rows existed. Since then, a skill forbids "it doesn't exist" until field names, types, and time ranges are validated and an unfiltered probe proves data is present at all.

The $43-a-day bill (September 3). A recurring read over a growing table, added without estimation: 300 million read units a day. Plugging the numbers into the formula afterward matched the bill almost exactly — one line of arithmetic beforehand would have prevented it. Now, database designs cannot proceed without a monthly figure from "reads × unit price."

The prose that got walked past (September 6). Three days after we canonicalized the cost discipline as skill prose, a billing-affecting configuration change executed with zero estimation anyway. Documents — and skill prose — do not work unless they are read at the moment of execution. That failure promoted the cost rules into a gate that interrupts at the moment the command runs: billing-affecting commands are now refused outright unless an estimation ledger exists.

Every failure becomes a mechanism that makes the same failure impossible. 165 skills is also a record of how many times we have failed.

The catalog guards itself too

At 165, even the author cannot remember every skill's purpose and invocation. The count was 90 on July 2, 121 by July 20, 154 in August, 165 now. A hand-maintained list of a growing set will drift from reality, guaranteed.

So the catalog is auto-generated. Every skill's registration requires four fields — category, purpose, when-to-use, invocation — and the catalog is generated from those alone, in 16 categories. Invocation styles are classified three ways: 69 skills trigger automatically, 68 are called explicitly by name, 28 are use-when-you-like. The guidance for humans is simple: "if in doubt, remember only the 68 explicit ones."

The mechanism is doubly enforced: generation halts if required fields are missing, and CI fails if the stored catalog is stale. We once had a hand-written list quietly coexisting with reality — four numbers diverged — and that story is written up separately.

Scale — 289 releases in four and a half months

This plugin is developed under its own discipline (using it on itself daily is where the quality comes from). First commit April 28, 2026; since then 547 commits and 289 version bumps. 139 architecture decision records. Over 26,000 lines of skill procedure text.

There is a second layer besides skills: 134 "instincts" — behavioral lessons from past sessions compressed into one line each, auto-injected at the start of every session in confidence order. Skills are the procedures to follow; instincts are the situational reflexes from past failures. Ship the rules, and ship the experience too. With AI teammates, cross-project learning needs both layers.

What this means for your team

The plugin itself is internal, but the approach transfers.

Pick one rule your team keeps breaking and move it from a document into a gate inside the workflow: a pre-commit hook, a required CI check, a template's required field. The location matters less than the property "you cannot proceed without it."
If AI writes your code, start with a gate that demands evidence (measured logs, test output) for AI completion reports. That is where our 165 began.

Lessons you can take with you

Development standards are not followed as documents. They are followed when they are built into the flow and block progress otherwise.
With AI development, "gates that verify AI claims" become the core of quality. Start at four spots: completion reports, debugging, review, and cost.
Keep mechanical gates structural; delegate meaning-judgment to an independent reviewer. Measure false positives before you let a gate block anyone.
If your rules multiply, auto-generate the rule catalog. Hand-written lists always drift.
Concentrate discipline in one place (a plugin), and every new project starts day one with the same quality bar.

One Place for Payments, Features in Each App — ELN Billing, One Platform for Multi-SaaS Billing

uehara — Tue, 15 Sep 2026 07:39:16 +0000

The short version

What we built: ELN Billing, a platform that consolidates billing for multiple in-house SaaS products into one place. It wraps Stripe and handles subscriptions, coupons, one-time charges, referral rewards, and refunds through a single API family. When a payment or cancellation happens, it notifies each service through signed webhooks (automated server-to-server notifications).
Why we built it: payment code is heavy. One existing site alone had more than 30 files related to subscriptions. Copying that into every new SaaS is not realistic.
The key point: for a consolidation service, deciding what not to own matters more than what to own. ELN Billing owns only the source of truth for "is this user subscribed"; the interpretation — "what does the subscription allow" — stays in each service. Six months of development, 172 commits, 17 spec documents; a pre-GA version is live in production today.

The full story (about a 14-minute read)

Last time I wrote about our notification platform. This is the same series: features every SaaS needs, moved out of each product and into a platform. This time, billing.

Thirty files is not something you copy

It started with counting. On one existing site, the subscription-related code alone exceeded 30 files: Stripe integration, contract state management, coupons, refund operations. Duplicating that for every new SaaS is not just a typing problem. Refund handling and billing disputes scatter across sites, and users face a different billing screen for every product.

The model we aimed for is the App Store's: payments in one place, features in each app. The platform owns payment and contract management; each service decides what the contract enables.

What ELN Billing does

From a consuming SaaS, the platform looks like this.

Subscriptions, coupons, and one-time charges through one API family. Services carry no payment screens and no Stripe integration of their own. Need a checkout page? Send a plan ID and return URLs, get a Stripe-hosted checkout URL back. The public API for registered services spans 22 endpoints: subscription lookup and cancellation, coupon validation, credit purchase/consumption/balance, referral codes and conversion tracking, refunds.
API keys are per service. The key identifies the calling service, and every response is automatically scoped to that service's data. Another service's contracts are invisible even with a valid key.
Payment events arrive as signed webhooks. Fourteen event types — subscription created/updated/canceled, payment failed, credit balance low, and more — with tamper detection.
A self-service portal ships with the platform. Users check their subscription, cancel, and apply coupons in a screen the platform owns, localized in six languages (Japanese, English, Chinese, Korean, Thai, Spanish). Services do not build their own cancellation screens.
Operators see all revenue in one console. Services, plans, coupons, subscriptions, credits, referrals, and MRR/ARR analytics. The operator console is physically separated from the user-facing portal — different domain, different deployment.
Services register their own plans. Each service creates and updates its plans with its own API key; plans are never created from the platform console. Plan revisions are versioned — a new plan ID chained to its predecessor, never an in-place overwrite.

The most important decision — do not own "what the plan allows"

Mid-development, one service asked us to store and enforce plan-level feature limits (say, a minimum check interval) inside the billing platform. It sounds natural — it's the billing platform, why not keep the plan-to-features table there?

I did not implement it. The moment the platform understands what a value means, it carries that service's domain knowledge. The next service brings a different limit concept, and the platform grows again. Billing sits where breakage stops payments for every service; letting per-service specifications snowball there is a future I wanted to avoid.

Instead, plans carry a free-form field that each service writes in its own vocabulary — a monitoring product writes "50 monitors max, 60-second minimum interval"; an AI workspace writes "top-up rate and unlimited artifacts." The meaning, the values, and the enforcement belong to each service; the platform distributes the bytes without interpreting them.

The boundary is also documented as prohibitions: "do not hardcode prices in individual sites," "do not define permissions or feature limits in the billing platform," "never identify plans by name-string matching — always by plan ID." Up to "is the user subscribed" is the platform; from "so what can they do" is the service. That single line is why consolidating billing did not turn the platform into a warehouse of everyone's feature specs.

Not owning authentication either

The other thing we refused to own is authentication. A billing platform knows who subscribes to what, so building its own login is tempting. We delegated it to our identity platform (ELN ID): user IDs are unified across all services, and billing only consumes the authentication result.

Saying "delegation made it easy" would be inaccurate, so let me add the cost: every new environment we spin up needs its return URL registered on the identity platform's side. Delegation removes implementation, but adds coupling to operate.

One Stripe account for everything

The Stripe topology was a compared decision too. One account per service defeats the point of consolidation — rejected. Stripe Connect changes the fee structure and complicates operations — rejected. What we chose: one Stripe account for all services, with the service ID, plan ID, and user ID burned into each payment's metadata. One Stripe dashboard shows all revenue, and every payment routes mechanically by its metadata.

Duplicate event processing is handled platform-side as well: every Stripe event ID is stored with a seven-day expiry, and re-delivered events are skipped safely. Payment events must be assumed to occasionally arrive twice.

Webhook craftsmanship — sign, retry, never drag payments down

Notifications to services are the accident-prone boundary, so the spec is nailed down tightly.

The signature formula is written in the spec as one line. The signature covers the timestamp and body joined with a period, and three headers carry signature, timestamp, and event type. Why so explicit? A story below.
Retries at 1 s → 5 s → 15 s, three attempts max. Only receiver-side transient failures (5xx) are retried; receiver bugs (4xx) are not. Each attempt times out at ten seconds.
Notification failure never blocks payment processing. Even if every delivery fails, Stripe gets its acknowledgment. A notification hiccup must not push the payment itself into a redelivery loop.

"The webhooks aren't arriving" — a five-battle war

The most instructive stretch of this project was April 2026: five consecutive battles with webhooks that would not arrive.

Battle one: signature mismatch. The sender signed the body alone; the receiver expected timestamp-plus-body. Every delivery was rejected as tampered. Root cause: the spec never stated the formula, so both sides invented one. We wrote the formula into the spec and added a contract test that the two sides match.

Battle two: three layers of missing configuration. Signatures fixed, still nothing. After three wrong hypotheses, the real answer was a chain: Stripe had zero webhook endpoints registered; the hosting environment lacked the verification secret; and the receiver URL in the database still said http://localhost:3000. Any one of them silences the whole path. We banned localhost-type URLs at the API, wrote a script that verifies all three layers in one command, and made a production-setup checklist.

Battle three: fire-and-forget. One case remained. The send call was written fire-and-forget, and the serverless runtime froze the environment the moment the handler returned — cutting off in-flight sends so the SaaS never received a single request. We awaited every send and added a regression test asserting "notification resolved before handler return." Fire-and-forget breaks on serverless.

Battle four: the receiver. The last missing delivery: the receiving service lacked its secret — and its build pipeline had been broken for two days, so adding the secret changed nothing. That week produced our two standing rules: "read the receiver's response body first" and "a live end-to-end check outranks every hypothesis." A script that actually pushes a delivery through is now a mandatory deployment gate.

Guess, fix, miss — four times. The one lesson: for anything that crosses a boundary, do not claim it is fixed until you have seen both ends for real.

The discipline of money-handling code

Billing has no "mostly correct." In September 2026 we swept the payment paths end to end: 16 defects fixed, all seven payment routes verified live. Two representative fixes:

Exactly-once referral rewards. Reward granting could double-pay under concurrency — and, worse, permanently lose a reward on a transient database error, because the "granted" flag was set before the grant and failures were swallowed. We rebuilt it on conditional writes (claim exactly one grant slot first) and verified the concurrency with integration tests against a real local database.
Coupon service boundaries. One service's coupons validated successfully against another service's checkout — a missed application of the "scope everything by API key" rule. Boundary checks are an every-endpoint discipline; one gap is an incident.

Behind the scenes — a working whole on day one, 32,000 lines in six months

The first commit is March 1, 2026. That same day, 19 commits laid down the working skeleton: identity integration, database, Stripe, checkout, webhook processing, subscription management, operator console, revenue analytics, signed notifications, and deployment. Building the entire working shape on day one and then fixing what real usage reveals — that rhythm fits AI-driven development well.

Six months later: 172 commits, 233 TypeScript files, about 32,000 lines, 17 spec documents, 258 tests (including integration tests against a real local database) plus four browser-driven end-to-end specs. Billing earns deliberately thicker tests than our other products.

It lives in its own repository. We considered co-locating with the corporate site or the shared monorepo, and rejected both: "billing stops every SaaS when it breaks; it does not belong anywhere it can be dragged down by unrelated changes." Note the contrast with our notification platform, which chose the shared monorepo — the same word "platform" lands in different homes depending on its blast radius.

Where it stands

A pre-GA version (v0.9) went to production in September 2026, with the first SaaS running real billing through it and a second mid-integration; six services are registered to onboard, and the GA (v1.0) scope is fixed at eight features. The user portal and the operator console run separately, on separate domains and deployments.

Lessons you can take with you

Decide what a consolidation service will not own, first. Keep the source of truth in the platform and the interpretation in the consumers, and the platform stays out of everyone's change stream. Write the prohibitions down.
Anything that stops everything when it breaks (payments) deserves an isolated home, with dependencies pointing one way only: service → platform.
Sign and auto-retry webhooks from day one — and put the signature formula itself in the spec. Two sides inventing formulas independently halts every delivery.
Debug boundary failures by looking at both ends, not by hypothesis. Configuration fails as a chain, not a point — make one command verify every layer, and gate deployments on it.
Guarantee "exactly once" for money movements with conditional writes, and prove it with concurrency tests against a real database.

Distributing secrets safely — we built dotvault, which delivers .env files with one command

uehara — Mon, 14 Sep 2026 08:22:32 +0000

The short version

What we built: a service that distributes secrets — .env files (files that bundle settings such as connection targets and passwords as environment variables), .npmrc (credentials for fetching packages), kubeconfig (connection information for server clusters), certificates, and so on — to each person's machine or to Continuous Integration (automation that continuously verifies changes, CI) with a single command: dotvault pull.
Why we built it: API keys and .env files get you a security violation if you put them on GitHub, and "the key is exposed, rotate it" if you hand them to an AI. You cannot distribute them, yet you have to distribute them. We built dotvault to resolve that contradiction at the root.

The article (about 9 min read)

This article is the first in a series that walks through why we built the secret-management service dotvault, what features it has, and how we developed it. Let me start with "why we built it."

You cannot distribute them, yet you have to distribute them

Development always requires sharing API keys and .env files (files that bundle settings such as connection targets and passwords as environment variables). The .env needed to start the app has to be shared among developers, and CI (automation that continuously verifies changes) won't run without it either.

But when you try to push these to GitHub (a code-sharing service), you're told it's a security violation — put them in .gitignore. When you try to hand them to an AI service like Claude Code, you're told the API key is exposed — rotate it (recreate the key). You cannot distribute them, yet you have to distribute them. I felt this contradiction was the real problem.

In the end you resort to distributing them through other services like Slack or Notion, which is very awkward. On top of that, when someone changes a .env, the change never lands in Git, so the other developers never hear about it. Development stalling because "an environment variable needed for startup was missing" or "a value had changed" was a constant occurrence. And this is not just our company. While consulting on development at various sites, I ran into the same problem again and again.

So: don't keep a .env file at all

That's why we changed our thinking at the root. The choice was: the project does not keep a .env file.

All secrets are registered in dotvault. Once a user logs in a single time via the npm CLI, they can then use dotvault pull and dotvault run. There is no need to hold a secret key per project.

Where this really pays off is that environment variables grow multiplicatively. The number of things to manage balloons as projects × environments (development, staging, production). With dotvault, no matter how many projects or environments there are, one login keeps the same secrets synchronized for everyone. That "synchronized for everyone" is what I wanted.

Let me start from what is running today

First, look at the finished form.

dotvault is a service for distributing secrets. It is running today as Software as a Service (business software used over the cloud, SaaS).

The admin console lists projects, service tokens (API keys for machines), members, and the audit log. Who touched which secret, and when — it's all recorded here.

We built it as a service rather than getting by with a short script, because for an organization, a procedure for distributing secrets is not enough. Who is allowed what. How much you hand to machines. Who touched things after you handed them over. How usage limits per plan are enforced. These four are managed as a system. External payment processing and automated billing are not part of this system.

Why does merely distributing secrets need this much of an admin screen? Because for organizational use, storage alone is not enough — you also need permissions and auditing. Let me first explain why we built it this way.

Why storage alone is not enough

Let's be clear about what is protected and where it is delivered. Secrets only become usable once they reach permitted people or execution environments, in only the scope that is needed.

One of the main reasons secrets leak is that there are too many places they live. Chat, email, each person's laptop, several servers. The more copies there are, the more paths there are to a leak. Rather than adding more procedures for hiding each copy, narrowing the storage down to one place is easier to manage.

The values in a .env are distributed to permitted people and servers, and used there.

dotvault doesn't just store secrets — it distributes them to permitted parties, in only the scope needed.

One place to manage storage, permissions, and auditing

So dotvault separates the storage of secrets from the code repository.

The server-side source of truth lives in dotvault, encrypted with KMS (Key Management Service). Because dotvault pull writes the fetched values into a local .env file, keeping that file out of Git's tracking is a separate operational step you still need.

On top of that, environments are separated: development, staging, and production — plus _shared, which every environment can reference. Even settings with the same name are different things in development and in production. We do not mix them.

And every single key carries an audit log. Who read which value, and when. Who changed it. Everything is recorded.

Fix a value in one place, and users get the new value on their next dotvault pull or at process startup. However, old .env files already fetched with dotvault pull are not deleted automatically, and values in already-running processes do not change until a restart. By concentrating the update source in one place and standardizing the re-fetch procedure, we reduce configuration drift between environments.

In the admin console you manage projects, service tokens, members, and the audit log. These are what you use to control where secrets are delivered and in what scope.

Members assign permissions to people. This person, this project, up to this environment — that's the line you draw.
Service tokens assign permissions to CI and servers. Machines, too, get only what they need.

The audit log records who touched what after it was handed over. It's not "hand it over and done."

Storage location and permissions for secrets are managed here. That addresses the storage side of the problem.

Receiving takes one command

Next, the distribution side.

No matter how safely you store secrets, if receiving them is a hassle, people will go back to copying by hand. So receiving has to be finished in one command.

In dotvault, environments that need a .env file use dotvault pull. One command writes the secrets that person is permitted to have into the declared file on their machine or in CI.

What dotvault pull can fetch is only the permitted scope. If someone who holds only development permissions runs dotvault pull, production secrets do not come down. Because the fetched values are written to a local file, you still need configuration and practices to keep it out of the repository. For execution paths that should not create a file, use dotvault run -- <command>.

dotvault manages destinations, scope, and operation records as a system. Rather than not sharing secrets, we replaced sharing with a distribution path where you can trace who was given what.

Users don't need an AWS account. No difficult setup either. A person who joins simply pulls.

A newcomer, once granted permissions, can run dotvault pull themselves, cutting the time spent waiting for someone to send a .env. In CI that doesn't need a file, dotvault run -- <command> passes the key values into the child process's environment variables; we confirmed by implementation and tests that it does not create or modify a .env file. The key values do not persist in that process environment after the child process exits. On the other hand, automatically deleting files already created by dotvault pull, or automatically revoking already-issued secrets, are things we have not verified.

You can distribute not only to people, but to AI as well. Situations where you want an AI service to reference secrets during development are increasing, but pasting them into chat is exposure. dotvault has API and MCP (Model Context Protocol, a standard for AI tools to connect safely to external services) integrations, so an AI service can safely read and write secrets, and pull out an API key to access another service.

At design time we compared against Doppler and Infisical (existing secret-management services). We chose to build our own in order to match our organizational structure, our permission boundaries, and our per-plan usage limits. "Plans" here means usage caps and feature restrictions — not external payment processing like Stripe or automated billing.

We started writing from the spec at the end of May, and in the single month of June built up organizations, permissions, audit logging, and per-plan usage limits in 127 commits. Stripe integration, automated billing, and self-service billing were out of scope for this first version (v1). We decided the scope from the problem itself: "it's a secret, yet it has to be distributed."

The manual work we removed, and the judgment humans keep

Before dotvault, we would paste and send secrets, communicate the password through a separate channel, and confirm receipt. With dotvault pull, the needed values can be written out with one command after permissions are checked, so that manual work can be reduced.

Who is allowed which secrets is a human judgment. After the judgment, dotvault handles file generation, environment-variable injection into child processes, and operation records. The division of labor is: humans decide permissions and scope, and the routine distribution work is left to the service.

Consolidating authentication and authorization into one platform shared by all our products

uehara — Sun, 13 Sep 2026 18:17:27 +0000

Conclusion

ELN ID is an in-house project run by EarthLink Network as a company, developed by a team of one human owner with AI doing the implementation. What started as an organization-management app grew into the authentication and authorization platform (an Identity Provider, IdP) shared by our products. When we audited which of our seven products recognized whom as an administrator, the number of products using the ServiceRole we had designed for that purpose was zero. So we made the pattern that was actually in use — globalRole === 'ADMIN' || groups.includes('<serviceId>-admin') — the shared rule.

Who counts as an administrator is decided in exactly one place. Whether a token is being issued for the first time or refreshed later, the same resolveScimGroupNames() runs. The UI does not make its own judgment; it simply displays the isAdmin the server computed. This is how we fixed two bugs: administrator-group information disappearing after a token refresh, and the admin screen rejecting users whose API calls were succeeding.

The story (about 8 minutes)

In June 2026, ELN ID — the organization-management app I maintained — had grown over three months into the IdP used by multiple products in the company. In a short period, requests like "please issue us an OAuth client" came in more than fifteen times, and it became clear we needed a shared specification for integrations instead of inventing the rules per product.

Three decisions we made when it became the authentication platform

The trigger was reactive. One product asked us: "we want silent Single Sign-On (SSO — using one authentication across multiple services) plus terms-of-service consent, handled together." That put us at a fork: handle it as a one-off, or turn it into shared rules. I chose the latter and wrote the integration rules down as design records. We decided three things first.

The first was how to implement silent authentication. Silent authentication means checking in the background — without asking the user to type anything — whether they are still logged in, and if so, logging them in automatically. The goal is to avoid showing a login screen every time a user opens an integrated product. Initially we considered the classic approach: place a small invisible window (a hidden iframe) and have it ask "is this user currently logged in?". But on real devices, the answer was always "not logged in". The cause: the cookie ELN ID uses to remember the login state is issued with SameSite=Lax. With that setting, the cookie is not sent to a window embedded in another site. So we restricted silent authentication to a method that briefly navigates the whole page to ELN ID and back (a top-level redirect), banned embedded iframes, and recorded that in the design records.

The second was how to manage consent to the terms of service. A "consent version" here is the marker of which version of the terms a user has agreed to. When the terms are updated, users must consent again. That only works if ELN ID and the integrated products judge "has this user consented to the current version?" identically. If an integrated product starts comparing numbers on its own — "version 2 is newer than version 1, so no re-consent needed" — its judgment will drift from ELN ID's. So we made the version an opaque string that ELN ID decides, with no meaning attached to its contents. The only thing integrated products are allowed to do is check whether the strings match exactly; converting to numbers or comparing order is forbidden. Whether a user has consented is not embedded in the login token (no claim in the id_token); it must always be fetched through the internal verification API (verify-token) and compared, character for character, with the current version.

The third was the decision not to migrate existing consent records. Before consolidating on ELN ID, each product kept its own records of "this user has agreed to the terms". We could have copied those into ELN ID, but we chose not to. Instead, on the first login after the switch, everyone was asked to consent once more. Migrating old records carries the risk of omissions and mix-ups, plus the labor. One round of re-consent at first login avoided all of it. These decisions were finalized as design records in June. The consent mechanism and silent authentication were built in four stages using test-driven development (TDD — writing a failing test first, then implementing), and at the end all 41 tests passed.

Splitting authorization into four parts

As integrations grow, the next thing that matters is deciding who is an administrator. If that stays vague and scattered across products, accidents are guaranteed. So we split authorization into four parts.

globalRole ('ADMIN' | 'USER', stored on the user record in DynamoDB, default 'USER') marks a global administrator.
ServiceRole marks a per-service role.
ServiceTeam marks team membership within a service.
SCIM groups are groups created through SCIM (System for Cross-domain Identity Management — the mechanism that automatically syncs users and groups with external identity systems).

We then audited, one by one, which of our seven in-house products recognized whom as an administrator. On paper, ServiceRole was supposed to drive each product's admin decision. Reality was different.

Admin-decision audit in the field (7 products):
- product-A : globalRole === 'ADMIN' only
- product-B : globalRole + its own RBAC
- product-C : JWT groups claim (<service>-admin) primary, globalRole as legacy fallback
- product-D : JWT groups claim (<service>-admin)
- product-E : globalRole OR staffRole on ServiceTeam
- product-F : no role check at all (valid session = admin / serious authz flaw)
- product-G : not even connected to SSO (hard-coded user/pass)
→ products using ServiceRole for admin decisions: zero

The design records described an ideal, but nobody in the field used ServiceRole; the two products that had gone live first had each independently settled on "is this user a member of the SCIM group <serviceId>-admin?". I abandoned the ideal and promoted the pattern reality had converged on to the standard. This is the new standard form.

// The standard admin check shared by all products (aligned with what the audit found)
const isAdmin =
  globalRole === 'ADMIN' ||
  groups.includes(`${serviceId}-admin`)

globalRole === 'ADMIN' is reserved for emergency global administrators. Day-to-day, per-service administrators are granted through the <serviceId>-admin SCIM group.

We also fixed the deployment order so that administrators never get locked out. If you change the admin-check code first, then until the corresponding SCIM group and its members exist, every administrator is locked out. So this order became mandatory:

Create the group
Assign the members
Change the admin-check code
Deploy and verify

Why it kept breaking somewhere else after every fix

While moving to this model, authorization bugs kept appearing — all with the same shape: the same decision was being built in two places, and the two drifted.

The first was the bug where administrator-group information disappears when a token is refreshed. When you log in, you receive a token that acts as your ID card. It expires after a while, so a refresh token (refresh_token) is used to get a new one without logging in again. The token carries the list of admin groups the user belongs to (groups), which is how "administrator or not" is decided. But while the initial issuance path (authorization_code) looked up the groups and built the groups claim, the refresh path (issueTokensForRefresh()) had no such lookup and passed undefined. So a user who was an administrator right after login lost every group-granted privilege the moment their token refreshed. Because it only happens on refresh, ordinary tests that verify the first login never caught it.

Our internal security review had flagged exactly this shape. The original finding (in English):

issueTokensForRefresh() (line 586) passes `undefined` as 5th arg to
generateAccessToken — SCIM groups are never fetched in the refresh path
→ any user with group-based access loses JWT groups claim after first
  token refresh

The second was the bug where the frontend and the backend look at different places. The backend admin check (withAdminAuth) had been fixed to accept members of the admin group, but the frontend guard (AdminGuard) still looked only at globalRole. As a result, users made administrators via a group could call the API successfully while the screen showed "Access Denied". Building the same decision twice, separately, is what caused the split.

The root cause was that who counts as an administrator was being decided in many places, independently. After the fix, it is decided in one place. In the token code we added a new function, resolveScimGroupNames(userId), shared by issuance and refresh. Both authorization_code (issuance) and refresh_token (refresh) call the same function.

// sso/token/route.ts — both issuance and refresh go through the same resolver
async function resolveScimGroupNames(userId: string): Promise<string[]> {
  const { ScimGroupMemberEntity, ScimGroupEntity } =
    await import('@/infrastructure/dynamodb/entities')
  const membershipResult =
    await ScimGroupMemberEntity.query.byUser({ userId }).go()
  // ...resolve group names from memberships...
}

// authorization_code grant side
const groups = await resolveScimGroupNames(userId)          // line ~309
const accessToken = generateAccessToken(userId, clientId, scopes, globalRole, groups, ttl)

// refresh grant side (inside issueTokensForRefresh)
const groups = await resolveScimGroupNames(userId)          // line ~588
const accessToken = generateAccessToken(/* ... */ groups /* ... */)

On the frontend, the server now includes its computed isAdmin in the API response, and the screen only displays it. The admin check itself is consolidated into the server-side userIsInScimGroup(...), leaning toward "deny unless verified".

// withAdminAuth.ts — allow admin-group members even when globalRole is not ADMIN
if (user.globalRole !== 'ADMIN') {
  const isOnionAdminGroupMember =
    await userIsInScimGroup(userId, ADMIN_GROUP)
  if (!isOnionAdminGroupMember) {
    return /* 403 */
  }
}

What happened next, and what remains

We made it possible to create the administrator-assignment groups (SCIM groups) and add members from the admin screen. In the security review before shipping that screen to production, two critical and two high-risk issues were found, and all were fixed before release. The most dangerous one: when adding a member, we were not checking whether the person belonged to the same organization as the group, so a user from org-B could be added to an org-A group — a path to mixing data across organizations. We closed it by checking that the target organization is included in the member's organizations (targetUser.organizations). This admin-group mechanism was then rolled out to the rest of the products, following the two that had adopted it first.

Meanwhile, the audit findings "product with no role check" and "product not even connected to SSO" remain as separate tasks as of this article. Writing the ideal authorization model that covers everything, and pulling each existing product toward it one by one, turned out to be entirely different kinds of labor — that lesson stung.

Four takeaways:

Write integration rules into design records. Deciding promises like "no iframes", "version is an opaque string with no meaning inside", and "no migration" before implementation prevents drift between you and the integrating products.
Standardize the permission shapes actually in use. The audit showed nobody used ServiceRole, so we changed the standard to SCIM groups.
Decide who is an administrator in one place, and hand it to everyone. Build it separately in the UI, the server, and the token issuance and refresh paths, and the answers will drift. Consolidate into one routine, and deny when you cannot verify.
For permission changes, decide the deployment order too. Create the groups and members first, then change the code. In the reverse order, every administrator gets locked out.

We let AI handle our whole legacy-site migration

uehara — Thu, 10 Sep 2026 09:02:09 +0000

Our company had a few old sites still running. One was built on WordPress, one was plain HTML, and for one we had lost the login. This is the record of what I actually did: starting by having AI migrate them, adding features and running them, and finally consolidating everything into our own admin panel (a CMS). Let me say up front that we did not set out to "build a CMS." Only by running the sites did the need become clear.

First, have AI do the migration

By "migrate" I don't mean I copied code by hand. All I did was create one project in Claude, hand it the URL of the site that was live, and ask: "Move everything at this URL so it can be hosted on AWS Amplify." If you don't have an AWS account, you create one, and after that you can migrate just by handing over the URL. This part I can recommend.

Why Amplify? Our company's main site doesn't get that much traffic, so I figured Amplify could run it within the free tier. It becomes easier to maintain and cheaper to run. That was the reason for moving. To be clear, this isn't "Amplify is always free" — it's an expectation given our site's traffic.

One thing to add: if it's a static web site with no running programs, you should be able to move it almost as-is this way. Sites with dynamic parts, like a database or logins, won't go over unchanged — but for a static site, you can migrate it the same way, just by handing over the URL.

Once it was running, we kept adding features

Once we could run it ourselves, we wanted more. We added an e-commerce feature, and because we wanted to send inquiry emails, we added an inquiry mechanism. We added them one at a time.

We were doing the same thing on four sites

While running them, I noticed something. Across four sites, we were repeating the same kind of work. Inquiries are a good example. We were building similar things, separately, on several sites. I started thinking there had to be a better way.

Claude is useful, but there are places where it doesn't fit operations

Here I remembered what had been convenient about WordPress: you could edit directly from the web, online.

Claude is useful in its own way. But there were places it didn't fit for day-to-day operations. Asking it for small tweaks takes effort, and there are rate limits, so sometimes you can't use it when you want to. AI was a great fit for the migration itself, but for the daily operations afterward, something else was needed.

Building our own WordPress-like admin panel

So, once we were into the operations phase, we decided to build an admin panel we could edit from the screen, like WordPress. We consolidated the four separately built sites into one Content Management System (CMS), so we could run their contents from a single admin panel. With that, day-to-day updates could be done with screen operations, without asking the AI.

This CMS we built ourselves has become our current product, Plovant (https://plovant.com ). It isn't in service yet, but it has a mechanism where AI runs SEO (for search engines) and AIO (optimization to be found by AI) automatically and on a regular schedule. I'll write about how the CMS itself is built in a separate article.

What we know now

We did not decide from the start to build a CMS. We migrated, we ran the sites, we noticed we were repeating the same work in four places — and only then did it become clear that we needed one admin panel to bring it together.

I'm glad we left the migration to AI. And I'm glad we made daily operations our own screen-editable system. Treating these two as separate things turned out to be the right call — that's what I felt from actually doing it.

Cross-reviewing the same deliverable with Claude Code and Codex

uehara — Tue, 08 Sep 2026 11:29:10 +0000

Conclusion

I had Claude Code and Codex build the same thing separately, then review each other. For the same deliverable — this blog's table of contents, its primary sources, and a sample article — I had two AIs build it independently, then cross-review across five categories: factual errors, missing evidence, secrets, prose, and medium-fit. A human makes the final call.
The two AIs found different kinds of defects. Codex caught misreadings of what numbers mean: "$15.76 cannot be asserted as the real bill," and "a 100% match rate is not proof that the LLM (Large Language Model, a model that generates responses from large amounts of text) is correct on its own." Claude Code caught that the model's own size, stated as "90GB," was actually about 47GB, and found a Slack workspace name and an absolute path left inside an HTML (HyperText Markup Language, the language that describes the structure of a web page) comment.
Even so, human review cannot be skipped. If both AIs share the same error, cross-review will not catch it. And the business decision of whether something is safe to publish still remains.

Body (about an 8-minute read)

Don't trust either AI on its own

The articles on this blog are built by two AIs — Claude Code and Codex — that both use an LLM. From the start there was one premise: never take either AI's output at face value.

The reason is simple. An AI tends to overrate what it wrote as "correct" and "ready to use." Have the same model look again, and it reads back with the same assumptions it wrote with. The same blind spots survive.

So I split generation and review across different models. Two AIs build the same deliverable independently and review each other. A human decides last whether it is safe to publish externally. That is the three-stage setup.

I once mentioned, in a single line in another article, that "I have one AI doubt another AI." This article opens up what was inside that one line — how the work is categorized, who is made to look at what, and what the two AIs actually flagged.

Decide roles and categories first

The target is the same deliverable. The sample I ran it on this time was a hands-on account of two things: tallying cloud costs, and operating a local AI that runs on a GPU. The dollar figure ($15.76) and the model sizes (72B, 90GB) that come up later are all content from inside that sample.

Let the AIs review however they like and the findings never line up. So in the production plan (PLAN.md) I wrote, up front, who looks at what and how findings are categorized.

# Cross-review role split (from PLAN.md)
Codex      : article structure & readability / claim-to-evidence mapping / misreadings of numbers / spin-off ideas per medium
Claude Code: consistency with the repo's latest state / gaps in ADRs, work logs, observations /
             technical errors / project-specific disclosure risk
Human      : what is OK to publish externally / whether customers, pricing, incidents can be disclosed /
             paid vs. free split / final title and where to post

# Sort review findings into 5 kinds
factual error / missing evidence / secret / prose / medium-fit

Here an ADR (Architecture Decision Record) is a document that records a design decision. These five categories paid off. Sorting each finding into "the fact is wrong," "the evidence is thin," "a secret leaked," "the prose is poor," or "it doesn't fit the medium" lets you line up, afterward, which AI tends to catch which kind of defect.

What Codex found (verdict: NEEDS_REVISION)

Reviews were kept not as chat but as structured documents a machine can process. Each document carries a reviewer name and a verdict, and findings are tracked in a correspondence table (ID, severity, owner, resolution, status).

The verdict was "needs revision." The table held ten items. The main findings against the Claude Code version were:

[P0 secret] Three real screenshots in the sample article still showed the internal network, the scale of operations, and per-project uptime. The point: an API (Application Programming Interface, the defined way software components exchange input and output) key not surfacing under image OCR is not the same thing as the image being safe to publish.
[P1 factual error] The article asserted "$15.76" as "the real cloud bill," but the implementation computes it from registered unit prices, and when a model with an unknown unit price is mixed in, it returns a lower bound covering only the known part. The nominal converted figure and the actual bill do not match.
[P1 factual error] It summarized a 100% match rate in the purchase-decision Proof of Concept (PoC, running something for real to confirm the idea) as "the LLM being correct on its own," but the primary sources show the LLM's JSON (JavaScript Object Notation, a text format for structured data) formatting succeeded 69% of the time and fell back to an alternate path 31% of the time. Zero dangerous-side errors happened because a deterministic final-stage safety rule (a fixed rule that overrides the LLM's output) absorbed the broken LLM output — not because the LLM was flawless.
[P1 logic] It said the 72B (72-billion-parameter) model was routed to a work lane (a processing track that assigns tasks) "because its quality is superior," but on the generation benchmark 14B (14-billion-parameter) and 72B tied on success rate, 72B was about six times slower, and the conclusion was "do not promote it." The real reason it was adopted was to use an existing asset as a free, low-risk lane.

What Claude Code found (32 items)

Next, Claude Code reviewed the Codex version. This is the second stage. It checked six angles in parallel — benchmark numbers, memory and routing, the measured 50.3% throughput, table-of-contents coverage, secrets, and prose and medium — and re-verified each finding against primary sources, using 18 agents in total. There were 32 findings: 3 factual errors, 4 secrets, 10 missing-evidence items, 11 prose items, and 4 medium-fit items. For the 12 items with medium-to-high impact, the grounds were re-confirmed. 77 items were confirmed as having no problem.

[factual error] "The 72B model itself is about 90GB" is wrong. 90GB is the resident value with a 32k-token context loaded; the model itself was about 47GB. Within the article, the "model itself" field and the "including context" field both read 90GB, which is internally inconsistent.
[secret] An HTML comment in the body (it stays in the Markdown source but is not shown on screen) still held a Slack workspace name, a permalink to a channel, and several local absolute paths. Even if they are invisible on screen, they are readable from the published Git repository and the Markdown source.
[missing evidence] The generation benchmark's denominator n=7 (a sample size of 7) was absent from the body; local 50.3% and cloud 48.2% sum to 98.5%, and the remaining 1.5% (unknown, the share that couldn't be classified) went unexplained; and projects with high observation counts were missing from the table of contents. It raised ten such evidence gaps.

Even under the same category name, the two AIs found different things. Both flagged "secret," but Codex found internal information captured in a published image; Claude Code found absolute paths and a Slack name buried in an HTML comment. Both flagged how 72B was handled, but Codex flagged the logic — "the stated reason is the opposite of the benchmark" — while Claude Code flagged the number, "90GB versus 47GB."

The difference the two models flagged this time

A single reviewer leaves that reviewer's own blind spots in place. The Claude Code version's sample article had a strong narrative and plenty of real screenshots. But it "put a real dollar figure in without vetting it" and "generalized beyond what had been observed." The Codex version handled the meaning of numbers and disclosure risk rigorously. But it dropped the later developments and buried evidence inside HTML comments, mixed into the body.

For this deliverable, Codex flagged the meaning of numbers and disclosure risk; Claude Code flagged consistency with the repo's latest state, dropped observations, and secrets inside HTML comments. That the two models flagged different kinds of defects is as far as this record lets me claim.

I have not run a controlled comparison against self-review, so I cannot assert that a different model finds more defects. What I can confirm is that this time Codex pointed out that "OCR not surfacing a key is not the same as being safe to publish" — an angle distinct from the side that generated the work.

The limits of cross-review

There are three limits.

First, cross-review is not exhaustive. The Codex-side review itself states its scope limit: "I did not re-verify every individual number across all 96 article candidates against the originals. I prioritized the sample article and the publishing pipeline."

Second, if the two AIs share the same error, cross-review will not catch it. That is why I kept the final human review — what is OK to publish externally, whether customers, pricing, and incidents can be disclosed — as a mandatory third stage.

Third, the theme taxonomy used to sort the articles diverged between the two. This is not the five review categories (factual error, missing evidence, secret, prose, medium-fit); it is a separate scheme for what themes the article catalog is grouped by. Because each version built that theme taxonomy separately, the Codex review records that the Claude Code version's six English categories and the Codex version's seven Japanese categories did not match. Reconciling them required a human ruling.

The human judgment remains.

As a byproduct, this cross-review has fed several corrections. Findings are put into a correspondence table (ID, severity, owner, status). The public tree and the non-public evidence are physically separated. Top articles get a per-claim evidence ledger. This very article you are reading went through the same process.

From running this, three things became clear:

Use a different model for generation and for review. This time, different models flagged different kinds of defects. That said, whether that catches more than self-review is something I have not compared.
Keep a shared taxonomy, but let each AI go deep in its strong area. Even under the same category name "secret," one finds a published image while the other finds an absolute path in an HTML comment. Line them up on a shared axis like the five categories and that difference becomes visible.
A human decides publishability last. Errors the two AIs share, and the business judgment of whether it is OK to publish, are not filled in by AI review alone.

Aggregating scattered dev-infrastructure state into a single dashboard

uehara — Tue, 08 Sep 2026 06:24:13 +0000

The short version

The state of my company's development infrastructure used to live in separate places. The continuous-integration runners, the security scans that fire on every code push, the vulnerability ledger, the metrics-and-logs monitoring, the 24/7 high availability that keeps everything up when any node dies, the local LLM running on a GPU node — each one had its own place to look and its own way of looking. I aggregated all of that state into a single dashboard.

That dashboard is Infra Portal. But the thing I most want to convey in this article isn't the screen itself. It's a way of thinking: what matters in dev infrastructure is not "where you run it" but the mechanism of "choosing the information you decide on, gathering it into one dashboard, and backing those numbers with tests on real hardware."

Here are the key points up front.

What I aggregated is a real platform — Infra Portal only renders the display; behind it run my own CI/CD, security scanning, monitoring, high availability, a local LLM, and infrastructure-as-code.
The screen returns a verdict, not raw numbers — not "12 of 12 runners" but "is capacity short right now, or is routing misassigned?" The screen takes on the work of interpreting the numbers so a human doesn't have to.
The passing line is a real-hardware test, not a design doc — I can only say high availability is "achieved" when I actually kill one node and everything survives. Even with every design review and unit test green, there were bugs that only showed up when I dropped a node for real.
Both the monitoring and the safety mechanisms grew after the fact, incident by incident — I didn't design this perfectly from the start. Every time something broke, I added the one thing I hadn't been watching. What I have now is the accumulation of that. This article traces those additions by the date of each failure.

It's about a 20-minute read. It's long, so feel free to jump to whichever layer interests you (CI / security / monitoring / high availability / local LLM).

The body (about a 20-minute read)

What runs behind the dashboard

This is not "I moved GitHub Actions to a home server to cut costs," and it's not "a dashboard that just collects scattered information." A single repository (an infra/ops monorepo) holds all of the following, living together:

My own CI/CD runners — running GitHub Actions on my own servers
A security-scanning platform and a vulnerability ledger — inspecting for vulnerabilities on every push and aggregating results into a ledger
Fully self-hosted monitoring — metrics, logs, and traces, held entirely by me with nothing outsourced to a SaaS
24/7/365 high availability — if any one node dies, the rest keep every service alive
A local LLM — models running on a GPU node, monitored, routed by purpose, and used in actual development
Infrastructure as code — reproducing a second (and later) server wholesale with one command

In numbers: this repository has 62 architecture decision records (ADRs), about 930 commits, and covers 7 hosts in total — 2 CI servers, 1 GPU node, and several Macs. I've written down architecture decisions 62 times because if I don't leave "why I did it this way" for my future self (and whoever eventually takes over), I won't be able to read my own work six months later.

Infra Portal is the dashboard that aggregates this whole thing into one. It aggregates 26 products across 10 tabs, and it opens on an internal subdomain (inside Cloudflare Access — unreachable from outside). It's a hand-written static page with no framework and no build step: about 3,900 lines of front-end script and about 1,800 lines of API server. Changes ship by copying files, not by building. To keep grasping the whole thing on one screen, this kind of simplicity breaks less.

Why I built it — gathering decision inputs into one place, riding on the CI server

The company I run stands on "run every business function with AI." Humans concentrate on judgment; the actual work moves to AI. For that, the material you decide on has to be right in front of you, with the fewest steps. If I'm SSH-ing into dashboards all over the place to scrape numbers together, I spend more time on work than on judgment. So I needed a dashboard that gathers the state of my dev infrastructure into one place and returns not raw numbers but "what's the bottleneck right now." I didn't stand up a new box for it — I rode it on the local server where CI was already running, because that was simply more efficient.

The trigger was a mundane cost cut. I wanted to reduce the bill for GitHub's hosted runners, so I stood up my own runners on a home server. But that kept exposing operational gaps, one after another. A runner ran away and ate the whole server. A security gate turned out never to have run at all. A spot I thought I was monitoring turned out to be one I "wasn't watching" — I only learned that when it broke. One machine going down took a service with it. The back half of this article is the record of fixing those gaps one at a time. I'm not dismissing the cost cut itself. On the contrary, the essential operational gaps came out of it, and fixing them is what grew this into a platform.

Triage — telling causes apart without SSH or LAN-only Grafana

The most important design in Infra Portal is that panels return a verdict, not raw numbers.

Take a stalled CI build. An ordinary dashboard lines up numbers — "3 queued, 0 runners busy" — and leaves the rest to a human. Portal is different. Looking only at two axes, queue and idle, a pure function decides:

queued, and idle is 0 → capacity (not enough machines; add runners)
queued, but idle exists → routing (there are machines that could run it but the work isn't flowing; suspect a label mismatch)
queued, and 0 machines running → offline (no executor at all)
nothing queued → healthy

And at the very top of the screen it says, in one line, "capacity is the bottleneck right now." This decision logic is pinned against regression by 16 tests, confirmed not to break as I vary the combinations of labels and queues. Grafana is only visible on the internal LAN, but even on the road, this dashboard alone lets me tell "should I add runners, or fix the config?" apart.

Portal's settings tab has one more thing I was strict about: it never displays a secret. For a linked service's token or URL, it shows only a diagnosis — "does it exist," "do I have permission" — never the value itself, which can be neither displayed nor entered. Not putting secrets on screen even when I'm the only viewer is a structural way to prevent screen-share and screenshot accidents.

Layer ① CI/CD runners — closing the gaps that going free opened, failure by failure

On my own runners, every single safety setting is there as a response to an accident that already happened.

The first accident was a runaway. Back when I hadn't set a CPU ceiling, one runner for lightweight jobs monopolized 8 cores and ballooned to 830% CPU usage. One container running away drags the whole server in with it. That's when I started enforcing per-container CPU and memory ceilings at the kernel level. Cross the ceiling and only that container gets throttled; its neighbors are fine.

Today the runners are split into 6 role-based pools (light, security, browser, Docker, cost, backup). Only the Docker pool is handed the Docker socket. Every runner is ephemeral — thrown away after a single job — isolated with tmpfs and no privilege escalation. On a 16-core CI server, 13 containers run in the standard configuration, and in this area alone I've stacked up 8 architecture decisions.

I build the runner images myself, too. Once, GitHub's code-download infrastructure had an outage and Actions checkouts failed all at once. So I baked the commonly used dependencies (runtime, package manager, cloud CLI, browser automation, IaC tooling) into the image ahead of time and hold my own caches. So that builds keep going even when something external is down, the cache layer is "pass through if you can't fetch (fail-open)," while the security-relevant dependency mirror is the opposite — "stop if you can't fetch (fail-closed)." I change the behavior by nature.

Onboarding CI to a new repo is self-service from Portal. Edit a runner's count or CPU ceiling on the screen, and the change to the configuration-management (Ansible) inventory becomes a pull request automatically. Instead of hand-touching a server, it always lands as a code change in the history. The fewer people you operate with, the more that "you can trace it later" property pays off.

Layer ② Security — the gate that kept failing 100% of the time

This is the failure I should write about most honestly.

Up to a certain point, pull requests had a security-check step, and green checks lined up. I believed I was protected by it. But the reality was that the step was invoking a subcommand that didn't exist. Because the command didn't exist, it failed with an error every time, regardless of the code's contents. Even a docs-only change went red. And the same broken line kept running across all 29 repos where it was installed.

So it wasn't "green = safe" — the gate had never once held. Red became normal, and people started manually ignoring the red and merging. This is the worst state a gate can be in. It's worse than not existing, because it creates the illusion of "being protected."

I recorded this failure as an architecture decision, deleted the broken gate, and rebuilt a meaningful one in its place. It now runs in three lanes: a "stop only newly introduced issues" gate for pull requests, re-ingestion on push to main, and a daily scheduled scan. Four open-source scanners (static analysis, secret detection, vulnerability scanning, dependency audit) run on my own runners, so the SaaS bill is zero.

The pull-request gate has a trick to it. An existing repo always has a backlog of findings piled up from the past. Block all of them and no one can merge. So I take the before/after diff of the pull request and fail only what the change newly introduced, while showing the existing backlog non-blocking. Stop new defects; make old defects visible — both at once.

The results aggregate into the vulnerability ledger (DefectDojo). That ledger is LAN-only, unreachable from the internet. I keep it as an audit-trail set with ISO 27001 and SOC 2 in view, and I triple its backups: a daily dump → off-site storage → a daily restore into a standby DB.

Layer ③ Monitoring — the spots I wasn't seeing, I only noticed when they broke

The monitoring stack brings up Grafana, Prometheus, Loki, Tempo, Alloy, various exporters, and alert management from a single definition, with every image version pinned. On top of that, I keep a separate full-text search stack for cross-cutting search by request ID or trace ID. There are 24 alert rules in 8 groups and 5 dashboards.

But this configuration wasn't designed from the start. You can't know in advance "what you should be monitoring." This platform's alerts are the trace of adding, after the fact, "the metric I wasn't watching at the time" on each real incident. It's something like a museum of monitoring.

The one that hurt most was on 2026-08-07. A split in the synchronous disk replication (DRBD) went unnoticed by anyone for 18 hours, and during that time the active role bounced back and forth 4 times. The standard exporters weren't watching this sync state at all. So I turned DRBD's state into metrics with my own exporter.

I have 5 home-grown exporters that fill the monitoring stack's blind spots (disk sync, external probing, fencing, backup, runner exhaustion) — the spots the standard exporters don't see (the GPU/model-server exporters below are a separate set). And I've put one important design into them: on a collection failure, stop emitting the metric rather than leaving a stale value (fail-loud). Further, each exporter has a paired alert for "has the exporter itself gone stale?" Monitoring the death of monitoring. It's the monitoring of monitoring, to prevent "I thought I was watching, but the monitoring itself had stopped."

There are 2 CI servers, and both Prometheus instances collect independently of each other. If one dies, the other keeps monitoring. Monitoring, of all things, must not be a single point of failure.

Layer ④ The mechanism that never stops for 24 hours — even with every test green, it isn't achieved yet

The mechanism that keeps running no matter which node dies was the hardest area to judge in this whole platform.

Let me state the principle first. The passing verdict is not a design doc or a unit test — it's a kill-test that drops one node on real hardware. Stop any one node, and every endpoint responds on the surviving node, no data is lost, no double write (split-brain) occurs, and it recovers properly. Only when that goes green on real hardware do I call it "high availability achieved."

I learned this principle from a brutal failure. It began with a split-brain incident that ran for 26 hours. Both nodes became active (and both were returning 200 on health checks, no less), two workers fought over the same file-backed queue, writes were truncated mid-way, and job data was corrupted. Health checks green, data corrupt inside. A surface 200 is no proof of health.

As the fix, I consolidated the sole backing for "who is active" into a single lease. Only the lease holder can be active. It's a scheme that makes dual-active impossible in principle by design. The disk aims for zero loss through synchronous replication, and the GPU node acts as a third voter so a split is decided by 2/3 majority. The edge side, too, has a doubled tunnel that switches over automatically.

And then the real-hardware kill-test. In the test on 2026-07-12, I force-stopped the active CI server container-and-all, and the standby switched to active automatically in 18 seconds with zero human involvement. The stop and the confirmed-up times are kept as evidence in real UTC timestamps. In the same test I confirmed self-fencing and takeover from a network partition, agreement of the data diff, and fail-back.

One honest caveat here. This 18 seconds is a measured value on the NFS-based configuration of the time (2026-07-12). I later migrated the lease foundation to DynamoDB on 2026-08-22, so the old configuration's 16–18 seconds does not carry over to the current one as-is. Keeping the number as "a record of the configuration at the time" and not asserting it as a current value is a discipline I hold on this platform.

One more honest note. There were 3 bugs that the design and unit tests couldn't detect and that appeared only in the real-hardware kill-test: a livelock during the grace period, a problem where a hard-mount's I/O blocks infinitely so self-fencing misfires, and a problem where blindly grabbing the lease rolls the generation number backward. All were green on paper. They only came out when I dropped a node for real. That's why I say "until you drop it for real, I don't call it achieved."

And I'll write the limit that still remains. In the reboot-involving kill-test on 2026-09-03, I measured that automatic recovery fails. The causes are startup races and an asymmetry under packet loss where "reads succeed but acquires fail." I've recorded this honestly as an architecture decision and put it on the ledger as a weakness I haven't fully fixed. High availability isn't a binary of "achieved / not achieved" — it's the work of counting weaknesses with evidence and clearing them incrementally. In this area alone there are 196 automated tests for real-hardware behavior.

Layer ⑤ Local LLM — a GPU that is both the monitored and the worker

On the GPU node I run a local LLM. It has two faces here. It is the monitored side and, at the same time, the side that actually works.

I wrote 2 exporters for monitoring myself: about 100 lines for the GPU and about 230 lines for the model server. Both use only the Python standard library, so they need no extra install. On a platform operated by a small number of people, not adding dependencies is itself availability.

I also face the not-so-clean parts head-on. This GPU has a unified-memory configuration, and the standard GPU-monitoring command won't return the model's memory usage. So I made the value the model server's API returns the source of truth. The model server has no metrics-output port, so I pick up the processing-time lines that appear in the logs with a regex and measure tokens generated per second for real. If there's no clean API, measure from what you have.

I route by purpose, too. Light purposes like classification go to one endpoint; heavy purposes like writing code go to the GPU node. The classifier side has passed the HA kill-test as well — I confirmed on real hardware that dropping one Mac switches transparently to the other.

And here's the most symbolic part. The large model running on this GPU actually writes commits to this infrastructure's repository. As the execution backend for automation, the AI maintains its own infrastructure. In the log, 5 commits remain that you can tell were written by the AI, by their prefix. The company's thinking of "run every business function with AI" appears here at once, both in a glowing card on the monitoring screen and in the commit history.

Reproducing everything from code, so it continues even when the person changes

Anything operated by a small number of people stops if that person falls. So I was thorough about "reproducible from code." The thoroughness comes in two stages.

Stage one is building the second and later servers. One configuration-management command brings up every service. The configuration roles (24 Ansible, 17 playbooks, 14 Compose service layers, 11 Terraform stacks) all derive from a single source-of-truth file that lists 25 products. The catalog shown in Portal is a derivative of that source, too. Because there's a single source, discrepancies don't arise.

Stage two goes further. It's a "clean-room export" that lets me export this whole platform safely without leaving any company-internal values. Even as nodes increase (ci3, ci4…), or when someone takes over, it can be handed off safely by the same procedure. It makes a copy with the history zeroed out, keeps only what's needed via an allow-list, mechanically substitutes company-internal values (domains, real IPs, real names, colleagues' names, task-tracker IDs), and finally runs it through a secret scan. If that scan catches even one item, it exports zero bytes (fail-closed). It's a one-way pipeline that doesn't leave unless it passes both gates. The same thinking as the security lesson in the earlier chapter runs through here.

Backing the numbers on real hardware — the kill-test I keep as a record, and the hard cap on cost

Let me return once more to this article's core. A number only becomes a number once you back it on real hardware.

The HA kill-test becomes the record directly. When, which node, dropped how, switched over in how many seconds. Kept with a UTC timestamp. Not "probably fine" but "on this day, on this configuration, it was this many seconds."

Cost is the same — held down by measurement and structure. The log platform stores the full volume on AWS, but I put a 10 GB (about 10.7 billion bytes) scan cap per query on the search platform, enforced so the user side can't lift it. Billing is metered per TB, so however large a single query is, it converges to about $0.05. I designed "store," "search," and "ship" as separate mechanisms: store long-term on S3, search short-term on the local search platform, and ship sorted by importance.

Honest here too. The policy says "forward only important logs," but the running forwarder configuration actually ships container logs and system logs in full, with no filter. There's a gap between policy and current implementation. When you match numbers against design on real hardware, discrepancies like this always surface. Measuring for real is how you find and fix them.

Updating what kinds of information you keep and how you verify them — scripts that do nothing without certainty

Finally, this is about operation that doesn't cause accidents even when no human is glued to it.

The scripts that run automatically share a common principle: do nothing without certainty. Destructive operations default to a dry run, place a two-stage grace period, and require typing an uppercase "YES" to execute.

Take disk cleanup. On 2026-07-02, a CI server's root filesystem filled to 97% (the running runners' write layers summed to over 60 GB), and out of that the disk-guardian script was born. This script cleans only when a threshold is crossed, and it absolutely protects any runner holding a running job. Deleting a live job just to clean up would defeat the purpose.

The runner reapers split into 3, dividing responsibility strictly. The first is the disk guardian, protecting live jobs. The second deletes only the offline ephemeral ghosts. The third reaps deadlocks that look like they're running but have actually frozen. Even the same "broken runner" has a different owner depending on its state. Making one script do all of it makes the collateral damage large when it misjudges, so I split them on purpose.

The HA switchover verdict follows the same idea. When it can't decide "which one is active," it tips both to the stopped side. Stopping and calling a human is safer than forcing it into a double write. fail-safe — when in doubt, tip to the safe side — is something I hold to thoroughly.

Holding infrastructure with a small number of people might sound like the height of a bus-factor problem. But what I'm actually doing is the opposite: gather the decision material into one dashboard and return a verdict, back the passing line on real hardware, make everything reproducible from code, and have the scripts stop when in doubt — adding "hard to break" one piece at a time is the result. Each one of those pieces corresponds to a concrete failure date: the day it ran away, the day the gate wasn't running, the day 18 hours went unnoticed, the day data was corrupt for 26 hours, the day the disk hit 97%, the day a reboot didn't recover.

Infrastructure doesn't become safe because the design is beautiful. It becomes safe only by however much you drop it, break it, count the gaps you weren't noticing, and fix them one at a time with evidence. A single dashboard is the foundation for continuing that work.

How to take over a design built in Figma Make and develop it with Claude Code

uehara — Mon, 07 Sep 2026 03:21:45 +0000

From February to April 2026, I launched four web apps, each starting from a code bundle that Figma Make (Figma's AI feature that generates a working front-end code bundle from a design) had spat out: a beauty-curation site, a gift-record app, a plush-toy album, and a UI mock for an AI development tool. Every one of them starts its repository in a state where "the look is already finished." In this article I look back — from the actual config files and commit history — at what I did to get those generated outputs into a state where I could take over development in Claude Code (Anthropic's CLI coding agent) and start working on them, and at how far each of the four repositories progressed or stalled.

The starting point: what shape does a Figma Make output come in?

A Figma Make export runs as-is with npm run dev. The README tells the story.

# Beauty Information Curation Site

This is a code bundle for Beauty Information Curation Site.
The original project is available at https://www.figma.com/design/<id>/...

## Running the code
Run `npm i` to install the dependencies.
Run `npm run dev` to start the development server.

A README that says "the original lives in Figma." That symbolizes the character of the output: the code is a projection of the Figma design, and the code is not the source of truth. On top of that, if you look at package.json, every dependency is exact-pinned.

{
  "dependencies": {
    "next": "15.3.4",
    "react": "19.1.0",
    "react-dom": "19.1.0",
    "lucide-react": "0.487.0",
    "motion": "12.23.24",
    "tailwind-merge": "3.2.0"
  }
}

Fixed versions with no ^. As a snapshot of the moment it was generated, it is highly reproducible, but leave it as-is and it grows stale with no one ever updating it. There is no data layer either. The screens are pretty, but behind them everything is mock data — no persistence, no authentication. "It runs, but there is no foundation to grow it on" — this was the common starting point across all four repositories.

[画像: The README and package.json of a Figma Make output (screen reconstructed and anonymized; contents are as measured). The README states plainly "the original lives in Figma / this is a code bundle," and dependencies are exact-pinned like next 15.3.4. The code is a projection of the Figma design, and as-is the source of truth is not on the code side]（画像は別途ホスティング予定）

How far each of the four repositories got

Even from the same Figma Make starting point, how far I pushed toward a product varied with what each one was.

The UI mock for an AI development tool I stopped at a mock, as the name says. Its README stayed the boilerplate from generation time. Its purpose was "a look for reviewing the UI," and there was never any intention to turn it into a product, so I didn't force it to grow. Deciding to throw something away is a decision too.

The beauty-curation site went the furthest. It works as a static site, so I set Next.js (a React web framework) to export statically and put it on Amplify (AWS's web hosting service).

// next.config.ts — export as a static site
const nextConfig: NextConfig = {
  output: "export",
  images: { unoptimized: true },
  trailingSlash: true,
};

# amplify.yml — serve out/ as the artifact (no SSR)
frontend:
  phases:
    preBuild: { commands: [ "npm ci" ] }
    build:    { commands: [ "npm run build" ] }
  artifacts:
    baseDirectory: out
    files: [ "**/*" ]

With output: "export" I made it a pure static site with no SSR (server-side rendering), and with images.unoptimized I also dropped Next.js's image-optimization Lambda. Do this and Amplify just serves out/, which makes operations very light. In March I switched the repository's deploy method to "automatic deploy via GitHub integration," carrying it from a generated output all the way to a "site that gets served."

The gift-record app and the plush-toy album had dense product specs, yet stalled at the data layer. The former's README spells out the MVP (Minimum Viable Product) scope concretely — dashboard, people management, gift records, calendar, anniversary management, and five business-facing screens, all implemented in Atomic Design (a UI-component method that builds screens up from small parts). But at the top of the "deferred / undecided" list, it says this:

Actual data persistence (currently mock data)

Authentication and user management

Notifications

The look and the screen transitions are finished, yet the widest gap of all sits between the mock data and a real backend. The plush-toy album, too, was built up to Vite (a fast front-end build tool) + React 18 + a five-layer Atomic Design, and even had its design philosophy nailed down (a gentle earth-tone palette specified as concrete values) — but it likewise stalled before persistence.

[画像: How far each of the four Figma Make-based repositories got (screen reconstructed and anonymized; states are as measured). The UI mock stayed a mock, the beauty-curation site reached static hosting, and the gift-record app and plush-toy album stopped just short of the data layer]（画像は別途ホスティング予定）

Putting the outputs under governance in "one place"

If you grow four repositories separately, CI (continuous integration; running tests automatically on each change), security, and quality all end up scattered and inconsistent per repo. So at the end of June I put a group of repositories including these ones onto the same self-hosted security CI all together. The traces remain in the commit logs, in the same form across every repository.

chore: onboard ELN self-hosted security CI (#1)          (2026-06-27)
chore(dependabot): group+monthly to cut hosted CI cost   (2026-06-27)
fix(security-pr): rename sticky comment script to .cjs
    for ESM repo compat                                  (2026-06-29)

That last rename to .cjs addresses a problem where the comment script posted on security PRs couldn't be loaded in ESM-configured repositories ("type": "module"). Figma Make outputs assume ESM (ECMAScript Modules, the modern JavaScript module system), so placing a CommonJS (the older Node.js module system) script as a plain .js makes it fail with require is not defined-type errors. Because a whole group of repositories hit this at once, I renamed them to .cjs across the board to fix it. "Turning an output into a product" is not only about adding features — it is also about putting them onto this kind of governance foundation and getting them in step.

[画像: The shared commits left in the same form across the four repositories (screen reconstructed and anonymized; commits are as measured). I onboarded them all at once onto the same self-hosted security CI, and fixed the problem of CommonJS scripts not loading in ESM repositories with a cross-cutting rename to .cjs]（画像は別途ホスティング予定）

Why an output as-is can't be grown

The reasons sort into three. First, where the source of truth lives. As the README says, "the original lives in Figma" — the code is a projection. The moment you start editing the code here, Figma and the code become double-managed, and unless you decide which is authoritative, a regeneration will overwrite your work by accident. So the first step toward productization was to make up my mind: "from now on, the code is the source of truth." Second, frozen dependencies. Exact pins are righteous at generation time, but with no one owning updates they rot. Only once you add a lockfile and CI do you reach a state where you "can update." Third, the missing data layer. The output is closed around its look and its mocks, so the "backbone of a product" — persistence, authentication, notifications — is entirely untouched. Even when the look is 90% done, this backbone, which looks like the remaining 10%, actually accounts for the bulk of the work.

Where things stand and what's left

The endpoints split cleanly. The mock I decided to discard stayed a mock; the site that works as static reached hosting; the product that needs a data layer marks time just short of the mock data. And every repository did get onto the shared CI and security gate. "With Figma Make the look stands up in a day" is true, but misjudge what comes after it — handing over the source of truth, thawing the dependencies, the data layer — and something that looks "almost done" can stop moving for months.

Lessons you can carry over

The first step in turning a design output into a product is deciding "the code is the source of truth." As long as the original lives elsewhere, the more you edit, the more double-management accidents you invite.
For anything that works as static, use static export (output: "export") to keep serving it light. If you need neither SSR nor an image-optimization Lambda, don't carry them.
An exact-pinned output only becomes "updatable" once you add a lockfile and CI. Leave it alone and it grows stale bit by bit.
Don't be fooled by how finished the look is. The foundation — persistence, authentication, notifications — looks like the remaining 10% but is the bulk of the work. Recognize that a wide gap sits here, and scope accordingly.

Hand one person's good way to the whole team — team development in a new era

uehara — Sat, 05 Sep 2026 05:54:27 +0000

What I wanted was not a system that only works when it is written down. What I wanted was a team that ships the same quality. While using AI, company-wide.

Someone comes up with a good practice. Someone sets a standard. Someone builds discipline. And then it stays with that one person. A common story. But if it stays there, there is no point in developing as a team.

One person builds a good way of working. That way reaches every project. It reaches every member. Only then does it become the team's value. In the age of AI, I believe this "spread" is what value looks like.

This article is about how we turned that spread into a mechanism.

What I wanted: a state where everyone ships the same quality

Why not documents?

Documents are kept only by the people who read them. People who do not read them do not keep them. A style guide sitting on a shelf moves nobody's hands. What I wanted was not more documents. It was the same quality, reproduced in everyone's hands.

Left alone, good practices become personal property. One person's project is careful; the neighboring project is sloppy. Same company, uneven output. One side writes tests; the other does not. One side keeps evidence; the other runs on gut. This is a loss for the team. The effort of the one person who built a good way stops with that one person.

For a long time, the assumption was: write it down and it will be kept. But documents alone are not kept. They are kept only when they are built into the working steps. So I decided not only to write, but to distribute. To every project, we distribute the same spec process. The same quality standards. The same operating discipline. The distribution channel is Claude Code. We built it as an in-house plugin platform, used only by our company. A plugin is a component you attach to a tool afterward to add capability.

"Distribute" may sound like pushing from above. It is the opposite. What gets pushed is rules nobody can keep. What gets distributed is a way of working that someone already used on the ground, and that worked. Not an ideal decided in a meeting room. Only the practices that were actually used, and turned out good, get distributed. So they do not float above the floor. They fit the user's hands. Distributing what worked in reality, not what sounds ideal — that difference decides whether it lasts.

Start from "looks convenient," stop with evidence

Why did we start using AI in the first place? The reason is simple. It looked convenient.

We did not start from a difficult theory. It looked faster; it looked like it would free our hands. That was the level of motivation, and I think that is fine. You do not need a cool reason. But if you run on "looks convenient" alone, there is one pit waiting.

AI says "done." Smoothly, and with full confidence. And that "done" is sometimes not to be trusted. It says something works when it does not. It says something is fixed when it is not. So many tests passed; the build succeeded; the state is such and such. It reads those proxy numbers as completion. Without looking at the evidence, the report runs ahead. Leave this alone, and quality actually drops. You just make mistakes faster.

So I decided not to believe this "done." Instead of believing, I look at evidence. I look at logs. I look inside the database. I look at test output. Only what was actually observed counts as done. Counts and statuses are not substitutes for completion. I wrote about this in the published article "AI's 'done' cannot be trusted — a quality gate that stops false completion with evidence."

Why do numbers fool us? Because numbers look exactly like evidence. Hear "a hundred tests passed," and you relax. But those hundred may not touch the one case that matters. A successful build does not mean it works. Numbers reflect a slice of the state. They guarantee nothing about the whole. So I stop just short of the number. What did this number actually verify? Only after that question does it become evidence. Looking reassured and having verified are different things.

"Done" does not pass without evidence

Saying "stop it with evidence" is easy. Words alone are not kept. People hurry. AI wants to move on too. So we made it a shape where you cannot move on without keeping it.

Concretely, we made it a skill — a small executable procedure. A skill is a small set of steps that works automatically inside AI development. When something tries to declare completion, this procedure cuts in. Are there logs? Are there database records? Is there test output? Declare a high-risk completion with none of them, and it stops right there.

The stopping is the point. If it only prints a notice, people skim past it. A yellow warning is invisible to hurried eyes. Even if seen, if you can still move on, the false completion flows downstream. The next person builds on top of it. Rework costs more the later it is found. So we made it impossible to move on. Until the evidence exists, you cannot enter the next step. Quality changed from a verbal promise into a gate you cannot pass.

Some people find this stifling. But with the gate, you actually move faster. You can look only forward, without worrying that things will collapse later. It is not a restraint. It is the ground that lets you move fast with peace of mind.

What is evidence, concretely? It differs per task. If you changed data, it is the database itself, after the change. If you fixed a screen, it is a record of actually opening the fixed screen. If you ran a process, it is the log that process emitted. What they share: they sit on reality's side, not on the report's side. It is not a person saying "I did it." It is reality showing "this is how it is." The gate passes only that reality. It does not pass words.

One person's way, distributed to everyone

One gate alone is still one person's effort. Here is the main point.

One person builds a good way. They register it on the platform. Then it is distributed to every project via Claude Code. The next person who starts a new project has that way from the start. No need to reinvent it from zero. The discipline a teammate found yesterday works in your hands today. On the receiving side, the only operation is a single approval, once.

This is different from training. Training spends the time of both the teacher and the learner. The more people, the more it costs to convey. Word of mouth changes shape with every retelling. A distribution mechanism is different. Build a good way once, and the platform distributes it. Headcount grows; the cost of conveying does not. The shape does not drift. One person's good judgment becomes everyone's starting state.

Think about a new joiner. Install the platform on day one, and the company's quality standards work in their hands from that day. No months of watching a senior's back to absorb how things are done. Good ways stop being something you memorize; they are something you start with. The loss called "practices becoming personal property" is closed here. A good way someone found is not theirs alone. From the next day, it is everyone's. That is what I think team development means.

Good ways compound as they grow. Add one, and the next person to join gains one more. Add ten, and they gain ten. The later you arrive, the richer the ground you stand on. This does not happen when you work alone. Your improvement helps the project of someone you have never met. Someone else's improvement helps your today. This give-and-take runs automatically on the platform. The longer time passes, the thicker the ground gets. That is exactly why building as a team is worth it. The team becomes larger than the sum of its members.

This article was made by this very mechanism

Saying one thing and doing another — that is the worst. So this article itself was made with the same mechanism.

An AI wrote it. But we do not ship what an AI wrote as-is. We make another AI doubt it. Another AI asks back: are the facts right, are there grounds, are there leaps? The writer and the doubter are deliberately different AIs. Let the same AI write and grade itself, and the grading goes soft.

And the final decision belongs to a human. A human reads, a human judges, a human decides to publish. The AIs are crossed so that what one misses, the other picks up. On top of that, a human holds final responsibility. We borrow writing speed from AI. We borrow the doubting eye from AI too. But judgment is the one thing humans do not let go. This order holds for article-making as well. The very text that preaches the mechanism came out through that mechanism. To me, that is the evidence.

Why set up a doubter at all? This approach is called adversarial review. It reads what was written on the assumption that an error is hiding somewhere, and goes looking for counter-evidence. The reason to appoint a doubter is that the author cannot read their own text that way. The grounds you wrote look correct to you. Human or AI, it is the same. So the doubting eye comes from outside. The other AI does not know our circumstances. It does not defer. It calls the odd parts odd. We take the findings and rewrite. If it does not pass, we rewrite again. This back-and-forth raises quality. Passing on the first try is not the goal. The value is in not passing.

Quality is kept by mechanisms, not by attention

Finally, the claim I care about most. Quality is kept not by human attention, but by mechanisms. When we find a contradiction, we do not leave it alone — we turn it into a mechanism, on the spot, and close it.

I did not want more rules. I wanted quality to be the same in everyone's hands. Ideals do not spread as ideals. Good ways stop with one person while they stay personal. The word "done" cannot be trusted without evidence. Efficiency alone does not return time. Every one of these, left alone, stays a contradiction.

So we closed them one by one. "It does not spread" — closed with a distribution mechanism. Verbal promises — closed with a gate you cannot pass. False completion — closed with an evidence gate. The softness of self-grading — closed with another AI and a human's judgment. Saying it keeps nothing. Things are kept only when they are built into the steps.

There is one more good thing about mechanisms. You stop blaming people. When quality drops, hunting for a culprit prevents nothing next time. What you should hunt for is which step was missing. Find the gap, add a gate there. Then the same failure does not repeat from there. Fix the mechanism instead of blaming the person. I believe that is the way that lasts.

Contradictions do not disappear on willpower. They disappear through mechanisms. Turning one person's improvement into everyone's quality — that spread is the value of a team in the age of AI. That is why I built this platform.

Run your AI subscription 24 hours a day — use the quota you already pay for

uehara — Thu, 03 Sep 2026 00:25:35 +0000

Let me start with a question. Why did I fear development done by artificial intelligence?

The answer is plain. AI can build software, and on top of that, it never rests.

AI has no labor law

People rest. There are labor laws. We sleep at night. We need weekends. Work too many days in a row and the body breaks. So there is a ceiling on how much work a person can move forward in a day. For a long time, we treated that ceiling as a given.

But AI has no labor law. It works at night. It works on weekends. Give it an instruction once, and it does not stop until morning. It never says it is tired. It takes no breaks. It keeps working for hours at the same quality.

This difference did not fit inside the word "convenient." What I felt was fear. This was not a story about one more handy tool. It was a story about the ground under the speed of work changing at the root.

Claude Code came out about a year and a half ago. That is when I understood. The company that runs it 24 hours takes the first-mover advantage. And the company that can punch with money wins. This is not a cynical take. It is the obvious consequence.

The first mover wins — that story is not new. Whoever enters a market early takes the ground. They set the standard. Everyone after them chases the gap. AI widens that gap by the day. A company that moved ten hours forward overnight and a company that stood still overnight are ten hours apart by morning. The gap compounds daily.

Can you catch up by hiring more people? You cannot. Hiring takes time. Post the opening, interview, teach, wait for people to settle in. That takes months. Meanwhile, the other side's AI keeps moving through the night. The speed of adding people cannot match the speed of adding AI. So the moment a small company steps into a contest of headcount, it loses. It was a ring we should never have entered.

Companies that can punch with money win — obviously

Why can I say it becomes a contest of money? Because there is no ceiling on how fast you can add more.

The AI that does the implementation can be added endlessly, after the fact. Add one connection key — an API key — and you have one more worker, then another. None of the time that hiring takes. No job postings, no interviews, no training. Add a key, and it works through the night starting that day. A company with money can line up as many of these workers as it likes.

There is one more obvious fact. Companies sitting on surplus compute are strong. There are companies that stockpiled graphics processing units (GPUs) for Bitcoin mining. When the market drops, those machines sit idle. Idle machines can be turned to AI work. The side that owns them runs AI cheaper and longer than the side that does not. They have nothing left to buy.

Put these together and the picture is clean. Companies that can buy more AI with money. Companies that can run AI cheap on leftover compute. Companies with both push development through the night, and widen the first-mover gap every day. I found that picture terrifying. I was standing on the small-company side.

The one resource left to a small company: the unused subscription quota

Here, most people drop their shoulders. If it is a contest of money, a small company has no chance.

But I did not stop there. I asked the question again. Is there really nothing a small company is failing to use up?

There was. The subscription quota.

A monthly subscription comes with a usage quota. Most companies use that quota for a few daytime hours. At night, nobody touches it. On weekends, it sits still. The quota is there every month, and most of it gets thrown away. This is not a problem of having no money. It is a problem of not using what you already have.

Factories have a word for this: utilization. The same equipment pays itself off faster the longer it runs. A plant running at half capacity and a plant running flat out get different returns on the same investment. A subscription quota is the same. The payment is already made. Use it or not, the cost does not change. Then there is no reason not to use it. An unused quota is idle equipment. "What a waste" does not cover it. It is a reason you lose the competition.

When I saw this, the top priority fixed itself: use up the leftover quota. In other words, keep the subscription running 24 hours a day. Fail to do this, and you fall behind the world. You drown in the AI wave. Not drowning in that wave — that was the first reason I started building.

History teaches the same lesson. In the middle of the industrial revolution, cotton mills could only run by day. At night it was too dark to see the thread. Then gas lamps came in, and later electric light. Mills could run at night. And the mills that ran nights overtook the mills that ran only days. Same equipment. Similar headcount. The only difference was how many hours they ran. How many hours you run what you already own — that is what decided the contest.

Economics has a famous observation called the Jevons paradox. In 1865, William Stanley Jevons pointed out that as coal was used more efficiently, coal consumption went up, not down. When efficiency makes something cheaper, people use more of it. AI is the same. AI that has become cheap and useful is not something to cut back on. It is something to use more. So there is no reason to leave quota on the table. Leftover quota simply expires. It does not come back next month. Quota you did not use up is quota you threw away.

Judgment goes local; implementation goes to the cloud

That said, running 24 hours blindly is not the point. If it runs all night, mistakes pile up all night too. Misguided implementation can march forward in the hours nobody is watching. Close that hole, or morning brings nothing but regret.

Do not leave contradictions alone. Close them with structure. That is my number one rule. I do not believe in "we will push through on grit." If a structure cannot close the hole, that structure is unfinished.

So I built an AI development orchestrator that puts a local large language model (LLM) in the judge's seat. Its name is local-commander. Its role is not command — it is sorting. What gets processed where. It decides that first.

Every incoming task gets sorted three ways:

What can be done locally
What needs the cloud
What requires human approval

The sorting judgment is done by the local LLM. This is the crux. If every judgment went out to an external service, it would eat both quota and money. What can be decided locally gets decided locally. That is faster, cheaper, quieter. There is one more reason the judge sits at home: judgment happens many times. Put the highest-frequency step somewhere expensive, and it becomes your biggest burden. So I put the most-run judgment in the cheapest seat.

After the judgment, things just flow. Work that needs implementation goes to Codex or Claude Code. Steps that need no human eyes move straight ahead. Merging the change, opening the pull request, review, and syncing the task after merge — the whole chain is automated. Humans touch only what requires approval. Everything else, the structure carries to the end.

Let it work the night; stack only approvals in the morning

A day looks like this.

At night, I leave instructions and step away. The structure runs on its own. Locally doable work finishes locally. Work that needs the cloud gets implemented by Codex or Claude Code — push, pull request, review, merge, all the way through. Only the approval-required items stop, just short of the line. They hold there and wait for morning. Nothing moves past that point on its own. If it did, mornings would greet me with things I cannot take back.

In the morning, what sits in front of me is one kind of thing only: the items that require approval. Not the work that finished overnight. The one point only a human can decide. I look at just that, and answer yes or no.

With this, the quota works nights too. And in the morning, my job is judgment only. What comes back to me is not the work. It is time. This is exactly what I want from efficiency. Not speed itself — the time that returns. With the returned time, I think about what to build next. That is the part only a human can do.

Remember where this started. I feared that AI never rests. Now I hand the night to the AI that never rests. Its nature has not changed. What changed is one thing: how I use that nature.

No gut purchases: DGX Spark was decided on 16 measured cases

Decide to run 24 hours, and the next question arrives. Is the compute at hand enough? Running the judge LLM locally through the night takes real hardware. My candidate was the DGX Spark.

But here too, I did not buy on gut. "Looks good, let's buy it" is not a way I buy. To tell myself a purchase is allowed, I need evidence. The more irreversible the purchase, the thicker the evidence.

So I built 16 test cases and measured. I lined up the numbers and grounded the decision in them. I do not buy on impressions. I buy on measurements.

Why go this far? To keep the quality of judgment from sinking to the level of a hunch. I believe human work is judgment. Tasks can be handed to AI. But what to build, what to buy, where a human must stop — that judgment stays with people. If judgment drifts into hunches, the whole 24-hour structure rots with it. Build automation on top of a hunch, and everything above it is a product of that hunch. So the one-shot judgment of a purchase gets fixed in numbers, more than anything else.

Contradictions get closed with structure, not left alone

Let me answer the questions that always come back, before they are asked.

"If AI runs all night, can you keep the quality?" The sorting answers this. Steps that need human eyes get routed to approval-required. What is delegated is only the work that needs no human eyes. Not everything is delegated.

"What if it runs wrong until morning — isn't that an accident?" The stop point answers this. Approval-required items stop before implementation. In front of every irreversible step, the structure waits for a human. What moves at night is only what can be undone.

"Is a local LLM good enough for the judgment?" The division of roles answers this. The local LLM does light judgment: sorting. Heavy implementation goes to Codex and Claude Code. Hard approvals go to humans. The local seat holds only the judgment it can carry. It is never asked to overreach. That is why "not good enough" does not happen.

"Doesn't 24-hour operation cost too much?" The order of consumption answers this. What gets used first is the quota already paid for. As long as it runs inside the quota, no extra cost appears. Before the quota runs out, the local-versus-cloud sorting kicks in — the cheap stuff finishes locally. I do not start saving after the costs rise. I put the design that keeps them from rising first.

Every answer has the same shape. No difficulty is crossed on grit. Each is closed with structure. Find a contradiction, and do not leave it alone. That is the one thing this article most wants to say.

The concrete inside of this approach is in the related article, "Half of AI development moved to a local LLM — sorted by role." What sorting by role means, and how much can be trusted to local — that is where the details live.

One last thing. A 24-hour structure is not reserved for big companies. If anything, the small companies throwing away their quota have the most room to grow. You cannot punch with money. But you can use up the quota you are leaving on the table. You no longer have to be on the drowning side.

Running the whole company on AI: the big picture

uehara — Tue, 01 Sep 2026 19:37:55 +0000

Let me tell you our starting point, precisely.

Our beginning was not "people make the judgment." Our beginning was "do everything with AI." Deciding to become an AI company — that was the starting line. Run all of EarthLink Network's work on AI. That is what we put down first.

"People make the judgment" is something that was derived afterward. It is not the starting point. Get the order wrong, and the whole picture blurs. So first, I will fix the order. First, run all the work on AI. As a result, the routine work goes to AI. And judgment stays with people. This article is here to let you see that structure on a single page.

In the sister article, "The 18 products EarthLink Network builds in-house," I introduced the 18 products one by one. I listed what each product does, product by product. But just listing products does not show why there came to be this many. Seen one by one, they feel like a lot. Seen as a whole, they converge on a single aim. This article is for seeing that aim, and the connections between the products.

The starting point was "do everything with AI"

Why everything with AI? The answer is simple. Because it looked convenient.

We did not start from some grand ideology. The work in front of us — hand it to AI, and it is faster. Hand over what can be handed over. Keep only what cannot. We did not draw that line from the start. First we tried to hand over everything. We tried to run all the work on AI. In that process, what could and could not be handed over separated out, as real things.

"Because it looked convenient" does not mean sloppy. If anything, the opposite. Raise a grand mission first, and you start wanting to bend facts to fit it. Convenience cannot be bent. Did it get faster? Did the effort shrink? Did the person using it become more comfortable? You find out by running it. So we made convenience, not ideology, our measuring stick. Because the stick is honest, our judgment does not waver.

Isn't it reckless, people ask. But we think the reckless way is the opposite one. Draw the line "people here, AI there" first, and that line is decided by armchair hunches. A line drawn by hunch is usually off. People end up hugging work they could have handed over. They throw judgment they should never hand over at AI. So we did not draw the line by hunch. We handed over everything first, ran it, and waited for the line to appear on its own.

The result of that separation is "people make the judgment." This is not an ideology. It is a conclusion left over from trying to hand over everything, confirmed by running it. So we do not treat judgment as special up front. First, hand over. Hand over all you can, then look at what is left. The order is always this way.

"People make the judgment" is a conclusion, not a start

People often ask: "So you don't let AI make judgments?" But that is only half right.

We did not begin by deciding not to let AI judge. We began by deciding to do everything with AI. We ran it, and judgment alone stayed on the human side. The order is reversed. People who put "people make the judgment" at the start usually do not hand over the work. They leave AI aside, as an auxiliary tool. That way, the whole company does not run on AI. People's own work barely shrinks.

So what went to AI, and what stayed as judgment? What went over is work with fixed steps. Gather numbers. Clean up text. Reformat into a set shape. Repeat the same thing many times. AI is fast here. What stayed is work outside the steps. What to prioritize. What to drop. Who takes responsibility. This stayed on the human side. The boundary is whether it can be turned into steps. If it can, hand it over. If it cannot, keep it.

This boundary is not drawn once and done. What AI can do grows, little by little. Work we held yesterday becomes handable today. So the boundary moves. Chase the moving boundary and re-hand what can be handed over. This, too, is human judgment. The boundary is not fixed. It is something you keep moving.

We handed over all the work first. Then the work began to run and run on the AI side. The more it ran, the clearer judgment's turn became. What to entrust, what to hold — the running floor taught us that, not the logic in our heads.

So "people make the judgment" is not a premise at the start. It is a place we reached after running everything. We did not make light of judgment. We only placed it where it belongs.

Three cores: work, quality, and the base

Running all the work does not mean one big AI does it all. Layers with different roles are stacked. There are three cores.

local-commander: the layer that hands over work. It feeds fixed jobs into AI.
claude-plugins: the layer that keeps quality. It watches whether what comes out is right.
eln-infra-ops: the layer that runs the base. It supports the ground on which the other layers keep moving.

And on the very top rides human judgment. The three layers — work, quality, base — run on AI, and on top of them a human judges. This overlap of four is our backbone.

Why these three? Handing over work alone does not make it run. Without a layer to see whether the handed-over work is right, mistakes flow straight through. So a quality layer is needed. And both work and quality stop without a running base. So a base layer is needed. Work, quality, base. With these three together, all the work finally keeps running. Remove any one, and the rest stop too.

Why split into layers? Give one AI everything, and you lose sight of where it went wrong. Did the work stop? Was it rejected at quality? Did the base fall? Split into layers, and the cause shows up by layer. Without splitting, everything blurs at once. Split, and you can tell which layer it happened in, and fix it.

Each need bred the next

The three cores were not built from a blueprint drawn in advance. Each need called the next. There is an order here.

First, we built. We handed work to AI and had it make things. Once it made things, we wanted to see whether they ran correctly. So watching was needed. Once we watched, broken spots showed. So repair was needed. As repair continued, managing what was fixed and how was needed. As management grew, a layer to manage that management itself was needed.

This nesting is the lineage of the products. The motive was always "because it looked convenient." There is no grand plan. Fill what is missing in front of you. Fill it, and the next missing thing shows. Repeating that, we tried to run all of EarthLink Network's work on AI, and came as far as 18 products running.

The last one, "manage the management," may be hard to grasp. Let me add a little. As products grow, the management mechanisms themselves grow. As management grows, the managements collide with each other. Which management to run first. Where to stop. A layer to bundle that is needed. Managing the management means this. The more the number grows, the more the bundling layer is worth.

We did not conceive of 18 at once. One need simply called the next one. So this lineage has no waste. We did not build in features we do not use ahead of time. It was not born from a plan. It was born from need.

Placing the 18 products into six areas

Lined up as 18, from the outside it feels like a lot. But re-placed into six areas, it fits on one page.

Base: supports the ground on which the other five areas run.
Growth: makes the entrance for reaching people.
Run: runs the fixed work.
Watch: watches whether things are moving.
Support: helps the people and the AI doing the running.
Test: confirms whether what comes out is right.

Six is a number we found by counting afterward. We did not prepare six boxes first and stuff products into them. Trace the lineage, and it naturally split into six clusters. There is a base, we gather people, we run the work, we watch it, we support the people and the AI, and we test the result. With these six, all the work goes around once. Because it goes around, missing areas can be found.

The three cores from earlier also fit into these six areas. eln-infra-ops is the base. local-commander is run. claude-plugins is test. The remaining areas hold their own products too. The whole picture is also laid out at https://www.eln.ne.jp/products.

Here, let me tell you the difference from the sister article once more. That article is a catalog of 18 products. It lists, one by one, what each does. This article has a different role. It is not an explanation of individual products. It is a hub for seeing, on a single page, how they mesh to run all the work. It is not an article that counts products. It is an article that sees the connections between them.

We close contradictions with structure, not willpower

Run all the work on AI, and contradictions always appear. AI makes mistakes. Fast, and many. Take this lightly, and everything collapses. This section is the most important part of this article.

But we do not hold contradictions down with human grit. We close them with structure. The build layer makes a mistake. So the watch layer watches. The watch finds an anomaly. So the repair layer fixes it. claude-plugins, which keeps quality, takes on this watching and fixing. We do not go back to an operation where a human stares at a screen all day, chasing it with their eyes. We build the mechanism to find mistakes, and the mechanism to fix them, into the AI side.

History has a good precedent. Double-entry bookkeeping. In 1494, a mathematician named Luca Pacioli compiled the method of double-entry bookkeeping into a book. In double-entry bookkeeping, one transaction is always written in two places. Debit and credit. Their totals must always match. If they do not match, something is wrong somewhere. It is not that a careful person notices the mistake. The structure — "it does not add up" — makes them notice. Our way is the same idea. We do not rely on human attention. We rely on a structure that stops when things do not match. More than 500 years on from those ledgers, what we are doing has not changed.

Recall the lineage from earlier. Watching and repair were not added on a whim. Each time a contradiction appeared, we added one more layer to close it. The lineage itself is a record of closing contradictions with structure. Put the other way: the spots we now run on human grit are the spots not yet turned into structure. So the moment we find them, we turn them into structure. Grit is used as a marker for the next spot to close.

We do not think "AI is unusable because there are contradictions." Precisely because contradictions appear, we build a layer to close them. If there is a structure that can close it, that work may be run on AI. If it cannot be closed, we do not hand it over yet. Our criterion for judgment is right there.

The point of efficiency is getting time back

Finally, let me tell you why we go this far.

In 19th-century Britain there was an economist named William Stanley Jevons. In 1865, he wrote of a paradox about coal. As steam-engine efficiency rose, coal consumption should have fallen. But in reality, it rose. When efficiency rises, the cost of using it falls. When it gets cheaper, people use more. So overall consumption, in fact, rose. This is called the Jevons paradox, and it is still cited today as a historical fact.

The same thing happens to our time. When AI makes work faster, the freed-up time does not idle. It is used to fill the next need. That is why 18 products were born. Even after making things efficient, the things to do do not shrink. If anything, they grow.

Let me add, just in case. Pour the returned time all back into new work, and the time for judgment does not grow. So we decide, in advance, where the returned time goes. Not to work, but to judgment. Unless we fix this with structure too, time gets sucked back into work again.

So is efficiency meaningless? No. Put the aim on "reducing work," and the Jevons paradox betrays you. But that is not the aim. The aim is for time to return to our hands. We hand work to AI, and the returned time, people spend on judgment. Fixed work to AI. Returned time to judgment. The structure for running all the work on AI is built for this one point.

We did not build it to erase work. We built it to take back, into human hands, the time for judgment.