The local AI demo loop is seductive. Get a model to fit on a laptop, show it running, and declare the hard part solved.
The product review is less fun. How much memory can it consume? Where do intermediate files go? Can a user edit the result? What may the agent touch? How does anyone know the job finished correctly?
"It runs locally" answers none of those questions. It tells you where one computation happened.
Local-first has to survive the entire workflow. The model call, interface, files, tools, approvals, exports, and completion evidence all need boundaries. Miss one, and the architecture can quietly contradict the label on the box.
Locality has six boundaries
Before choosing a model or framework, write down this contract:
- Resource boundary: What memory, compute, storage, and wait time may the feature consume?
- Residency boundary: Which inputs and intermediate artifacts stay on the device? Which network dependencies remain?
- Handoff boundary: What editable artifact does generation return?
- Interface boundary: Which state transitions and behaviors are explicit and testable?
- Authority boundary: What may the agent read, change, call, or publish without approval?
- Evidence boundary: What report, diff, export, trace, or test result proves the step completed?
This is not a vendor standard. It is an architecture-review shortcut. If a team cannot answer one of these questions, it has found an unresolved product decision rather than a documentation problem.
Start with the resource envelope
Local-model discussions tend to collapse into a single number: how large a model can fit into how little VRAM.
Projects such as AirLLM make that constraint visible by documenting very large model execution under constrained VRAM. That is useful engineering. It still does not tell a product team whether a feature is usable.
"Can run" and "can support this workflow" are different claims. The latter needs decisions about:
- acceptable wait time for an interactive action
- storage available for model weights and generated artifacts
- what else must run on the device at the same time
- how the feature behaves when the resource ceiling is reached
- whether the application queues work, degrades the capability, or stops visibly
Those are routing and UX decisions. A model that technically loads may still be wrong for a hot interaction path. A slower local path may be perfectly reasonable for a queued task with a visible progress state. The hardware limit does not settle the product design; it forces the product design to become explicit.
The residency boundary needs the same precision. A model can execute on-device while telemetry, retrieval, authentication, asset storage, or a later processing step still uses the network. That may be an acceptable system. It just should not hide behind a broad "local" claim.
Map every input, intermediate artifact, and outbound request. Then describe the boundary you actually built.
Generation is not a handoff
A raw model response is rarely the end of a real workflow.
MiniMax H3's ComfyUI integration moves the conversation one step past model availability. The announcement pairs an open-weight multimodal model, native stereo audio, and 2K video with a local, optimized ComfyUI path. The model matters, but so does the operator workflow around it. Generation becomes more useful when the result can move through inspection, revision, export, and the next production step.
A good handoff is editable and boring. It might be a video plus audio tracks, a still image, a project graph, a structured specification, or a set of exported files. The right artifact depends on the job. A final preview with no recoverable state is usually a dead end wearing a nice UI.
Consider a generated thumbnail or video still. The generation step may be local, but publishing still needs deterministic dimensions and an export the creator can inspect. For an Instagram asset, Resize Image for Instagram provides browser-local fit, fill, preview, and export across square, portrait, landscape, Story, and Reel presets. That is a small step, but it preserves the point of the workflow: the image remains an artifact the user can see and control before publishing.
Here is the harsher handoff test: remove the model after generation. If the user cannot continue with what it returned, the feature produced a dependency on another model call, not a durable artifact.
Local-first includes the frontend
Developers sometimes apply rigor to model placement and then treat the interface as a screenshot around a prompt box.
This gets the priorities backwards. A local system has more visible resource limits, longer-running operations, partial results, and more failure states to explain. Its frontend needs a stronger state model, not a prettier loading animation.
Bonsai takes the opposite route. It models reactive web applications with functional state machines, incremental recomputation, and executable DOM behavior tests. That does not mean every local AI product should use Bonsai or OCaml. It shows what "explicit" looks like: state has an owner, transitions have rules, and behavior can be exercised rather than admired in a screenshot.
The same distinction matters in Generative UI. A catalog of Generative UI resources can help a team compare schema-driven renderers, trusted component catalogs, protocols, and implementation patterns. Those examples are design input. They do not prove that a generated interface preserves state, handles interruption, respects permissions, or returns the right action to the application.
Treat these as separate jobs:
- inspiration decides what kinds of interface might be useful
- generation selects or composes an allowed surface
- the application owns state and permissions
- behavioral tests prove the important transitions
For a long local task, the interface should be able to distinguish queued, loading, generating, awaiting approval, exporting, completed, interrupted, and failed states when those states exist in the workflow. Do not flatten them into one spinner and a hopeful success toast.
A plausible screen is not proof. The user needs to know what happened, what can still change, and what the system expects next.
Local execution does not grant safe authority
Where a model runs and what an agent may do are separate decisions.
Nightcrawler's documented design keeps those questions apart. It puts a scope proxy, small-step loop, dashboard, and structured report around a local mobile agent. Together, those pieces define the allowed surface, expose progress, and leave something an operator can inspect afterward.
The project documentation is not an independent security audit, and the pattern does not guarantee safety. What is worth borrowing is the shape of the design: locality is not used as a substitute for authorization.
A local agent can still delete the wrong file, call an unintended tool, publish too early, or operate with credentials broader than its task. Keeping inference on-device does not reduce the need for scoped capabilities and approval gates.
Write the authority boundary as operations, not adjectives:
may_read:
- project/media/inbox/**
may_write:
- project/media/exports/**
may_call:
- local-video-generator
- local-image-resizer
requires_approval:
- publish
- overwrite-source
forbidden:
- read-credentials
- upload-source-media
The exact syntax is unimportant. The test is whether the runtime can enforce the policy and whether the interface can explain an approval request without making the user reconstruct the agent's history.
Receipts are part of the product
A completed animation is not evidence that a multi-step job completed.
For a media workflow, the receipt may name the source artifact, generation settings, exported files, failed outputs, and the directory where results were written. For a coding agent, it may be a diff plus test results. For an interface-generating agent, it may be the accepted component specification and behavioral checks.
Keep the receipt narrow. Recording every prompt, image, and intermediate state by default can create a second data-handling problem. Capture what a reviewer needs to verify the operation, redact sensitive values, and give the evidence a retention policy.
This is also where local-first claims become testable. A receipt can record which steps stayed on-device, which network calls occurred, which approvals were granted, and which artifact crossed the final boundary. Configuration describes intent. The receipt describes the run.
Apply the contract before picking the stack
Take a hypothetical local creator tool that generates a short video and prepares a still for social publishing. This example is not a claim about any of the projects above.
The team should be able to answer six questions before debating models:
- Resources: What device envelope is supported, and what happens when generation exceeds it?
- Residency: Do prompts, source media, previews, and exports stay local? Which checks or services still need a network connection?
- Handoff: Does the user receive editable media and a predictable export, or only a rendered preview?
- Interface: Can the UI represent partial output, cancellation, approval, retry, and failure without losing state?
- Authority: Can the agent prepare files but never publish or overwrite a source without approval?
- Evidence: Does the run end with an export manifest or report that names what happened?
A benchmark score answers none of those questions.
Benchmarks may help choose an implementation after the workflow is defined. They cannot define the workflow for you. Neither can an impressive hardware demo.
Local is the whole chain
A coherent local-first product makes a series of small, verifiable promises. It names the resource ceiling and shows where data moves. It returns an artifact the user can edit. The frontend exposes state, the runtime limits what the agent may do, and the workflow leaves evidence when the work ends.
One locally running model does not rescue a workflow that breaks those promises elsewhere. At that point, local execution is a placement detail rather than an architecture.
Ask the team to explain all six boundaries without naming a model benchmark. If the answer still begins with VRAM, the architecture review is not done.
Top comments (0)