How we navigated the engineering challenges of embedding local AI models and agentic CLIs directly into a native desktop environment.
RexIDE started as a personal frustration.
Modern IDEs are powerful, but they weren’t designed for a world where AI agents are active participants in your workflow. They assume short lived commands, stateless tools, and human only context switching. That model breaks down the moment you introduce long running AI agents, real terminals, and multi project execution.
This post walks through how RexIDE was designed, the tradeoffs behind its architecture, and why a local first, execution centric approach became the core principle.
Building Persistent Terminal State
The primary goal was simple:
Keep context alive, across projects, terminals, and AI agents, without forcing the developer to think about infrastructure.
That goal immediately shaped every technical decision that followed.
Technical Tradeoffs of Local Execution
One of the earliest decisions was whether AI execution should happen in the cloud or directly on the developer’s machine. Cloud models offer excellent quality, but they introduce friction through API keys and billing management, trust concerns around proprietary code, and a heavy dependency on latency and availability.
Local models remove those concerns entirely. They keep code on the machine, work offline, and feel instant when integrated correctly.
RexIDE was designed local first by default, with the option to layer in cloud models only when the user explicitly opts in. Privacy and control are the baseline, not premium features.
A Note on Codex and the Recent Shift
Recently, OpenAI launched the Codex desktop app, which meaningfully validates the direction RexIDE took early on: local execution with persistent context.
Codex today focuses on a single toolchain, the Codex ecosystem, and does a solid job at solving the local, long running AI workflow problem within that scope.
RexIDE takes a broader approach. Instead of committing to a single AI provider or tool, it was designed from the start to act as an orchestrator for multiple local AI CLIs across platforms, including Claude Code, Codex CLI, and OpenCode. All of these run locally on macOS, Windows, and Linux, side by side, inside the same execution centric environment.
This reflects how many developers already work today: using multiple AI tools side by side, depending on the task at hand. The environment should adapt to that reality rather than force consolidation.
Model Selection and Resource Constraints
Running AI models locally isn’t free: CPU, memory, and energy usage matter, especially on a machine you actively work on. RexIDE intentionally uses multiple layers of local AI execution. It utilizes external local CLIs such as Claude Code, Codex CLI, and similar tools for full reasoning and agent-driven workflows, while also employing embedded lightweight local models for smaller, fast tasks like snippet analysis, summarization, and structural understanding directly inside the app.
Instead of chasing the largest model possible, RexIDE follows a simple rule:
Use the smallest model that reliably meets the task’s requirements.
Lightweight embedded models handle frequent, low-latency tasks without context switching, while heavier reasoning is delegated to specialized local CLIs that already excel at those workflows.
Multiple model sizes were tested against real workflows including transcription, summarization, and code understanding while monitoring latency, sustained CPU usage, and memory pressure. The selected models stay well within acceptable resource bounds, ensuring they don’t interfere with compilers, editors, or other foreground tasks.
Native PTY Execution and State Persistence
Most IDEs optimize for editing, but RexIDE optimizes for execution. That means providing real terminals rather than simulated ones, maintaining long running processes that don’t reset when focus changes, and enabling AI agents that operate inside the same execution context as the developer.
This approach eliminates a huge amount of mental overhead. You don’t restart tasks, re-explain context, or reconstruct state — everything stays alive.
Engineering Stateless Backend Boundaries
RexIDE doesn’t require a backend to function, but it was designed with one in mind. If a backend were introduced, it would follow a few strict principles: stateless request handling, explicit separation between compute, user state, and storage, and strong session isolation to prevent data leakage.
The client would remain the source of truth for execution context, with the backend acting only as an optional accelerator — never a dependency.
Resource Management and Background Throttling
Performance isn’t something you optimize later, it is a core part of the user experience. RexIDE treats system resources with respect by ensuring heavy work runs off the main thread and AI workloads throttle when the app is backgrounded.
If the tool ever feels like it’s “in the way,” it has failed.
Reversible Architectural Decisions
Early design decisions are rarely perfect. RexIDE was built with reversibility in mind.
Short, time boxed prototypes were preferred over long debates. Decisions were explicitly labeled as reversible or irreversible, which made it easier to move fast without locking the project into bad paths. That mindset allowed rapid iteration without accumulating architectural debt.
The Result
RexIDE isn’t trying to be another editor with AI bolted on. It’s an execution environment where context persists, AI agents feel native, and the developer stays in control.
Everything else is a consequence of that choice.
If you’re building tools for developers today, the question isn’t whether to add AI — it’s where it lives, how much context it gets, and who ultimately controls it.
RexIDE represents one way to approach that problem.
Top comments (0)