What if you could have an AI coding agent that reads your files, writes code, executes commands, and fixes errors without sending a single model request to the cloud?
No OpenAI API. No Anthropic API. No per-token billing.
Just an open-weight AI model running on your own hardware.
I recently decided to test exactly how practical this setup has become.
Using Qwen 3.8 27B, LM Studio, and Pi Coding Agent, I built a fully local AI coding workflow and challenged it to create an entire web application from scratch.
And the results were surprisingly good.
The agent built a React + TypeScript application, executed commands, tested its work, identified errors, and fixed them without requiring my intervention.
All while running a 27-billion-parameter model locally on my MacBook Pro.
But there were some interesting trade-offs, especially around inference speed, hardware requirements, and context management.
In this article, I’ll walk through the complete setup, explain why I chose these tools, and share what actually happened when I put the agent to work.
🎥 Full video walkthrough
In the video, I cover:
- Installing LM Studio and downloading Qwen 3.8 27B.
- Understanding 4-bit quantization.
- Starting the local inference server.
- Installing Pi Coding Agent.
- Connecting Pi to LM Studio.
- Testing local tool calling.
- Building the Mission Control dashboard.
- Reviewing the results, performance, and limitations.
🚀 What Did I Build?
Before getting into the setup, here’s what I asked my local AI coding agent to create.
An AI Agent Mission Control dashboard for monitoring and managing a team of autonomous AI agents.
The application needed several features:
- Agent Overview: Monitor active agents, running tasks, completed tasks, and failures.
- Agent Management: View agents, their roles, current tasks, and statuses.
- Kanban Task Board: Organize tasks across Backlog, In Progress, Review, and Completed.
- Live Activity Feed: Track agent activity and task execution.
- Analytics Dashboard: Visualize token usage, estimated costs, success rates, and task performance.
- Modern UI: A responsive dark-themed interface with charts, animations, and polished interactions.
And I didn’t want a static HTML mockup.
I wanted a working React application with realistic sample data.
The interesting part?
The entire coding workflow ran locally.
The AI model wasn’t hosted by OpenAI, Anthropic, or another cloud inference provider.
It was running directly on my computer.
🧠 The Architecture: Three Tools, One Local AI Coding Agent
The architecture is relatively simple.
Developer
|
v
+----------------+
| Pi Coding Agent|
| |
| Read Files |
| Write Code |
| Execute Bash |
| Edit Files |
+----------------+
|
| OpenAI-Compatible API
v
+----------------+
| LM Studio |
| |
| Local Inference|
| Server |
+----------------+
|
v
+----------------+
| Qwen 3.8 27B |
| |
| 4-bit Quantized|
| Local AI Model |
+----------------+
|
v
Your Computer
Each component has a specific responsibility.
1. Qwen3.8 27B — The Brain
Qwen 3.8 27B is the language model responsible for understanding instructions, reasoning about coding tasks, and generating code.
For this experiment, I used a 4-bit quantized version of the model, approximately 16GB in size.
This makes it much more practical to run a large model on consumer hardware.
2. LM Studio — The Local Inference Server
LM Studio makes running open-weight AI models locally much easier.
Instead of manually setting up inference frameworks, Python environments, or model-serving infrastructure, you get a desktop application where you can:
- Discover and download models.
- Load models into memory.
- Chat with models directly.
- Configure inference settings.
- Expose models through an OpenAI-compatible API.
That last feature is particularly important because it allows external applications to communicate with your locally running model.
3. Pi Coding Agent — The Hands
A language model alone cannot automatically inspect your project files, modify code, or execute terminal commands.
That’s where Pi comes in.
Pi is a lightweight, extensible terminal-based AI coding agent.
It provides the tools and agent loop needed to interact with your development environment.
Instead of simply generating code in a chat window, Pi can use the model to work through actual coding tasks.
And because Pi is relatively minimal, it’s particularly interesting for local model experiments.
⚙️ Step 1: Install LM Studio and Download Qwen 3.8 27B
First, download LM Studio from its official website:
LM Studio: https://lmstudio.ai/
Install the application for your operating system and hardware.
Once installed, open LM Studio and navigate to the model search section.
Search for:
Qwen 3.8 27B
You’ll notice that multiple versions of the same model may be available.
For this experiment, I selected a 4-bit quantized version.
Why 4-bit Quantization?
Running a 27-billion-parameter model at full precision can require substantial memory.
Quantization reduces the precision used to store model weights.
For example:
| Precision | Approximate Raw Weight Storage |
|---|---|
| FP16 (16-bit) | 54 GB |
| INT8 (8-bit) | 27 GB |
| 4-bit | 13.5 GB |
These are theoretical weight-storage estimates. Actual model files and runtime memory requirements can be higher because of quantization metadata, model architecture, KV cache, and other overhead.
The 4-bit model I downloaded was approximately 16GB.
That’s a significant reduction compared with FP16.
Of course, quantization comes with trade-offs.
Reducing precision can affect model accuracy, especially on tasks involving complex reasoning or code generation.
However, modern quantization techniques can often preserve much of the original model’s capabilities while substantially reducing memory requirements.
For running larger models locally, this is extremely useful.
Load the Model
After downloading:
- Open My Models in LM Studio.
- Select your downloaded Qwen model.
- Click Load Model.
- Configure the context length according to your available memory.
- Load the model.
For my experiment, I configured a context length of 200,000 tokens.
Keep in mind that configuring a large context window doesn’t mean your hardware can efficiently process that entire context. Actual memory usage and performance depend on the model, runtime, and KV-cache configuration.
Once the model is loaded, you can test it using LM Studio’s built-in chat interface.
Try sending a simple message to verify that inference is working.
Now we need to make this model accessible to our coding agent.
🔌 Step 2: Enable LM Studio’s Local API Server
LM Studio can expose your local model through an OpenAI-compatible API.
This is what allows Pi Coding Agent to communicate with Qwen.
Inside LM Studio:
- Navigate to the Developer tab.
- Enable the local server.
- Verify that your model is available.
By default, LM Studio uses port 1234.
The local API endpoint is:
http://localhost:1234/v1
You can verify that the server is responding by running:
curl http://localhost:1234/v1/models
This should return information about the models exposed by your LM Studio server.
At this point, your computer is acting as its own AI inference server.
And the best part is that you don’t need to send inference requests to an external model provider.
Note: Keep the server bound to localhost unless you intentionally need remote access. Exposing an inference endpoint to your network introduces additional security considerations.
🛠️ Step 3: Install Pi Coding Agent
Now that Qwen is running locally, we need an agent that can actually work with our codebase.
For this, I’m using Pi Coding Agent.
Official project:
Pi is designed around a relatively small set of core capabilities.
Out of the box, it provides four essential tools:
| Tool | Purpose |
|---|---|
read |
Read files from your project |
write |
Create files and write content |
edit |
Modify existing files |
bash |
Execute terminal commands |
That’s it.
And these four tools are enough to handle a surprising amount of coding work.
Pi also supports extensions, allowing you to add functionality when needed.
Install Pi
Follow the installation instructions on Pi’s official website.
After installation, open your terminal and run:
pi
If a new Pi session opens, the installation is working.
Now let’s connect it to our local model.
🔗 Step 4: Connect Pi to LM Studio
There are multiple ways to configure Pi to use a locally hosted model.
You can manually configure a custom provider, but I used a community extension called pi-lmstudio.
The extension discovers models exposed by LM Studio and makes them available inside Pi.
You can find it through Pi’s package directory.
Install the extension using the command provided on its package page.
Once installed, restart Pi:
pi
Then open the model selector:
/model
You should see the model currently exposed by LM Studio.
Select your Qwen 3.8 27B model.
And that’s it.
Pi is now using your locally running Qwen model instead of a hosted inference API.
The workflow looks like this:
Your Prompt
|
v
Pi Coding Agent
|
| Sends model request
v
LM Studio API (localhost:1234)
|
v
Qwen 3.8 27B
|
| Returns response / tool calls
v
Pi Coding Agent
|
| Executes tools
v
Your Local Project
One important distinction: Pi executes tools locally, while LM Studio handles model inference.
The model generates the instructions and tool calls, but Pi is responsible for carrying them out.
Verify That Tool Calling Works
Before assigning a complicated task, I recommend testing the setup with something simple.
For example, create a Python file:
print("Hello, World!")
Then ask Pi:
Read the Python file in this directory, explain what it does, and create a document containing the explanation.
This tests whether the model can:
- Understand the instruction.
- Select the appropriate tools.
- Read a local file.
- Generate a summary.
- Write a new file.
In my experiment, Pi successfully completed this test.
Now it was time for something more challenging.
💻 Step 5: Can a Local 27B Model Build an Entire Web Application?
This was the part I was most interested in.
Getting a model to answer questions is one thing.
Getting it to work through a multi-step software development task is something else entirely.
I started with an empty project directory.
Then I asked Pi to build an AI Agent Mission Control dashboard.
Here’s a condensed version of the prompt:
Build a modern, polished AI Agent Mission Control dashboard for managing and monitoring a team of autonomous AI agents.
The dashboard should include:
* Overview cards for Active Agents, Running Tasks, Completed Tasks, and Failed Tasks
* A list/grid of AI agents with names, roles, status, current task, and model being used
* A Kanban-style task board with Backlog, In Progress, Review, and Completed columns
* A Live Activity Feed showing agents starting tasks, using tools, completing work, and handing tasks to other agents
* System metrics such as token usage, estimated cost, success rate, and average task duration
* A sidebar with Overview, Agents, Tasks, Activity, Analytics, and Settings
Use realistic mock data for a small AI-agent team such as Research Agent, Coding Agent, Browser Agent, Content Agent, and QA Agent.
Make it feel like a premium SaaS mission-control product: clean dark theme, excellent typography, subtle animations, status indicators, charts, responsive layout, and polished interactions.
Build the complete working website, not just a static mockup.
Then I let the agent work.
What Happened?
The agent created a React + TypeScript project and started implementing the application.
It performed multiple tool calls, created files, wrote components, and executed terminal commands.
But what impressed me most wasn’t just the generated code.
It actually attempted to test and fix its own work.
During the process, the agent:
- Created the application structure.
- Generated React components.
- Implemented the dashboard interface.
- Ran the application locally.
- Encountered errors.
- Inspected the results using screenshots.
- Made additional changes to address issues.
And it completed this workflow without requiring me to manually fix the errors it encountered.
The final result included the agent overview, task board, activity feed, analytics dashboard, and settings interface.
To be clear, this was a demonstration application using sample data, not a production-ready autonomous agent management platform.
But as an experiment in local AI-assisted software development, it was encouraging.
📊 The Results: Performance, Hardware, and Cost
Here’s the actual setup I used.
| Component | Configuration |
|---|---|
| Computer | MacBook Pro |
| Chip | Apple M5 Max |
| Memory | 48GB unified memory |
| AI Model | Qwen3.8 27B |
| Quantization | 4-bit |
| Model Size | Approximately 16GB |
| Inference Runtime | LM Studio |
| Coding Agent | Pi |
| Observed Speed | 10–15 tokens/second |
| Task Duration | A few hours |
| Paid Model API Cost | $0 |
The agent managed to build the application, but it wasn’t particularly fast.
At approximately 10–15 tokens per second, complex coding tasks can take quite a while.
And when the agent needs to perform multiple rounds of reasoning, tool execution, and error correction, that time adds up.
Still, the fact that this entire workflow ran on consumer hardware is interesting.
There was no per-token API bill.
However, zero API cost doesn’t mean zero cost.
You still need suitable hardware, electricity, and time.
And depending on your workload, a hosted model might still be more economical or productive.
🧩 Why I Chose Pi Instead of a Larger Coding Agent
This is one of the more interesting technical aspects of the experiment.
When running AI models locally, context management becomes especially important.
Every instruction, conversation message, tool definition, and tool result consumes tokens.
And those tokens aren’t free from a computational perspective.
Even when you’re not paying an API provider, your hardware still needs to process them.
For local models, large contexts can increase:
- Memory consumption.
- Time to process prompts.
- KV-cache requirements.
- Overall task execution time.
This is where Pi’s design becomes interesting.
A Minimal Agent Harness
Pi starts with four core tools:
read
write
edit
bash
Rather than beginning with a large collection of built-in capabilities, you can extend Pi as needed.
This gives you more control over the tools and functionality available to your local model.
For example, you might eventually add:
- Browser automation.
- Web search.
- Custom development tools.
- Project-specific workflows.
- Additional integrations.
But you don’t necessarily need all of those capabilities loaded from the beginning.
A smaller tool surface can help keep the setup simpler.
Important: This doesn’t automatically mean Pi uses fewer tokens or performs better than Claude Code, Codex, or other coding agents in every scenario.
That would require a controlled benchmark using the same model, tasks, and evaluation criteria.
For my experiment, Pi was simply a good fit because I wanted a lightweight, extensible agent environment.
⚠️ What Are the Limitations of Running AI Coding Agents Locally?
As promising as the results were, there are some important limitations.
1. Hardware Requirements
A 27B model, even when quantized, requires a significant amount of memory.
And the model weights are only part of the equation.
You also need memory for inference overhead, context processing, and the operating system.
Not everyone has a machine capable of comfortably running models this large.
2. Inference Can Be Slow
My setup generated approximately 10–15 tokens per second.
That’s usable, but considerably slower than many hosted inference services.
For complicated tasks involving hundreds of tool calls or repeated debugging cycles, performance becomes a major consideration.
3. Tool-Calling Reliability Varies
An AI coding agent needs more than good code generation.
It must reliably decide when to use tools, generate valid tool calls, interpret results, and continue working.
A model might perform well on a single coding prompt but struggle with a long sequence of tool interactions.
This is something I want to evaluate more systematically in future experiments.
4. Local Models Aren’t Always Competitive With Frontier Models
Strong hosted models still have advantages on many complex software engineering tasks.
Smaller local models may struggle with:
- Large codebases.
- Complex architectural changes.
- Long debugging sessions.
- Multi-step reasoning.
- Maintaining consistency across many files.
And quantization can introduce additional accuracy trade-offs.
So I wouldn’t consider this a universal replacement for cloud-based coding agents just yet.
5. Local Inference Doesn’t Automatically Make an Agent Safe
Running the model locally helps keep inference data under your control.
But an agent that can execute shell commands and modify files still has significant permissions.
A local model can make mistakes, delete files, or execute unintended commands.
For sensitive projects, consider using isolated development environments, restricted permissions, version control, and approval mechanisms.
Also remember that extensions or tools with network access may still communicate externally.
A fully local inference endpoint is not, by itself, a guarantee that every part of an agent workflow stays offline.
🔮 Are Fully Local AI Coding Agents Becoming Practical?
I think we’re getting closer.
Not because local models have suddenly become better than every cloud model.
But because the overall ecosystem has improved significantly.
We now have:
Open-weight models capable of increasingly sophisticated coding tasks.
Local inference tools like LM Studio that make downloading and running models accessible.
Lightweight agent frameworks like Pi that can turn those models into useful coding assistants.
And hardware capable of running models that would have been impractical for many developers just a few years ago.
What’s particularly interesting is that these components are independent.
You can swap the model without necessarily replacing your coding agent.
You can experiment with different quantization levels.
You can add or remove agent capabilities.
And you can choose how much of your development workflow runs locally.
For developers who care about privacy, experimentation, offline workflows, or avoiding recurring API costs, that’s a compelling direction.
The next challenge isn’t just making local models generate better code.
It’s making them reliable enough to complete long-running software engineering tasks with minimal human intervention.
And that requires evaluating the entire agent workflow, not just the model’s benchmark scores.
💬 Final Thoughts
My biggest takeaway from this experiment wasn’t that a local 27B model could replace Claude Code or Codex.
It was that a complete AI coding workflow can now run on a personal computer without relying on paid model APIs.
And that opens up some interesting possibilities.
Local-first development environments.
Private AI coding assistants.
Offline software development workflows.
And customizable agents that developers can adapt to their own hardware and requirements.
There’s still plenty of room for improvement, particularly around speed, reliability, and long-context performance.
But I’m excited to see how far this ecosystem progresses.
Top comments (1)
I’m curious: Are you already using local models for coding?
If so, which model and coding agent combination has worked best for you?
And would you accept slower inference in exchange for keeping your entire AI coding workflow local?
I’d love to hear what others are experimenting with.