Metrics
Once I had Aider working with a local Ollama instance, there was still one thing missing: understanding what was actually happening under the hood.
I wanted to see:
- how many tokens were being processed
- how long each response took
- how fast text generation was
- whether one model was slower than another
- whether the context window was growing too large
Aider alone didn't provide enough visibility. However, Ollama exposes several useful metrics in the final response returned by its API.
So I decided to add a small local proxy.
Architecture
The request flow became:
Aider
↓
Local proxy on port 11435
↓
Ollama on Windows (11434)
↓
qwen2.5-coder:14b
The proxy performs two simple tasks:
- forwards every request to Ollama
- stores the execution statistics in a JSONL file
The resulting log file is:
~/ollama_usage.jsonl
Creating the Proxy
Inside WSL:
cat > ~/ollama_usage_proxy.py <<'PY'
#!/usr/bin/env python3
import json
import time
import urllib.request
from datetime import datetime, timezone
from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler
UPSTREAM = "http://127.0.0.1:11434"
LOG_FILE = "/home/xxxxx/ollama_usage.jsonl"
def ns_to_s(value):
if isinstance(value, (int, float)):
return value / 1_000_000_000
return None
def safe_div(a, b):
if not a or not b:
return None
return a / b
class OllamaProxy(BaseHTTPRequestHandler):
protocol_version = "HTTP/1.0"
def log_message(self, format, *args):
return
def do_GET(self):
self.forward()
def do_POST(self):
self.forward()
def forward(self):
length = int(self.headers.get("Content-Length", "0") or "0")
body = self.rfile.read(length) if length else None
url = UPSTREAM + self.path
headers = {
k: v for k, v in self.headers.items()
if k.lower() not in {
"host",
"content-length",
"connection",
"accept-encoding",
}
}
request = urllib.request.Request(
url,
data=body,
headers=headers,
method=self.command,
)
started = time.perf_counter()
final_obj = None
try:
with urllib.request.urlopen(request, timeout=None) as response:
self.send_response(response.status)
self.send_header(
"Content-Type",
response.headers.get("Content-Type", "application/json"),
)
self.end_headers()
for line in response:
self.wfile.write(line)
self.wfile.flush()
try:
obj = json.loads(line.decode("utf-8"))
if obj.get("done") is True:
final_obj = obj
except Exception:
pass
except Exception as exc:
self.send_error(502, f"Proxy error: {exc}")
return
wall_time_s = time.perf_counter() - started
if not final_obj:
return
prompt_tokens = final_obj.get("prompt_eval_count")
response_tokens = final_obj.get("eval_count")
prompt_eval_s = ns_to_s(final_obj.get("prompt_eval_duration"))
eval_s = ns_to_s(final_obj.get("eval_duration"))
total_s = ns_to_s(final_obj.get("total_duration"))
load_s = ns_to_s(final_obj.get("load_duration"))
row = {
"ts": datetime.now(timezone.utc).isoformat(),
"endpoint": self.path,
"method": self.command,
"model": final_obj.get("model"),
"done_reason": final_obj.get("done_reason"),
"prompt_tokens": prompt_tokens,
"response_tokens": response_tokens,
"total_tokens": (prompt_tokens or 0) + (response_tokens or 0),
"wall_time_s": round(wall_time_s, 3),
"ollama_total_s": round(total_s, 3) if total_s is not None else None,
"load_s": round(load_s, 3) if load_s is not None else None,
"prompt_eval_s": round(prompt_eval_s, 3) if prompt_eval_s is not None else None,
"generation_s": round(eval_s, 3) if eval_s is not None else None,
"prompt_tokens_per_s": round(safe_div(prompt_tokens, prompt_eval_s), 2)
if safe_div(prompt_tokens, prompt_eval_s) else None,
"generation_tokens_per_s": round(safe_div(response_tokens, eval_s), 2)
if safe_div(response_tokens, eval_s) else None,
}
with open(LOG_FILE, "a", encoding="utf-8") as f:
f.write(json.dumps(row, ensure_ascii=False) + "\n")
if __name__ == "__main__":
server = ThreadingHTTPServer(("127.0.0.1", 11435), OllamaProxy)
print("Ollama usage proxy listening on http://127.0.0.1:11435")
print(f"Forwarding requests to {UPSTREAM}")
print(f"Writing statistics to {LOG_FILE}")
server.serve_forever()
PY
Then make it executable:
chmod +x ~/ollama_usage_proxy.py
Starting the Proxy
In a WSL terminal:
cd ~
python3 ~/ollama_usage_proxy.py
Leave this terminal running while using Aider.
Running Aider Through the Proxy
In a second WSL terminal:
cd ~/repos/my-project
OLLAMA_API_BASE=http://127.0.0.1:11435 aider --model ollama_chat/qwen2.5-coder:14b --no-show-model-warnings
The distinction is important:
11434 = Direct connection to Ollama (no metrics)
11435 = Proxy enabled (metrics collected)
Inspecting the JSONL Log
After Aider generates a response:
tail -n 5 ~/ollama_usage.jsonl
To make the output easier to read, I installed jq, a powerful open-source command-line tool for parsing, filtering, formatting, and manipulating JSON data:
sudo apt install -y jq
Then:
tail -n 5 ~/ollama_usage.jsonl | jq .
A typical log entry looks like this:
{
"model": "qwen2.5-coder:14b",
"prompt_tokens": 1280,
"response_tokens": 310,
"total_tokens": 1590,
"wall_time_s": 42.3,
"generation_s": 37.8,
"generation_tokens_per_s": 8.2
}
Creating an Alias
To avoid accidentally connecting Aider to the wrong port, I created a shell alias:
echo 'alias aider14stats="OLLAMA_API_BASE=http://127.0.0.1:11435 aider --model ollama_chat/qwen2.5-coder:14b --no-show-model-warnings"' >> ~/.bashrc
source ~/.bashrc
From that point on, starting Aider was as simple as:
cd ~/repos/my-project
aider14stats
What I Learned
Logging should be part of the design from the very beginning.
Without metrics, a local model may simply feel slow. With proper measurements, I can understand:
how much input I'm sending
how much output I'm receiving
how long each request takes
how many tokens per second are being generated
whether the model was already loaded into memory
Having access to this information completely changes the way I can evaluate and compare locally hosted LLMs.
Top comments (0)