DEV Community

Cover image for Setting Up a Local AI Coding Agent with Ollama and Aider (part 3)
eleonorarocchi
eleonorarocchi

Posted on

Setting Up a Local AI Coding Agent with Ollama and Aider (part 3)

#ai #llm

Dashboard

After collecting the metrics in ollama_usage.jsonl, I wanted a way to visualize them.

I used two tools:

Streamlit → quick, operational view
Grafana   → more stable dashboards and long-term monitoring
Enter fullscreen mode Exit fullscreen mode

Both read the same data, but in different ways.

The final architecture looks like this:

Aider → proxy on 11435 → ollama_usage.jsonl
                            ├─ Streamlit
                            └─ exporter → Prometheus → Grafana
Enter fullscreen mode Exit fullscreen mode

Streamlit

Streamlit was the fastest way to build a local user interface.

Inside WSL:

python3 -m venv ~/.venvs/aider-dashboard
source ~/.venvs/aider-dashboard/bin/activate
pip install streamlit pandas streamlit-autorefresh
Enter fullscreen mode Exit fullscreen mode

Then I created the dashboard:

cat > ~/aider_usage_dashboard.py <<'PY'
import json
from pathlib import Path

import pandas as pd
import streamlit as st
from streamlit_autorefresh import st_autorefresh

LOG_PATH = Path.home() / "ollama_usage.jsonl"

st.set_page_config(
    page_title="Aider / Ollama usage dashboard",
    layout="wide",
)

st_autorefresh(interval=5000, key="aider_dashboard_refresh")

st.title("Aider / Ollama usage dashboard")
st.caption("Local statistics: tokens, response times, and generation speed.")

if not LOG_PATH.exists():
    st.warning(f"No file found: {LOG_PATH}")
    st.stop()

rows = []
with LOG_PATH.open("r", encoding="utf-8") as f:
    for line in f:
        line = line.strip()
        if not line:
            continue
        try:
            rows.append(json.loads(line))
        except json.JSONDecodeError:
            pass

if not rows:
    st.warning("The file exists, but it does not contain any valid metrics yet.")
    st.stop()

df = pd.DataFrame(rows)

if "ts" in df.columns:
    df["ts"] = pd.to_datetime(df["ts"], errors="coerce")
    df = df.sort_values("ts")
    df["run"] = range(1, len(df) + 1)
else:
    df["run"] = range(1, len(df) + 1)

numeric_cols = [
    "prompt_tokens",
    "response_tokens",
    "total_tokens",
    "wall_time_s",
    "ollama_total_s",
    "load_s",
    "prompt_eval_s",
    "generation_s",
    "generation_tokens_per_s",
]

for col in numeric_cols:
    if col in df.columns:
        df[col] = pd.to_numeric(df[col], errors="coerce")

total_calls = len(df)
total_tokens = int(df["total_tokens"].sum()) if "total_tokens" in df.columns else 0
avg_time = df["wall_time_s"].mean() if "wall_time_s" in df.columns else None
avg_tps = df["generation_tokens_per_s"].mean() if "generation_tokens_per_s" in df.columns else None

c1, c2, c3, c4 = st.columns(4)

c1.metric("Recorded responses", total_calls)
c2.metric("Total tokens", f"{total_tokens:,}")
c3.metric("Average response time", f"{avg_time:.1f}s" if pd.notna(avg_time) else "n/a")
c4.metric("Average tokens/s", f"{avg_tps:.2f}" if pd.notna(avg_tps) else "n/a")

st.divider()

left, right = st.columns(2)

with left:
    st.subheader("Tokens per response")
    token_cols = [c for c in ["prompt_tokens", "response_tokens", "total_tokens"] if c in df.columns]
    if token_cols:
        st.line_chart(df.set_index("run")[token_cols])

with right:
    st.subheader("Response times")
    time_cols = [c for c in ["wall_time_s", "ollama_total_s", "generation_s", "prompt_eval_s"] if c in df.columns]
    if time_cols:
        st.line_chart(df.set_index("run")[time_cols])

left, right = st.columns(2)

with left:
    st.subheader("Generation speed")
    if "generation_tokens_per_s" in df.columns:
        st.line_chart(df.set_index("run")[["generation_tokens_per_s"]])

with right:
    st.subheader("Input vs output tokens")
    cols = [c for c in ["prompt_tokens", "response_tokens"] if c in df.columns]
    if cols:
        st.bar_chart(df.set_index("run")[cols])

st.divider()

st.subheader("Latest responses")
show_cols = [
    c for c in [
        "ts",
        "model",
        "prompt_tokens",
        "response_tokens",
        "total_tokens",
        "wall_time_s",
        "ollama_total_s",
        "generation_s",
        "generation_tokens_per_s",
        "done_reason",
    ]
    if c in df.columns
]

st.dataframe(
    df[show_cols].tail(50).sort_index(ascending=False),
    use_container_width=True,
)

st.caption(f"Reading data from: {LOG_PATH}")
PY
Enter fullscreen mode Exit fullscreen mode

To start it:

source ~/.venvs/aider-dashboard/bin/activate
streamlit run ~/aider_usage_dashboard.py --server.address 127.0.0.1 --server.port 8501
Enter fullscreen mode Exit fullscreen mode

Then open:

http://localhost:8501
Enter fullscreen mode Exit fullscreen mode

I used this as a quick operational dashboard. Of course, this was only the starting point; I later customized it with all the metrics I actually needed.

Grafana

Grafana, a widely used data visualization and analytics tool, requires a few additional components:

exporter → Prometheus → Grafana
Enter fullscreen mode Exit fullscreen mode

Docker Desktop was already installed on my Windows PC, but I first had to enable WSL integration from Docker Desktop:

Settings → Resources → WSL Integration
Enter fullscreen mode Exit fullscreen mode

Then, inside WSL, I fixed the Docker permissions:

sudo groupadd docker 2>/dev/null || true
sudo usermod -aG docker $USER
newgrp docker
Enter fullscreen mode Exit fullscreen mode

To verify the setup:

docker version
docker compose version
docker ps
Enter fullscreen mode Exit fullscreen mode

Monitoring Directory 

mkdir -p ~/monitoring/llm-grafana
cd ~/monitoring/llm-grafana

mkdir -p prometheus
mkdir -p grafana/provisioning/datasources
mkdir -p ollama-jsonl-exporter
Enter fullscreen mode Exit fullscreen mode

Prometheus Exporter

The exporter reads the JSONL file and exposes Prometheus metrics on port 9108.

cat > ollama-jsonl-exporter/Dockerfile <<'DOCKER'
FROM python:3.12-slim
WORKDIR /app
COPY exporter.py /app/exporter.py
ENV LOG_PATH=/data/ollama_usage.jsonl
ENV PORT=9108
EXPOSE 9108
CMD ["python", "/app/exporter.py"]
DOCKER
Enter fullscreen mode Exit fullscreen mode

The exporter.py file is the component that converts the JSONL data into Prometheus metrics.

Prometheus

Prometheus is the monitoring engine. It is responsible for collecting and storing metrics such as CPU usage, memory consumption, and application-specific measurements.

cat > prometheus/prometheus.yml <<'YAML'
global:
  scrape_interval: 5s
  evaluation_interval: 5s

scrape_configs:
  - job_name: "prometheus"
    static_configs:
      - targets: ["prometheus:9090"]

  - job_name: "aider-ollama"
    static_configs:
      - targets: ["ollama-jsonl-exporter:9108"]
YAML
Enter fullscreen mode Exit fullscreen mode

Grafana Data Source 

cat > grafana/provisioning/datasources/prometheus.yml <<'YAML'
apiVersion: 1

datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://prometheus:9090
    isDefault: true
YAML
Enter fullscreen mode Exit fullscreen mode

Docker Compose

The working configuration mounts the entire prometheus directory rather than the individual configuration file.

cat > docker-compose.yml <<'YAML'
services:
  ollama-jsonl-exporter:
    build:
      context: ./ollama-jsonl-exporter
    container_name: ollama-jsonl-exporter
    restart: unless-stopped
    volumes:
      - ${HOME}/ollama_usage.jsonl:/data/ollama_usage.jsonl:ro
    ports:
      - "127.0.0.1:9108:9108"

  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus-llm
    restart: unless-stopped
    depends_on:
      - ollama-jsonl-exporter
    command:
      - "--config.file=/etc/prometheus/prometheus.yml"
      - "--storage.tsdb.path=/prometheus"
      - "--storage.tsdb.retention.time=30d"
    volumes:
      - ./prometheus:/etc/prometheus:ro
      - prometheus_data:/prometheus
    ports:
      - "127.0.0.1:9090:9090"

  grafana:
    image: grafana/grafana:latest
    container_name: grafana-llm
    restart: unless-stopped
    depends_on:
      - prometheus
    environment:
      - GF_SECURITY_ADMIN_USER=admin
      - GF_SECURITY_ADMIN_PASSWORD=admin
      - GF_USERS_ALLOW_SIGN_UP=false
    volumes:
      - grafana_data:/var/lib/grafana
      - ./grafana/provisioning:/etc/grafana/provisioning:ro
    ports:
      - "127.0.0.1:3001:3000"

volumes:
  prometheus_data:
  grafana_data:
YAML
Enter fullscreen mode Exit fullscreen mode

To start the stack:

cd ~/monitoring/llm-grafana
docker compose up -d --build
Enter fullscreen mode Exit fullscreen mode

To verify it:

docker compose ps
curl http://127.0.0.1:9108/metrics | head -n 20
curl http://127.0.0.1:9090/-/ready
Enter fullscreen mode Exit fullscreen mode

Grafana is available at:

http://localhost:3001
Enter fullscreen mode Exit fullscreen mode

On the first login, you need to configure a user account.

Useful Grafana Queries

In the Grafana panels, I used Code mode instead of the visual query builder.

Recorded responses:

sum(aider_ollama_calls_total)
Enter fullscreen mode Exit fullscreen mode

Total tokens:

sum(aider_ollama_tokens_total)
Enter fullscreen mode Exit fullscreen mode

Latest response time:

aider_ollama_last_wall_time_seconds
Enter fullscreen mode Exit fullscreen mode

Tokens per second for the latest response:

aider_ollama_last_generation_tokens_per_second
Enter fullscreen mode Exit fullscreen mode

For KPIs, I used Stat panels, while for trends I used Time series panels.

Daily Workflow

Terminal 1:

cd ~
python3 ~/ollama_usage_proxy.py
Enter fullscreen mode Exit fullscreen mode

Terminal 2:

cd ~/repos/my-project
aider14stats
Enter fullscreen mode Exit fullscreen mode

Terminal 3:

cd ~/monitoring/llm-grafana
docker compose up -d
Enter fullscreen mode Exit fullscreen mode

Terminal 4:

source ~/.venvs/aider-dashboard/bin/activate
streamlit run ~/aider_usage_dashboard.py --server.address 127.0.0.1 --server.port 8501
Enter fullscreen mode Exit fullscreen mode

With this setup, I ended up with two complementary views:

Streamlit → operational debugging
Grafana   → stable monitoring
Enter fullscreen mode Exit fullscreen mode

This is still an experimental setup. For a more mature environment, I would probably consolidate everything into a single tool.

Top comments (0)