DEV Community

PEPPERCORN
PEPPERCORN

Posted on

[Day 19] Staff day at the cat AI company: qwen2.5:7b holds the fort, Claude opens a branch office

#localllm #ai #dgxspark #agents

Intro

Day 19!

In Day 18 I set up a company of cats to run my task management. Today is staff day — I put Rusuban-neko, the caretaker, to actual work 🐱

What I used: Ollama (qwen2.5:7b) / Claude Code / DGX Spark / Slack

Today's plan

  • What I want: four things — get Rusuban-neko running, flesh out the four staff cats, prepare for Claude's usage limit, and fix the Slack inbox watcher
  • Done means: I drop a memo in a folder, and Rusuban-neko has done the prep work by the time I look
  • Result:
    • Rusuban-neko is live. No extra API cost, replies in seconds. The snag: it slips into Chinese
    • Limit prep is in place (a usage readout plus a handoff to Codex)
    • Took the inbox watcher off the LLM. No API cost while idle, and reaction time went from up to 4 minutes down to seconds
    • Branch-office Necco now lives on the DGX, so Slack works even when my laptop is asleep

① Rusuban-neko is live

What's Rusuban-neko? = A staff member that is a local LLM (qwen2.5:7b) living on the DGX. Drop a memo in a folder and it writes back a summary, tags, and candidate next actions.

It runs entirely inside the DGX, so there's no API cost. The contents of my memos never leave the house either.

Three things to build.

  1. Foundation: Ollama (the standard local-LLM runtime) was already on the DGX, so I used it as-is. The model is the lightweight qwen2.5:7b
  2. Persona: a system prompt saying "summarize the memo, write tags and the next move"
  3. Plumbing: a script that sweeps the folder and attaches a reply to anything unprocessed 
drop a memo into input/
  → Rusuban-neko reads it
  → leaves a summary, tags, and candidate actions next to it
Enter fullscreen mode Exit fullscreen mode

One snag. Partway through a reply it switched to Chinese. I told it "output in Japanese only", locked the reply into a fixed template, and turned the variance setting to its minimum. That settled it.

Rusuban-neko's output

It even picked up deadlines like "decide by next Wednesday", and replies came back in seconds.

② Fleshed out the four staff cats

I added what I'd learned from actually running the company to the four staff definitions from Day 18.

Staff What I added
Soroban-neko How to hand work to Rusuban-neko, made concrete against the real thing
Shirabe-neko Three angles for research (market, legal, feasibility), plus "don't render a verdict — go as far as the evidence"
Mekiki-neko Added inspecting subcontracted output to its duties
Tsukuri-neko Reports must now state whether the work was actually saved

Tsukuri-neko's line comes from an incident where something was running fine but had never been written to disk.

③ Hand off to Codex when the limit gets close

Claude has a usage limit. Getting cut off mid-task is painful, so I built two things.

  • The readout: usage displayed at all times along the bottom of the screen (a tool called ccusage)
  • The handoff: when the limit gets close, the remaining routine work goes into a handoff document and over to Codex, the subcontractor

The handoff document has three parts: what's done, what I want done (with acceptance criteria), and what not to touch. When I'm back, Mekiki-neko inspects the result against those criteria.

That was under an hour of work in total.

④ Took the Slack inbox watcher off the LLM

The Slack inbox I built in Day 18 worked by having Necco go look every 4 minutes.

  • That's 360 checks a day, against a handful of actual messages
  • The patrol conversation drags its history along, so each round costs more than the last

I switched it to have Slack tell me when something arrives (Socket Mode). A small Python script handles the receiving, and Necco only wakes up once something has actually landed.

Go and look (old) Get told (new)
LLM calls 360/day (even with nothing new) a few a day
Cost while waiting billed none
Reaction time up to 4 min seconds

I posted from my phone and it landed in the file within seconds. Two snags (a truncated config paste and a missing private-channel permission) are in the details below.

⑤ Branch-office Necco moved into the DGX

I wanted Slack to work from the road even with my laptop asleep, so I put a replying agent — branch-office Necco — on the DGX, which runs 24/7.

phone → Slack → the DGX notices the new message
  → branch-office Necco reads the ledger and the journal, replies in Slack
Enter fullscreen mode Exit fullscreen mode

Every reply summons a brand-new Necco (carrying only the last 8 exchanges). That sidesteps the "longer conversation, higher cost" problem. It's read-only, and it won't reply to anything more than 30 minutes old.

One snag. When I ran receivers on both my laptop and the DGX at once, the notifications got split between them. One receiver in the world, that's the rule.

So now Slack works without my laptop in the picture at all.

The current state of the cat company

Where things stand at the end of Day 19.

                    🧑 me
                   /        \
          (talk)  /           \ (post from phone)
                 /             \
   ┌────────────/──────┐    💬 Slack inbox
   │  MY LAPTOP        │        │      ▲
   │  🐈 Necco         │        ▼      │
   │   (chief)         │   ┌───────────┴──────────────┐
   │    │              │   │  DGX  (24/7)             │
   │    ├─ 🐾 4 staff  │   │  📮 receiver (script)    │
   │    │   soroban    │   │       │                  │
   │    │   shirabe    │   │       ▼                  │
   │    │   mekiki     │   │  🐈 branch-office Necco  │
   │    │   tsukuri    │   │       (read-only)        │
   │    │              │   │                          │
   │    └─ 🤖 Codex    │   │  😺 Rusuban-neko         │
   │       (at limit)  │   │     qwen2.5:7b           │
   └───────────────────┘   └──────────────────────────┘
              \                     /
               \                   /
                📒 the ledger & journal (files)
Enter fullscreen mode Exit fullscreen mode

The chief secretary stays on my laptop; holding the fort and handling me on the road happens on the DGX. Nothing on the DGX side costs API money while it waits.

Today's takeaways

  • Make up for a small model with instructions: the Chinese leakage was suppressible with prompt work alone
  • Prep work belongs local: no API cost, seconds, and the data stays home
  • Separate watching from thinking: a script does the waiting, the LLM starts after something arrives
  • An AI's memory is a file: branch-office Necco's replies are only as good as the ledger and journal are written

The details

Surveying the DGX, and picking a model

which ollama                 # → /usr/local/bin/ollama
systemctl list-units | grep ollama   # → ollama.service active (running)
ss -tlnp                     # → LISTEN on 127.0.0.1:11434
ollama list                  # → qwen2.5:7b / qwen2.5vl / qwen2.5-coder:32b / qwen2.5:72b
Enter fullscreen mode Exit fullscreen mode
  • The plan was to stand up a new server (port 8591), but the survey found Ollama already running, so it wasn't needed
  • Ollama serves an OpenAI-compatible API (/v1/chat/completions) on 11434. That became Rusuban-neko's front door as-is
  • It binds to 127.0.0.1 only, which already satisfied the "not exposed externally" requirement
  • Installed models: qwen2.5:7b (4.7GB) / qwen2.5vl (6.0GB) / qwen2.5-coder:32b (19GB) / qwen2.5:72b (47GB). For summarizing, sorting, and tagging I picked the lightest 7b
  • I considered switching to llama.cpp server, but quality comes from the model rather than the server software, so the existing Ollama stayed
  • Aside: nvidia-smi shows [N/A] in the memory column. The DGX Spark (GB10) shares memory between CPU and GPU, so GPU-only usage doesn't appear

Inside the Chinese fix

  • The system prompt now states "output in Japanese only. Chinese and English are forbidden"
  • Output format locked to a Markdown template (summary / tags / candidate actions) — less room for free composition means more stable language
  • temperature 0.3 → 0.1
  • After the fix, usage was 158 prompt / 110 completion tokens. It's a 7b, so replies take seconds

Inside the caretaker script

  • It's a single bash script with python3 embedded to assemble JSON safely (no jq dependency)
  • Walks input/ → asks qwen about any file without a .digest.md → writes <original name>.digest.md next to it
  • Originals untouched, processed files skipped, empty files marked SKIP
  • Recovery steps (restarting Ollama) went into the setup runbook

Why Rusuban-neko's persona is not a staff definition file

The four staff cats are defined in agents/, and an install script places them into ~/.claude/ as Claude Code subagents.

It's tempting to put Rusuban-neko there too, but I didn't. It's a local LLM, so a definition in agents/ would grow a "Rusuban-neko" on the Claude Code side that isn't actually qwen. The persona lives inside the script instead, keeping one source of truth.

The handoff document (HANDOFF.md)

  • Written to the project root (when the budget is nearly out, this file takes priority over explaining anything)
  • Four parts: current state (what's done, what's verified) / the request list (with acceptance criteria, spec-complete work only) / what not to touch (no design decisions, no spec changes) / where to report when finished
  • It targets the usage limit only. The context limit is handled by auto-compaction, so this skill isn't for that
  • On return, Mekiki-neko checks against the acceptance criteria → if it passes, HANDOFF.md is deleted and one line goes into the record

Inside "get told" (Socket Mode)

  • Uses Slack's Socket Mode, so Slack can push over WebSocket without exposing my machine
  • I created the Slack app by pasting a manifest (a JSON config). Socket Mode, event subscriptions, and scopes all land in one shot. The App-Level Token was generated automatically
  • Minimal scopes, two of them: channels:history (public) plus groups:history (private)
  • The receiver script is Python plus slack-sdk only. It appends human posts, one JSON line at a time
  • Bot messages and edit events are ignored mechanically, which stops the loop where a reply triggers its own reaction
  • flush + fsync on every append, so the watcher notices immediately
  • The ack goes back within 3 seconds. Slack re-sends the notification if you're late

Snags in the inbox switchover

  1. Truncated manifest: pasting a heavily-indented JSON into the browser cut off partway and threw a JSON error. Re-pasting a compact version fixed it
  2. Private channel scopes: Slack splits scopes into public (channels:*) and private (groups:*). Channel IDs start with C either way, so there's no telling them apart — a missing_scope error is what finally pinned it down
    • Note: after changing scopes you need to reinstall the app and restart the receiver. The token string didn't change

Inside branch-office Necco

  • Receiver and replier scripts run as systemd services on the DGX (enable --now plus loginctl enable-linger, so they come back after a reboot)
  • The replier notices the file append → launches claude -p exactly once (read-only, Read/Grep/Glob permitted) → reads the ledger and journal → posts to Slack as a bot
  • A fresh short context every time (only the last 8 exchanges included), so per-reply cost stays flat and nothing accrues while idle
  • Safety rails, fixed in the prompt: skip anything present at startup / no replies to messages older than 30 minutes / batch consecutive posts into one / hold write requests and pass them to the main Necco

Tomorrow: Day 20

Six video-generation tools, head to head (2026 edition). Same cat image into all six, comparing quality and time.

Thanks for reading!

Top comments (0)