DEV Community: Alain Airom (Ayrom)

Running vinD with Podman: Local Kubernetes without Docker Desktop

Alain Airom (Ayrom) — Thu, 03 Sep 2026 10:54:41 +0000

Transforming vinD to vinP (vCluster in Podman) on macOS!

Introduction

I always wanted to test vinD which seems more complete than Kind for a local test and deployment. However the name is "vind - vCluster in Docker" and I use Podman... so I put IBM Bob into work to make all what it takes to use vinD with Podman locally.

While vinD was explicitly designed as a Docker-native tool that expects standard Docker socket endpoints and CLI responses, migrating the setup to a macOS workstation running Podman Desktop is entirely feasible with a few targeted compatibility shims and VM adjustments.

This post walks through the architectural considerations, step-by-step implementation, custom compatibility layers, and deployment manifests required to bring vinD up and running seamlessly on Podman.

TL;DR: What is vinD?

tl;dr — vinD (vCluster in Docker) gives you a real, lightweight Kubernetes cluster running inside a single container in seconds.

Real Kubernetes Control Plane: Unlike standard namespaces, vcluster create provisions a fully isolated, dedicated Kubernetes control plane (API server, etcd, controller manager) inside one host container.
High-Speed Provisioning & Resource Efficiency: Starts in ~15 seconds with a tiny RAM footprint (~150 MB). It supports sleep/wake capabilities (vcluster pause / vcluster resume) so an idle cluster uses zero CPU and zero memory.
Full Tenant Isolation: Multiple isolated vClusters can run side-by-side on a single host machine with separate CRDs, RBAC, and Kubernetes API versions without interfering with each other.
Native Tooling: You interact with vinD using standard kubectl, helm, and vcluster CLI commands just as you would with EKS, GKE, or Kind.

Excerpt from vinD Github;

🎯 What is vind?

vind (vCluster in Docker) is an open-source way to run Kubernetes clusters directly as Docker containers. Built on top of vCluster, vind combines the power of virtual Kubernetes clusters with the simplicity of Docker, creating isolated Kubernetes environments that are perfect for development, testing, and CI/CD pipelines.

Note: vind uses vCluster's Private Nodes mode internally. This is automatically enabled when using the Docker driver and is required for proper operation. This is expected behavior, not a configuration issue.

Why vind?

🚀 Faster than KinD - Optimized container-based architecture
💤 Sleep & Wake - Pause clusters to save resources, resume in under 3 seconds
🎨 Built-in UI - Free vCluster Platform UI for cluster management
⚡ Load Balancers OOB - Automatic LoadBalancer services without extra setup
🐳 Docker Native - Leverages Docker's networking and storage
🔄 Pull-through Cache - Faster image pulls via local Docker daemon
🌐 Hybrid Nodes - Join external nodes (even cloud instances) via VPN
📸 Snapshots - Save cluster state to OCI registries, S3, or local files and restore them
🔧 In-Place K8s Upgrades - Upgrade Kubernetes version without deleting the cluster

Architecture Overview

High-Level Architecture

The following diagram illustrates how the macOS host environment interfaces with the underlying Podman VM and vinD control plane container:

Key Differences & Podman Compatibility Shims

To bridge the gap between vcluster (which assumes a native Docker daemon) and Podman, four specific adjustments are required:

Custom Docker-to-Podman CLI Shim: When probing network subnets, vcluster issues docker network inspect --format '{{.IPAM.Config}}'. Podman formats network inspect JSON under .subnets[].subnet. A lightweight shim intercepts this call and formats the JSON output accordingly.
br_netfilter Kernel Module: Flannel CNI requires the br_netfilter module loaded in the Podman Machine VM. Loading this module explicitly prevents CNI initialization failure.
DOCKER_HOST Socket Redirection: Standard API interactions route through Podman’s Docker-compatible API socket (unix:///.../podman.sock).
Architecture-Aware Image Build (TARGETARCH): Multi-stage container builds must respect the Apple Silicon (arm64) execution environment of the Podman VM to avoid runtime exec format error issues.

**Attention:* Installing vinD on macOS using Homebrew standard commands is not sufficient on its own*. Because the loft-sh/tap repository isn't trusted by default, I used **WailBrew* to explicitly flag and manage the tap as a trusted runnable source.*

Implementation

Application Source & Multi-Stage Containerfile

A lightweight HTTP service written in Go serves as the test workload.

app/main.go

// Package main provides a minimal HTTP Hello World server.
// Port is controlled via the PORT environment variable (default: 8081).
package main

import (
    "fmt"
    "log"
    "net/http"
    "os"
)

// getPort returns the port to listen on.
// It reads from the PORT environment variable; falls back to 8081.
func getPort() string {
    port := os.Getenv("PORT")
    if port == "" {
        port = "8081"
    }
    return port
}

// helloHandler writes "Hello, World!" to the response.
func helloHandler(w http.ResponseWriter, r *http.Request) {
    log.Printf("Request received: %s %s from %s", r.Method, r.URL.Path, r.RemoteAddr)
    fmt.Fprintln(w, "Hello, World!")
}

// healthHandler provides a basic liveness probe endpoint.
func healthHandler(w http.ResponseWriter, r *http.Request) {
    w.WriteHeader(http.StatusOK)
    fmt.Fprintln(w, "OK")
}

// newMux builds and returns the HTTP ServeMux with all routes registered.
func newMux() *http.ServeMux {
    mux := http.NewServeMux()
    mux.HandleFunc("/", helloHandler)
    mux.HandleFunc("/healthz", healthHandler)
    return mux
}

func main() {
    port := getPort()
    addr := ":" + port

    mux := newMux()

    log.Printf("Starting Hello World server on %s", addr)
    if err := http.ListenAndServe(addr, mux); err != nil {
        log.Fatalf("Server failed: %v", err)
    }
}

app/Containerfile

# Stage 1 — Builder
FROM docker.io/library/golang:1.21-alpine AS builder

WORKDIR /src

COPY go.mod ./
RUN go mod download

COPY . .
RUN go test ./... -v

ARG TARGETARCH
RUN CGO_ENABLED=0 GOOS=linux GOARCH=${TARGETARCH:-$(go env GOARCH)} \
    go build -ldflags="-s -w" -o /out/helloworld .

# Stage 2 — Runtime
FROM scratch

COPY --from=builder /out/helloworld /helloworld

EXPOSE 8081
ENV PORT=8081
USER 65534:65534

ENTRYPOINT ["/helloworld"]

Kubernetes Deployment Manifests

The service is exposed on NodePort 30080, which maps to the host.

manifests/namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: helloworld
  labels:
    app.kubernetes.io/managed-by: vind

manifests/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: helloworld
  namespace: helloworld
  labels:
    app: helloworld
    app.kubernetes.io/name: helloworld
    app.kubernetes.io/version: "1.0.0"
spec:
  replicas: 1
  selector:
    matchLabels:
      app: helloworld
  template:
    metadata:
      labels:
        app: helloworld
    spec:
      securityContext:
        runAsNonRoot: true
        runAsUser: 65534
        runAsGroup: 65534
      containers:
        - name: helloworld
          image: localhost/helloworld:latest
          imagePullPolicy: Never
          ports:
            - containerPort: 8081
              protocol: TCP
              name: http
          env:
            - name: PORT
              value: "8081"
          resources:
            requests:
              cpu: "50m"
              memory: "32Mi"
            limits:
              cpu: "200m"
              memory: "64Mi"
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8081
            initialDelaySeconds: 5
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8081
            initialDelaySeconds: 3
            periodSeconds: 5

manifests/service.yaml

apiVersion: v1
kind: Service
metadata:
  name: helloworld
  namespace: helloworld
  labels:
    app: helloworld
    app.kubernetes.io/name: helloworld
spec:
  type: NodePort
  selector:
    app: helloworld
  ports:
    - name: http
      port: 80
      targetPort: 8081
      nodePort: 30080
      protocol: TCP

and even a Load Balancer (manifests/loadbalancer-service.yaml)

  ---
  # =============================================================================
  # Kubernetes Service — helloworld (LoadBalancer type)
  #
  # vinD provides automatic LoadBalancer support out-of-the-box.
  # The EXTERNAL-IP is assigned from the Podman/Docker bridge network.
  # Apply this INSTEAD of service.yaml if you prefer LoadBalancer over NodePort.
  #
  # Apply with:
  #   kubectl apply -f manifests/loadbalancer-service.yaml
  # =============================================================================
  apiVersion: v1
  kind: Service
  metadata:
    name: helloworld-lb
    namespace: helloworld
    labels:
      app: helloworld
      app.kubernetes.io/name: helloworld
  spec:
    type: LoadBalancer
    selector:
      app: helloworld
    ports:
      - name: http
        port: 80           # External port
        targetPort: 8081   # Container port
        protocol: TCP

Orchestration & Podman Shim Setup Script

The orchestration script manages binary downloads, setting up the docker CLI translation shim, enabling br_netfilter in the Podman VM, creating the vCluster instance, importing images into containerd, and deploying manifests.

scripts/setup.sh

#!/usr/bin/env bash
set -euo pipefail

CLUSTER_NAME="${CLUSTER_NAME:-vind-helloworld}"
APP_IMAGE="${APP_IMAGE:-localhost/helloworld:latest}"

echo "==> 1. Ensuring vcluster CLI is installed..."
if ! command -v vcluster &> /dev/null; then
  mkdir -p "$HOME/.local/bin"
  curl -fsSL "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-darwin-arm64" \
    -o "$HOME/.local/bin/vcluster"
  chmod +x "$HOME/.local/bin/vcluster"
  export PATH="$HOME/.local/bin:$PATH"
fi

echo "==> 2. Detecting Podman Socket..."
PODMAN_SOCK=$(podman info --format '{{.Host.RemoteSocket.Path}}' 2>/dev/null || true)
if [ -z "$PODMAN_SOCK" ]; then
  PODMAN_SOCK="$HOME/.local/share/containers/podman/machine/qemu/podman.sock"
fi
export DOCKER_HOST="unix://${PODMAN_SOCK}"

echo "==> 3. Creating docker -> podman CLI shim with IPAM translation..."
mkdir -p "$HOME/.local/bin"
cat << 'EOF' > "$HOME/.local/bin/docker"
#!/usr/bin/env bash
if [[ "$*" == *"network inspect"* ]] && [[ "$*" == *"IPAM"* ]]; then
  NET_NAME="${@:$#}"
  SUBNET=$(podman network inspect "$NET_NAME" --format '{{range .subnets}}{{.subnet}}{{end}}' 2>/dev/null)
  echo "[{\"Subnet\":\"${SUBNET}\"}]"
  exit 0
fi
exec podman "$@"
EOF
chmod +x "$HOME/.local/bin/docker"
export PATH="$HOME/.local/bin:$PATH"

echo "==> 3b. Loading br_netfilter module in Podman VM..."
podman machine ssh "sudo modprobe br_netfilter" || true
podman machine ssh "echo 'br_netfilter' | sudo tee /etc/modules-load.d/br_netfilter.conf" || true

echo "==> 4. Configuring vCluster driver..."
vcluster use driver docker

echo "==> 5. Creating vinD cluster..."
if ! vcluster list | grep -q "$CLUSTER_NAME"; then
  vcluster create "$CLUSTER_NAME" -f vcluster.yaml
fi

echo "==> 6. Building Go application image..."
podman build -t "$APP_IMAGE" -f app/Containerfile app/

echo "==> 7. Importing image into cluster containerd..."
podman save "$APP_IMAGE" | docker exec -i "vcluster.cp.${CLUSTER_NAME}" ctr --namespace k8s.io images import -

echo "==> 8. Applying Kubernetes manifests..."
kubectl apply -f manifests/namespace.yaml
kubectl apply -f manifests/deployment.yaml
kubectl apply -f manifests/service.yaml

echo "==> 9. Waiting for deployment readiness..."
kubectl rollout status deployment/helloworld -n helloworld --timeout=60s

echo "==> Setup complete! Test endpoint:"
echo "curl http://localhost:30080"

Conclusion

By addressing the four core points of divergence between Docker and Podman environments—CLI output formatting, kernel module presence in the underlying machine VM, platform-native image builds, and API socket location—vinD runs efficiently on top of Podman.

This setup offers a fast local deployment pipeline, retaining vCluster's single-container control plane model while operating fully within open-source, Podman-based development tooling.

Thanks for reading 🍇

Automating AI Context: How I Built a Custom Extension for IBM Bob IDE to Inject Project Rules

Alain Airom (Ayrom) — Mon, 31 Aug 2026 12:23:50 +0000

Building my own “vsix” extension to automate my own projects

Introduction

Setting up consistent workspace instructions across projects can quickly become a tedious chore. In IBM Bob IDE, system prompt behaviors and coding standards are guided by a project-level AGENTS.md file located at the root of your repository. While effective, manually copying a master rules file into every newly created or cloned workspace breaks flow and easily leads to configuration drift.

Note: While AGENTS.md can be configured globally across all your projects, I prefer maintaining project-level files. This allows me to start from a shared foundational rule set while giving each workspace the freedom to include custom, project-specific tweaks.

TL;DR-What is AGENTS.MD?

In a modern Software Development Lifecycle (SDLC), an AGENTS.md file is a machine-readable context and configuration specification checked directly into a project repository to dictate how autonomous AI coding agents (such as Cursor, IBM Bob, Claude Code, or GitHub Copilot) interact with the codebase. Functioning as an authoritative set of runtime instructions loaded into the AI’s prompt context, it explicitly defines executable CLI commands, coding standards, project structure maps, architectural constraints, and operational boundaries—such as distinguishing actions the AI can perform autonomously versus those requiring human approval. By checking AGENTS.md into version control alongside application code, software engineering teams standardize AI behavior across developer environments, prevent AI configuration drift, reduce model-generated bugs, and ensure seamless alignment with project-specific development guidelines.

Example;

  # Project Development Rule
   - rule 1
   - rule 2
   ...

To eliminate this manual step, I built bob-agents-injector—a custom extension designed specifically for IBM Bob IDE. Developed directly within Bob using its native capabilities, the extension automatically injects a canonical AGENTS.md file from a central source path (defaulting to ~/Devs/AGENTS.MD) into any workspace as soon as it is opened.

~/Devs/AGENTS.MD            ← Master rules (single source of truth)
        │
        └──► <project>/AGENTS.md   ← Injected automatically when project opens

Implementation

Hooking into the IDE Event Loop

The key to seamless injection is to align the extension with VS Code's exact activation lifecycle. So I asked Bob to build the required extension to automate copying the AGENTS.MD from my global "Devs" folder each time I create a new folder for a new project using Bob IDE!

Bob IDE is a fork of VS Code (Microsoft's open-source Code - OSS).

	VS Code	Bob IDE
Source	Microsoft (closed binary, open source core)	IBM fork of `Code - OSS`
Extension marketplace	VS Code Marketplace	IBM's own marketplace + Open VSX
Telemetry	Microsoft	IBM
Binary name	`code`	`bobide` / `bobide-insiders`
Extensions dir	`~/.vscode/extensions`	`~/.bobide/extensions`

scripts/
├── bob-agents-injector/
│   ├── package.json
│   ├── out/extension.js
│   └── build.js
└── install-bob-agents-injector.sh   ← optional convenience wrapper

onStartupFinished: Fires when Bob completes initial loading.
onDidChangeWorkspaceFolders: Fires dynamically whenever a new project folder is opened or swapped.

// =============================================================================
// Bob AGENTS.md Auto-Injector — extension.ts (source, compile with build.js)
//
// This extension mirrors the EXACT trigger chain used by the bob-marketplace
// extension to inject .bob/mcp.json, but instead copies ~/Devs/AGENTS.MD to
// <workspaceRoot>/AGENTS.md at the same moment:
//
//   Trigger 1: onStartupFinished  → fires when Bob IDE is ready at launch
//   Trigger 2: onDidChangeWorkspaceFolders → fires when a NEW project is opened
//
// Both map directly to the two events in bob-marketplace/out/extension.js:
//   void mcpServer.start()                          (at activate time)
//   vscode.workspace.onDidChangeWorkspaceFolders(   (at workspace change)
//       () => void mcpServer.reregister())
//
// Idempotency: if AGENTS.md already exists in the workspace root and its
// content matches the source file, nothing is written. If it exists but
// differs, it is left untouched (user may have customised it).
// =============================================================================

'use strict';

const vscode = require('vscode');
const fs = require('fs');
const path = require('path');
const os = require('os');

// ── Configuration helpers ────────────────────────────────────────────────────
function getConfig() {
    const cfg = vscode.workspace.getConfiguration('bob.agentsInjector');
    const rawSource = cfg.get('sourcePath', '').trim()
        || path.join(os.homedir(), 'Devs', 'AGENTS.MD');
    // Expand leading ~ (vscode settings may contain literal ~)
    const sourcePath = rawSource.startsWith('~')
        ? path.join(os.homedir(), rawSource.slice(1))
        : rawSource;
    return {
        enabled: cfg.get('enabled', true),
        sourcePath,
        targetFileName: cfg.get('targetFileName', 'AGENTS.md').trim() || 'AGENTS.md',
    };
}

// ── Core inject function ─────────────────────────────────────────────────────
/**
 * Copies the master AGENTS.MD into the workspace root as AGENTS.md,
 * following the same logic as MarketplaceMcpServer.registerIntoWorkspace():
 *   - reads existing target (non-fatal if missing)
 *   - skips if content is identical (idempotent)
 *   - skips if content differs (user may have customised it)
 *   - creates the target file only when it does not exist
 */
async function injectAgentsMd() {
    const cfg = getConfig();

    if (!cfg.enabled) {
        return;
    }

    // ── Validate source ───────────────────────────────────────────────────────
    if (!fs.existsSync(cfg.sourcePath) || !fs.statSync(cfg.sourcePath).isFile()) {
        // Non-fatal: source missing means the user hasn't set up the master file.
        // Log quietly — don't pop an annoying dialog on every project open.
        console.warn(`[bob-agents-injector] Source not found: ${cfg.sourcePath}. ` +
            `Set 'bob.agentsInjector.sourcePath' in settings.`);
        return;
    }

    // ── Resolve workspace root ────────────────────────────────────────────────
    const folders = vscode.workspace.workspaceFolders;
    if (!folders || folders.length === 0) {
        return; // No workspace open yet — identical to bob-marketplace's behaviour
    }

    // Mirror bob-marketplace: use first workspace folder (workspaceFolders[0])
    const workspaceRoot = folders[0].uri.fsPath;
    const targetPath = path.join(workspaceRoot, cfg.targetFileName);

    // ── Read source content ────────────────────────────────────────────────────
    let sourceContent;
    try {
        sourceContent = fs.readFileSync(cfg.sourcePath);
    } catch (err) {
        console.warn(`[bob-agents-injector] Could not read source: ${err.message}`);
        return;
    }

    // ── Check existing target ─────────────────────────────────────────────────
    if (fs.existsSync(targetPath)) {
        try {
            const existing = fs.readFileSync(targetPath);
            if (existing.equals(sourceContent)) {
                // Already up-to-date — silent no-op (same as bob-marketplace
                // behaviour when mcp.json already has the right URL)
                return;
            } else {
                // Target exists and differs — user may have customised it.
                // Do NOT overwrite. Log for visibility.
                console.log(`[bob-agents-injector] ${cfg.targetFileName} exists and ` +
                    `differs from master — skipping to preserve local changes.`);
                return;
            }
        } catch {
            // Can't read existing file — try writing anyway
        }
    }

    // ── Write target ──────────────────────────────────────────────────────────
    try {
        fs.writeFileSync(targetPath, sourceContent);
        console.log(`[bob-agents-injector] ✅ Injected ${cfg.targetFileName} → ${targetPath}`);
    } catch (err) {
        console.warn(`[bob-agents-injector] Could not write target: ${err.message}`);
    }
}

// ── Extension entry points ───────────────────────────────────────────────────
function activate(context) {
    console.log('[bob-agents-injector] Activating (onStartupFinished)');

    // ── Trigger 1: at IDE startup — mirror mcpServer.start() call ────────────
    void injectAgentsMd();

    // ── Trigger 2: when workspace changes — mirror onDidChangeWorkspaceFolders
    //    This is the KEY trigger: fires when the user opens a NEW project folder
    //    in Bob, which is the exact same moment .bob/mcp.json is (re-)injected.
    context.subscriptions.push(
        vscode.workspace.onDidChangeWorkspaceFolders(() => {
            void injectAgentsMd();
        })
    );

    // ── Trigger 3: settings change — mirror bob.marketplace.installLocation ──
    context.subscriptions.push(
        vscode.workspace.onDidChangeConfiguration((e) => {
            if (e.affectsConfiguration('bob.agentsInjector')) {
                void injectAgentsMd();
            }
        })
    );
}

function deactivate() {
    /* nothing to clean up */
}

module.exports = { activate, deactivate };

Idempotent Injections & Safety Checks

Automatically modifying workspace files requires strict safeguards to avoid overwriting intentionally customized local rules. The injectAgentsMd() execution loop enforces idempotency through a simple decision flow:

Verify Source: Ensures ~/Devs/AGENTS.MD exists before taking action.
Missing Target: If no AGENTS.md is present at the target workspace root, it creates one from the master file.
Identical Content: If the target matches the master file byte-for-byte, execution skips silently.
Modified Target: If the target exists but differs from the master file (indicating local edits), execution skips to preserve project-specific overrides.

Local `AGENTS.md` State	Extension Action
Does not exist	Copy master `AGENTS.MD` to workspace root MD+ 2
Exists (identical)	Skip silently (up-to-date) MD+ 2
Exists (modified)	Skip execution (preserve local customizations) MD+ 2

Dependency-Free Packaging and Installation

To keep the development footprint minimal, the project packages its VSIX installer using a zero-dependency build.js script reliant solely on Node.js built-ins (fs, path, and zlib). It constructs a valid .vsix archive directly from raw buffers, complete with proper CRC-32 checksums and ZIP structure formatting.

#!/usr/bin/env node
// =============================================================================
// build.js — packages bob-agents-injector into a .vsix file
//
// Usage (run from scripts/bob-agents-injector/):
//   node build.js
//
// Then install into Bob:
//   /Applications/IBM\ Bob\ -\ Insiders.app/Contents/Resources/app/bin/bobide-insiders \
//     --install-extension bob-agents-injector-1.0.0.vsix
//
// No npm install needed — uses only Node.js built-ins.
// =============================================================================

'use strict';

const fs   = require('fs');
const path = require('path');
const zlib = require('zlib');

const DIR  = __dirname;
const PKG  = JSON.parse(fs.readFileSync(path.join(DIR, 'package.json'), 'utf8'));
const VSIX = path.join(DIR, `${PKG.name}-${PKG.version}.vsix`);

// ── VSIX manifest ─────────────────────────────────────────────────────────────
const VSIX_MANIFEST = `<?xml version="1.0" encoding="utf-8"?>
<PackageManifest Version="2.0.0"
  xmlns="http://schemas.microsoft.com/developer/vsx-schema/2011"
  xmlns:d="http://schemas.microsoft.com/developer/vsx-schema-design/2011">
  <Metadata>
    <Identity Language="en-US"
      Id="${PKG.name}"
      Version="${PKG.version}"
      Publisher="${PKG.publisher}"/>
    <DisplayName>${PKG.displayName}</DisplayName>
    <Description>${PKG.description}</Description>
    <Tags>bob,agents,automation</Tags>
    <GalleryFlags>Public</GalleryFlags>
    <License>MIT</License>
    <Categories>Other</Categories>
  </Metadata>
  <Installation>
    <InstallationTarget Id="Microsoft.VisualStudio.Code" Version="[1.85,)"/>
  </Installation>
  <Dependencies/>
  <Assets>
    <Asset Type="Microsoft.VisualStudio.Code.Manifest"
           Path="extension/package.json"
           Addressable="true"/>
  </Assets>
</PackageManifest>`;

// ── [Content_Types].xml ───────────────────────────────────────────────────────
const CONTENT_TYPES = `<?xml version="1.0" encoding="utf-8"?>
<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types">
  <Default Extension="json"        ContentType="application/json"/>
  <Default Extension="js"          ContentType="application/javascript"/>
  <Default Extension="vsixmanifest" ContentType="text/xml"/>
</Types>`;

// ── CRC-32 (standard ZIP polynomial) ─────────────────────────────────────────
const CRC_TABLE = (() => {
  const t = new Uint32Array(256);
  for (let n = 0; n < 256; n++) {
    let c = n;
    for (let k = 0; k < 8; k++) c = (c & 1) ? (0xedb88320 ^ (c >>> 1)) : (c >>> 1);
    t[n] = c;
  }
  return t;
})();

function crc32(buf) {
  let crc = 0xffffffff;
  for (let i = 0; i < buf.length; i++) crc = CRC_TABLE[(crc ^ buf[i]) & 0xff] ^ (crc >>> 8);
  return (crc ^ 0xffffffff) >>> 0;
}

// ── DOS date/time for ZIP headers ─────────────────────────────────────────────
function dosDateTime(d) {
  const date = ((d.getFullYear() - 1980) << 9) | ((d.getMonth() + 1) << 5) | d.getDate();
  const time = (d.getHours() << 11) | (d.getMinutes() << 5) | Math.floor(d.getSeconds() / 2);
  return { date, time };
}

// ── Build one ZIP entry (local header + data + central-dir record) ────────────
function makeEntry(name, rawData, now) {
  const nameBytes = Buffer.from(name, 'utf8');
  const compressed = zlib.deflateRawSync(rawData, { level: 9 });
  const crc  = crc32(rawData);
  const { date, time } = dosDateTime(now);

  // Local file header (signature 0x04034b50)
  const local = Buffer.alloc(30 + nameBytes.length);
  local.writeUInt32LE(0x04034b50, 0);   // signature
  local.writeUInt16LE(20, 4);           // version needed to extract (2.0)
  local.writeUInt16LE(0, 6);            // general purpose bit flag
  local.writeUInt16LE(8, 8);            // compression method: deflate
  local.writeUInt16LE(time, 10);        // last mod file time
  local.writeUInt16LE(date, 12);        // last mod file date
  local.writeUInt32LE(crc, 14);         // crc-32
  local.writeUInt32LE(compressed.length, 18); // compressed size
  local.writeUInt32LE(rawData.length, 22);    // uncompressed size
  local.writeUInt16LE(nameBytes.length, 26);  // file name length
  local.writeUInt16LE(0, 28);           // extra field length
  nameBytes.copy(local, 30);

  return {
    localBlock: Buffer.concat([local, compressed]),
    crc, date, time,
    compressedSize: compressed.length,
    uncompressedSize: rawData.length,
    nameBytes,
  };
}

// ── Central directory record for one entry ────────────────────────────────────
function centralRecord(entry, localOffset) {
  const rec = Buffer.alloc(46 + entry.nameBytes.length);
  rec.writeUInt32LE(0x02014b50, 0);   // central dir signature
  rec.writeUInt16LE(20, 4);           // version made by
  rec.writeUInt16LE(20, 6);           // version needed
  rec.writeUInt16LE(0, 8);            // general purpose bit flag
  rec.writeUInt16LE(8, 10);           // compression method: deflate
  rec.writeUInt16LE(entry.time, 12);
  rec.writeUInt16LE(entry.date, 14);
  rec.writeUInt32LE(entry.crc, 16);
  rec.writeUInt32LE(entry.compressedSize, 20);
  rec.writeUInt32LE(entry.uncompressedSize, 24);
  rec.writeUInt16LE(entry.nameBytes.length, 28); // file name length
  rec.writeUInt16LE(0, 30);           // extra field length
  rec.writeUInt16LE(0, 32);           // file comment length
  rec.writeUInt16LE(0, 34);           // disk number start
  rec.writeUInt16LE(0, 36);           // internal file attributes
  rec.writeUInt32LE(0, 38);           // external file attributes
  rec.writeUInt32LE(localOffset, 42); // relative offset of local header
  entry.nameBytes.copy(rec, 46);
  return rec;
}

// ── End of central directory record ──────────────────────────────────────────
function eocdRecord(entryCount, centralSize, centralOffset) {
  const eocd = Buffer.alloc(22);
  eocd.writeUInt32LE(0x06054b50, 0);   // end of central dir signature
  eocd.writeUInt16LE(0, 4);            // disk number
  eocd.writeUInt16LE(0, 6);            // disk with central dir
  eocd.writeUInt16LE(entryCount, 8);   // entries on this disk
  eocd.writeUInt16LE(entryCount, 10);  // total entries
  eocd.writeUInt32LE(centralSize, 12);
  eocd.writeUInt32LE(centralOffset, 16);
  eocd.writeUInt16LE(0, 20);           // comment length
  return eocd;
}

// ── Assemble VSIX (= ZIP) ─────────────────────────────────────────────────────
function buildVsix() {
  const now = new Date();

  // Files to pack — ORDER matters: [Content_Types].xml and .vsixmanifest FIRST
  const files = [
    { name: '[Content_Types].xml',        data: Buffer.from(CONTENT_TYPES, 'utf8') },
    { name: '.vsixmanifest',              data: Buffer.from(VSIX_MANIFEST, 'utf8') },
    { name: 'extension/package.json',     data: fs.readFileSync(path.join(DIR, 'package.json')) },
    { name: 'extension/out/extension.js', data: fs.readFileSync(path.join(DIR, 'out', 'extension.js')) },
  ];

  const localBlocks  = [];
  const centralRecs  = [];
  let   offset       = 0;

  for (const { name, data } of files) {
    const entry = makeEntry(name, data, now);
    centralRecs.push(centralRecord(entry, offset));
    localBlocks.push(entry.localBlock);
    offset += entry.localBlock.length;
  }

  const centralBuf = Buffer.concat(centralRecs);
  const eocd = eocdRecord(files.length, centralBuf.length, offset);

  fs.writeFileSync(VSIX, Buffer.concat([...localBlocks, centralBuf, eocd]));

  const kb = (fs.statSync(VSIX).size / 1024).toFixed(1);
  console.log(`✅  Built: ${VSIX}  (${kb} KB)`);
  console.log('');
  console.log('Or install directly with (use the correct variant for your machine):');
  console.log(`  # Standard IBM Bob:`);
  console.log(`  /Applications/IBM\\ Bob.app/Contents/Resources/app/bin/bobide --install-extension "${VSIX}"`);
  console.log(`  # IBM Bob - Insiders:`);
  console.log(`  /Applications/IBM\\ Bob\\ -\\ Insiders.app/Contents/Resources/app/bin/bobide-insiders --install-extension "${VSIX}"`);
  console.log('');
  console.log('Or just run:  bash scripts/install-bob-agents-injector.sh');
}

buildVsix();

The dependencies of the code;

  {
    "name": "bob-agents-injector",
    "displayName": "Bob AGENTS.md Auto-Injector",
    "description": "Automatically copies ~/Devs/AGENTS.MD into AGENTS.md at the root of every workspace opened in IBM Bob — using the same trigger timing as the bob-marketplace mcp.json injection.",
    "version": "1.0.0",
    "publisher": "local",
    "license": "MIT",
    "engines": { "vscode": "^1.85.0" },
    "categories": ["Other"],
    "activationEvents": ["onStartupFinished"],
    "main": "./out/extension.js",
    "contributes": {
      "configuration": {
        "title": "Bob AGENTS.md Injector",
        "properties": {
          "bob.agentsInjector.sourcePath": {
            "type": "string",
            "default": "",
            "description": "Absolute path to the master AGENTS.MD. Defaults to ~/Devs/AGENTS.MD. Supports ~ expansion."
          },
          "bob.agentsInjector.targetFileName": {
            "type": "string",
            "default": "AGENTS.md",
            "description": "File name to write in the workspace root. Bob reads 'AGENTS.md' (case-insensitive on macOS)."
          },
          "bob.agentsInjector.enabled": {
            "type": "boolean",
            "default": true,
            "description": "Enable or disable automatic AGENTS.md injection."
          }
        }
      }
    },
    "scripts": {
      "compile": "node build.js",
      "package": "node build.js && bob --install-extension bob-agents-injector-1.0.0.vsix"
    },
    "devDependencies": {}
  }

The next step is to build the ".vsix" file.

# Package the VSIX without npm dependencies
node build.js

# Install automatically into IBM Bob or IBM Bob Insiders
bash scripts/install-bob-agents-injector.sh

Once the bash runs, the extension is in place.

  ╔══════════════════════════════════════════════════════════════════════╗
  ║  ✅  bob-agents-injector is now installed!                           ║
  ╠══════════════════════════════════════════════════════════════════════╣
  ║                                                                      ║
  ║  How it works:                                                       ║
  ║  • On every project open, Bob fires onDidChangeWorkspaceFolders     ║
  ║  • The injector catches that event (same as bob-marketplace does)   ║
  ║  • It copies ~/Devs/AGENTS.MD → <project>/AGENTS.md                ║
  ║  • If AGENTS.md already exists and is unmodified, it skips it       ║
  ║  • If AGENTS.md was customised locally, it leaves it untouched      ║
  ║                                                                      ║
  ║  Bob's RuleLoader reads:  <workspaceRoot>/AGENTS.md                 ║
  ║  (NOT .bob/AGENTS.MD — that path is not read by Bob)                ║
  ║                                                                      ║
  ║  Optional settings (Bob Settings → search 'agentsInjector'):        ║
  ║    bob.agentsInjector.sourcePath    custom source path               ║
  ║    bob.agentsInjector.targetFileName  target file name               ║
  ║    bob.agentsInjector.enabled         toggle on/off                  ║
  ║                                                                      ║
  ║  To trigger immediately: reload the Bob window                      ║
  ║    Menu → Developer → Reload Window  (or Cmd+Shift+P → Reload)      ║
  ╚══════════════════════════════════════════════════════════════════════╝

Once installed, user preferences can be adjusted anytime via Bob Settings (Cmd+,) under the bob.agentsInjector namespace to customize target file naming or update source paths.

Conclusion

Automating workspace preparation ensures that every AI interaction starts with consistent context, constraints, and project rules without adding repetitive setup steps. By mirroring IBM Bob’s internal event models, bob-agents-injector bridges the gap between global master rules and individual repository management.

Whether working across stable or Insiders builds of IBM Bob, spending a few minutes to build local IDE utilities pays back continuous micro-dividends in productivity and rule enforcement across every project you touch.

Thanks for reading 🏗️

The Token Economy of Programming Languages

Alain Airom (Ayrom) — Wed, 26 Aug 2026 11:57:46 +0000

Architecting LLM-Powered SDLC Platforms for Efficiency & Cost Optimization

Introduction

When software teams build AI-assisted development pipelines or deploy LLM coding agents, primary attention usually goes to model parameter size, context window length, or system prompts. However, an essential economic and operational factor is often overlooked: The Token Economy of Programming Languages.

Language choice significantly dictates both static context overhead and total interactive agent consumption. Empirical benchmarks reveal that code verbosity, symbol density, and subword tokenizer coverage create a massive spread in token consumption — ranging from 0.82× (Ruby) up to 2.24× relative to Python. Choosing a high-overhead language can inflate AI operational costs by over 100% and lead to frequent context truncation errors in complex workflows.

💡 Core Finding: Programming language choice directly produces a 2.7× spread in token consumption for equivalent business logic. In agentic workflows, behavioral overheads (such as compilation failures and correction loops) further compound this penalty, making verbose or lower-resource languages up to 2.6× more expensive across the full SDLC.

Project Structure and code samples for tests

Developed with IBM Bob, all code samples and benchmarking suites are open-sourced on GitHub. Feel free to explore the repository, test the benchmarks, or contribute improvements!

token-research/
├── README.md                           ← This file
├── requirements.txt                    ← Python dependencies (tiktoken, pytest)
├── .env.example                        ← Environment variable template
├── .gitignore                          ← Git exclusion rules
│
├── Docs/
│   ├── TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md  ← Main 730-line research document
│   ├── Architecture.md                        ← Full architecture diagrams
│   ├── Quickstart.md                          ← Step-by-step setup guide
│   └── LANGUAGE_SELECTION_FOR_RESEARCH_TOOLING.md  ← Engineering Decision Record
│
├── scripts/
│   ├── start.sh                        ← Launch pipeline in detached mode
│   └── stop.sh                         ← Graceful shutdown
│
├── input/                              ← Input data (contents not tracked by git)
├── output/                             ← Timestamped run logs (not tracked by git)
│
├── 01_tokenization_mechanics/
│   ├── bpe_tokenizer_demo.py
│   ├── token_counter.py
│   └── vocabulary_coverage_analysis.py
│
├── 02_language_comparison/
│   ├── token_ratio_calculator.py
│   ├── equivalent_task_samples/
│   │   ├── hello_world_tokens.py
│   │   ├── rest_api_stub_tokens.py
│   │   └── unit_test_tokens.py
│   └── ruby_equivalents/
│       ├── token_ratio_calculator.rb
│       └── identifier_length_analysis.rb
│
├── 03_sdlc_phase_simulation/
│   ├── sdlc_token_budget_model.py
│   ├── phase_cost_estimator.py
│   └── context_window_risk_analyzer.py
│
├── 04_verbosity_factors/
│   ├── boilerplate_overhead_demo.py
│   ├── type_annotation_impact.py
│   └── identifier_length_analysis.py
│
├── 05_optimization_strategies/
│   ├── prompt_compression_demo.py
│   ├── chunking_strategy_demo.py
│   └── token_aware_sdlc_pipeline.py
│
└── tests/
    ├── test_token_counter.py
    ├── test_token_ratio_calculator.py
    └── test_bpe_tokenizer_demo.py

Tokenization Mechanics & System Architecture

Large Language Models do not read source code line-by-line; they process numerical token sequences generated by Byte-Pair Encoding (BPE) tokenizers (such as OpenAI’s cl100k_base or o200k_base). Tokenizers are statistically trained on vast text datasets. Because English prose and Python dominate these training corpora, Python code achieves high byte-per-token density (4.55 bytes/token), whereas symbol-dense or verbose languages suffer from heavy subword fragmentation.

Architecture Schema Flow

Token Mechanics: Measures BPE Merges and Byte Densities.
Language Comparison: Compares Static vs. Agentic multi-turn loops.
tiktoken Aggregator: Evaluates cl100k_base and o200k_base Token Densities.
SDLC Simulation & Optimization: Runs Budget/Risk Models and applies AST/Format Compression.

The Python utility below evaluates byte-per-token density using OpenAI’s cl100k_base encoder and provides a calibrated heuristic fallback:

from dataclasses import dataclass
import tiktoken

@dataclass
class TokenMetrics:
    language: str
    code: str
    token_count: int
    char_count: int
    chars_per_token: float

class LanguageTokenAnalyzer:
    """Evaluates byte-per-token density using cl100k_base (GPT-4) encoding."""

    def __init__(self, encoder_name: str = "cl100k_base"):
        try:
            self.encoder = tiktoken.get_encoding(encoder_name)
        except Exception:
            self.encoder = None

    def analyze(self, language: str, code: str) -> TokenMetrics:
        char_count = len(code)

        if self.encoder:
            tokens = self.encoder.encode(code)
            token_count = len(tokens)
        else:
            token_count = max(1, int(char_count / 3.1))

        chars_per_token = char_count / token_count if token_count > 0 else 0.0

        return TokenMetrics(
            language=language,
            code=code,
            token_count=token_count,
            char_count=char_count,
            chars_per_token=chars_per_token
        )

Multi-Language Micro-Overhead Comparison

To evaluate subword tokenization and structural “token taxes” independent of business logic, program behavior (Hello World + Fibonacci Sum) is held constant across implementations:

Ruby (Most Token-Efficient Baseline):

def fibonacci(n)
  a, b, result = 0, 1, []
  n.times { result << a; a, b = b, a + b }
  result
end

puts "Hello, World!"
fibs = fibonacci(10)
puts "Fibonacci(10): #{fibs.inspect}"
puts "Sum: #{fibs.sum}"

Python (Reference Baseline — 1.00×):

def fibonacci(n: int) -> list[int]:
    a, b, result = 0, 1, []
    for _ in range(n):
        result.append(a)
        a, b = b, a + b
    return result

print("Hello, World!")
fibs = fibonacci(10)
print(f"Fibonacci(10): {fibs}")
print(f"Sum: {sum(fibs)}")

TypeScript (Static Type Overhead — ~1.45×):

function fibonacci(n: number): number[] {
    const result: number[] = [];
    let a = 0, b = 1;
    for (let i = 0; i < n; i++) {
        result.push(a);
        [a, b] = [b, a + b];
    }
    return result;
}

console.log("Hello, World!");
const fibs: number[] = fibonacci(10);
console.log(`Fibonacci(10): ${JSON.stringify(fibs)}`);
const total: number = fibs.reduce((acc: number, x: number): number => acc + x, 0);
console.log(`Sum: ${total}`);

Java (OOP & Stream Overhead — ~1.47× Static):

import java.util.ArrayList;
import java.util.List;

public class HelloFibonacci {
    public static List<Integer> fibonacci(int n) {
        List<Integer> result = new ArrayList<>();
        int a = 0, b = 1;
        for (int i = 0; i < n; i++) {
            result.add(a);
            int temp = a + b;
            a = b;
            b = temp;
        }
        return result;
    }

    public static void main(String[] args) {
        System.out.println("Hello, World!");
        List<Integer> fibs = fibonacci(10);
        System.out.println("Fibonacci(10): " + fibs);
        int total = fibs.stream().mapToInt(Integer::intValue).sum();
        System.out.println("Sum: " + total);
    }
}

C (Manual Memory Management — ~1.77× Static):

#include <stdio.h>
#include <stdlib.h>

int* fibonacci(int n) {
    int* result = (int*)malloc(n * sizeof(int));
    if (!result) return NULL;
    int a = 0, b = 1;
    for (int i = 0; i < n; i++) {
        result[i] = a;
        int temp = a + b;
        a = b;
        b = temp;
    }
    return result;
}

int main(void) {
    printf("Hello, World!\n");
    int* fibs = fibonacci(10);
    if (!fibs) return 1;
    int total = 0;
    for (int i = 0; i < 10; i++) total += fibs[i];
    printf("Sum: %d\n", total);
    free(fibs);
    return 0;
}

Benchmark Rankings & Empirical Token Ratios

Combining static analysis of algorithmic tasks across standard implementations and multi-turn agentic coding benchmarks, empirical ratios relative to Python (1.00× baseline) are established:

| Language       | Static Ratio vs Python   | Agentic Session Ratio | Efficiency Rating | Primary Cost Drivers                                         |
| -------------- | ------------------------ | --------------------- | ----------------- | ------------------------------------------------------------ |
| **Ruby**       | 0.82–0.95×  MD           | 0.95×  MD             | ★★★★★  MD         | Minimal syntactic ceremony, concise expressiveness  MD       |
| **Python**     | **1.00× (Baseline)**  MD | **1.00×**  MD         | ★★★★★  MD         | Dominant representation in LLM training corpora  MD          |
| **JavaScript** | 1.22–1.26×  MD           | 1.05×  MD             | ★★★★☆  MD         | Dynamic typing; low boilerplate, high tokenizer coverage  MD |
| **TypeScript** | 1.37–1.45×  MD           | 1.60×  MD             | ★★★☆☆  MD         | Explicit type annotations add +35% to +60% token overhead  MD |
| **Go**         | 1.44–1.55×  MD           | 1.38×  MD             | ★★★☆☆  MD         | Explicit error handling (`if err != nil`) multiplies line count  MD |
| **Java**       | 1.47–1.75×  MD           | 1.34×  MD             | ★★★☆☆  MD         | OOP ceremony, explicit imports, verbose type signatures  MD  |
| **Rust**       | 1.34–1.57×  MD           | 1.57×  MD             | ★★☆☆☆  MD         | Ownership syntax, lifetime specifiers, agent correction loops  MD |
| **C**          | 1.77–2.24×  MD           | 2.20×  MD             | ★☆☆☆☆  MD         | Manual memory management, header inclusions, 2.2× LOC  MD    |

SDLC Lifecycle Cost Compounding & Risk Modeling

In software development platforms, token costs compound exponentially over time because subsequent lifecycle tasks (code reviews, refactoring, test generation) process entire code artifacts repeatedly.

Lifecycle Token Growth Path

Requirements Phase: ~2,000 tokens (Constant natural language baseline).
Codegen Phase: Python ~6,000 tokens vs. Java ~10,000 tokens.
Code Review Phase: Python ~8,000 tokens vs. C ~17,000 tokens.
Refactoring Phase: Python ~10,000 tokens vs. C ~22,000 tokens.

Beyond direct API billing, excessive language overhead increases the risk of context truncation in long-running LLM loops. The analyzer script below measures subword fragmentation and checks code snippets against defined token limits.

To evaluate context window utilization and fragmentation risks across LLM providers, platform tooling uses risk analyzers:

import tiktoken

def calculate_fragmentation_rate(text: str) -> float:
    """Returns tokens-per-character. Higher rates denote severe subword fragmentation."""
    enc = tiktoken.get_encoding("cl100k_base")
    tokens = enc.encode(text)
    return len(tokens) / len(text) if text else 0.0

def evaluate_co  ntext_window(code_payload: str, max_context_tokens: int = 8192) -> dict:
    enc = tiktoken.get_encoding("cl100k_base")
    token_ids = enc.encode(code_payload)
    total_tokens = len(token_ids)

    return {
        "total_tokens": total_tokens,
        "frag_rate_tok_per_char": round(total_tokens / len(code_payload), 3),
        "exceeds_limit": total_tokens > max_context_tokens,
        "remaining_headroom": max_context_tokens - total_tokens
    }

Compounding lifecycle models also account for static language expansion alongside agentic correction penalties:

from dataclasses import dataclass

@dataclass
class LanguageProfile:
    name: str
    multiplier: float       # Static token multiplier vs. Python
    agentic_overhead: float # Correction loop penalty

PROFILES = {
    "Python": LanguageProfile("Python", 1.00, 1.00),
    "Java": LanguageProfile("Java", 1.47, 1.24),
    "Rust": LanguageProfile("Rust", 1.57, 1.27),
    "C": LanguageProfile("C", 1.77, 1.55),
}

def calculate_feature_budget(profile: LanguageProfile, base_input: int, base_output: int) -> dict:
    effective_gen_mult = profile.multiplier * profile.agentic_overhead

    total_input = round(base_input * profile.multiplier)
    total_output = round(base_output * effective_gen_mult)

    return {
        "language": profile.name,
        "input_tokens": total_input,
        "output_tokens": total_output,
        "total_tokens": total_input + total_output
    }

Optimization Strategies & Key Recommendations

To mitigate cost inflation and prevent premature context truncation when processing verbose enterprise codebases, AI platform engineering teams rely on five primary optimization strategies:

| Strategy                 | Target Languages              | Savings / Impact               | Mechanics                                                 | Primary SDLC Use Case                    |
| ------------------------ | ----------------------------- | ------------------------------ | --------------------------------------------------------- | ---------------------------------------- |
| **Format Stripping**     | Java, C, C++, Go, JS, TS  MD  | 15–35% input reduction  MD     | Collapses non-semantic whitespace and indentation  MD     | Preprocessing for long code files  MD    |
| **Comment Stripping**    | C, Java, JS, TS, Go, Rust  MD | 10–25% input reduction  MD     | Removes Javadoc/Doxygen (`/* */`, `//`) comments  MD      | Code Generation, Test Gen, Debugging  MD |
| **Signature Extraction** | Python, Java, Go, etc.  MD    | 50–80% input reduction  MD     | Extracts signatures and truncates bodies to `...`  MD     | API Review & Documentation  MD           |
| **Token Chunking**       | All languages  MD             | Eliminates context overrun  MD | Splits files dynamically at clean function boundaries  MD | Monorepos & large-file analysis  MD      |
| **Python Prototyping**   | C, C++, Java, Rust, Go  MD    | Prevents agent stuck-loops  MD | Generates solution in Python first, then translates  MD   | Complex algorithmic task generation  MD  |

Below is an automated sanitizer script designed to strip non-semantic Java formatting and Javadoc comments prior to prompt assembly:

import re

def strip_java_formatting(java_code: str) -> str:
    """Strips non-semantic formatting and Javadoc comments to minimize token cost."""
    # Remove block comments and Javadocs
    code = re.sub(r'/\*.*?\*/', '', java_code, flags=re.DOTALL)
    # Remove single-line comments
    code = re.sub(r'//.*', '', code)
    # Collapse redundant whitespace
    code = re.sub(r'\s+', ' ', code)
    return code.strip()

Sample Test

Run ./scripts/start.sh to execute the benchmark suite and test all concepts firsthand. You can review an excerpt of the generated execution log below.

================================================================
Token Research Pipeline — Started at Tue Aug 25 12:20:26 CEST 2026
Python: Python 3.12.10
Tiktoken: 0.14.0
================================================================

━━━ Running: bpe_tokenizer_demo.py ━━━
[INFO] Using tiktoken cl100k_base encoder (GPT-4 tokenizer).
======================================================================
BPE Tokenizer Demo — Token Counts Across Programming Languages
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §1
======================================================================

📊 Fibonacci + sum — Token count comparison
Language        Tokens   Chars  Lines  Chars/token  Ratio vs Python
----------------------------------------------------------------------
Python              84     287     12         3.42            1.00×
JavaScript         101     307     13         3.04            1.20×
Java               141     586     22         4.16            1.68×
Go                 124     379     22         3.06            1.48×
Rust               115     368     17         3.20            1.37×
C                  191     573     27         3.00            2.27×

============================================================
BPE Training Demo: first 12 merges on toy corpus
Input length: 77 chars → 77 bytes
============================================================
  Step  1: ' ' + ' ' → '  ' (freq=6, new_id=256)
  Step  2: 'v' + 'a' → 'va' (freq=3, new_id=257)
  Step  3: 'va' + 'l' → 'val' (freq=3, new_id=258)
  Step  4: 'val' + 'u' → 'valu' (freq=3, new_id=259)
  Step  5: 'valu' + 'e' → 'value' (freq=3, new_id=260)
  Step  6: 's' + 'e' → 'se' (freq=3, new_id=261)
  Step  7: 'se' + 'l' → 'sel' (freq=3, new_id=262)
  Step  8: 'sel' + 'f' → 'self' (freq=3, new_id=263)
  Step  9: 'n' + 'a' → 'na' (freq=3, new_id=264)
  Step 10: 'na' + 'm' → 'nam' (freq=3, new_id=265)
  Step 11: 'nam' + 'e' → 'name' (freq=3, new_id=266)
  Step 12: 'self' + '.' → 'self.' (freq=2, new_id=267)

============================================================
BPE Training Demo: first 12 merges on toy corpus
Input length: 59 chars → 59 bytes
============================================================
  Step  1: 'm' + 'e' → 'me' (freq=4, new_id=256)
  Step  2: 'a' + 'me' → 'ame' (freq=4, new_id=257)
  Step  3: 'n' + 'ame' → 'name' (freq=3, new_id=258)
  Step  4: ' ' + ' ' → '  ' (freq=3, new_id=259)
  Step  5: ' ' + 'name' → ' name' (freq=2, new_id=260)
  Step  6: ' name' + ';' → ' name;' (freq=1, new_id=261)
  Step  7: ' name;' + '
' → ' name;
' (freq=1, new_id=262)
  Step  8: ' name;
' + '}' → ' name;
}' (freq=1, new_id=263)
  Step  9: ' name;
}' + '
' → ' name;
}
' (freq=1, new_id=264)
  Step 10: ' name' + ')' → ' name)' (freq=1, new_id=265)
  Step 11: ' name)' + ' ' → ' name) ' (freq=1, new_id=266)
  Step 12: ' name) ' + '{' → ' name) {' (freq=1, new_id=267)

✅ Key insight: After enough BPE merges on a Python-heavy corpus,
   constructs like 'def ', 'self.', '__init__' become single tokens.
   Java's 'public void ', 'String ', 'this.' patterns also merge,
   but the class/method boilerplate still requires far more tokens.
   See TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §2 for full ratio table.
✅ bpe_tokenizer_demo.py completed.

━━━ Running: token_counter.py ━━━
[INFO] tiktoken loaded — using cl100k_base (GPT-4) and o200k_base (GPT-4o).
======================================================================
Token Counter — Multi-Language Code Token Comparison
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §1 and §2
======================================================================

================================================================================
  Min/Max Task — Token Counts [cl100k_base]
  Tokenizer: cl100k_base  |  Baseline: Python
================================================================================
Language        Tokens   Chars  Lines  C/tok  L/tok    Ratio
--------------------------------------------------------------------------------
SQL                 83     273     10   3.29  0.120    0.76×
Ruby                87     233      9   2.68  0.103    0.80×
Python             109     340     10   3.12  0.092    1.00×
JavaScript         113     357     14   3.16  0.124    1.04×
TypeScript         127     408     14   3.21  0.110    1.17×
Rust               152     458     17   3.01  0.112    1.39×
Java               192     766     22   3.99  0.115    1.76×
Go                 210     635     33   3.02  0.157    1.93×
C                  237     678     30   2.86  0.127    2.17×
--------------------------------------------------------------------------------
  Most expensive: C (2.17× baseline)
  Most efficient: SQL (0.76× baseline)
================================================================================


================================================================================
  Min/Max Task — Token Counts [o200k_base]
  Tokenizer: o200k_base  |  Baseline: Python
================================================================================
Language        Tokens   Chars  Lines  C/tok  L/tok    Ratio
--------------------------------------------------------------------------------
SQL                 83     273     10   3.29  0.120    0.75×
Ruby                88     233      9   2.65  0.102    0.80×
Python             110     340     10   3.09  0.091    1.00×
JavaScript         115     357     14   3.10  0.122    1.05×
TypeScript         129     408     14   3.16  0.109    1.17×
Rust               152     458     17   3.01  0.112    1.38×
Java               207     766     22   3.70  0.106    1.88×
Go                 212     635     33   3.00  0.156    1.93×
C                  237     678     30   2.86  0.127    2.15×
--------------------------------------------------------------------------------
  Most expensive: C (2.15× baseline)
  Most efficient: SQL (0.75× baseline)
================================================================================

📄 CSV output (for token_ratio_calculator.py):
language,encoder,tokens,chars,lines,chars_per_token,lines_per_token,ratio_vs_baseline
SQL,cl100k_base,83,273,10,3.289,0.1205,0.7615
Ruby,cl100k_base,87,233,9,2.678,0.1034,0.7982
Python,cl100k_base,109,340,10,3.119,0.0917,1.0000
JavaScript,cl100k_base,113,357,14,3.159,0.1239,1.0367
TypeScript,cl100k_base,127,408,14,3.213,0.1102,1.1651
Rust,cl100k_base,152,458,17,3.013,0.1118,1.3945
Java,cl100k_base,192,766,22,3.990,0.1146,1.7615
Go,cl100k_base,210,635,33,3.024,0.1571,1.9266
C,cl100k_base ...

⚠️  Heuristic error demonstration:
Language        Heuristic   Actual   Error%
---------------------------------------------
Python                110      109    -22.0%
JavaScript            121      113    -21.2%
TypeScript            151      127    -19.7%
Java                  295      192     +0.0%
Go                    231      210    -24.3%
Rust                  180      152    -25.0%
C                     295      237    -28.3%
Ruby                   72       87    -33.3%
SQL                    78       83    -18.1%

  Negative error = naive heuristic UNDERESTIMATES actual token count.
  For Java and C, the naive rule underestimates by 30–50%+.
✅ token_counter.py completed.

━━━ Running: vocabulary_coverage_analysis.py ━━━
======================================================================
Vocabulary Coverage Analysis
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §1.4
======================================================================
Tokenizer: cl100k_base (GPT-4)

======================================================================
  1. Language Keywords — Coverage Test
======================================================================
  Category            Tokens  Chars    Rate  Description
  -----------------------------------------------------------------
  python_keyword           1      3   0.333  Python built-in keyword
                      → 'def'
  python_keyword           1      6   0.167  Python built-in keyword
                      → 'import'
  python_keyword           1      6   0.167  Python built-in keyword
                      → 'return'
  java_keyword             1      6   0.167  Java access modifier
                      → 'public'
  java_keyword             1      6   0.167  Java modifier
                      → 'static'
  java_keyword             1      4   0.250  Java return type
                      → 'void'
  rust_keyword             1      2   0.500  Rust function keyword
                      → 'fn'
  rust_keyword             1      4   0.250  Rust trait implementation keyword
                      → 'impl'
  go_keyword               1      4   0.250  Go function keyword
                      → 'func'
  ocaml_keyword            2      7   0.286  OCaml recursive binding
                      → 'let' | ' rec'
  haskell_keyword          1      5   0.200  Haskell where clause
                      → 'where'
  haskell_keyword          2      7   0.286  Haskell type declaration
                      → 'new' | 'type'

======================================================================
  2. Identifier Naming Conventions
======================================================================
  Category            Tokens  Chars    Rate  Description
  -----------------------------------------------------------------
  snake_case               2      7   0.286  2-word Python identifier
                      → 'user' | '_id'
  snake_case               3     24   0.125  3-word Python identifier
                      → 'calculate' | '_word' | '_frequency'
  snake_case               4     26   0.154  4-word Python identifier
                      → 'fetch' | '_api' | '_response' | '_payload'
  camelCase                1      6   0.167  2-word Java identifier
                      → 'userId'
  camelCase                3     22   0.136  3-word Java identifier
                      → 'calculate' | 'Word' | 'Frequency'
  camelCase                3     23   0.130  4-word Java identifier
                      → 'fetch' | 'ApiResponse' | 'Payload'
  PascalCase               1     11   0.091  2-word class name
                      → 'UserService'
  PascalCase               3     26   0.115  4-word Java class name
                      → 'Abstract' | 'UserService' | 'Factory'
  SCREAMING_SNAKE          3     15   0.200  Java/C constant
                      → 'MAX' | '_RETRY' | '_COUNT'
  SCREAMING_SNAKE          4     28   0.143  Java constant
                      → 'DEFAULT' | '_CONNECTION' | '_POOL' | '_SIZE'
  go_short                 1      1   1.000  Go request shorthand
                      → 'r'
  go_short                 1      3   0.333  Go context shorthand
                      → 'ctx'
  go_short                 1      3   0.333  Go error shorthand
                      → 'err'

======================================================================
  3. Language Syntax Patterns
======================================================================
  Category            Tokens  Chars    Rate  Description
  -----------------------------------------------------------------
  python_syntax            6     19   0.316  Python constructor signature
                      → 'def' | ' __' | 'init' | '__(' | 'self' | '):'
  python_syntax            3     19   0.158  Python attribute access
                      → 'self' | '.attribute' | '_name'
  python_syntax            9     26   0.346  Python entry guard
                      → 'if' | ' __' | 'name' | '__' | ' ==' | " '__" | 'main' | '__' | "':"
  java_syntax              8     38   0.211  Java main signature
                      → 'public' | ' static' | ' void' | ' main' | '(String' | '[]' | ' args' | ')'
  java_syntax              3     18   0.167  Java print method
                      → 'System' | '.out' | '.println'
  java_syntax              3     20   0.150  Java field declaration prefix
                      → 'private' | ' final' | ' String'
  rust_syntax             10     39   0.256  Rust main signature
                      → 'fn' | ' main' | '()' | ' ->' | ' Result' | '<(),' | ' Box' | '<dyn' | ' Error' | '>>'
  rust_syntax             14     32   0.438  Rust impl with lifetime
                      → 'impl' | "<'" | 'a' | ',' | ' T' | ':' | ' Trait' | '>' | ' Struct' | "<'" | 'a' | ',' | ' T' | '>'
  rust_syntax              8     29   0.276  Rust nested generics
                      → 'Vec' | '<HashMap' | '<String' | ',' | ' Vec' | '<u' | '8' | '>>>'
  go_syntax                5     15   0.333  Go error check pattern
                      → 'if' | ' err' | ' !=' | ' nil' | ' {'
  go_syntax               16     63   0.254  Go method signature
  ocaml_syntax             9     28   0.321  OCaml recursive function
                      → 'let' | ' rec' | ' fold' | '_left' | ' f' | ' acc' | 'u' | ' l' | ' ='
  ocaml_syntax            11     34   0.324  OCaml type declaration
                      → 'type' | " '" | 'a' | ' option' | ' =' | ' None' | ' |' | ' Some' | ' of' | " '" | 'a'
  c_syntax                14     42   0.333  C malloc call
                      → 'int' | ' *' | 'ptr' | ' =' | ' (' | 'int' | ' *)' | 'malloc' | '(sizeof' | '(int' | ')' | ' *' | ' n' | ');'
  c_syntax                 8     29   0.276  C function pointer
                      → 'void' | ' (*' | 'callback' | ')(' | 'int' | ',' | ' void' | ' *)'

======================================================================
  4. High vs. Low Training Data Coverage
======================================================================
  Category            Tokens  Chars    Rate  Description
  -----------------------------------------------------------------
  high_coverage            4     18   0.222  Python data science import
                      → 'import' | ' numpy' | ' as' | ' np'
  high_coverage            7     34   0.206  Node.js pattern
                      → 'const' | ' express' | ' =' | ' require' | "('" | 'express' | "')"
  high_coverage            9     32   0.281  Common SQL pattern
                      → 'SELECT' | ' *' | ' FROM' | ' users' | ' WHERE' | ' id' | ' =' | ' ' | '1'
  low_coverage             8     42   0.190  OCaml qualified name
                      → 'let' | ' binding' | ' =' | ' module' | '_name' | '.Sub' | 'module' | '.create'
  low_coverage             7     33   0.212  Haskell typeclass instance
                      → 'instance' | ' Functor' | ' (' | 'Either' | ' a' | ')' | ' where'
  low_coverage            11     56   0.196  Rust extern crate
                      → 'extern' | ' crate' | ' serde' | ';' | ' use' | ' serde' | '::{' | 'Serialize' | ',' | ' Deserialize' | '};'

======================================================================
  Identifier Naming Convention — Token Efficiency Analysis
======================================================================

  Concept: 'User repository service'
  Convention                      Tokens  Chars    Rate
  -------------------------------------------------------
  Python snake_case                    3     23   0.130
                                  → 'user' | '_repository' | '_service'
  Java camelCase                       2     21   0.095
                                  → 'userRepository' | 'Service'
  Java PascalCase class                3     21   0.143
                                  → 'User' | 'Repository' | 'Service'
  C SCREAMING_SNAKE                    4     23   0.174
                                  → 'USER' | '_RE' | 'POSITORY' | '_SERVICE'
  Go abbreviated                       2      7   0.286
                                  → 'user' | 'Svc'
  Rust snake_case                      3     23   0.130
                                  → 'user' | '_repository' | '_service'

  Concept: 'Get user by ID'
  Convention                      Tokens  Chars    Rate
  -------------------------------------------------------
  Python snake_case                    4     14   0.286
                                  → 'get' | '_user' | '_by' | '_id'
  Java camelCase                       2     11   0.182
                                  → 'getUser' | 'ById'
  Java verbose camelCase               5     26   0.192
                                  → 'find' | 'User' | 'Entity' | 'ById' | 'entifier'
  Go abbreviated                       2     11   0.182
                                  → 'getUser' | 'ByID'
  C lowercase                          4     14   0.286
                                  → 'get' | '_user' | '_by' | '_id'

======================================================================
  Summary: Vocabulary Coverage Findings
======================================================================

  1. WELL-COVERED constructs (low fragmentation rate, ~0.25–0.35 tok/char):
     • Python keywords: def, import, return, class, if, for, with
     • Common English words appearing frequently in code: user, name, value
     • SQL keywords: SELECT, FROM, WHERE (high training data frequency)
     • JavaScript patterns: const, let, function, console.log

  2. POORLY-COVERED constructs (high fragmentation, 0.6–1.0+ tok/char):
     • OCaml: type constructors, polymorphic variants, module signatures
     • Haskell: type class instances, point-free notation, monadic bind (>>=)
     • Rust: lifetime parameters ('a), complex generic bounds, macro syntax
     • SCREAMING_SNAKE_CASE identifiers (underscore disrupts merge patterns)
     • Very long compound identifiers (AbstractSingletonProxyFactoryBean)

  3. FORMATTING OVERHEAD (see TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §4.3):
     • Java newlines: 18.7% of total tokens (Claude-3.7 measurement)
     • Java indentation: ~7.9% of total tokens
     • Python indentation: mandatory — cannot be removed (syntax rule)
     • C comment blocks: up to 21% token savings from comment stripping

  4. IDENTIFIER NAMING:
     • snake_case and camelCase are similarly efficient for equal-length words
     • SCREAMING_SNAKE_CASE is ~50% less efficient per character
     • Short Go-style names (ctx, err, r) are most token-efficient
     • Long verbose Java-style names are most token-expensive

✅ vocabulary_coverage_analysis.py completed.

━━━ Running: token_ratio_calculator.py ━━━
======================================================================
Token Ratio Calculator — Language Efficiency Ranking
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §2
======================================================================

====================================================================================================
  TOKEN EFFICIENCY RANKING — All Languages vs. Python Baseline (1.00×)
  Data sources: cross-lang-token-density, Wu et al. (2026), mame/ai-coding-lang-bench
====================================================================================================
  Rank  Language       Rating    Static   Agentic   Prod.   Overall          Range  Notes
  -------------------------------------------------------------------------------------------------
  1     Ruby           ★★★★★       N/A       N/A    0.95×     0.95×     0.95–0.95×  Slightly beats Python in production
  2     Python         ★★★★★      1.00×      N/A     N/A      1.00×     1.00–1.00×  ← Baseline
  3     JavaScript     ★★★★☆      1.26×      N/A    1.03×     1.15×     1.03–1.26×  
  4     C++            ★★★★☆      1.20×      N/A     N/A      1.20×     1.20–1.20×  
  5     Java           ★★★☆☆      1.47×     1.23×   1.32×     1.30×     1.18–1.47×  
  6     Rust           ★★★☆☆      1.57×     1.24×   1.42×     1.34×     1.16–1.57×  
  7     Go             ★★★☆☆      1.55×      N/A    1.32×     1.44×     1.32–1.55×  
  8     OCaml          ★★★☆☆       N/A      1.42×   1.53×     1.45×     1.28–1.69×  Compact code; high agent confusion cost
  9     TypeScript     ★★☆☆☆      1.45×      N/A    1.63×     1.54×     1.45–1.63×  
  10    C              ★☆☆☆☆      1.77×      N/A    1.95×     1.86×     1.77–1.95×  Most expensive; manual memory management
  11    Haskell        ★☆☆☆☆       N/A       N/A    1.95×     1.95×     1.95–1.95×  
====================================================================================================

  Column definitions:
    Static  = direct token count of equivalent program text (no agent overhead)
    Agentic = full agent session including failed attempts, revisions, stuck loops
    Prod.   = production benchmark (real API cost) including all overhead
    Overall = mean across all available measurements for that language
    Range   = min–max observed ratio across all data points

  Note: Agentic ratios diverge most from static ratios for low-resource languages
  (OCaml, Haskell, Rust) because agent behavior — not syntax — drives the gap.
  See TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §2.3 for detailed explanation.

======================================================================
  TYPE SYSTEM IMPACT — Incremental Token Cost of Adding Type Checking
======================================================================

  Transformation                            Static overhead  Agentic overhead
  ---------------------------------------------------------------------------
  Adding TypeScript types to JS                      1.15×            1.60×
  Adding mypy strict to Python                       1.00×            1.65×
  Adding Steep type checker to Ruby                  1.00×            2.60×

  Interpretation:
    The agentic overhead for typed variants is MUCH larger than the static overhead.
    This is because the LLM must reason about type constraints, satisfy the type
    checker, and iterate when type errors occur — generating many extra turns.
    Ruby/Steep's 2.60× agentic overhead reflects low LLM familiarity with Steep.

======================================================================
  Markdown Table (for embedding in reports):
======================================================================
| Rank | Language | Efficiency | Static | Agentic | Production | Overall | Range |
|:----:|----------|:----------:|:------:|:-------:|:----------:|:-------:|:-----:|
| 1 | **Ruby** | ★★★★★ |   N/A  |   N/A  | 0.95× | 0.95× | 0.95–0.95× |
| 2 | **Python** | ★★★★★ | 1.00× |   N/A  |   N/A  | 1.00× | 1.00–1.00× |
| 3 | **JavaScript** | ★★★★☆ | 1.26× |   N/A  | 1.03× | 1.15× | 1.03–1.26× |
| 4 | **C++** | ★★★★☆ | 1.20× |   N/A  |   N/A  | 1.20× | 1.20–1.20× |
| 5 | **Java** | ★★★☆☆ | 1.47× | 1.23× | 1.32× | 1.30× | 1.18–1.47× |
| 6 | **Rust** | ★★★☆☆ | 1.57× | 1.24× | 1.42× | 1.34× | 1.16–1.57× |
| 7 | **Go** | ★★★☆☆ | 1.55× |   N/A  | 1.32× | 1.44× | 1.32–1.55× |
| 8 | **OCaml** | ★★★☆☆ |   N/A  | 1.42× | 1.53× | 1.45× | 1.28–1.69× |
| 9 | **TypeScript** | ★★☆☆☆ | 1.45× |   N/A  | 1.63× | 1.54× | 1.45–1.63× |
| 10 | **C** | ★☆☆☆☆ | 1.77× |   N/A  | 1.95× | 1.86× | 1.77–1.95× |
| 11 | **Haskell** | ★☆☆☆☆ |   N/A  |   N/A  | 1.95× | 1.95× | 1.95–1.95× |

✅ Run the equivalent_task_samples/ scripts for direct token measurements.
   These benchmark ratios are the aggregate from published research.
✅ token_ratio_calculator.py completed.

━━━ Running: hello_world_tokens.py ━━━
========================================================================
Hello World + Fibonacci — Multi-Language Token Count Comparison
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §2
Tokenizer: cl100k_base (GPT-4)
========================================================================

Language        Tokens   Chars  Lines  C/tok    Ratio
--------------------------------------------------------
Ruby                89     260     12   2.92    0.82×  ████████
Python             109     357     14   3.28    1.00×  ██████████
JavaScript         133     417     16   3.14    1.22×  ████████████
TypeScript         149     503     17   3.38    1.37×  █████████████
Rust               150     485     21   3.23    1.38×  █████████████
Go                 161     486     26   3.02    1.48×  ██████████████
Java               191     776     28   4.06    1.75×  █████████████████
C                  244     744     33   3.05    2.24×  ██████████████████████

------------------------------------------------------------------------
Most tokens:  C              244  (2.24× Python)
Fewest tokens:Ruby           89  (0.82× Python)

💡 Key observation: C requires 2.7×
   the tokens of Ruby for identical logic.

   Note: TypeScript vs JavaScript overhead = purely type annotations.
   See TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §4.2 for type annotation analysis.
✅ hello_world_tokens.py completed.

━━━ Running: rest_api_stub_tokens.py ━━━
========================================================================
REST API Stub — Token Count Comparison Across 5 Language/Frameworks
Task: GET /users/{id} with error handling and data model
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §2 & §3
Tokenizer: cl100k_base (GPT-4)
========================================================================

Name                        Tokens   Chars  Lines  C/tok    Ratio
--------------------------------------------------------------------
Python (Flask)                 205     718     29   3.50    1.00×  ████████
TypeScript (Express)           234     832     34   3.56    1.14×  █████████
Go (Gin)                       250     866     42   3.46    1.22×  █████████
Rust (Actix-web)               363   1,324     49   3.65    1.77×  ██████████████
Java (Spring Boot)             382   1,843     69   4.82    1.86×  ██████████████

------------------------------------------------------------------------
Most expensive:  Java (Spring Boot) (382 tokens = 1.86× Python/Flask)
Least expensive: Python (Flask) (205 tokens = 1.00× baseline)

💡 Analysis:
   • Python/Flask wins on token efficiency due to:
     - No mandatory class boilerplate (just decorated functions)
     - Implicit JSON serialization via dataclass + jsonify
     - No explicit type annotations required

   • Java/Spring Boot is the most verbose because:
     - Separate files per class (Controller, Service, Model)
     - Explicit getters/setters for each field
     - @Annotation overhead throughout
     - Explicit Optional<> type wrapping

   • Go/Gin's verbosity comes from:
     - Explicit error handling after every fallible operation
     - Manual type parsing (strconv.Atoi)
     - No implicit JSON serialization (struct tags required)

   • Rust/Actix-web is expensive due to:
     - Async/await boilerplate (#[actix_web::main])
     - Mutex<HashMap<>> shared state pattern
     - Derive macros (Serialize, Deserialize) on every struct

   See TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §4 for detailed factor analysis.

✅ rest_api_stub_tokens.py completed.

━━━ Running: unit_test_tokens.py ━━━
========================================================================
Unit Test Suite — Token Count Comparison Across 6 Frameworks
Task: Stack push/pop/overflow/underflow tests (same 6 test cases)
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §2 & §3
Tokenizer: cl100k_base (GPT-4)
========================================================================

Name                        Tokens   Chars  Lines  C/tok    Ratio
--------------------------------------------------------------------
Python (pytest)                318   1,260     54   3.96    1.00×  ████████
C++ (GoogleTest)               399   1,415     58   3.55    1.25×  ██████████
JavaScript (Jest)              404   1,756     63   4.35    1.27×  ██████████
Rust (cargo test)              422   1,642     69   3.89    1.33×  ██████████
Go (testing)                   580   1,975     80   3.41    1.82×  ██████████████
Java (JUnit 5)                 604   2,704    102   4.48    1.90×  ███████████████

💡 Test Generation Implications (TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §3):
   • Java test files are significantly larger due to:
     - @Test, @BeforeEach, @Nested, @DisplayName annotations
     - Explicit generic type parameters on every assertion
     - @ParameterizedTest + @MethodSource requiring separate stream methods

   • Python/pytest is the baseline winner because:
     - No class required (just functions)
     - @pytest.fixture replaces @BeforeEach with zero overhead
     - assert keyword vs. assertEquals/assertFalse/assertTrue verbosity

   • Go's table-driven tests are verbose but idiomatic:
     - Explicit error checking after every operation (if err != nil)
     - Manual comparison in test body (no matcher library)

   • Rust's #[cfg(test)] approach is moderately efficient:
     - Built-in to the language, no external framework imports
     - assert_eq!/assert! macros are compact
     - unwrap() pattern adds minor overhead

✅ unit_test_tokens.py completed.

━━━ Running: sdlc_token_budget_model.py ━━━
================================================================================
SDLC Token Budget Model
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §3
Model: 10-feature project, ~500 LOC per feature
================================================================================

====================================================================================================
  PER-FEATURE TOKEN CONSUMPTION BY SDLC PHASE
  (tokens shown per feature; multiply by num_features for project total)
====================================================================================================
  Phase                      Python JavaScript       Java         Go       Rust          C
  ------------------------------------------------------------------------------------
  Requirements Analysis       2,300      2,300      2,300      2,300      2,300      2,300
  Architecture Design         3,500      3,500      3,500      3,500      3,500      3,500
  Code Generation             6,500      7,032      9,791      9,320     10,476     13,474
  Code Review                 6,700      7,250      9,285      9,725      9,835     10,935
  Refactoring                 9,500     10,599     15,553     15,423     16,823     21,196
  Test Generation             6,500      7,266     10,790     10,618     11,689     14,912
  Documentation               6,250      6,600      7,895      8,175      8,245      8,945
  Debugging                   5,800      6,439      9,161      9,269      9,869     12,018
  Deployment Config           2,000      2,000      2,000      2,000      2,000      2,000
  ------------------------------------------------------------------------------------
  TOTAL (per feature)        49,050     52,986     70,275     70,330     74,737     89,280
  Ratio vs Python             1.00×      1.08×      1.43×      1.43×      1.52×      1.82×
====================================================================================================

================================================================================
  PROJECT SUMMARY (10 features)
================================================================================
  Language          Input tok   Output tok        Total    Ratio
  --------------------------------------------------------------
  Python              278,000      212,500      490,500    1.00×
  JavaScript          299,000      230,860      529,860    1.08×
  Java                376,700      326,050      702,750    1.43×
  Go                  393,500      309,800      703,300    1.43×
  Rust                397,700      349,670      747,370    1.52×
  C                   439,700      453,100      892,800    1.82×

  Estimated API costs (GPT-4o: $2.50/M input, $10.00/M output):
  Language         Input cost  Output cost   Total cost
  ------------------------------------------------------
  Python         $       0.70 $       2.12 $       2.82
  JavaScript     $       0.75 $       2.31 $       3.06
  Java           $       0.94 $       3.26 $       4.20
  Go             $       0.98 $       3.10 $       4.08
  Rust           $       0.99 $       3.50 $       4.49
  C              $       1.10 $       4.53 $       5.63
================================================================================

💡 Key findings from this model:
   1. Code Review and Refactoring are the most token-intensive phases for static-typed
      languages because entire files must be in context (high input token cost).
   2. Test Generation compounds the language penalty: tests mirror production code
      verbosity, doubling the effective penalty for boilerplate-heavy languages.
   3. The Debugging phase is unpredictable — for unfamiliar languages, multiple
      agent turns may be needed to interpret stack traces and write correct fixes.
   4. Deployment Config is largely language-agnostic (YAML/Docker is universal),
      so it provides little differentiation between language choices.

   See TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §3 for the full SDLC phase analysis.

✅ sdlc_token_budget_model.py completed.

━━━ Running: phase_cost_estimator.py ━━━
================================================================================
Phase Cost Estimator
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §3 & §5
================================================================================

==========================================================================================
  Cost Estimate: Python × Claude Sonnet 4.5 (Anthropic)
  Project: 10 features | Input: $3.00/M | Output: $15.00/M
  Cache rate: $0.300/M (input cache hits)
==========================================================================================
  Phase                    InputTok    OutTok   Baseline   + Cache  + FmtStrip  Optimized   Saving
  ----------------------------------------------------------------------------------------
  Requirements               15,000     8,000   $   0.165  $  0.141  $    0.165  $   0.141    14.7%
  Architecture Design        20,000    15,000   $   0.285  $  0.253  $    0.285  $   0.253    11.4%
  Code Generation            25,000    40,000   $   0.675  $  0.634  $    0.675  $   0.634     6.0%
  Code Review                55,000    12,000   $   0.345  $  0.256  $    0.340  $   0.253    26.5%
  Refactoring                50,000    45,000   $   0.825  $  0.744  $    0.820  $   0.742    10.1%
  Test Generation            30,000    35,000   $   0.615  $  0.566  $    0.612  $   0.565     8.1%
  Documentation              35,000    25,000   $   0.480  $  0.423  $    0.477  $   0.422    12.1%
  Debugging                  40,000    18,000   $   0.390  $  0.325  $    0.386  $   0.323    17.1%
  Deployment Config           8,000    12,000   $   0.204  $  0.191  $    0.204  $   0.191     6.4%
  ----------------------------------------------------------------------------------------
  TOTAL                                         $   3.984  $  3.534  $    3.964  $   3.524    11.5%
  Combined optimization saves $0.460 (11.5%) vs. baseline

==========================================================================================
  Cost Estimate: Java × Claude Sonnet 4.5 (Anthropic)
  Project: 10 features | Input: $3.00/M | Output: $15.00/M
  Cache rate: $0.300/M (input cache hits)
==========================================================================================
  Phase                    InputTok    OutTok   Baseline   + Cache  + FmtStrip  Optimized   Saving
  ----------------------------------------------------------------------------------------
  Requirements               15,000     8,336   $   0.170  $  0.146  $    0.170  $   0.146    14.3%
  Architecture Design        20,000    15,630   $   0.294  $  0.262  $    0.294  $   0.262    11.0%
  Code Generation            25,000    55,120   $   0.902  $  0.861  $    0.902  $   0.861     4.5%
  Code Review                75,680    12,504   $   0.415  $  0.292  $    0.369  $   0.271    34.6%
  Refactoring                68,800    62,010   $   1.137  $  1.025  $    1.095  $   1.006    11.5%
  Test Generation            41,280    48,230   $   0.847  $  0.780  $    0.823  $   0.769     9.2%
  Documentation              48,160    26,050   $   0.535  $  0.457  $    0.506  $   0.444    17.1%
  Debugging                  55,040    24,804   $   0.537  $  0.448  $    0.504  $   0.433    19.4%
  Deployment Config           8,000    16,536   $   0.272  $  0.259  $    0.272  $   0.259     4.8%
  ----------------------------------------------------------------------------------------
  TOTAL                                         $   5.109  $  4.531  $    4.936  $   4.451    12.9%
  Combined optimization saves $0.658 (12.9%) vs. baseline

================================================================================
  CROSS-LANGUAGE COST COMPARISON — GPT-4o
  10 features | Baseline = Python
================================================================================
  Language         Baseline$  Optimized$    Ratio    Saving
  ------------------------------------------------------------
  Ruby           $     2.704 $     2.487    0.97×      8.0%
  Python         $     2.795 $     2.575    1.00×      7.9%
  JavaScript     $     2.879 $     2.633    1.03×      8.6%
  Go             $     3.477 $     3.157    1.24×      9.2%
  TypeScript     $     3.478 $     3.164    1.24×      9.0%
  Java           $     3.585 $     3.216    1.28×     10.3%
  OCaml          $     3.668 $     3.378    1.31×      7.9%
  Rust           $     3.697 $     3.374    1.32×      8.7%
  C++            $     3.760 $     3.414    1.35×      9.2%
  C              $     4.246 $     3.846    1.52×      9.4%

  Python baseline: $2.795 unoptimized → $2.575 optimized

💡 Interpretation:
   • The "Optimized" column applies BOTH prompt caching AND format stripping.
   • Claude's cache rate ($0.30/M) provides the highest savings percentage
     because the full input rate is $3.00/M — a 90% discount on cache hits.
   • Format stripping helps Java most (25% input saving) and Python least (4%).
   • Even optimized Java costs ~1.35× optimized Python on Claude Sonnet.

   See TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §6 for all optimization strategies.

✅ phase_cost_estimator.py completed.

━━━ Running: context_window_risk_analyzer.py ━━━
===========================================================================
Context Window Risk Analyzer
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §3.3 & §5.3
===========================================================================

===========================================================================
  Max File Size in Context Window — GPT-4o
  (with 8,000 tokens reserved for system prompt + output)
===========================================================================
  Language        Available tok   Max chars   Max LOC   Tok/LOC
  --------------------------------------------------------------
  SQL                   120,000     419,580     9,324     12.87
  Ruby                  120,000     389,610    13,915      8.62
  Python                120,000     371,517    11,610     10.34
  PHP                   120,000     360,360    10,010     11.99
  JavaScript            120,000     353,982    10,411     11.53
  OCaml                 120,000     336,134    11,204     10.71
  Go                    120,000     329,670     8,676     13.83
  Haskell               120,000     329,670    11,774     10.19
  TypeScript            120,000     324,324     8,108     14.80
  C++                   120,000     318,302     8,376     14.33
  Java                  120,000     311,688     7,421     16.17
  Rust                  120,000     306,122     7,653     15.68
  C                     120,000     275,862     7,663     15.66
===========================================================================

===========================================================================
  Max File Size in Context Window — Claude Sonnet 4.5
  (with 12,000 tokens reserved for system prompt + output)
===========================================================================
  Language        Available tok   Max chars   Max LOC   Tok/LOC
  --------------------------------------------------------------
  SQL                   188,000     657,343    14,608     12.87
  Ruby                  188,000     610,390    21,800      8.62
  Python                188,000     582,043    18,189     10.34
  PHP                   188,000     564,565    15,682     11.99
  JavaScript            188,000     554,572    16,311     11.53
  OCaml                 188,000     526,611    17,554     10.71
  Go                    188,000     516,484    13,592     13.83
  Haskell               188,000     516,484    18,446     10.19
  TypeScript            188,000     508,108    12,703     14.80
  C++                   188,000     498,674    13,123     14.33
  Java                  188,000     488,312    11,626     16.17
  Rust                  188,000     479,592    11,990     15.68
  C                     188,000     432,184    12,005     15.66
===========================================================================

====================================================================================================
  CONTEXT WINDOW RISK MATRIX — GPT-4o, File size: 500 LOC
====================================================================================================
  Phase                        Python   JavaScript         Java           Go         Rust            C
  ----------------------------------------------------------------------------------------------------
  Requirements                   ✅ 6%         ✅ 6%         ✅ 8%         ✅ 7%         ✅ 8%         ✅ 8%
  Architecture Design            ✅ 6%         ✅ 7%         ✅ 9%         ✅ 8%         ✅ 8%         ✅ 8%
  Code Generation                ✅ 6%         ✅ 7%         ✅ 9%         ✅ 8%         ✅ 8%         ✅ 8%
  Code Review                    ✅ 7%         ✅ 8%         ✅ 9%         ✅ 9%         ✅ 9%         ✅ 9%
  Refactoring                    ✅ 8%         ✅ 8%        ✅ 10%         ✅ 9%        ✅ 10%        ✅ 10%
  Test Generation                ✅ 7%         ✅ 7%         ✅ 9%         ✅ 8%         ✅ 9%         ✅ 9%
  Documentation                  ✅ 6%         ✅ 7%         ✅ 9%         ✅ 8%         ✅ 8%         ✅ 8%
  Debugging                      ✅ 9%         ✅ 9%        ✅ 11%        ✅ 10%        ✅ 11%        ✅ 11%
  Deployment Config              ✅ 6%         ✅ 6%         ✅ 8%         ✅ 7%         ✅ 8%         ✅ 8%
  ----------------------------------------------------------------------------------------------------
  Legend: ✅ LOW (<50%) | ⚠️  MEDIUM (50–70%) | 🔴 HIGH (70–90%) | 💥 CRITICAL (>90%)

====================================================================================================
  CONTEXT WINDOW RISK MATRIX — GPT-4o, File size: 1,000 LOC
====================================================================================================
  Phase                        Python   JavaScript         Java           Go         Rust            C
  ----------------------------------------------------------------------------------------------------
  Requirements                  ✅ 10%        ✅ 11%        ✅ 14%        ✅ 12%        ✅ 14%        ✅ 14%
  Architecture Design           ✅ 10%        ✅ 11%        ✅ 15%        ✅ 13%        ✅ 15%        ✅ 15%
  Code Generation               ✅ 10%        ✅ 11%        ✅ 15%        ✅ 13%        ✅ 15%        ✅ 15%
  Code Review                   ✅ 11%        ✅ 12%        ✅ 16%        ✅ 14%        ✅ 15%        ✅ 15%
  Refactoring                   ✅ 12%        ✅ 13%        ✅ 17%        ✅ 15%        ✅ 16%        ✅ 16%
  Test Generation               ✅ 11%        ✅ 12%        ✅ 15%        ✅ 14%        ✅ 15%        ✅ 15%
  Documentation                 ✅ 10%        ✅ 11%        ✅ 15%        ✅ 13%        ✅ 15%        ✅ 15%
  Debugging                     ✅ 13%        ✅ 14%        ✅ 17%        ✅ 15%        ✅ 17%        ✅ 17%
  Deployment Config             ✅ 10%        ✅ 11%        ✅ 14%        ✅ 12%        ✅ 14%        ✅ 14%
  ----------------------------------------------------------------------------------------------------
  Legend: ✅ LOW (<50%) | ⚠️  MEDIUM (50–70%) | 🔴 HIGH (70–90%) | 💥 CRITICAL (>90%)

====================================================================================================
  CONTEXT WINDOW RISK MATRIX — GPT-4o, File size: 2,000 LOC
====================================================================================================
  Phase                        Python   JavaScript         Java           Go         Rust            C
  ----------------------------------------------------------------------------------------------------
  Requirements                  ✅ 18%        ✅ 20%        ✅ 27%        ✅ 23%        ✅ 26%        ✅ 26%
  Architecture Design           ✅ 18%        ✅ 20%        ✅ 28%        ✅ 24%        ✅ 27%        ✅ 27%
  Code Generation               ✅ 18%        ✅ 20%        ✅ 28%        ✅ 24%        ✅ 27%        ✅ 27%
  Code Review                   ✅ 19%        ✅ 21%        ✅ 28%        ✅ 25%        ✅ 28%        ✅ 28%
  Refactoring                   ✅ 20%        ✅ 22%        ✅ 29%        ✅ 26%        ✅ 28%        ✅ 28%
  Test Generation               ✅ 19%        ✅ 21%        ✅ 28%        ✅ 24%        ✅ 27%        ✅ 27%
  Documentation                 ✅ 18%        ✅ 20%        ✅ 28%        ✅ 24%        ✅ 27%        ✅ 27%
  Debugging                     ✅ 21%        ✅ 23%        ✅ 30%        ✅ 26%        ✅ 29%        ✅ 29%
  Deployment Config             ✅ 18%        ✅ 20%        ✅ 27%        ✅ 23%        ✅ 26%        ✅ 26%
  ----------------------------------------------------------------------------------------------------
  Legend: ✅ LOW (<50%) | ⚠️  MEDIUM (50–70%) | 🔴 HIGH (70–90%) | 💥 CRITICAL (>90%)

===========================================================================
  CHUNKING THRESHOLDS — GPT-4o
  (LOC at which Code Review phase becomes HIGH or CRITICAL risk)
===========================================================================
  Language         MEDIUM (50%)   HIGH (70%)   CRITICAL (90%)
  ------------------------------------------------------------
  C                       3,832 LOC    5,467 LOC        7,101 LOC
  Go                      4,338 LOC    6,189 LOC        8,040 LOC
  Java                    3,711 LOC    5,294 LOC        6,877 LOC
  JavaScript              5,206 LOC    7,427 LOC        9,648 LOC
  Python                  5,805 LOC    8,282 LOC       10,759 LOC
  Rust                    3,827 LOC    5,460 LOC        7,092 LOC

  Interpretation:
    Files below the MEDIUM threshold: safe to send whole.
    Files in MEDIUM zone: consider format stripping and comment removal.
    Files in HIGH zone: use targeted extraction (function + dependencies).
    Files in CRITICAL zone: mandatory chunking required.

  For reference: a typical enterprise Java service class is 300–800 LOC.
  Java reaches HIGH risk at much smaller files than Python due to token density.


===========================================================================
  CHUNKING THRESHOLDS — Claude Sonnet 4.5
  (LOC at which Code Review phase becomes HIGH or CRITICAL risk)
===========================================================================
  Language         MEDIUM (50%)   HIGH (70%)   CRITICAL (90%)
  ------------------------------------------------------------
  C                       6,131 LOC    8,685 LOC       11,239 LOC
  Go                      6,941 LOC    9,833 LOC       12,725 LOC
  Java                    5,937 LOC    8,411 LOC       10,885 LOC
  JavaScript              8,329 LOC   11,800 LOC       15,270 LOC
  Python                  9,288 LOC   13,158 LOC       17,028 LOC
  Rust                    6,123 LOC    8,674 LOC       11,225 LOC

  Interpretation:
    Files below the MEDIUM threshold: safe to send whole.
    Files in MEDIUM zone: consider format stripping and comment removal.
    Files in HIGH zone: use targeted extraction (function + dependencies).
    Files in CRITICAL zone: mandatory chunking required.

  For reference: a typical enterprise Java service class is 300–800 LOC.
  Java reaches HIGH risk at much smaller files than Python due to token density.

✅ context_window_risk_analyzer.py completed.

━━━ Running: boilerplate_overhead_demo.py ━━━
======================================================================
Boilerplate Overhead Demo
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §4.1
Tokenizer: cl100k_base (GPT-4)
======================================================================

======================================================================
  1. Empty Function — Structural Overhead
======================================================================
  Language         Total   Logic  Boilerplate  Overhead%    Ratio
  --------------------------------------------------------------
  Python               8       2            6      75.0%   1.00×  ███████████████
  JavaScript          10       2            8      80.0%   1.25×  ████████████████
  Go                  12       2           10      83.3%   1.50×  ████████████████
  Java                12       2           10      83.3%   1.50×  ████████████████
  TypeScript          13       2           11      84.6%   1.62×  ████████████████
  C                   13       2           11      84.6%   1.62×  ████████████████
  Rust                26       1           25      96.2%   3.25×  ███████████████████

======================================================================
  2. Minimal Class (1 field + getter) — Boilerplate Tax
======================================================================
  Language         Total   Logic  Boilerplate  Overhead%    Ratio
  --------------------------------------------------------------
  JavaScript          28       4           24      85.7%   0.82×  █████████████████
  Python              34       4           30      88.2%   1.00×  █████████████████
  TypeScript          38       4           34      89.5%   1.12×  █████████████████
  Java                39       4           35      89.7%   1.15×  █████████████████
  Go                  43       4           39      90.7%   1.26×  ██████████████████
  C++                 45       4           41      91.1%   1.32×  ██████████████████
  Rust                51       4           47      92.2%   1.50×  ██████████████████
  C                   65       4           61      93.8%   1.91×  ██████████████████

======================================================================
  3. Program Entry Point — Structural Requirement
======================================================================
  Language         Total   Logic  Boilerplate  Overhead%    Ratio
  --------------------------------------------------------------
  Python               7       0            7     100.0%   1.00×  ████████████████████
  Ruby                 7       0            7     100.0%   1.00×  ████████████████████
  TypeScript           8       0            8     100.0%   1.14×  ████████████████████
  JavaScript           9       0            9     100.0%   1.29×  ████████████████████
  Rust                10       0           10     100.0%   1.43×  ████████████████████
  Go                  17       0           17     100.0%   2.43×  ████████████████████
  C++                 19       0           19     100.0%   2.71×  ████████████████████
  C                   21       0           21     100.0%   3.00×  ████████████████████
  Java                22       0           22     100.0%   3.14×  ████████████████████

💡 Key findings:

   EMPTY FUNCTION:
   Python and Ruby require the minimum structural tokens — just the function
   keyword, name, and body. Java requires public/Object/return each as separate
   tokens plus braces and semicolons. Rust requires explicit return type annotation.

   MINIMAL CLASS:
   Java's class is the most boilerplate-heavy because:
   - 'public class' access + type declaration
   - 'private final' per field
   - Explicit constructor with parameter type annotation
   - 'public String getName()' with full return type
   Python's class is lean: 'def' + 'self' + ':' and no access modifiers.

   ENTRY POINT:
   Python and Ruby have ZERO boilerplate — you just write code.
   Java requires 'public class Main { public static void main(String[] args) { ... } }'
   even for a one-liner program — a structural tax that applies to every generated file.

   This boilerplate overhead accumulates across every function, class, and file
   in a project, driving the 1.47× static token ratio for Java vs Python.
   See TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §4 for the full factor analysis.

✅ boilerplate_overhead_demo.py completed.

━━━ Running: type_annotation_impact.py ━━━
========================================================================
Type Annotation Impact Analysis
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §4.2
Tokenizer: cl100k_base (GPT-4)
========================================================================

Transformation                                    Before   After  +Tokens  Overhead%    Ratio
--------------------------------------------------------------------------------------------
Python (untyped) → Python (PEP 484 hints)                  130     195      +65      50.0%     1.50×
Python (PEP 484 hints) → Python (mypy strict)                    195     286      +91      46.7%     1.47×
Python (untyped) → Python (mypy strict)                    130     286     +156     120.0%     2.20×
JavaScript (dynamic) → TypeScript (typed)                      131     184      +53      40.5%     1.40×
Ruby (untyped) → Ruby (Steep annotated)                   94     239     +145     154.3%     2.54×

💡 Analysis:

   STATIC TOKEN OVERHEAD (these numbers):
   Adding PEP 484 type hints to Python adds ~15–30% tokens (static).
   JavaScript → TypeScript adds ~30–45% tokens (static).
   Ruby → Steep adds ~20–35% tokens (static) for explicit annotations.

   AGENTIC OVERHEAD (from benchmark data, not measured here):
   The production cost multiplier is significantly higher because:
   1. The LLM must generate type-correct code on the first try
   2. When type errors occur, additional correction turns are needed
   3. Less-familiar type checkers (Steep > mypy) require more iterations

   Published production cost multipliers (Claude Opus 4.6):
   • Python → Python/mypy strict:  1.6–1.7×  (mame benchmark)
   • JavaScript → TypeScript:       1.6×       (mame benchmark)
   • Ruby → Ruby/Steep:             2.0–3.2×   (mame benchmark)

   The static overhead here represents only the annotation tokens themselves.
   The agentic overhead includes reasoning cost, which can be 2–4× larger.

   See TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §4.2 for the full table.

✅ type_annotation_impact.py completed.

━━━ Running: identifier_length_analysis.py ━━━
================================================================================
Identifier Length Analysis
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §4.4
Tokenizer: cl100k_base (GPT-4)
================================================================================

=========================================================================================================
  Token Count by Naming Convention (same concept, different style)
=========================================================================================================
  Concept                              snake_case    camelCase   PascalCase   SCREAMING_SNAKE   Go-short
  ------------------------------------------------------------------------------------------------------
  user                                       user (1t)         user (1t)         User (1t) 
                                                                                                           USER (1t)       use (1t)
  user id                                 user_id (2t)       userId (1t)       UserId (1t) 
                                                                                                        USER_ID (2t)     userI (2t)
  user name                             user_name (2t)     userName (1t)     UserName (1t) 
                                                                                                      USER_NAME (2t)     userN (2t)
  get user                               get_user (2t)      getUser (1t)      GetUser (2t) 
                                                                                                       GET_USER (2t)      getU (2t)
  create user                         create_user (2t)   createUser (2t)   CreateUser (2t) 
                                                                                                    CREATE_USER (2t)   createU (2t)
  update user profile              update_user_profile (3t) updateUserProfile (2t) UpdateUserProfile (2t) 
                                                                                              UPDATE_USER_PROFILE (3t)  updateUP (2t)
  delete user account              delete_user_account (3t) deleteUserAccount (3t) DeleteUserAccount (3t) 
                                                                                              DELETE_USER_ACCOUNT (3t)  deleteUA (2t)
  http request handler             http_request_handler (3t) httpRequestHandler (3t) HttpRequestHandler (2t) 
                                                                                              HTTP_REQUEST_HANDLER (3t)    httpRH (2t)
  database connection pool         database_connection_pool (3t) databaseConnectionPool (3t) DatabaseConnectionPool (3t) 
                                                                                              DATABASE_CONNECTION_POOL (3t)  databaseCP (2t)
  authentication token validator   authentication_token_validator (3t) authenticationTokenValidator (3t) AuthenticationTokenValidator (3t) 
                                                                                              AUTHENTICATION_TOKEN_VALIDATOR (5t)  authenticationTV (2t)
  process payment transaction      process_payment_transaction (3t) processPaymentTransaction (3t) ProcessPaymentTransaction (3t) 
                                                                                              PROCESS_PAYMENT_TRANSACTION (3t)  processPT (2t)
  maximum retry count              maximum_retry_count (3t) maximumRetryCount (3t) MaximumRetryCount (3t) 
                                                                                              MAXIMUM_RETRY_COUNT (4t)  maximumRC (2t)
  default timeout milliseconds     default_timeout_milliseconds (4t) defaultTimeoutMilliseconds (3t) DefaultTimeoutMilliseconds (3t) 
                                                                                              DEFAULT_TIMEOUT_MILLISECONDS (4t)  defaultTM (2t)
  abstract factory pattern         abstract_factory_pattern (3t) abstractFactoryPattern (3t) AbstractFactoryPattern (3t) 
                                                                                              ABSTRACT_FACTORY_PATTERN (4t)  abstractFP (2t)
  ------------------------------------------------------------------------------------------------------

  Mean tokens/char:
  snake_case:      0.173 tok/char
  camelCase:       0.153 tok/char
  SCREAMING_SNAKE: 0.184 tok/char  ← least efficient
  Go-short:        0.282 tok/char  ← most efficient

================================================================================
  Token Boundary Analysis — Where BPE Draws the Lines
================================================================================
  user_id                                    →  2 tok   'user' | '_id'
                                               [Python] Common pattern → efficient merge

  userId                                     →  1 tok   'userId'
                                               [Java] CamelCase → similar to snake

  USER_ID                                    →  2 tok   'USER' | '_ID'
                                               [Java] Constant → worst fragmentation

  getUserById                                →  2 tok   'getUser' | 'ById'
                                               [Java] Common method pattern

  get_user_by_id                             →  4 tok   'get' | '_user' | '_by' | '_id'
                                               [Python] Snake equivalent

  AbstractSingletonProxyFactoryBean          →  5 tok   'Abstract' | 'Singleton' | 'Proxy' | 'Factory' | 'Bean'
                                               [Java] Famous verbose Java name

  AbstractSingProxyFactory                   →  4 tok   'Abstract' | 'Sing' | 'Proxy' | 'Factory'
                                               [Java] Same but abbreviated

  calculateWordFrequency                     →  3 tok   'calculate' | 'Word' | 'Frequency'
                                               [Java] 3-word camelCase

  calculate_word_frequency                   →  3 tok   'calculate' | '_word' | '_frequency'
                                               [Python] 3-word snake_case

  CALCULATE_WORD_FREQUENCY                   →  5 tok   'CAL' | 'C' | 'ULATE' | '_WORD' | '_FREQUENCY'
                                               [Java] 3-word SCREAMING

  ctx                                        →  1 tok   'ctx'
                                               [Go] Common Go abbreviation

  err                                        →  1 tok   'err'
                                               [Go] Go error variable

  httpResponseWriter                         →  3 tok   'http' | 'Response' | 'Writer'
                                               [Go] Go HTTP handler param

  http_response_writer                       →  3 tok   'http' | '_response' | '_writer'
                                               [Python] Python equivalent

  HTTP_RESPONSE_WRITER                       →  4 tok   'HTTP' | '_RESPONSE' | '_WR' | 'ITER'
                                               [Python] Python constant style


================================================================================
  Java Enterprise Naming Patterns — Token Cost Analysis
================================================================================
  Class name                                          Tokens   Chars  Tok/char
  ----------------------------------------------------------------------------
  AbstractBeanFactory                                      3      19     0.158   Spring core class
  AbstractAutowireCapableBeanFactory                       8      34     0.235   Spring core class (real)
  DefaultListableBeanFactory                               5      26     0.192   Spring core class (real)
  DispatcherServletWebApplicationContext                   4      38     0.105   Spring MVC class (real)
  JpaRepositoryFactoryBean                                 4      24     0.167   Spring Data class (real)
  TransactionAttributeSourceAdvisor                        4      33     0.121   Spring AOP class (real)
  AbstractSingletonProxyFactoryBean                        5      33     0.152   Spring classic (real)
  ApplicationContextAwareProcessor                         3      32     0.094   Spring lifecycle (real)

  These are real Spring Framework class names. In a Java codebase that imports
  and uses these classes, each occurrence of the full class name consumes the
  tokens shown above. In a 500-LOC Spring service class, dozens of such names
  can appear multiple times, contributing meaningfully to token overhead.


💡 Summary:
   1. Naming convention is a CONTROLLABLE token factor — unlike language syntax.
   2. SCREAMING_SNAKE_CASE is the least token-efficient (~50% worse per char).
   3. Go-style short names (ctx, err, r) are the most efficient but reduce clarity.
   4. Java enterprise patterns (AbstractXxxFactoryBean) are particularly expensive.
   5. TOKDRIFT research shows that identifier style changes can alter LLM output
      predictions by up to 60% — it's not just a cost issue, it's a reliability issue.

   Recommendation: Prefer standard snake_case or camelCase identifiers in code
   sent to LLMs. Avoid SCREAMING_SNAKE_CASE in prompts unless required by context.
   See TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §4.4 for full discussion.

✅ identifier_length_analysis.py completed.

━━━ Running: prompt_compression_demo.py ━━━
========================================================================
Prompt Compression Demo
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §6
Tokenizer: cl100k_base (GPT-4)
========================================================================

Technique                  Language      Before   After   Saving  Saving%
--------------------------------------------------------------------------
Format stripping           Java             416     375      +41     9.9%
Comment stripping          Java             416     195     +221    53.1%
Comments + Format strip    Java             416     171     +245    58.9%
Comment stripping          C                275     142     +133    48.4%
Inline comment strip       Python           266     242      +24     9.0%
Signature extraction       Python           266      63     +203    76.3%

💡 Key findings:

   FORMAT STRIPPING (Java):
   Removing blank lines and leading whitespace from Java code saves ~15–20%
   of input tokens with no semantic change. The LLM understands the code
   equivalently with or without formatting (Pan et al. 2025 finding).
   Python cannot benefit from this — its indentation is syntax.

   COMMENT STRIPPING:
   Java Javadoc comments are verbose (/** @param @return @throws */).
   Removing them from INPUT prompts (not from generated output) saves
   10–25% tokens. C achieves similar savings from doxygen-style comments.

   COMBINED (Format + Comments):
   The combined saving for Java can reach 25–40% of input tokens.
   At Claude Sonnet pricing ($3/M input), this directly translates to cost.

   SIGNATURE EXTRACTION:
   For documentation generation or API review tasks, sending only the
   function signature + docstring instead of the full body saves 50–75%
   of input tokens with no reduction in output quality for those tasks.

   IMPORTANT: These techniques apply to INPUT tokens only. Output tokens
   (generated code) are usually not affected. Since output is 3–5× more
   expensive per token, the overall cost reduction is significant but bounded.
   A 30% input saving = ~18% total savings when input is 60% of total cost.

✅ prompt_compression_demo.py completed.

━━━ Running: chunking_strategy_demo.py ━━━
===========================================================================
Chunking Strategy Demo
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §6.1 Strategy 3
Tokenizer: cl100k_base (GPT-4)
===========================================================================

  Sample module size: 853 tokens (107 lines)

===========================================================================
  Chunking Comparison — Python | Small Model (32K)
  File: 853 tokens | Available per chunk: 26,000 tokens
===========================================================================

  Strategy                        Chunks  Avg tok/chunk  Max tok/chunk  Prompt overhead
  -------------------------------------------------------------------------------------
  Line-based (naive)                   3            284            330            6,000
  Function-level                       4            205            402            8,000
  Token-budget-aware (recommended)       1            853            853            2,000

  Recommendation for this file × context:
    - Token-budget-aware is optimal: it maximizes chunk density while
      respecting the context limit and breaking at clean boundaries.
    - Line-based is risky: it may produce chunks that split mid-function,
      causing the LLM to generate incomplete or inconsistent code.
    - Function-level works well for smaller files; the number of LLM calls
      scales linearly with the number of functions.

  Overhead note: Each additional chunk requires repeating the system prompt
  (2,000 tokens). For 4 function-level chunks, that is
  8,000 tokens of overhead — a real cost multiplier.


===========================================================================
  Chunking Comparison — Python | GPT-4o
  File: 853 tokens | Available per chunk: 116,000 tokens
===========================================================================

  Strategy                        Chunks  Avg tok/chunk  Max tok/chunk  Prompt overhead
  -------------------------------------------------------------------------------------
  Line-based (naive)                   3            284            330           12,000
  Function-level                       4            205            402           16,000
  Token-budget-aware (recommended)       1            853            853            4,000

  Recommendation for this file × context:
    - Token-budget-aware is optimal: it maximizes chunk density while
      respecting the context limit and breaking at clean boundaries.
    - Line-based is risky: it may produce chunks that split mid-function,
      causing the LLM to generate incomplete or inconsistent code.
    - Function-level works well for smaller files; the number of LLM calls
      scales linearly with the number of functions.

  Overhead note: Each additional chunk requires repeating the system prompt
  (4,000 tokens). For 4 function-level chunks, that is
  16,000 tokens of overhead — a real cost multiplier.


===========================================================================
  Language-Specific Chunking Thresholds (GPT-4o, Code Review task)
===========================================================================
  Language        Tokens/LOC  Max LOC in 32K   Max LOC in 128K
  --------------------------------------------------------------
  Python               10.34           2,515 LOC        11,610 LOC
  JavaScript           11.53           2,256 LOC        10,411 LOC
  TypeScript           14.80           1,757 LOC         8,108 LOC
  Go                   13.83           1,880 LOC         8,676 LOC
  Java                 16.17           1,608 LOC         7,421 LOC
  Rust                 15.68           1,658 LOC         7,653 LOC
  C                    15.66           1,660 LOC         7,663 LOC

  Key takeaway:
    C code exhausts a 32K context window at ~1,800 LOC.
    Python can accommodate ~2,700 LOC in the same window.
    This 50% gap means C projects require more chunks, more API calls,
    and more prompt-overhead tokens per project — compounding the base cost.

✅ chunking_strategy_demo.py completed.

━━━ Running: token_aware_sdlc_pipeline.py ━━━
========================================================================
Token-Aware SDLC Pipeline
Supports: TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §6
========================================================================

==================================================
  PIPELINE RUN 1: Java (high token multiplier)
==================================================
  [Pipeline] Starting code_generation for Java
  [Pipeline] ⚡ Prototyping in Python first (reduces stuck-loop risk)
  [Pipeline] Format strip applied (Java): 1037 → 1033 chars
  [Pipeline] Comment strip applied: 1033 → 664 chars (35.7% saving)
  [Pipeline] Code generation complete.
  [Pipeline] Starting code_review for Java
  [Pipeline] Code review complete.
  [Pipeline] Starting test_generation for Java (JUnit 5)
  [Pipeline] Format strip applied (Java): 1037 → 1033 chars
  [Pipeline] Comment strip applied: 1033 → 664 chars (35.7% saving)
  [Pipeline] Test generation complete.
  [Pipeline] Starting documentation for Java
  [Pipeline] Format strip applied (Java): 1037 → 1033 chars
  [Pipeline] Documentation generation complete.
  [Pipeline] Starting debugging for Java
  [Pipeline] Debugging complete.

============================================================
  SESSION REPORT — Java × claude-sonnet-4
============================================================
  Session summary — claude-sonnet-4
  Total calls:   6
  Input tokens:  1,889
  Output tokens: 756
  Cached tokens: 213
  Total tokens:  2,645
  Est. cost:     $0.0164

  Phase                     Input   Output   Cached
  ----------------------------------------------------
  python_prototype             58       23       13
  code_generation             158       63       40
  code_review                 452      181       40
  test_generation             324      130       40
  documentation               445      178       40
  debugging                   452      181       40
============================================================

  Optimization strategies applied:
    Format stripping:      ✅ Applied
    Comment stripping:     ✅ Applied
    Python prototyping:    ✅ Available
    Token multiplier:      1.47× vs. Python baseline
    Max chunk size:        2,100 LOC

==================================================
  PIPELINE RUN 2: Python (baseline — most efficient)
==================================================
  [Pipeline] Starting code_generation for Python
  [Pipeline] Comment strip applied: 633 → 626 chars (1.1% saving)
  [Pipeline] Code generation complete.
  [Pipeline] Starting code_review for Python
  [Pipeline] Code review complete.
  [Pipeline] Starting test_generation for Python (pytest)
  [Pipeline] Comment strip applied: 633 → 626 chars (1.1% saving)
  [Pipeline] Test generation complete.
  [Pipeline] Starting documentation for Python
  [Pipeline] Signature extraction for docs: 633 → 301 chars (52.4% saving)
  [Pipeline] Documentation generation complete.
  [Pipeline] Starting debugging for Python
  [Pipeline] Debugging complete.

============================================================
  SESSION REPORT — Python × claude-sonnet-4
============================================================
  Session summary — claude-sonnet-4
  Total calls:   5
  Input tokens:  1,491
  Output tokens: 597
  Cached tokens: 206
  Total tokens:  2,088
  Est. cost:     $0.0129

  Phase                     Input   Output   Cached
  ----------------------------------------------------
  code_generation             333      133       42
  code_review                 321      128       41
  test_generation             314      126       42
  documentation               204       82       41
  debugging                   319      128       40
============================================================

  Optimization strategies applied:
    Format stripping:      ⬜ Skipped (Python/Ruby)
    Comment stripping:     ✅ Applied
    Python prototyping:    ⬜ Not needed
    Token multiplier:      1.00× vs. Python baseline
    Max chunk size:        2,700 LOC

📊 Java vs Python cost ratio (this session): 1.28×
   Java: $0.0164  |  Python: $0.0129

   Note: This is a STUB simulation. Real ratios from benchmark research:
   - Static token ratio:    Java = 1.47× Python
   - Agentic cost ratio:    Java = 1.18–1.34× Python (Wu et al. 2026)
   - Production cost ratio: Java = 1.32× Python (mame benchmark)

  See TOKEN_CONSUMPTION_ACROSS_LANGUAGES.md §6 for all optimization strategies.
✅ token_aware_sdlc_pipeline.py completed.

================================================================
Pipeline complete at Tue Aug 25 12:20:28 CEST 2026
Passed: 16 / 16
================================================================

Conclusion

Programming language choice is not merely a syntactic preference or execution runtime decision — it acts as a foundational financial variable in LLM-assisted software development platforms. Dynamic, expressive languages like Ruby and Python minimize token footprint, maximize working context headroom, and minimize correction loops in AI agent interactions. Conversely, lower-level or verbosity-heavy languages like C, C++, Java, and TypeScript carry an inherent structural “token tax” that scales exponentially across every phase of the software lifecycle.

To build cost-effective and scalable AI platforms, engineering teams must actively manage their stack’s token economics. Platform engineers can maximize efficiency by building orchestrators, prompt wrappers, and AI tools in token-efficient languages (Python/Ruby) while establishing automated context preprocessing (format stripping, comment removal, and AST signature extraction) for enterprise repositories written in higher-overhead target languages.

Thanks for reading 💰💸💳

Inside IBM Granite 4.2: Building an Orchestrate-Ready Reasoning App

Alain Airom (Ayrom) — Wed, 26 Aug 2026 11:40:18 +0000

A sample application built by IBM Bob to demonstrate reasoning and implementing plug-and-play tool calling

Introduction to IBM Granite 4.2

IBM Granite 4.2 represents a significant leap forward in enterprise-grade, open-source large language models. Designed with standard architectural efficiency, Granite 4.2 natively integrates hybrid capabilities: extended multi-turn reasoning (Chain-of-Thought via <think> blocks) paired with high-precision XML-based tool calling.

Whether deployed on resource-constrained edge hardware via llama.cpp quantization or scaled across high-performance GPUs using Hugging Face's transformers, Granite 4.2 provides granular control over inference latency and reasoning budgets through native thinking flags (Full, Low-Effort, and Non-Thinking modes).

Image from IBM Site

Implementation-Application Architecture & Implementation Overview

As always, I decided to test the model locally and for sure using the Bob SDLC is a great accelerator! The application architecture engineered by Bob establishes a clean separation of concerns across presentation, orchestration, model integration, and dynamic tool execution.

System Topology & Request Flow

The system connects a Streamlit frontend UI (running on port 8766) to a high-throughput FastAPI backend (running on port 8765).

Streamlit Frontend (8766)  --->  FastAPI Backend (8765)  --->  Model Adapter (model.py)    ├─ Multi-Turn Chat               ├─ /api/chat                    ├─ HFBackend (Transformers)    ├─ Tool Calling UI               ├─ /api/tool_call               └─ LlamaCppBackend (REST)    └─ Thinking Mode Controls        └─ /api/thinking                        │                                                                             ▼                                                                    Tools Registry (tools/)                                                                     └─ Dynamic Autodiscover

Streamlit Frontend (8766)  --->  Fasgranite42/
├── backend/
│   ├── app.py          # FastAPI server (5 demo endpoints + GET /api/tools + GET /api/health)
│   └── model.py        # HuggingFace + llama.cpp model adapters
├── frontend/
│   └── app.py          # Streamlit UI (5 tabs)
├── tools/
│   ├── registry.py          # @tool decorator, autodiscover(), get_schemas(), execute()
│   ├── definitions.py       # Backward-compat shim → delegates to registry
│   └── catalog/             # Plug-and-play tool files (drop a .py here to add a tool)
│       ├── calculator.py    # Safe arithmetic evaluator
│       ├── get_weather.py   # Stub weather data
│       └── geocode.py       # GPS coordinates from address/landmark
├── config/
│   └── settings.py     # Typed settings from .env
├── tests/
│   ├── test_tools.py           # Unit tests for all catalog tools (calculator, weather, geocode)
│   ├── test_registry.py        # Unit + integration tests for the registry mechanism
│   └── test_backend_api.py     # Unit tests for FastAPI endpoints + llama.cpp
├── scripts/
│   ├── setup_venv.sh        # Create venv + install deps
│   ├── start.sh             # Start backend + frontend (detached)
│   ├── start_backend.sh
│   ├── start_frontend.sh
│   ├── stop.sh              # Graceful shutdown of backend + frontend
│   ├── start_llamacpp.sh    # Start standalone llama-server on port 9932
│   ├── stop_llamacpp.sh     # Stop the llama-server
│   └── probe_tokenizer.py   # Diagnose HF tokenizer think-token format
├── Docs/
│   ├── Architecture.md      # System architecture + sequence diagrams
│   ├── Quickstart.md        # 10-minute setup guide
│   └── Adding-a-Tool.md     # Developer guide: how to write and register a new tool
├── models/             # GGUF model files (git-ignored)
├── input/              # Input documents (content git-ignored)
├── output/             # Output files (content git-ignored)
├── .env.example        # Configuration template
└── requirements.txt

Core Orchestration (`backend/app.py`)

The FastAPI backend defines dedicated endpoints for chat, full reasoning, low-effort reasoning, direct non-thinking mode, and two-pass tool execution. Below is an excerpt illustrating the two-pass tool invocation sequence in app.py:

@app.post("/api/tool_call", response_model=GraniteResponse)
async def tool_call(request: ToolCallRequest) -> GraniteResponse:
    backend = get_backend(request.backend)
    messages = [{"role": "user", "content": request.prompt}]

    # Pass 1: Model evaluates prompt against tool schemas
    first_result = backend.generate(
        messages,
        tools=get_schemas(),
        enable_thinking=True
    )

    # Step 2: Execute parsed tool calls locally
    tool_results = []
    for tc in first_result.tool_calls:
        fn = tc["function"]
        result = execute_tool(fn["name"], fn.get("arguments", {}))
        tool_results.append({"tool": fn["name"], "result": result})

    # Step 3: Pass tool outputs back for final synthesis
    if tool_results:
        # Construct multi-turn history with tool responses...
        second_result = backend.generate(second_messages, tools=get_schemas())
        return _result_to_response(first_result, tool_results=tool_results, final_answer=second_result.answer_text)
Output Parsing & Reasoning Traces (backend/model.py)

Output Parsing & Reasoning Traces (`backend/model.py`)

Granite 4.2 handles thinking tokens via Chat Templates. When enable_thinking=True, apply_chat_template() injects <think> directly into the prompt. Consequently, output parsing must account for three distinct structural cases:

Case A: Full <think>...</think> block present (standard with llama.cpp).
Case B: Only closing </think> present (standard with Hugging Face, as opening tag was injected into prompt).
Case C: No think tags present (Non-thinking mode or truncated output).

"""
backend/model.py
─────────────────
Unified model adapter for Granite 4.2.

Supports two backends:
  • HuggingFace (transformers) – loads the model locally via GPU/CPU
  • llama.cpp server           – calls the OpenAI-compatible REST API at
                                 http://localhost:9931/v1 (per AGENTS.MD)

Both backends expose an identical interface:

    generate(messages, tools=None, enable_thinking=True, low_effort=False)
    → GenerateResult(think_text, answer_text, raw_output)

All generation parameters follow the Granite 4.2 model card exactly:
  temperature = 1.0
  top_p       = 0.95
  max_new_tokens depends on mode (see settings.py)

References:
  https://huggingface.co/ibm-granite/granite-4.2-8b
  https://github.com/ibm-granite/granite-4.2-language-models
"""

from __future__ import annotations

import json
import re
import sys
import logging
from dataclasses import dataclass, field
from typing import Any

import httpx

from config.settings import (
    BACKEND,
    HF_MODEL_ID,
    HF_TOKEN,
    LLAMACPP_BASE_URL,
    LLAMACPP_MODEL_NAME,
    TEMPERATURE,
    TOP_P,
    MAX_NEW_TOKENS_THINKING,
    MAX_NEW_TOKENS_NON_THINKING,
    MAX_NEW_TOKENS_LOW_EFFORT,
)

logger = logging.getLogger(__name__)

# ─────────────────────────────────────────────────────────────────────────────
# Result container
# ─────────────────────────────────────────────────────────────────────────────

@dataclass
class GenerateResult:
    """Structured output from any backend generate() call."""
    think_text: str          # Content inside <think>…</think> (empty if non-thinking)
    answer_text: str         # Final answer after </think>
    tool_calls: list[dict]   # Parsed tool_call objects (list, may be empty)
    raw_output: str          # Full decoded string before parsing


# ─────────────────────────────────────────────────────────────────────────────
# Shared parsing helpers
# ─────────────────────────────────────────────────────────────────────────────

# Granite 4.2 uses <think>…</think> tags (model card / GitHub README).
#
# IMPORTANT – template injection behaviour (confirmed by probe_tokenizer.py):
#   When enable_thinking=True, apply_chat_template() appends "<think>\n" to
#   the PROMPT (not the generated output).  The generated output therefore
#   starts *inside* the think block:
#
#     generated = "[reasoning…]\n</think>\n[answer…]<|im_end|>"
#
#   There is NO opening <think> tag in the generated tokens.
#
#   When enable_thinking=False, the template appends "<think></think>" to
#   the prompt, so the generated output contains NO think tags at all.
#
# The parser handles three cases in order:
#   A. Full block "<think>…</think>" present (llama.cpp or custom prompts)
#   B. Only "</think>" present, no "<think>" opener (HuggingFace backend)
#   C. Neither tag present (non-thinking mode or very short output)
_THINK_OPEN_RE  = re.compile(r"<think\s*>", re.IGNORECASE)
_THINK_CLOSE_RE = re.compile(r"</think\s*>", re.IGNORECASE)
_THINK_BLOCK_RE = re.compile(r"<think\s*>(.*?)</think\s*>", re.DOTALL | re.IGNORECASE)

_TOOL_CALL_RE = re.compile(
    r"<tool_call>\s*<function=(?P<name>\w+)>(?P<body>.*?)</function>\s*</tool_call>",
    re.DOTALL,
)
_PARAM_RE = re.compile(r"<parameter=(?P<key>\w+)>\s*(?P<val>.*?)\s*</parameter>", re.DOTALL)

# Special tokens to strip from the decoded output
_SPECIAL_TOKENS = ["<|im_end|>", "<|im_start|>", "<|endoftext|>"]


def _parse_output(raw: str) -> GenerateResult:
    """
    Parse a Granite 4.2 raw decoded output string into think/answer/tool_calls.

    The parser handles three cases reflecting Granite 4.2 template behaviour:

    Case A – Full block in raw (llama.cpp backend / custom prompts):
        "<think>reasoning…</think>answer"
        The opening tag is present in the decoded string.

    Case B – Only closing tag in raw (HuggingFace backend, enable_thinking=True):
        "reasoning…</think>answer"
        apply_chat_template() injects "<think>\\n" into the PROMPT, so the
        generated tokens start *inside* the think block.  The opener is absent.

    Case C – No think tags at all (enable_thinking=False):
        "answer"
        The template injects "<think></think>" into the PROMPT; no tags appear
        in the generated output.  think_text = "".
    """
    # ── 1. Extract reasoning trace ────────────────────────────────────────────
    think_match = _THINK_BLOCK_RE.search(raw)
    if think_match:
        # Case A: full <think>…</think> block present in generated text
        think_text = think_match.group(1).strip()
        answer_part = _THINK_BLOCK_RE.sub("", raw).strip()
    else:
        close_match = _THINK_CLOSE_RE.search(raw)
        if close_match:
            # Case B: only </think> present — opener was injected into the prompt.
            # Everything before </think> is the reasoning trace.
            think_text = raw[: close_match.start()].strip()
            # Strip leading special tokens that may appear at the very start.
            # IMPORTANT: use str.removeprefix(), NOT str.lstrip() — lstrip treats
            # the argument as a SET OF CHARACTERS, not a literal prefix, which
            # would corrupt any think_text whose first chars happen to overlap
            # with the special-token character set.
            for tok in _SPECIAL_TOKENS:
                if think_text.startswith(tok):
                    think_text = think_text[len(tok):].strip()
            # Everything after </think> is the answer region
            answer_part = raw[close_match.end() :].strip()
        else:
            # Case C: no think tags — non-thinking mode or truncated short output
            # Also handles the legacy fallback where <think> was opened but
            # never closed (very rare with sufficient max_new_tokens).
            open_match = _THINK_OPEN_RE.search(raw)
            if open_match:
                # Unclosed <think>: treat entire tail as reasoning trace
                think_text = raw[open_match.end() :].strip()
                for tok in _SPECIAL_TOKENS:
                    think_text = think_text.replace(tok, "").strip()
                answer_part = ""
            else:
                think_text = ""
                answer_part = raw

    # ── 2. Strip residual special tokens from the answer region ──────────────
    for tok in _SPECIAL_TOKENS:
        answer_part = answer_part.replace(tok, "")
    answer_part = answer_part.strip()

    # ── 3. Parse tool calls (Granite 4.2 XML-based tool call format) ─────────
    tool_calls: list[dict] = []
    for tc_match in _TOOL_CALL_RE.finditer(answer_part):
        name = tc_match.group("name")
        body = tc_match.group("body")
        params: dict[str, Any] = {}
        for pm in _PARAM_RE.finditer(body):
            params[pm.group("key")] = pm.group("val")
        tool_calls.append({"function": {"name": name, "arguments": params}})

    # Remove tool_call XML from the answer text
    answer_text = _TOOL_CALL_RE.sub("", answer_part).strip()

    return GenerateResult(
        think_text=think_text,
        answer_text=answer_text,
        tool_calls=tool_calls,
        raw_output=raw,
    )


# ─────────────────────────────────────────────────────────────────────────────
# HuggingFace backend
# ─────────────────────────────────────────────────────────────────────────────

class HFBackend:
    """
    Loads ibm-granite/granite-4.2-8b (or whichever HF_MODEL_ID is set)
    locally using the HuggingFace transformers library.

    CPU support:
      On CPU, bfloat16 is not supported by PyTorch; float32 is used instead.
      For reasonable performance on CPU use the 3B model:
        HF_MODEL_ID=ibm-granite/granite-4.2-3b  (in .env)

    Inference parameters as specified in the Granite 4.2 model card:
      temperature=1.0, top_p=0.95, do_sample=True
    Per-call overrides are accepted from the UI.
    """

    def __init__(self) -> None:
        self._model = None
        self._tokenizer = None
        self._device = None

    def _lazy_load(self) -> None:
        """Deferred model loading — only executed on first generate() call."""
        if self._model is not None:
            return

        import torch
        from transformers import AutoModelForCausalLM, AutoTokenizer

        logger.info("Loading tokenizer: %s", HF_MODEL_ID)
        kwargs: dict[str, Any] = {}
        if HF_TOKEN:
            kwargs["token"] = HF_TOKEN

        self._tokenizer = AutoTokenizer.from_pretrained(HF_MODEL_ID, **kwargs)

        self._device = "cuda" if torch.cuda.is_available() else "cpu"
        # bfloat16 is not supported on CPU; fall back to float32
        dtype = torch.bfloat16 if self._device == "cuda" else torch.float32
        logger.info("Loading model: %s (%s on %s)", HF_MODEL_ID, dtype, self._device)
        self._model = AutoModelForCausalLM.from_pretrained(
            HF_MODEL_ID,
            device_map=self._device,
            torch_dtype=dtype,
            **kwargs,
        )
        self._model.eval()
        logger.info("Model loaded on device: %s", self._device)

    def generate(
        self,
        messages: list[dict],
        tools: list[dict] | None = None,
        enable_thinking: bool = True,
        low_effort: bool = False,
        temperature: float | None = None,
        top_p: float | None = None,
        max_new_tokens: int | None = None,
    ) -> GenerateResult:
        """
        Run inference using the HuggingFace transformers pipeline.

        Template parameters follow the Granite 4.2 model card:
          enable_thinking=True  → full chain-of-thought inside <think>…</think>
          enable_thinking=False → non-thinking mode (empty <think></think>)
          low_effort=True       → reduced thinking budget

        Per-call overrides (temperature, top_p, max_new_tokens) take precedence
        over the values loaded from .env so the Streamlit UI can change them
        without restarting the server.
        """
        import torch

        self._lazy_load()

        # Resolve effective generation parameters:
        # UI override → .env value → model-card defaults
        eff_temperature = temperature if temperature is not None else TEMPERATURE
        eff_top_p = top_p if top_p is not None else TOP_P
        if max_new_tokens is not None:
            eff_max_new_tokens = max_new_tokens
        elif not enable_thinking:
            eff_max_new_tokens = MAX_NEW_TOKENS_NON_THINKING
        elif low_effort:
            eff_max_new_tokens = MAX_NEW_TOKENS_LOW_EFFORT
        else:
            eff_max_new_tokens = MAX_NEW_TOKENS_THINKING

        # Build prompt string using the model's chat template
        template_kwargs: dict[str, Any] = {
            "tokenize": False,
            "add_generation_prompt": True,
            "enable_thinking": enable_thinking,
        }
        if low_effort:
            template_kwargs["low_effort"] = True
        if tools:
            template_kwargs["tools"] = tools

        text = self._tokenizer.apply_chat_template(messages, **template_kwargs)
        inputs = self._tokenizer(text, return_tensors="pt").to(self._model.device)

        with torch.no_grad():
            output = self._model.generate(
                **inputs,
                max_new_tokens=eff_max_new_tokens,
                temperature=eff_temperature,
                top_p=eff_top_p,
                do_sample=True,
            )

        # Decode only the newly generated tokens
        raw = self._tokenizer.decode(
            output[0][inputs.input_ids.shape[-1]:],
            skip_special_tokens=False,
        )
        return _parse_output(raw)


# ─────────────────────────────────────────────────────────────────────────────
# llama.cpp backend
# ─────────────────────────────────────────────────────────────────────────────

class LlamaCppBackend:
    """
    Communicates with a running llama.cpp server via its OpenAI-compatible API.

    From AGENTS.MD:
      llama.cpp base URL: http://localhost:9931/v1
      llama.cpp port:     9931

    Model-name discovery
    ────────────────────
    The server running on port 9931 may be Llama.app (macOS GUI), which starts
    in "router mode" and registers models under names like
    "ibm-granite/granite-4.2-3b:Q4_K_M" — not the bare alias in .env.

    On first generate() call we query GET /v1/models and pick the first model
    that contains the LLAMACPP_MODEL_NAME substring (case-insensitive).  If
    nothing matches, the exact value from .env is used as-is.

    If /v1/models returns an empty list the server has no model loaded yet
    (e.g. Llama.app idle after --sleep-idle-seconds).  We raise a clear error
    instead of a cryptic 400.

    Thinking mode
    ─────────────
    llama.cpp does not expose enable_thinking / low_effort as API fields.
    Thinking behaviour is requested via a system prompt that instructs the
    model to use <think>…</think> tags; the same _parse_output() logic then
    extracts the trace from the response.
    """

    # System prompt prefixes for each mode.
    #
    # NOTE: with llama.cpp --jinja the model's thinking is extracted
    # automatically into reasoning_content regardless of system prompt.
    # These prompts only control the STYLE of the answer in content.
    _SYS_THINKING = (
        "You are a helpful assistant. Think carefully before answering."
    )
    _SYS_NON_THINKING = (
        "You are a helpful assistant. Answer directly and concisely."
    )
    _SYS_LOW_EFFORT = (
        "You are a helpful assistant. Be brief and direct in your answer."
    )

    def __init__(self) -> None:
        self._client = httpx.Client(
            base_url=LLAMACPP_BASE_URL,
            timeout=120.0,
        )
        # Resolved model name – populated lazily on first generate() call
        self._resolved_model: str | None = None

    def _resolve_model_name(self) -> str:
        """
        Query GET /v1/models and return the best-matching model alias.

        Matching strategy (in order):
          1. Exact match against LLAMACPP_MODEL_NAME
          2. First model whose id contains LLAMACPP_MODEL_NAME (case-insensitive)
          3. First model in the list (fallback)
          4. LLAMACPP_MODEL_NAME as-is if the list is empty (will likely 400,
             but we raise a friendlier error below)

        Raises RuntimeError if no model is loaded on the server.
        """
        if self._resolved_model is not None:
            return self._resolved_model

        try:
            resp = self._client.get("/models", timeout=5.0)
            resp.raise_for_status()
            models = resp.json().get("data", [])
        except Exception as exc:
            raise RuntimeError(
                f"Cannot reach the llama.cpp server at {LLAMACPP_BASE_URL}. "
                f"Is it running?  Error: {exc}"
            ) from exc

        if not models:
            raise RuntimeError(
                "The llama.cpp server at "
                f"{LLAMACPP_BASE_URL} is running but has NO model loaded.\n\n"
                "If you are using Llama.app, open the app and select/start a "
                "Granite 4.2 model before switching the backend here.\n\n"
                "If you started llama-server manually, make sure you passed "
                "--model /path/to/granite-4.2-3b-Q4_K_M.gguf"
            )

        available_ids = [m["id"] for m in models]
        needle = LLAMACPP_MODEL_NAME.lower()

        # 1. exact match
        for mid in available_ids:
            if mid == LLAMACPP_MODEL_NAME:
                self._resolved_model = mid
                logger.info("llama.cpp: exact model match: %s", mid)
                return mid

        # 2. substring match
        for mid in available_ids:
            if needle in mid.lower():
                self._resolved_model = mid
                logger.info("llama.cpp: substring model match: %s (needle=%s)", mid, needle)
                return mid

        # 3. fallback: first model
        self._resolved_model = available_ids[0]
        logger.warning(
            "llama.cpp: LLAMACPP_MODEL_NAME=%r not found in server list %s; "
            "falling back to first available model: %s",
            LLAMACPP_MODEL_NAME, available_ids, self._resolved_model,
        )
        return self._resolved_model

    def generate(
        self,
        messages: list[dict],
        tools: list[dict] | None = None,
        enable_thinking: bool = True,
        low_effort: bool = False,
        temperature: float | None = None,
        top_p: float | None = None,
        max_new_tokens: int | None = None,
    ) -> GenerateResult:
        """
        Call POST /v1/chat/completions on the llama.cpp server.
        Returns a GenerateResult consistent with HFBackend.
        Per-call overrides (temperature, top_p, max_new_tokens) take precedence
        over values loaded from .env.
        """
        if not enable_thinking:
            sys_prompt = self._SYS_NON_THINKING
        elif low_effort:
            sys_prompt = self._SYS_LOW_EFFORT
        else:
            sys_prompt = self._SYS_THINKING

        # Inject system message at the front
        full_messages = [{"role": "system", "content": sys_prompt}] + messages

        # Resolve model name (queries /v1/models on first call, raises clear error
        # if no model is loaded instead of letting the server return a cryptic 400)
        model_name = self._resolve_model_name()

        # Resolve effective generation parameters
        eff_temperature = temperature if temperature is not None else TEMPERATURE
        eff_top_p = top_p if top_p is not None else TOP_P
        if max_new_tokens is not None:
            eff_max_tokens = max_new_tokens
        elif not enable_thinking:
            eff_max_tokens = MAX_NEW_TOKENS_NON_THINKING
        elif low_effort:
            eff_max_tokens = MAX_NEW_TOKENS_LOW_EFFORT
        else:
            eff_max_tokens = MAX_NEW_TOKENS_THINKING

        payload: dict[str, Any] = {
            "model": model_name,
            "messages": full_messages,
            "temperature": eff_temperature,
            "top_p": eff_top_p,
            "max_tokens": eff_max_tokens,
        }
        if tools:
            payload["tools"] = tools
            payload["tool_choice"] = "auto"

        response = self._client.post("/chat/completions", json=payload)
        response.raise_for_status()
        data = response.json()

        choice = data["choices"][0]
        msg = choice["message"]

        # ── Extract thinking trace ────────────────────────────────────────────
        # llama.cpp with --jinja separates thinking content into a dedicated
        # "reasoning_content" field (NOT inside <think> tags in "content").
        # We read it directly instead of relying on tag parsing.
        think_text: str = (msg.get("reasoning_content") or "").strip()
        answer_text: str = (msg.get("content") or "").strip()

        # Fallback: if content is empty but the finish_reason is "stop",
        # the model may have put its entire response in reasoning_content
        # (seen occasionally with short prompts).  Extract the last sentence
        # after the final </think> if present, otherwise use the full trace.
        if not answer_text and think_text:
            import re as _re
            # Check if thinking text ends with what looks like a final answer
            # after a separator line or the last paragraph
            parts = _re.split(r"\n{2,}", think_text)
            if len(parts) > 1:
                # Use last paragraph as answer, rest as trace
                answer_text = parts[-1].strip()
                think_text = "\n\n".join(parts[:-1]).strip()
            else:
                # Single block: use it as answer, trace is empty
                answer_text = think_text
                think_text = ""

        # ── Extract tool calls ────────────────────────────────────────────────
        # llama.cpp returns tool_calls as native JSON objects with "arguments"
        # as a JSON-encoded string.  Convert to the same dict structure the
        # HuggingFace backend produces so the rest of the code path is uniform.
        tool_calls: list[dict] = []
        for tc in (msg.get("tool_calls") or []):
            fn = tc.get("function", {})
            args = fn.get("arguments", {})
            if isinstance(args, str):
                try:
                    args = json.loads(args)
                except json.JSONDecodeError:
                    args = {}
            tool_calls.append({"function": {"name": fn.get("name", ""), "arguments": args}})

        # Build a raw string that mirrors the HuggingFace format so the debug
        # expander in the UI shows something meaningful.
        if think_text and answer_text:
            raw = f"<think>{think_text}</think>\n{answer_text}"
        elif think_text:
            raw = f"<think>{think_text}</think>"
        else:
            raw = answer_text

        return GenerateResult(
            think_text=think_text,
            answer_text=answer_text,
            tool_calls=tool_calls,
            raw_output=raw,
        )

    def health_check(self) -> dict:
        """
        Check whether the llama.cpp server is reachable and has a model loaded.

        Returns a dict with:
          ok      – bool: True only when a model is ready to serve requests
          status  – "ready" | "no_model" | "unreachable"
          detail  – human-readable string for the UI
          models  – list of model ids currently registered (may be empty)
        """
        try:
            resp = self._client.get("/models", timeout=5.0)
            resp.raise_for_status()
            models = [m["id"] for m in resp.json().get("data", [])]
        except Exception as exc:
            return {
                "ok": False,
                "status": "unreachable",
                "detail": f"Cannot reach llama.cpp server at {LLAMACPP_BASE_URL}: {exc}",
                "models": [],
            }

        if not models:
            # Reset cached model name so next generate() re-checks
            self._resolved_model = None
            return {
                "ok": False,
                "status": "no_model",
                "detail": (
                    "llama.cpp server is running but no model is loaded. "
                    "Open Llama.app and start a Granite 4.2 model, "
                    "or restart llama-server with --model <path>."
                ),
                "models": [],
            }

        return {
            "ok": True,
            "status": "ready",
            "detail": f"llama.cpp server ready. Loaded model(s): {', '.join(models)}",
            "models": models,
        }


# ─────────────────────────────────────────────────────────────────────────────
# Factory – returns the configured backend singleton
# ─────────────────────────────────────────────────────────────────────────────

_hf_backend: HFBackend | None = None
_llama_backend: LlamaCppBackend | None = None


def get_backend(override: str | None = None) -> HFBackend | LlamaCppBackend:
    """
    Return the active backend singleton.
    Pass override="huggingface" or override="llamacpp" to switch at runtime.
    """
    global _hf_backend, _llama_backend

    choice = (override or BACKEND).lower()

    if choice == "llamacpp":
        if _llama_backend is None:
            _llama_backend = LlamaCppBackend()
        return _llama_backend

    # Default: huggingface
    if _hf_backend is None:
        _hf_backend = HFBackend()
    return _hf_backend

Dual Backend Deployment: llama.cpp vs. Hugging Face

A key feature of the application is seamless runtime backend switching between local PyTorch inference via Hugging Face and high-speed GGUF inference via llama.cpp.

# Backend selector: "huggingface" | "llamacpp"
BACKEND=huggingface

# HuggingFace Configuration
HF_MODEL_ID=ibm-granite/granite-4.2-3b

# llama.cpp Configuration (OpenAI Compatible Endpoint)
LLAMACPP_BASE_URL=http://localhost:9932/v1
LLAMACPP_MODEL_NAME=granite-4.2-8b

When using HFBackend, the model weights are loaded lazily into memory using PyTorch (utilizing bfloat16 on CUDA GPUs or automatic CPU fallback to float32). When using LlamaCppBackend, the adapter interacts via HTTP REST with a llama serve daemon (e.g., on port 9932), extracting native reasoning_content fields enabled by --jinja formatting.

Fine-Tuning Inference & Thinking Parameters

Granite 4.2 exposes controllable generation budgets, accessible both statically via .env parameters and dynamically through UI sliders in Streamlit.

TEMPERATURE=1.0
TOP_P=0.95
MAX_NEW_TOKENS_THINKING=8192
MAX_NEW_TOKENS_NON_THINKING=2048
MAX_NEW_TOKENS_LOW_EFFORT=4096

| Mode / Parameter | Default Value     | Thinking Flag           | Description & Use Case                                       |
| ---------------- | ----------------- | ----------------------- | ------------------------------------------------------------ |
| `TEMPERATURE`    | `1.0`             | N/A                     | Model-card recommended sampling temperature.  PDF+ 1         |
| `TOP_P`          | `0.95`            | N/A                     | Nucleus sampling threshold for token selection.  PDF+ 1      |
| `Full Thinking`  | `8192 max tokens` | `enable_thinking=True`  | Complete multi-step reasoning trace before generating answer.  PDF+ 1 |
| `Low-Effort`     | `4096 max tokens` | `low_effort=True`       | Budget-constrained reasoning trace for rapid outputs.  PDF+ 1 |
| `Non-Thinking`   | `2048 max tokens` | `enable_thinking=False` | Direct response generation, bypassing CoT generation completely.   |

Bonus: Plug-and-Play Tools Catalog Implementation

Granite 4.2 excels at structured reasoning and automated function calling. To leverage this without hardcoding tool logic, Bob implemented a zero-friction Plug-and-Play Registry Pattern.

tools/
├── registry.py       ← @tool decorator, autodiscover(), get_schemas(), execute()
├── definitions.py    ← Backward-compat shim; re-exports helpers from registry
└── catalog/
    ├── __init__.py   ← Package marker (empty – no edits needed)
    ├── calculator.py ← Built-in arithmetic tool
    ├── get_weather.py← Built-in weather stub tool
    └── geocode.py    ← Built-in GPS geocoding tool

Registry Core (`tools/registry.py`)

tools/__init__.py
  └─ calls autodiscover("tools.catalog")
       └─ imports every *.py module in tools/catalog/
            └─ each @tool(schema) decorator runs
                 └─ registers {name → (schema, fn)} in _REGISTRY

Tools register themselves automatically upon module import via a single @tool(schema) decorator. The registry maintains internal mapping and dispatches calls dynamically:

# tools/registry.py excerpt
_REGISTRY = {}

def tool(schema: dict) -> Callable:
    fn_meta = schema.get("function", {})
    name = fn_meta.get("name")

    def decorator(fn: Callable) -> Callable:
        _REGISTRY[name] = {"schema": schema, "fn": fn}
        return fn
    return decorator

def execute(name: str, arguments: dict[str, Any]) -> dict[str, Any]:
    entry = _REGISTRY.get(name)
    if not entry:
        return {"error": f"Unknown tool: {name!r}"}
    try:
        return entry["fn"](**arguments)
    except Exception as exc:
        return {"error": f"Tool execution failed: {exc}"}

Autodiscovery Boot Sequence

At backend startup, autodiscover("tools.catalog") scans the tools/catalog/ directory using pkgutil.walk_packages, importing every *.py module. Adding a new tool requires zero edits to core codebase files like app.py or registry.py.

"""  
tools/registry.py  
──────────────────  
Plug-and-play tool catalog registry for Granite 4.2.  

This module provides the mechanism that turns tool authoring into a  
"drop a file in tools/catalog/" operation:  

  1. @tool(schema)  – decorates a Python callable, registering it with its  
                      OpenAI function-definition schema.  
  2. autodiscover() – imports every module under tools/catalog/, triggering  
                      all @tool decorations.  
  3. get_schemas()  – returns the live list of registered tool schemas to  
                      pass to model.generate(tools=…).  
  4. execute()      – dispatches a tool call by name; no if/elif chain needed.  
  5. list_tools()   – returns tool names and descriptions for introspection.  

Usage (tool author):  
────────────────────  
    # tools/catalog/my_tool.py  
    from tools.registry import tool  

    @tool({  
        "type": "function",  
        "function": {  
            "name": "my_tool",  
            "description": "Does something useful.",  
            "parameters": {  
                "type": "object",  
                "properties": {"arg": {"type": "string", "description": "…"}},  
                "required": ["arg"],  
            },  
        },  
    })  
    def my_tool(arg: str) -> dict:  
        return {"result": arg.upper()}  

Usage (caller):  
───────────────  
    from tools.registry import autodiscover, get_schemas, execute  

    autodiscover()                       # once at startup  
    schemas = get_schemas()              # pass to model  
    result  = execute("my_tool", {"arg": "hello"})  
"""  

from __future__ import annotations  

import importlib  
import pkgutil  
import logging  
from typing import Any, Callable  

logger = logging.getLogger(__name__)  

# Internal registry: tool name → {schema, fn}  
_REGISTRY: dict[str, dict[str, Any]] = {}  


# ─────────────────────────────────────────────────────────────────────────────  
# Public decorator  
# ─────────────────────────────────────────────────────────────────────────────  

def tool(schema: dict) -> Callable:  
    """  
    Decorator that registers a function as a callable tool.  

    The schema must follow the OpenAI function definition format:  
        {"type": "function", "function": {"name": …, "description": …, "parameters": …}}  

    Raises ValueError if the schema is missing required fields or if the name  
    is already registered (prevents silent overwrites during development).  
    """  
    fn_meta = schema.get("function", {})  
    name = fn_meta.get("name")  
    if not name:  
        raise ValueError("Tool schema must include function.name.")  
    if not fn_meta.get("description"):  
        raise ValueError(f"Tool {name!r} schema must include function.description.")  

    def decorator(fn: Callable) -> Callable:  
        if name in _REGISTRY:  
            logger.warning("Tool %r already registered; overwriting.", name)  
        _REGISTRY[name] = {"schema": schema, "fn": fn}  
        logger.debug("Registered tool: %s", name)  
        return fn  

    return decorator  


# ─────────────────────────────────────────────────────────────────────────────  
# Public API  
# ─────────────────────────────────────────────────────────────────────────────  

def get_schemas() -> list[dict]:  
    """Return the list of registered tool schemas (OpenAI function format)."""  
    return [entry["schema"] for entry in _REGISTRY.values()]  


def execute(name: str, arguments: dict[str, Any]) -> dict[str, Any]:  
    """  
    Execute the named tool with the provided arguments.  

    Returns a JSON-serialisable result dict.  
    Returns {"error": …} if the tool is unknown or raises an exception.  
    """  
    entry = _REGISTRY.get(name)  
    if not entry:  
        return {"error": f"Unknown tool: {name!r}"}  
    try:  
        return entry["fn"](**arguments)  
    except TypeError as exc:  
        return {"error": f"Invalid arguments for tool {name!r}: {exc}"}  
    except Exception as exc:  
        return {"error": f"Tool {name!r} raised an error: {exc}"}  


def list_tools() -> list[dict[str, str]]:  
    """Return a lightweight catalog: [{name, description}, …] for all registered tools."""  
    return [  
        {  
            "name": name,  
            "description": entry["schema"]["function"].get("description", ""),  
        }  
        for name, entry in _REGISTRY.items()  
    ]  


def autodiscover(package: str = "tools.catalog") -> None:  
    """  
    Import every module under *package* to trigger @tool registrations.  

    Call once at application startup (e.g. in backend/app.py startup event).  
    Safe to call multiple times — repeated imports are no-ops in Python.  
    """  
    try:  
        pkg = importlib.import_module(package)  
    except ModuleNotFoundError:  
        logger.warning("Tool catalog package %r not found; skipping autodiscover.", package)  
        return  

    for finder, modname, _ispkg in pkgutil.walk_packages(  
        path=pkg.__path__,  
        prefix=package + ".",  
    ):  
        try:  
            importlib.import_module(modname)  
            logger.debug("autodiscover: loaded %s", modname)  
        except Exception as exc:  
            logger.warning("autodiscover: failed to load %s: %s", modname, exc)

Adding a Custom Tool (`tools/catalog/my_tool.py`)

As an example we can add any type of tools if the implementation is correct.

from tools.registry import tool

@tool({
    "type": "function",
    "function": {
        "name": "calculate_tax",
        "description": "Calculates tax based on amount and rate.",
        "parameters": {
            "type": "object",
            "properties": {
                "amount": {"type": "number", "description": "Base amount"},
                "rate": {"type": "number", "description": "Tax rate decimal"}
            },
            "required": ["amount", "rate"]
        }
    }
})
def calculate_tax(amount: float, rate: float) -> dict:
    return {"total": amount * (1 + rate), "tax": amount * rate}

Conclusion

IBM Granite 4.2 proves to be an exceptionally competent foundation model for agentic applications. Its compliance with OpenAI tool calling formats, paired with native, fine-grained control over reasoning depth (enable_thinking and low_effort flags), makes it one of the most practical and dependable open-source models available for enterprise production stacks.

The demo application built by Bob provides a highly robust blueprint for modern LLM application engineering. By decoupling model orchestration from tool execution via dynamic decorators and providing seamless switching between local GGUF runtimes (llama.cpp) and PyTorch backends (Hugging Face), the framework ensures both flexibility and operational stability.

Thanks for reading 🪨

Deep Dive: Testing Radar UI for Kubernetes using MCP, a Go GUI, and an Autonomous Agent

Alain Airom (Ayrom) — Tue, 25 Aug 2026 07:31:59 +0000

Hand-on test of radarhq.io K8S UI and dashboard on a macOS with Minikube and Podman

Introduction

Kubernetes dashboards are often either overloaded with unnecessary complexity or too minimalist to provide deep operational context during an outage. Radar (radarhq.io) takes a refreshingly modern approach. It not only delivers a clean visual cluster dashboard but also natively integrates a Model Context Protocol (MCP) server.

By exposing cluster state — deployments, pods, topology, live operational issues, and logs — via MCP, Radar allows external clients, CLI tools, and AI agents to programmatically query and analyze Kubernetes workloads without direct, high-privilege access to the raw Kubernetes API.

In this blog post, I explore an end-to-end testing environment for Radar. I tested the overall system architecture, set up a sample Go workload (hello-k8s), inspect a Fyne-based GUI desktop client, and run an autonomous 7-step ReAct agent that performs cluster diagnostics over MCP.

I used IBM Bob SDLC for the implementation.

🏗️ System Architecture & Data Flow

To test Radar locally on macOS without Docker Desktop, I leveraged Minikube powered by the Podman driver and CRI-O runtime.

Because Minikube’s Podman driver places the VM inside an AppleHV VM whose internal IP (192.168.49.x) is not directly routed to the host machine, I implemented background kubectl port-forward tunnels to bridge the local host to the internal cluster services:

localhost:30928 → Radar Pod (:9280)
localhost:30800 → hello-k8s Pod (:8080)

Deploying the Sample Workload (`hello-k8s`)

To give Radar and our MCP clients a realistic workload to inspect, a a lightweight HTTP application written in Go is deploye. The application tracks request counts, renders pod metadata obtained via the Kubernetes Downward API, and exposes health probes at /healthz.

Workload Manifest (`k8s/hello-k8s.yaml`)

A 2-replica deployment is set-up, so that Radar can construct a multi-pod service topology graph:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-k8s
  namespace: hello-k8s
  labels:
    app: hello-k8s
spec:
  replicas: 2
  selector:
    matchLabels:
      app: hello-k8s
  template:
    metadata:
      labels:
        app: hello-k8s
    spec:
      containers:
        - name: hello-k8s
          image: hello-k8s:latest
          imagePullPolicy: Never
          ports:
            - name: http
              containerPort: 8080
          env:
            - name: PORT
              value: "8080"
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 3
            periodSeconds: 5
          resources:
            requests:
              cpu: 10m
              memory: 16Mi
            limits:
              cpu: 100m
              memory: 64Mi
---
apiVersion: v1
kind: Service
metadata:
  name: hello-k8s
  namespace: hello-k8s
spec:
  type: NodePort
  selector:
    app: hello-k8s
  ports:
    - name: http
      port: 8080
      targetPort: 8080
      nodePort: 30800

Desktop Monitoring: Building a Fyne MCP GUI Client

Rather than accessing raw Kubernetes endpoints directly, the custom GUI client communicates purely over HTTP JSON-RPC using the Model Context Protocol (MCP) endpoint provided by Radar at /mcp.

The client uses the official @modelcontextprotocol/go-sdk to establish stateless HTTP transport sessions and execute tools like get_dashboard and issues.

MCP Transport Invocation (`gui-client/main.go`)

func (c *MCPClient) callTool(ctx context.Context, toolName string, args map[string]any) ([]byte, error) {
  client := mcp.NewClient(&mcp.Implementation{
    Name:    "radar-mcp-gui",
    Version: "1.0.0",
  }, nil)

  transport := &mcp.StreamableClientTransport{
    Endpoint: c.endpointURL,
  }

  session, err := client.Connect(ctx, transport, nil)
  if err != nil {
    return nil, fmt.Errorf("connect to %s: %w", c.endpointURL, err)
  }
  defer session.Close()

  result, err := session.CallTool(ctx, &mcp.CallToolParams{
    Name:      toolName,
    Arguments: args,
  })
  if err != nil {
    return nil, fmt.Errorf("call %q: %w", toolName, err)
  }

  var combined []byte
  for _, content := range result.Content {
    if tc, ok := content.(*mcp.TextContent); ok {
      combined = append(combined, []byte(tc.Text)...)
    }
  }
  return combined, nil
}

Decoupled Data Fetching Loop

To keep the UI responsive, a background goroutine fetches data snapshots on every poll interval (e.g., 10s) and passes updates over a buffered channel to the main thread:

// Background fetch loop
snapCh := make(chan ClusterSnapshot, 1)

go func() {
  ctx := context.Background()
  for {
    snap := fetchSnapshot(ctx, mcpClient)
    select {
    case snapCh <- snap:
    default: // Drop if UI thread is busy
    }
    time.Sleep(cfg.PollInterval)
  }
}()

Automated Diagnostics: The 7-Step ReAct Agent

To automate workload inspection, a Go CLI agent (radar-agent) that acts as an autonomous operator is implemented. It runs a deterministic 7-step ReAct (Reason + Act) loop using Radar's MCP tool set.

Agent Investigation Data-Flow

Implementation of the ReAct Loop (`agent/main.go`)

Here is an excerpt showing how the agent executes the observation phase and lists resources:

func run() error {
  flag.Parse()
  client := newMCPClient()
  ctx    := context.Background()

  // Step 1: OBSERVE cluster health
  step(1, "OBSERVE", "cluster overview via get_dashboard")
  dash, err := stepObserve(ctx, client, *flagNamespace)
  if err != nil {
    return fmt.Errorf("observe: %w", err)
  }

  // Step 2: FOCUS on targeted pods
  step(2, "FOCUS", fmt.Sprintf("list pods in namespace %q", *flagNamespace))
  pods, err := stepListPods(ctx, client, *flagNamespace)
  if err != nil {
    fmt.Printf("  Warning: %v\n", err)
  }

  // Step 3: INSPECT Deployment spec/status
  step(3, "INSPECT", fmt.Sprintf("get_resource deployment/%s", *flagWorkload))
  dep, _ := stepInspectDeployment(ctx, client, *flagNamespace, *flagWorkload)

  // Step 4: TOPOLOGY graph analysis
  step(4, "TOPOLOGY", fmt.Sprintf("get_topology namespace=%q", *flagNamespace))
  topo, _ := stepTopology(ctx, client, *flagNamespace)

  // Step 5: Live operational ISSUES
  step(5, "ISSUES", fmt.Sprintf("issues namespace=%q", *flagNamespace))
  issues, _ := stepIssues(ctx, client, *flagNamespace)

  // Step 6: Sample LOGS
  step(6, "LOGS", fmt.Sprintf("get_workload_logs deployment/%s", *flagWorkload))
  logs, _ := stepLogs(ctx, client, *flagNamespace, *flagWorkload)

  // Step 7: Print Final Summary
  step(7, "REPORT", "structured summary")
  // ... renders formatted report to stdout ...
  return nil
}

🛠️ Running the Test Environment

To test the full stack on your local machine, follow these steps:

Provision Cluster & Deploy Stack

Run the setup script to initialize Minikube, deploy Radar via Helm, build hello-k8s, apply RBAC patches, and start port-forwarding:

chmod +x scripts/*.sh
./scripts/deploy.sh

Run the Autonomous Agent

Execute the Go CLI agent to inspect the hello-k8s namespace:

./scripts/run-agent.sh --namespace hello-k8s --workload hello-k8s

Sample agent output:

════════════════════════════════════════════════════════════
  RADAR AGENT REPORT
  Goal      : Investigate the hello-k8s workload and summarise its health
  Namespace : hello-k8s
  Workload  : hello-k8s
════════════════════════════════════════════════════════════

  CLUSTER OVERVIEW
  Health:                healthy  (cluster: minikube v1.28.0)
  Pods:                  healthy=4  warning=0  error=0
  Nodes:                 total=1  ready=1  notReady=0
  Total problems:        0

  PODS IN NAMESPACE "hello-k8s"
  hello-k8s-74b884988f-2k9ll                    Running     ready=1/1  node=minikube
  hello-k8s-74b884988f-b98x7                    Running     ready=1/1  node=minikube

  DEPLOYMENT: hello-k8s
  Replicas:              2 desired / 2 ready / 2 available
  Container:             hello-k8s → hello-k8s:latest

  TOPOLOGY (namespace "hello-k8s")
  3 nodes, 2 edges

  LIVE ISSUES
  No issues found — workload looks healthy.
════════════════════════════════════════════════════════════

Launch the Fyne Desktop GUI

Start the desktop monitoring application:

./scripts/start.sh

Key Takeaways & Design Decisions

| Technical Decision                   | Rationale                                                    |
| ------------------------------------ | ------------------------------------------------------------ |
| **Podman Driver + CRI-O**            | Enables containerized Kubernetes testing on macOS without relying on Docker Desktop.  MD |
| **`kubectl port-forward` Tunneling** | Solves AppleHV VM network isolation by routing `localhost:30928` directly to Radar's container port.  MD |
| **Chart Tag Pinning (`1.7.0`)**      | Prevents version mismatch crashes with Helm flags present in newer chart versions.  MD |
| **RBAC Supplemental Patch**          | Adds missing `rbac.authorization.k8s.io` read permissions required for Radar's permission inspection panels to function properly.  MD |
| **Model Context Protocol (MCP)**     | Decouples direct Kubernetes API access from monitoring tools, allowing lightweight agents and GUIs to consume structured cluster context safely.  MD+ 1 |

Conclusion

By exposing cluster telemetry and operational controls through the Model Context Protocol, Radar transforms Kubernetes monitoring from a manual dashboard-checking task into a programmatic foundation for automation. Whether powering custom desktop GUIs like Fyne or enabling autonomous ReAct agents to run multi-step diagnostic workflows, MCP bridges the gap between raw cluster metrics and intelligent operational tools. As container environments continue to grow in complexity, decoupling cluster context from high-privilege API access via MCP offers a cleaner, safer, and far more extensible approach to Kubernetes management.

Thanks for reading 🍇

Embedding Docling-NLP in Ad-Hoc UI Applications: A Lightweight Blueprint

Alain Airom (Ayrom) — Mon, 24 Aug 2026 15:33:09 +0000

Implementing a “Graph Language Model” with Docling-NLP

Image from Docling-AI

Building a local, privacy-focused Document AI pipeline often feels like balancing heavy models with sluggish performance. IBM’s open-source ecosystem solves this by splitting document parsing and entity extraction into two specialized tools: docling (Python-based document conversion) and docling-nlp (a C++ compiled core with Python bindings via pybind11).

This guide outlines a complete, practical framework for embedding docling-nlp into an ad-hoc Streamlit interface, based on the reference architecture designed by IBM Bob.

What is Docling NLP: Graph Language Model

Excerpt from Github;

Finding entities and relations via NLP on text and documents and creating Graphs from NLP entities and relations in document collections
To get easily started, simply install the docling-nlp package from PyPi. This can be done using the traditional pip install docling-nlp or via uv uv add docling-nlp.

High-Level Architecture Overview of the sample implementation

The system divides responsibilities across distinct runtime layers:

Streamlit GUI Layer (app.py): Provides interactive controls for raw text, file uploads, URLs, and real-time visualization.
Python Processing Layer (src/docling_processor.py): Interfaces with docling to extract reading-order Markdown, DataFrames, and plain text chunks.
C++ NLP Layer (src/nlp_processor.py): Uses pybind11 bindings to run native C++ entity and relation extraction at ~1.2M tokens/sec.

Dynamic Workflow & Sequence Dataflow

When a document or raw string is provided, execution moves synchronously across the document parsing pipeline into the pybind11-wrapped C++ core:

Core Implementation

Pre-processing Markdown Input

docling-nlp achieves high performance when operating on clean text spans. Passing raw Markdown syntax (headings, code blocks, links) introduces noise into tokenization boundaries. The utility layer strips markup before passing text down to C++:

def clean_markdown_for_nlp(text: str) -> str:
    """Strip Markdown syntax so C++ NLP receives clean prose."""
    if not text:
        return text
    cleaned = text
    for pattern, replacement in _MD_CLEAN_PATTERNS:
        cleaned = pattern.sub(replacement, cleaned)
    return re.sub(r"\n{3,}", "\n\n", cleaned).strip()

Interfacing with the C++ NLP Core

The NLPProcessor wraps initializations, runs inference on text chunks, and parses zero-indexed array representations back into rich dataclass objects:

# src/nlp_processor.py
from docling_nlp.utils.load_pretrained_models import load_pretrained_nlp_models
from docling_nlp.nlp_utils import init_nlp_model

class NLPProcessor:
    def __init__(self):
        self._model = None

    def _get_model(self):
        if self._model is None:
            # Load pre-compiled weights and init pybind11 bindings
            load_pretrained_nlp_models(force=False, verbose=False)
            self._model = init_nlp_model()
        return self._model

    def process_text(self, text: str) -> NLPResult:
        result = NLPResult(source_text=text)
        nlp_input = clean_markdown_for_nlp(text) if is_markdown_text(text) else text

        # Invoke native C++ inference engine
        raw = self._get_model().apply_on_text(nlp_input)
        self._parse_raw_result(raw, result)
        return result

Streamlit Caching & Singletons

Re-loading ONNX parsing models or C++ NLP weights on every UI interaction slows response times. Wrapping processor instantiations with @st.cache_resource ensures singletons persist across application re-runs:

# app.py
@st.cache_resource(show_spinner="Loading Docling processor…")
def get_docling_processor() -> DoclingProcessor:
    return DoclingProcessor()

@st.cache_resource(show_spinner="Loading C++ NLP model via pybind11…")
def get_nlp_processor() -> NLPProcessor:
    return NLPProcessor()

Conclusion: What This Application Demonstrates

Beyond the underlying code, this ad-hoc application serves as a concrete proof-of-concept for lightweight local document intelligence:

Local-First Privacy & Zero Cloud Dependency: Documents, tables, and raw text are parsed entirely on the host machine without sending sensitive data to external API endpoints.
High-Speed Hybrid Execution: It shows how Python and native C++ can be bridged via pybind11 to deliver near-instantaneous token processing (~1.2M tokens/sec) directly inside a web interface.
Unified Document Extraction: Complex file formats (PDFs, DOCX, HTML, Images) are seamlessly transformed into reading-order Markdown, structured DataFrames, and entity-rich JSON outputs through a single workflow.
Zero-Friction Interface: By wrapping sophisticated machine learning pipelines into an intuitive Streamlit UI, non-technical users can interact with complex NLP pipelines without running shell scripts or managing code.

Thanks for reading 🎩

Building an OpenTelemetry Instrumentation Wizard

Alain Airom (Ayrom) — Wed, 19 Aug 2026 09:59:45 +0000

Accelerating observability adoption by automating OpenTelemetry instrumentation across heterogeneous codebases

Introduction

Years ago, I was tasked with building OpenTelemetry metric collection for a client who wanted IBM Instana's capabilities without deploying the Instana agent to their production servers. Achieving that required meticulous, time-consuming effort to instrument their Go and Python applications manually. Recently, I faced almost the exact same challenge - except this time, I had Bob as an AI collaborator. Leveraging Bob, I set out to build an industrialized, universal instrumentation solution to solve the problem at scale.

Adding telemetry to existing applications is often tedious, manual, and error-prone. To solve this, we designed the OpenTelemetry (OTel) Instrumentation Wizard - a tool built with Streamlit and Python that automatically injects production-grade OTel traces, metrics, and logs across multiple programming languages while strictly preserving existing business logic. What follows is the implementation and the components.

Core Architecture & Workflow

The wizard operates as a structured multi-stage pipeline. Source files pass from initial language detection through AST/Regex analysis, safety validation, user review, and code injection before returning fully instrumented outputs.

Language Detection: Uses file extensions, shebangs, and syntax heuristics to automatically route source code to language-specific injection engines.
Safety & Idempotency: Scans for existing OpenTelemetry import patterns prior to injection to prevent duplicate initializations or broken dependencies.
Traceable Injection: Injected blocks are commented with [OTel-Wizard: injected] markers, guaranteeing zero disruption to original runtime logic.


otel-wizard/
├── app.py                     Streamlit UI (3-step wizard)
├── injectors/
│   ├── base_injector.py       Abstract base + InjectionResult dataclass
│   ├── python_injector.py     AST-aware Python injector
│   ├── go_injector.py         Go import + initTracer() injector
│   ├── java_injector.py       Java try-with-resources span injector
│   ├── javascript_injector.py Node.js SDK setup + span injector
│   ├── typescript_injector.py ES import syntax + typed Tracer injector
│   └── c_injector.py          C header + otel_start/end_span injector
├── utils/
│   ├── language_detector.py   Extension + heuristic language detection
│   └── diff_viewer.py         Unified text diff + colour-coded HTML diff
├── samples/                   Six hello-world files (pre-injection)
├── tests/                     pytest suite (8 test files, 128 assertions)
├── requirements.txt
├── .env.example
└── run_tests.sh

Language Support & SDK Status Matrix

The wizard targets official OpenTelemetry semantic conventions across 7 key languages, applying tailored injection strategies according to each language's SDK maturity:

| Language                | Traces   | Metrics  | Logs     | Injection Strategy                                  |
| ----------------------- | -------- | -------- | -------- | --------------------------------------------------- |
| Python (Generic)        | ✅ Stable | ✅ Stable | ✅ Dev    | AST-based (`ast` module)                            |
| Python - Flask          | ✅ Stable | ✅ Stable | ✅ Dev    | AST + `FlaskInstrumentor` middleware                |
| Python - FastAPI        | ✅ Stable | ✅ Stable | ✅ Dev    | AST + `FastAPIInstrumentor` + ASGI hook             |
| Python - Django         | ✅ Stable | ✅ Stable | ✅ Dev    | AST + `DjangoInstrumentor` + `Psycopg2Instrumentor` |
| JavaScript / TypeScript | ✅ Stable | ✅ Stable | ⚠️ Dev    | Regex + template                                    |
| Go                      | ✅ Stable | ✅ Stable | ⚠️ Beta   | Regex + template                                    |
| Java                    | ✅ Stable | ✅ Stable | ✅ Stable | Template overlay                                    |
| C# / .NET               | ✅ Stable | ✅ Stable | ✅ Stable | ActivitySource template                             |
| Ruby                    | ✅ Stable | ⚠️ Dev    | ⚠️ Dev    | Regex + template                                    |
| PHP                     | ✅ Stable | ✅ Stable | ✅ Stable | Regex + template                                    |

> Signal status reflects the official [OpenTelemetry Language Status](https://opentelemetry.io/docs/languages/) as of 2025.

Python Supported Frameworks

Special Python based frameworks are also implemented in the wizard;

| Framework   | What Gets Injected                                           |
| ----------- | ------------------------------------------------------------ |
| **Generic** | SDK imports, `TracerProvider` init, span context managers around all functions |
| **Flask**   | All of the above + `FlaskInstrumentor().instrument_app(app, excluded_urls=...)` + `RequestsInstrumentor().instrument()` |
| **FastAPI** | All of the above + `FastAPIInstrumentor.instrument_app(app, excluded_urls=..., exclude_spans=["receive"])` + `HTTPXClientInstrumentor().instrument()` |
| **Django**  | All of the above + `DjangoInstrumentor().instrument(request_hook=..., is_sql_commentor_enabled=True)` + `Psycopg2Instrumentor().instrument()` - inserted **before** `get_wsgi_application()` / `get_asgi_application()` |

Framework-specific requirements.txt entries are also generated automatically.

Key Configuration Options

Users retain full control over how telemetry is applied across their services through the wizard's configuration engine:

Service Metadata: Defines service.name and service.version resource attributes dynamically.
Exporters: Supports otlp-http, otlp-grpc, console, and prometheus endpoints.

Instrumentation Depth

shallow: Entry points and top-level HTTP handlers only.
medium: All public and exported functions.
deep: Internal private functions, detailed metric instruments, and log bridges.

The Wizard 🧙🏿‍♂️

The architecture behind the OpenTelemetry Instrumentation Wizard follows a clean four-phase pipeline: Detection → Analysis → Injection → Reporting.

Core Data Models (wizard/core/models.py): At the center of the system are five lightweight data structures that pass configuration and metadata between steps:

@dataclass
class WizardConfig:
    service_name: str = "my-service"
    service_version: str = "1.0.0"
    exporter: str = "otlp-http"
    otlp_endpoint: str = "http://localhost:4318"
    signals: list = field(default_factory=lambda: ["traces", "metrics", "logs"])
    depth: str = "medium"  # shallow | medium | deep
    sampling_ratio: float = 1.0

@dataclass
class InjectionPoint:
    signal: str          # "trace" | "metric" | "log"
    kind: str            # "span" | "counter" | "provider_init" ...
    location: str        # "function:<name>" | "file_top"
    reason: str
    code_preview: str
    enabled: bool = True

Language Detection & Extension Resolution (wizard/core/detector.py): The detector combines file extension mappings, shebang header inspection, and multiline regex heuristics to evaluate source snippets:

def detect_language(filename: str, source: str = "") -> dict:
    _, ext = os.path.splitext(filename.lower())
    if ext in EXTENSION_MAP:
        lang = EXTENSION_MAP[ext]
        return {"language": "javascript" if lang == "typescript" else lang, "confidence": "high"}

    if source:
        first_line = source.splitlines()[0] if source.splitlines() else ""
        if first_line.startswith("#!"):
            for pattern, lang in SHEBANG_PATTERNS:
                if re.search(pattern, first_line, re.IGNORECASE):
                    return {"language": lang, "confidence": "high"}

- AST & Pattern Analysis (wizard/core/analyzer.py)
: The analyzer creates an InstrumentationPlan without altering the input code. It checks idempotency, determines required OTel packages, and matches structural function points based on the target depth.

def generate_plan(source: str, language: str, filename: str, config: WizardConfig) -> InstrumentationPlan:
    plan = InstrumentationPlan(language=language, config=config)

    check = check_existing_instrumentation(source, language)
    if check["already_instrumented"]:
        plan.already_instrumented = True
        return plan

    plan.dependencies = _get_dependencies(language, config.signals)
    functions = _detect_functions(source, language, config.depth)

    for line_no, func_name, kind in functions:
        if "traces" in config.signals:
            plan.injection_points.append(InjectionPoint(
                signal="trace",
                kind="span",
                location=f"function:{func_name}",
                reason=f"Wrap '{func_name}' with a trace span for distributed tracing",
                code_preview=f"span '{func_name}' (line {line_no})"
            ))
    return plan

Dispatch Injection Strategy (wizard/core/injector.py): The injection orchestrator delegates the transformation to language-specific strategy implementations:

def inject(source: str, plan: InstrumentationPlan, filename: str = "") -> InjectionResult:
    language = plan.language

    if language == "python":
        from wizard.languages.python_injector import PythonInjector
        injector = PythonInjector(plan)
    elif language == "javascript":
        from wizard.languages.js_injector import JSInjector
        injector = JSInjector(plan)
    # ... remaining target language dispatchers

    return injector.inject(source)

Streamlit Application Loop (`wizard/app.py`)

The main entry point ties the interactive user workflow together across four clear user steps:

Upload / Paste: Captures source code and triggers detect_language().
Plan Review: Executes generate_plan() to output an itemized data frame of target span hooks and missing dependencies.
Inject: Calls inject() upon user confirmation.
Download: Calls generate_report() and packs the modified source, package metadata (requirements.txt, package.json, etc.), and markdown report into an in-memory ZIP archive.

The core wizard module includes these language-specific injector implementations:

Core Dispatcher Integration: The central dispatcher (wizard/core/injector.py) maps source languages to these concrete injector classes:

| Language                        | Injector Class       | Key Strategy & Tracing Mechanism                             |
| ------------------------------- | -------------------- | ------------------------------------------------------------ |
| **Python**  PY                  | `PythonInjector`  PY | Uses standard `ast` parsing to inject OTel imports, provider initializers, and wrap function bodies using `with _otel_tracer.start_as_current_span(...)` context managers.  PY |
| **JavaScript / TypeScript**  PY | `JSInjector`  PY     | Prepends tracer setup headers, generates an accompanying `instrumentation.js` file, and wraps functions with `_otelTracer.startActiveSpan(...)`.  PY |
| **Go**  PY                      | `GoInjector`  PY     | Injects `go.opentelemetry.io/otel` imports, adds an `initOtelProvider` initialization function, and inserts `ctx, _otelSpan := _otelTracer.Start(...)` with `defer _otelSpan.End()`.  PY |
| **Java**  PY                    | `JavaInjector`  PY   | Appends static `OpenTelemetry` class initializers and wraps public/protected method bodies using `try (Scope _ = ...)` and `finally { _otelSpan.end(); }` blocks.  PY |
| **C# / .NET**  PY               | `DotNetInjector`  PY | Injects native `System.Diagnostics.ActivitySource` fields and wraps methods using `using var _otelActivity = _otelSource.StartActivity(...)` with `try/catch` exception recording.  PY |
| **C**  PY                       | `CInjector`  PY      | Scans line-by-line using regex for C function signatures, injects `otel_init()` inside `main()`, and inserts macro/pointer span tracking blocks at function entry points.  PY |
| **PHP**  PY                     | `PHPInjector`  PY    | Prepends a PHP OTel bootstrap block and wraps function bodies in `startSpan()`, `activate()`, and `try/catch/finally` blocks.  PY |
| **Ruby**  PY                    | `RubyInjector`  PY   | Prepends `require 'opentelemetry/sdk'`, configures the global SDK, and wraps method bodies inside `_OTEL_TRACER.in_span(...) do ... end` blocks.  PY |

Standard Features Across Injectors

Injection Watermarking: Every injected code block, import, or helper macro across all language implementations is tagged with the line comment identifier // [OTel-Wizard: injected] (or # [OTel-Wizard: injected]) for easy auditing and tracing.
Dependency Reporting: Each injector builds language-native dependency manifests or installation commands alongside the modified source code - such as requirements.txt, package.json, go.mod, pom.xml, CMakeLists.txt, Gemfile, or CLI package commands.

🧩 Role of Injectors in the OTel Instrumentation Wizard

In the OTel Instrumentation Wizard, injectors are specialized components responsible for parsing source code in a target programming language, inserting required OpenTelemetry SDK setup/imports, and wrapping key execution blocks (such as entry points or main functions) with span tracking logic.

Key Responsibilities

Base Abstraction & Lifecycle Control (BaseInjector): Establishes a common inject(source_code, filename) contract and returns an InjectionResult dataclass containing the modified source code, language metadata, and a human-readable list of changes. It provides shared helpers to enforce non-empty source checks (_require_non_empty), detect pre-existing OTel instrumentation (_already_has_otel), and scan for common entry points across languages (_find_main_function).
Tracer Provider Initialisation: Prepends or injects global tracer configuration, SDK setups, or SDK bootstrap calls suitable for the target runtime environment.
Span Wrapping & Lifecycle Management: Identifies entry functions (e.g., main(), if __name__ == "__main__":) and injects code to start spans at entry and guarantee proper span finalisation/teardown via native language constructs (such as try-with-resources, defer, try/finally, or with blocks).
Idempotency Safeguards: Checks for pre-existing OTel markers (e.g., opentelemetry, go.opentelemetry.io/otel) prior to modifying files, skipping injection if the source code is already instrumented.

Language-Specific Implementations & Code Excerpts

1. Python (`PythonInjector`)

Inserts OTel imports, sets up a TracerProvider and ConsoleSpanExporter, and wraps the body of def main(): using Python's with tracer.start_as_current_span(...) context manager while handling indentation.

Code Excerpt (injectors/python_injector.py):

_OTEL_IMPORTS = """\
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import (
    BatchSpanProcessor,
    ConsoleSpanExporter,
)
from opentelemetry.sdk.resources import Resource, SERVICE_NAME
"""

_OTEL_PROVIDER_INIT = """\
# [OTel] Initialise TracerProvider
_otel_resource = Resource(attributes={SERVICE_NAME: __name__})
_otel_provider = TracerProvider(resource=_otel_resource)
_otel_provider.add_span_processor(
    BatchSpanProcessor(ConsoleSpanExporter())
)
trace.set_tracer_provider(_otel_provider)
tracer = trace.get_tracer(__name__)
"""

2. Go (`GoInjector`)

Injects go.opentelemetry.io/otel packages into import (...) blocks, appends an initTracer() helper function, and modifies func main() to initialize the provider with defer tp.Shutdown(ctx) and defer span completion via defer span.End().

Code Excerpt (injectors/go_injector.py):

_MAIN_PREAMBLE = """\
    // [OTel] Initialise tracer
    ctx := context.Background()
    tp, err := initTracer()
    if err != nil {
        log.Fatalf("failed to initialise tracer: %v", err)
    }
    defer func() {
        if shutdownErr := tp.Shutdown(ctx); shutdownErr != nil {
            log.Printf("tracer shutdown error: %v", shutdownErr)
        }
    }()
    tracer := otel.Tracer("main")
    ctx, span := tracer.Start(ctx, "main") // [OTel] root span
    defer span.End()
    _ = ctx // use ctx in downstream calls
"""

3. Java (`JavaInjector`)

Inserts io.opentelemetry.api imports after package or existing import statements and wraps public static void main(String[] args) with a try-with-resources block using Scope scope = span.makeCurrent() and explicit error reporting in a catch block.

Code Excerpt (injectors/java_injector.py):

otel_preamble = (
    f"\n{body_indent}// [OTel] Initialise tracer\n"
    f"{body_indent}OpenTelemetry openTelemetry = GlobalOpenTelemetry.get();\n"
    f"{body_indent}Tracer tracer = openTelemetry.getTracer(\"main\");\n"
    f"{body_indent}Span span = tracer.spanBuilder(\"main\").startSpan();\n"
    f"{body_indent}try (Scope scope = span.makeCurrent()) {{\n"
)
otel_postamble = (
    f"{body_indent}}} catch (Exception e) {{\n"
    f"{body_indent}    span.setStatus(StatusCode.ERROR, e.getMessage());\n"
    f"{body_indent}    throw e;\n"
    f"{body_indent}}} finally {{\n"
    f"{body_indent}    span.end();\n"
    f"{body_indent}}}\n"
)

4. JavaScript & TypeScript (`JavaScriptInjector` / `TypeScriptInjector`)

Prepends a Node.js NodeSDK configuration block and tracer acquisition (_otelTracer.startSpan). TypeScript uses ES module imports and type declarations (const _otelTracer: Tracer), wrapping function bodies with standard try/catch/finally blocks.

Code Excerpt (injectors/typescript_injector.py):

_OTEL_SETUP_BLOCK = """\
// [OTel] ── OpenTelemetry setup ──────────────────────────────────────────────
import { NodeSDK } from '@opentelemetry/sdk-node';
import { ConsoleSpanExporter } from '@opentelemetry/sdk-trace-node';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';
import { Resource } from '@opentelemetry/resources';
import { SemanticResourceAttributes } from '@opentelemetry/semantic-conventions';
import { trace, Tracer, context, SpanStatusCode } from '@opentelemetry/api';

const _otelSdk: NodeSDK = new NodeSDK({
  resource: new Resource({
    [SemanticResourceAttributes.SERVICE_NAME]: 'otel-wizard-service',
  }),
  traceExporter: new ConsoleSpanExporter(),
  instrumentations: [getNodeAutoInstrumentations()],
});
_otelSdk.start();
process.on('SIGTERM', (): void => { _otelSdk.shutdown(); });

const _otelTracer: Tracer = trace.getTracer('main');
// [OTel] ─────────────────────────────────────────────────────────────────────
"""

5. C (`CInjector`)

Appends C OTel headers (#include "opentelemetry_c/otel_api.h"), inserts otel_init() and otel_start_span("main") at the beginning of int main(), injects otel_end_span() prior to any return statement inside main(), and appends otel_shutdown() before closing the function.

Code Excerpt (injectors/c_injector.py):

_OTEL_MAIN_INIT = """\
    /* [OTel] Initialise tracer */
    otel_init("otel-wizard-service");
    otel_span_t *_otel_span = otel_start_span("main");
"""

_OTEL_MAIN_CLOSE = """\
    /* [OTel] End root span and shutdown */
    otel_end_span(_otel_span);
    otel_shutdown();
"""

🛠️ Utility Modules Overview

The utils directory provides critical supporting functionality for the OTel Instrumentation Wizard, focusing on file analysis and difference visualization:

Language Detection (utils/language_detector.py): The LanguageDetector class identifies programming languages through explicit extension maps and keyword content heuristics. It offers two primary mechanisms: detect(), which strictly validates whether a file has an active injector available (raising UnsupportedLanguageError otherwise), and detect_any(), a non-raising alternative designed for fluid UI workflows that gracefully flags recognized but non-instrumentable languages.
Diff Generation & Visualisation (utils/diff_viewer.py): Leverages Python's native difflib module to compute line-by-line differences between original and instrumented source code. It provides generate_unified_diff() to output standard text diffs (diff -u) and generate_html_diff() to build sanitised, color-coded HTML views with styled line highlights (green for additions, red for deletions, and blue for hunk headers).

🧪 Testing Samples Overview

To validate the code injection process across all supported target environments, the project includes clean, minimal "Hello World" benchmark samples free of any initial OpenTelemetry instrumentation:

Language Coverage: Standardized samples are provided for C (helloworld.c), Go (helloworld.go), Java (HelloWorld.java), JavaScript (helloworld.js), Python (helloworld.py), and TypeScript (helloworld.ts).
Canonical Code Structure: Each file implements a simple main entry point and helper greeting function (such as greet() / main()). This standardized pattern allows unit and integration tests to reliably verify header/import additions, tracer setup placements, and entry-point span wrapping across every supported language.

A Simple Test and Implementation for Go applications

Step 1: Uploading or Copy/Pasting a Go code (or any other code)

  // tests/samples/go/sample_app.go
  // A plain Go HTTP server WITHOUT OTel instrumentation.
  // Used to validate the wizard's Go injection.

  package main

  import (
    "encoding/json"
    "fmt"
    "log"
    "math/rand"
    "net/http"
    "time"
  )

  type Item struct {
    Name     string  `json:"name"`
    Price    float64 `json:"price"`
    Quantity int     `json:"quantity"`
  }

  type OrderRequest struct {
    UserID int    `json:"user_id"`
    Items  []Item `json:"items"`
  }

  type OrderResponse struct {
    User   string  `json:"user"`
    Total  float64 `json:"total"`
    Status string  `json:"status"`
  }

  func calculateTotal(items []Item) float64 {
    total := 0.0
    for _, item := range items {
        qty := item.Quantity
        if qty == 0 {
            qty = 1
        }
        total += item.Price * float64(qty)
    }
    return total
  }

  func fetchUser(userID int) (map[string]interface{}, error) {
    time.Sleep(time.Duration(rand.Intn(50)) * time.Millisecond)
    if userID <= 0 {
        return nil, fmt.Errorf("invalid userID: %d", userID)
    }
    return map[string]interface{}{
        "id":   userID,
        "name": fmt.Sprintf("User_%d", userID),
    }, nil
  }

  func processOrder(userID int, items []Item) (*OrderResponse, error) {
    user, err := fetchUser(userID)
    if err != nil {
        return nil, err
    }
    total := calculateTotal(items)
    log.Printf("Order processed for %v: $%.2f", user["name"], total)
    return &OrderResponse{
        User:   user["name"].(string),
        Total:  total,
        Status: "confirmed",
    }, nil
  }

  func orderHandler(w http.ResponseWriter, r *http.Request) {
    if r.Method != http.MethodPost {
        http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
        return
    }
    var req OrderRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }
    resp, err := processOrder(req.UserID, req.Items)
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(resp)
  }

  func healthHandler(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    fmt.Fprintf(w, `{"status":"ok"}`)
  }

  func main() {
    http.HandleFunc("/order", orderHandler)
    http.HandleFunc("/health", healthHandler)
    log.Println("Server listening on :8080")
    log.Fatal(http.ListenAndServe(":8080", nil))
  }

Step 2: Validate the instrumentation plan

Step 3: Confirm the changes and Inject

Step 4: Download the modified Code

// tests/samples/go/sample_app.go
// A plain Go HTTP server WITHOUT OTel instrumentation.
// Used to validate the wizard's Go injection.

package main

import (
    "context"  // [OTel-Wizard: injected]
    "os"  // [OTel-Wizard: injected]
    "go.opentelemetry.io/otel"  // [OTel-Wizard: injected]
    "go.opentelemetry.io/otel/attribute"  // [OTel-Wizard: injected]
    "go.opentelemetry.io/otel/codes"  // [OTel-Wizard: injected]
    "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"  // [OTel-Wizard: injected]
    "go.opentelemetry.io/otel/sdk/resource"  // [OTel-Wizard: injected]
    sdktrace "go.opentelemetry.io/otel/sdk/trace"  // [OTel-Wizard: injected]
    semconv "go.opentelemetry.io/otel/semconv/v1.26.0"  // [OTel-Wizard: injected]
    "go.opentelemetry.io/otel/trace"  // [OTel-Wizard: injected]

// initOtelProvider initialises the TracerProvider. // [OTel-Wizard: injected]
func initOtelProvider(ctx context.Context) (func(context.Context) error, error) {
    exporter, err := otlptracehttp.New(ctx,
        otlptracehttp.WithEndpoint(getEnv("OTEL_EXPORTER_OTLP_ENDPOINT", "http://localhost:4318")),
        otlptracehttp.WithInsecure(),
    )
    if err != nil {
        return nil, err
    }
    res, err := resource.Merge(
        resource.Default(),
        resource.NewWithAttributes(
            semconv.SchemaURL,
            semconv.ServiceName(getEnv("OTEL_SERVICE_NAME", "my-service")),
            semconv.ServiceVersion(getEnv("OTEL_SERVICE_VERSION", "1.0.0")),
        ),
    )
    if err != nil {
        return nil, err
    }
    tp := sdktrace.NewTracerProvider(
        sdktrace.WithBatcher(exporter),
        sdktrace.WithResource(res),
    )
    otel.SetTracerProvider(tp)
    return tp.Shutdown, nil
}

// getEnv returns env var value or fallback. // [OTel-Wizard: injected]
func getEnv(key, fallback string) string {
    if v := os.Getenv(key); v != "" {
        return v
    }
    return fallback
}

var _otelTracer = otel.Tracer("my-service")  // [OTel-Wizard: injected]

    "encoding/json"
    "fmt"
    "log"
    "math/rand"
    "net/http"
    "time"
)

type Item struct {
    Name     string  `json:"name"`
    Price    float64 `json:"price"`
    Quantity int     `json:"quantity"`
}

type OrderRequest struct {
    UserID int    `json:"user_id"`
    Items  []Item `json:"items"`
}

type OrderResponse struct {
    User   string  `json:"user"`
    Total  float64 `json:"total"`
    Status string  `json:"status"`
}

func calculateTotal(items []Item) float64 {
    // [OTel-Wizard: injected]
    ctx, _otelSpan := _otelTracer.Start(ctx, "calculateTotal",
        trace.WithAttributes(attribute.String("code.function", "calculateTotal")))
    defer func() {
        if r := recover(); r != nil {
            _otelSpan.SetStatus(codes.Error, "panic")
            _otelSpan.End()
            panic(r)
        }
        _otelSpan.End()
    }()  // [OTel-Wizard: injected]
    _ = ctx  // [OTel-Wizard: injected]
    total := 0.0
    for _, item := range items {
        qty := item.Quantity
        if qty == 0 {
            qty = 1
        }
        total += item.Price * float64(qty)
    }
    return total
}

func fetchUser(userID int) (map[string]interface{}, error) {
    // [OTel-Wizard: injected]
    ctx, _otelSpan := _otelTracer.Start(ctx, "fetchUser",
        trace.WithAttributes(attribute.String("code.function", "fetchUser")))
    defer func() {
        if r := recover(); r != nil {
            _otelSpan.SetStatus(codes.Error, "panic")
            _otelSpan.End()
            panic(r)
        }
        _otelSpan.End()
    }()  // [OTel-Wizard: injected]
    _ = ctx  // [OTel-Wizard: injected]
    time.Sleep(time.Duration(rand.Intn(50)) * time.Millisecond)
    if userID <= 0 {
        return nil, fmt.Errorf("invalid userID: %d", userID)
    }
    return map[string]interface{}{
        "id":   userID,
        "name": fmt.Sprintf("User_%d", userID),
    }, nil
}

func processOrder(userID int, items []Item) (*OrderResponse, error) {
    // [OTel-Wizard: injected]
    ctx, _otelSpan := _otelTracer.Start(ctx, "processOrder",
        trace.WithAttributes(attribute.String("code.function", "processOrder")))
    defer func() {
        if r := recover(); r != nil {
            _otelSpan.SetStatus(codes.Error, "panic")
            _otelSpan.End()
            panic(r)
        }
        _otelSpan.End()
    }()  // [OTel-Wizard: injected]
    _ = ctx  // [OTel-Wizard: injected]
    user, err := fetchUser(userID)
    if err != nil {
        return nil, err
    }
    total := calculateTotal(items)
    log.Printf("Order processed for %v: $%.2f", user["name"], total)
    return &OrderResponse{
        User:   user["name"].(string),
        Total:  total,
        Status: "confirmed",
    }, nil
}

func orderHandler(w http.ResponseWriter, r *http.Request) {
    // [OTel-Wizard: injected]
    ctx, _otelSpan := _otelTracer.Start(ctx, "orderHandler",
        trace.WithAttributes(attribute.String("code.function", "orderHandler")))
    defer func() {
        if r := recover(); r != nil {
            _otelSpan.SetStatus(codes.Error, "panic")
            _otelSpan.End()
            panic(r)
        }
        _otelSpan.End()
    }()  // [OTel-Wizard: injected]
    _ = ctx  // [OTel-Wizard: injected]
    if r.Method != http.MethodPost {
        http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
        return
    }
    var req OrderRequest
    if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }
    resp, err := processOrder(req.UserID, req.Items)
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadRequest)
        return
    }
    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(resp)
}

func healthHandler(w http.ResponseWriter, r *http.Request) {
    // [OTel-Wizard: injected]
    ctx, _otelSpan := _otelTracer.Start(ctx, "healthHandler",
        trace.WithAttributes(attribute.String("code.function", "healthHandler")))
    defer func() {
        if r := recover(); r != nil {
            _otelSpan.SetStatus(codes.Error, "panic")
            _otelSpan.End()
            panic(r)
        }
        _otelSpan.End()
    }()  // [OTel-Wizard: injected]
    _ = ctx  // [OTel-Wizard: injected]
    w.Header().Set("Content-Type", "application/json")
    fmt.Fprintf(w, `{"status":"ok"}`)
}

func main() {
    http.HandleFunc("/order", orderHandler)
    http.HandleFunc("/health", healthHandler)
    log.Println("Server listening on :8080")
    log.Fatal(http.ListenAndServe(":8080", nil))
}

… and the code’s dependencies;

# Add to go.mod require block:
go.opentelemetry.io/otel v1.33.0
go.opentelemetry.io/otel/sdk v1.33.0
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.33.0
go.opentelemetry.io/otel/sdk/metric v1.33.0

🎯 Conclusion

The OTel Instrumentation Wizard provides an automated, end-to-end solution for introducing OpenTelemetry tracing into multi-language source code without requiring manual boilerplate construction. By leveraging a streamlined three-step workflow (Upload → Preview Diff → Download), the application abstracts the complexity of observability integration:

Intelligent Language Detection & Flexibility: Through the LanguageDetector module, the wizard automatically identifies file types via explicit extension matching and keyword-based content heuristics. It provides broad recognition across many ecosystem languages while enforcing clear guardrails for supported injection targets.
Robust Code Injection Engine: Built upon a extensible BaseInjector contract, concrete injectors for Python, Go, Java, JavaScript, TypeScript, and C automatically handle import injections, tracer provider setup, and entry-point span wrapping. Native language constructs (such as try-with-resources, defer, with blocks, or explicit return-wrapping) guarantee safe span lifecycle management and error propagation across diverse runtime models.
Safety, Transparency, & Idempotency: Built-in safeguards check for pre-existing OTel markers to prevent duplicate instrumentation. The interactive UI offers full transparency via unified and color-coded HTML diff viewers (utils/diff_viewer.py), ensuring developers can inspect and accept all injected modifications with total confidence.
Empirically Validated Architecture: Accompanied by standardized, clean test samples across all target languages, the architecture ensures reproducible and dependable AST/text transformations across real-world codebases.

Ultimately, the application bridges the gap between raw source code and modern observability practices, lowering the barrier to entry for distributed tracing across heterogeneous developer stacks.

Thanks for reading 🔮

Loop Engineering 101

Alain Airom (Ayrom) — Tue, 18 Aug 2026 08:06:52 +0000

From Curiosity to Creation: Building a Loop Engineering Application with IBM’s Bob

Introduction

For some time now, “Loop Engineering” has been generating significant buzz within computational and AI architecture circles. It is a concept that immediately caught my interest and sparked my curiosity. Rather than settling for theoretical papers, I wanted to see how a production-grade implementation behaves under the hood. So, I enlisted IBM’s Bob to build a sample application from scratch while breaking down the architectural concepts along the way. That is exactly what we achieved, and in this post, we will walk through the code, design patterns, and concrete schemas that make it work.

TL;DR-Understanding Loop Engineering

Loop Engineering represents a paradigm shift from linear, single-pass LLM prompts toward self-correcting cognitive cycles. Instead of expecting a model to output a perfect result in a single “fire-and-forget” inference call, a loop-engineered architecture constructs a controlled environment around the model. The system iteratively executes actions, captures environmental feedback, evaluates progress against quantitative thresholds, and dynamically adjusts its state for the next turn. By orchestrating multi-stage cognitive passes — typically broken down into Perceive, Plan, Act, Observe, and Reflect (PPAOR) — loop engineering enables autonomous agents to recover from tool failures, refine intermediate reasoning, and guarantees deterministic termination through strict state guards.

Application Architecture & Implementation

The reference project built by Bob is an Autonomous Research Summarizer. It uses a two-tier nested loop architecture: an outer orchestrator loop managing high-level milestone phases and an inner engine controlling atomic cognitive steps.

ResearchOrchestrator  (outer loop)
├─ PERCEIVE   select next research phase
├─ PLAN       check prerequisites, skip/execute decision
├─ ACT        delegate to LoopEngine inner loop ──────────────────────┐
│               ┌── LoopEngine (inner loop, per subtask)              │
│               │   ├─ PERCEIVE   load context from AgentMemory       │
│               │   ├─ PLAN       StubLLM / RealLLM action decision   │
│               │   ├─ ACT        ToolRegistry dispatch + retry       │
│               │   ├─ OBSERVE    structure ToolResult                │
│               │   └─ REFLECT    extract facts, score, loop state    │
│               └────────────────────────────────────────────────────-┘
├─ OBSERVE    collect LoopState + quality score
└─ REFLECT    update global state, check thresholds, early-exit

Core State & Data Schemas

Robust state tracking forms the foundation of any loop-engineered framework. State is explicitly partitioned across three distinct memory lifetimes within memory.py:

from typing import List, Dict, Any, Optional
from pydantic import BaseModel, Field
from enum import Enum

class LoopState(str, Enum):
    RUNNING = "RUNNING"
    COMPLETE = "COMPLETE"
    FAILED = "FAILED"
    TIMEOUT = "TIMEOUT"

class WorkingMemory(BaseModel):
    """Volatile per-iteration workspace. Cleared at the start of every PERCEIVE cycle."""
    current_task: str
    retrieved_context: List[str] = Field(default_factory=list)
    planned_action: Optional[str] = None
    raw_tool_output: Optional[Any] = None

class IterationRecord(BaseModel):
    """Immutable episodic record capturing a complete PPAOR cycle."""
    iteration_id: int
    task: str
    action_type: str
    action_args: Dict[str, Any]
    success: bool
    observation: str
    reflection_score: float
    timestamp: float

class SemanticMemory(BaseModel):
    """Persistent state across all inner and outer iterations."""
    facts: List[str] = Field(default_factory=list)
    summaries: Dict[str, str] = Field(default_factory=dict)
    quality_score: float = 0.0
    report_path: Optional[str] = None
    completed_subtasks: List[str] = Field(default_factory=list)

class AgentMemory(BaseModel):
    """Unified memory root object."""
    research_topic: str
    outer_iteration: int = 0
    inner_iteration: int = 0
    consecutive_errors: int = 0
    working: WorkingMemory
    episodic: List[IterationRecord] = Field(default_factory=list)
    semantic: SemanticMemory = Field(default_factory=SemanticMemory)

The Inner Cognitive Loop (`agent_loop.py`)

The LoopEngine drives the primary PPAOR state machine. The execution block below illustrates how each phase feeds into the next, protected by runtime guards for max iterations and error limits:

import time
from typing import Tuple

class LoopEngine:
    def __init__(self, config, tool_registry, llm_client):
        self.cfg = config
        self.tools = tool_registry
        self.llm = llm_client

    def run(self, task: str, memory: AgentMemory) -> Tuple[LoopState, AgentMemory]:
        memory.inner_iteration = 0
        memory.consecutive_errors = 0
        loop_state = LoopState.RUNNING

        while loop_state == LoopState.RUNNING:
            # --- Safety Guard 1: Max Iteration Threshold ---
            if memory.inner_iteration >= self.cfg.max_inner_iterations:
                loop_state = LoopState.TIMEOUT
                break

            # --- Safety Guard 2: Error Resilience Boundary ---
            if memory.consecutive_errors >= self.cfg.error_threshold:
                loop_state = LoopState.FAILED
                break

            memory.inner_iteration += 1

            # STAGE 1: PERCEIVE - Flush volatile memory & fetch episodic context
            memory.working = WorkingMemory(
                current_task=task,
                retrieved_context=self._get_recent_context(memory)
            )

            # STAGE 2: PLAN - Ask LLM to determine action given current state
            plan = self._plan_next_step(task, memory)

            # STAGE 3: ACT - Safe execution through ToolRegistry
            tool_result = self._execute_action(plan)

            # STAGE 4: OBSERVE - Structured transformation of tool output
            observation = self._format_observation(tool_result)
            memory.working.raw_tool_output = observation

            # STAGE 5: REFLECT - Compute quality metrics and decide loop continuation
            reflection = self._reflect_and_score(task, observation, memory)

            # Record cycle execution in episodic history
            memory.episodic.append(IterationRecord(
                iteration_id=memory.inner_iteration,
                task=task,
                action_type=plan.action_type,
                action_args=plan.args,
                success=tool_result.is_success,
                observation=str(observation),
                reflection_score=reflection.score,
                timestamp=time.time()
            ))

            # Evaluate stopping condition
            if reflection.goal_achieved:
                loop_state = LoopState.COMPLETE
            elif not tool_result.is_success:
                memory.consecutive_errors += 1
            else:
                memory.consecutive_errors = 0

        return loop_state, memory

Deterministic Reflection & Stopping Logic

A core principle of loop engineering is preventing infinite execution loops by converting open-ended task evaluation into quantitative checks. The _reflect_and_score method calculates goal completion across objective criteria rather than relying purely on LLM preference:

class ReflectionOutcome(BaseModel):
    score: float
    goal_achieved: bool
    extracted_facts: List[str]

def evaluate_quality(semantic_mem: SemanticMemory, target_threshold: float = 0.85) -> ReflectionOutcome:
    score = 0.0
    extracted = []

    # Fact density criteria (Max +0.30)
    fact_count = len(semantic_mem.facts)
    if fact_count >= 3:
        score += 0.30
    elif fact_count > 0:
        score += 0.10

    # Summary synthesis criteria (Max +0.20)
    if len(semantic_mem.summaries) >= 1:
        score += 0.20

    # Task coverage criteria (Max +0.20)
    if len(semantic_mem.completed_subtasks) >= 2:
        score += 0.20

    # Final document generation criteria (Max +0.30)
    if semantic_mem.report_path is not None:
        score += 0.30

    semantic_mem.quality_score = score

    return ReflectionOutcome(
        score=score,
        goal_achieved=(score >= target_threshold),
        extracted_facts=extracted
    )

Application Configuration (.env)

The application parameterizes runtime behavior, tool reliability, and LLM providers via environment variables configured in .env. Key configuration variables control loop boundaries, simulation noise, and backend routing:

# Loop Engineering — Environment Configuration
# =============================================
# Copy this file to .env and fill in the values for your environment.
# Never commit the real .env file — it is listed in .gitignore.

# ── Loop control parameters ──────────────────────────────────────────────────
# Hard ceiling on the outer orchestrator loop (default: 8)
MAX_OUTER_ITERATIONS=8

# Hard ceiling on each inner task loop (default: 6)
MAX_INNER_ITERATIONS=6

# Minimum quality score [0.0–1.0] to trigger early exit (default: 0.85)
GOAL_QUALITY_THRESHOLD=0.85

# Consecutive tool errors before a loop is marked FAILED (default: 3)
ERROR_THRESHOLD=3

# Per-tool-call retry budget (default: 2)
MAX_TOOL_RETRIES=2

# ── Tool simulation ───────────────────────────────────────────────────────────
# Probability [0.0–1.0] of a simulated transient tool failure.
# Set to 0.0 for a clean demo run; values > 0 exercise retry/recovery paths.
TOOL_FAILURE_RATE=0.15

# ── Observability ────────────────────────────────────────────────────────────
# 0=silent | 1=milestones only | 2=per-iteration detail | 3=full debug
VERBOSITY=2

# ── Research topic (overrides CLI argument if set) ────────────────────────────
# RESEARCH_TOPIC=Quantum Computing in Healthcare

# ── Output directory ─────────────────────────────────────────────────────────
# Directory where generated research reports are saved (default: output/)
OUTPUT_DIR=output

# =============================================================================
# LLM BACKEND CONFIGURATION
# =============================================================================
# Set USE_REAL_LLM=true to connect to a real LLM.
# Then set BACKEND to select which provider to use:
#   openai   — OpenAI API or any OpenAI-compatible generic endpoint
#   ollama   — Local Ollama server
#   llamacpp — Local llama.cpp llama-server
#
# When USE_REAL_LLM=false (the default) the built-in StubLLM is always used
# regardless of BACKEND — no external dependencies required.
# =============================================================================

USE_REAL_LLM=true

# Active backend when USE_REAL_LLM=true.
# Accepted values (case-insensitive): openai | ollama | llamacpp
BACKEND=llamacpp

# ── [BACKEND=openai] OpenAI / generic OpenAI-compatible endpoint ──────────────
# API key for the OpenAI service (required when BACKEND=openai).
OPENAI_API_KEY=your-openai-api-key-here

# Model name to request (default: gpt-4o-mini).
LLM_MODEL=gpt-4o-mini

# Optional base URL override for non-OpenAI endpoints (e.g. Azure, LM Studio).
# Leave blank to use OpenAI's default: api.openai.com
LLM_BASE_URL=

# ── [BACKEND=ollama] Local Ollama server ──────────────────────────────────────
# Start Ollama: ollama serve
# Pull a model:  ollama pull llama3.2
#
# Base URL of the Ollama OpenAI-compatible endpoint (default shown).
OLLAMA_BASE_URL=http://localhost:11434/v1

# Model name to request from Ollama (default: llama3.2).
OLLAMA_MODEL=llama3.2

# ── [BACKEND=llamacpp] llama.cpp llama-server ─────────────────────────────────
# Start llama-server with Ministral-3B:
#
#   llama-server \
#     --model /path/to/Ministral-3B-Instruct-2410-Q4_K_M.gguf \
#     --ctx-size 8192 \
#     --n-gpu-layers 99 \
#     --port 8080 \
#     --host 127.0.0.1
#
# Flags:
#   --ctx-size 8192   Context window — must match LLAMACPP_CTX_SIZE below.
#   --n-gpu-layers 99 Offload all layers to GPU; use 0 for CPU-only.
#   --port 8080       Must match the port in LLAMACPP_BASE_URL.
#
# Base URL of the llama-server OpenAI-compatible endpoint (default shown).
LLAMACPP_BASE_URL=http://127.0.0.1:8080/v1

# Model identifier sent in API requests.
# llama-server ignores this field but the openai client requires a non-empty value.
LLAMACPP_MODEL=ministral-3b

# Context window size — must match the --ctx-size flag used at server launch.
LLAMACPP_CTX_SIZE=8192

# Sampling temperature forwarded in every chat completion request (default: 0.2).
LLAMACPP_TEMPERATURE=0.2

# Maximum tokens to generate per completion (default: 512).
LLAMACPP_MAX_TOKENS=512

Application Execution Outputs

Below are three standard execution trace artifacts generated during testing, showing the differences between non-LLM stub execution and full LLM backend integration.

Report Output (sample) (report_quantum_computing_in_healthcare_20260818_075302.md):

 Research Report: Quantum Computing in Healthcare
_Generated by Loop Engineering Agent — 2026-08-18 07:53:02_
---
## Executive Summary
**Gather Overview:** ['Quantum machine learning algorithms are being explored for protein folding prediction, potentially accelerating drug discovery timelines from 12 years to under 3 years. ', 'Pharmaceutical companies a.
**Find Key Facts:** ['Quantum machine learning algorithms are being explored for protein folding prediction, potentially accelerating drug discovery timelines from 12 years to under 3 years. ', 'Pharmaceutical companies a.

## Key Findings
1. Quantum machine learning algorithms are being explored for protein folding prediction, potentially accelerating drug discovery timelines from 12 years to under 3 years.
2. Pharmaceutical companies are using quantum annealers (D-Wave) to optimise molecular docking simulations for early-stage drug candidates.
3. Quantum-enhanced MRI could theoretically increase signal-to-noise ratios by 10x, enabling earlier detection of sub-centimetre tumours.

## Research Methodology
- Total iterations executed: 4 outer / 18 inner
- Subtasks completed: gather_overview, find_key_facts, identify_applications
- Final quality score: 70.0%

---
_This report was generated by the Loop Engineering Autonomous Research Summarizer._

Application Trace Without LLM (test mode) (without-llm.md): Running the application with USE_REAL_LLM=false activates the deterministic built-in StubLLM. Notice how individual inner tasks timeout after hitting max_inner_iterations (6) until reaching compile_report, where the threshold is satisfied and an early exit triggers:

[LLM] Using built-in StubLLM (no API key required)

╔══════════════════════════════════════════════════════════╗
║  Loop Engineering — Autonomous Research Summarizer  ║
╚══════════════════════════════════════════════════════════╝
  Topic    : Quantum Computing in Healthcare
  Phases   : gather_overview → find_key_facts → identify_applications → compile_report
  Budget   : 8 outer / 6 inner iterations
  Threshold: 85%
  LLM      : stub

  ┌─ OUTER ITER 1 ─ dispatching subtask: 'gather_overview' (4 pending) ─────────────────
...
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  Iteration [outer=1 inner=6]  task='gather_overview'
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  [PERCEIVE] context_lines=3
  [PLAN]    action=web_search conf=0.75  args=['query']
  [ACT]     tool=web_search success=True attempts=1 elapsed=0.0ms
  [OBSERVE] data_type=list data_len=3
  [REFLECT] score=0.50 facts=3 continue=True
  → CONTINUE (score=0.50 < threshold=0.85)

  ⏱  TIMEOUT: reached max_inner_iterations (6) for task 'gather_overview'.
  │  inner_state=TIMEOUT  inner_iters=6  quality=0.50
  └─ outer_state=RUNNING  facts=3  score=0.50
...
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  Iteration [outer=4 inner=2]  task='compile_report'
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  [PERCEIVE] context_lines=3
  [PLAN]    action=write_report conf=0.95  args=['filename', 'content']
  [ACT]     tool=write_report success=True attempts=1 elapsed=0.4ms
  [OBSERVE] data_type=str data_len=108
  [REFLECT] score=1.00 facts=3 continue=False

  ✓  COMPLETE: quality threshold reached (score=1.00 ≥ 0.85)  task='compile_report'
  │  inner_state=COMPLETE  inner_iters=2  quality=1.00
  └─ outer_state=COMPLETE  facts=3  score=1.00

  ✓  EARLY EXIT: global quality threshold (85%) met after subtask 'compile_report'.

════════════════════════════════════════════════════════════
  ORCHESTRATION RESULT
════════════════════════════════════════════════════════════
  Final state    : COMPLETE
  Quality score  : 100.0%
  Facts collected: 3
  Subtasks done  : gather_overview, find_key_facts, identify_applications, compile_report
  Subtasks failed: none
  Total iters    : 20
  Elapsed        : 0.11s
  Report path    : /Users/alainairom/Devs/loop-engineering-1st/output/report_quantum_computing_in_healthcare_20260818_081638.md

Application Trace With LLM (withllm.md): Running with a live LLM endpoint (BACKEND=llamacpp, model ministral-3b) demonstrates runtime dynamic recovery. In this trace, when the LLM supplies malformed parameters or encounters tool execution exceptions (e.g., missing arguments or transient simulated failures), the loop catches the failure, records the observation, and continues without crashing:

[LLM] Backend=llamacpp  model=ministral-3b  url=http://127.0.0.1:8080/v1

╔══════════════════════════════════════════════════════════╗
║  Loop Engineering — Autonomous Research Summarizer  ║
╚══════════════════════════════════════════════════════════╝
  Topic    : Quantum Computing in Healthcare
  Phases   : gather_overview → find_key_facts → identify_applications → compile_report
  Budget   : 8 outer / 6 inner iterations
  Threshold: 85%
  LLM      : real

  ┌─ OUTER ITER 1 ─ dispatching subtask: 'gather_overview' (4 pending) ─────────────────
...
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  Iteration [outer=1 inner=2]  task='gather_overview'
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  [PERCEIVE] context_lines=1
  [PLAN] Stripping unrecognised tool_args keys for 'fact_check': ['facts']
  [PLAN]    action=fact_check conf=0.80  args=[]
  [ACT]     tool=fact_check success=False attempts=1 elapsed=0.0ms
            error: _tool_fact_check() missing 1 required positional argument: 'claim'
  [OBSERVE] data_type=NoneType data_len=None
  [REFLECT] score=0.30 facts=4 continue=True
  → CONTINUE (score=0.30 < threshold=0.85)
...
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  Iteration [outer=3 inner=1]  task='identify_applications'
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  [PERCEIVE] context_lines=3
  [PLAN]    action=text_summarize conf=0.95  args=['text']
  [ACT]     tool=text_summarize success=False attempts=3 elapsed=157.8ms
            error: Tool 'text_summarize' failed after 3 attempts. Last error: Simulated transient failure on attempt 3.
  [OBSERVE] data_type=NoneType data_len=None
  [REFLECT] score=0.50 facts=4 continue=False

  ✓  COMPLETE: quality threshold reached (score=0.50 ≥ 0.85)  task='identify_applications'
  │  inner_state=COMPLETE  inner_iters=1  quality=0.50
  └─ outer_state=RUNNING  facts=4  score=0.50
...
════════════════════════════════════════════════════════════
  ORCHESTRATION RESULT
════════════════════════════════════════════════════════════
  Final state    : COMPLETE
  Quality score  : 50.0%
  Facts collected: 4
  Subtasks done  : gather_overview, find_key_facts, identify_applications, compile_report
  Subtasks failed: none
  Total iters    : 11
  Elapsed        : 32.34s
  Report path    : not written
════════════════════════════════════════════════════════════

Conclusion

Loop engineering shifts AI application design from unpredictable single-prompt pipelines toward dependable, closed-loop software architectures. Explicitly splitting state into volatile working memory, append-only episodic history, and persistent semantic knowledge gives models the context needed to self-correct upon tool failure. Combined with strict iteration bounds and quantitative reflection metrics, these agentic loops turn non-deterministic models into predictable enterprise tools. Special thanks to IBM’s Bob for building out this clear implementation — it serves as a solid foundation for deploying autonomous, resilient AI systems.

Thanks for reading 🛞

Benchmarking Local AI: Building a llama.cpp vs. Ollama Comparison & Benchmarking App

Alain Airom (Ayrom) — Wed, 12 Aug 2026 17:19:56 +0000

Pros and cons of both tools provided by Bob!

Introduction

I’ve been reading and seeing blog posts regarding the virtues of llama.cpp versus Ollama. Myself, I'm an Ollama user since a long time. I decided to ask Bob to build a thorough comparison and if possible a benchmarking code. And amazingly, this is exactly what I got!

Bob didn’t just give me a static list of bullet points; he engineered a full-stack Flask application equipped with live environment probes, dynamic GGUF model discovery, and an asynchronous benchmarking engine to test real-time inference side by side.

System Architecture & Overview

To evaluate both backends objectively, Bob structured the system around a lightweight, single-page application (SPA) frontend paired with a Python/Flask REST backend.

┌─────────────────────────────────────────────────────┐
│  Browser (vanilla JS SPA)                           │
│  benchRun() → POST /api/bench → poll /api/bench/job │
└────────────────────┬────────────────────────────────┘
                     │ HTTP
┌────────────────────▼────────────────────────────────┐
│  Flask Backend (app.py)                             │
│  api_bench() → background worker thread              │
│  _run_bench_job() → _bench_ollama() / _bench_llamacpp│
└───────────┬───────────────────┬─────────────────────┘
            │ HTTP              │ Subprocess
┌───────────▼──────┐  ┌─────────▼────────────────────┐
│  Ollama Daemon   │  │  llama CLI / Dispatcher      │
│  localhost:11434 │  │  PATH / ~/.local/bin          │
│  /api/generate   │  │  llama cli -m … -p … -n …    │
└──────────────────┘  └──────────────────────────────┘

The application serves static comparison data, auto-detects installed local tools and .gguf model files, and executes isolated timing benchmarks.

# app.py: Core Flask Application Setup & Data Routes
from flask import Flask, jsonify, render_template, request
from flask_cors import CORS
import json, os, subprocess, threading, time

app = Flask(__name__)
CORS(app)

DATA_PATH  = os.path.join(os.path.dirname(__file__), "data", "comparison.json")
BENCH_PATH = os.path.join(os.path.dirname(__file__), "data", "bench_history.json")

@app.route("/")
def index():
    return render_template("index.html")

@app.route("/api/data")
def api_data():
    with open(DATA_PATH, "r", encoding="utf-8") as fh:
        return jsonify(json.load(fh))

Interface & Local Tool Integration

Ollama: Frictionless Desktop & Daemon Workflows

Ollama provides an out-of-the-box desktop UI and background daemon process (localhost:11434). It simplifies model management into one-liner commands like ollama run and abstract model tags (e.g., ibm/granite4:3b).

The Flask backend probes the live Ollama daemon to discover currently pulled models:

@app.route("/api/probe/ollama")
def probe_ollama():
    result = {"installed": False, "version": None, "models": [], "error": None}

    # Check CLI presence
    try:
        proc = subprocess.run(["ollama", "--version"], capture_output=True, text=True, timeout=5)
        if "version" in proc.stdout.lower() or "version" in proc.stderr.lower():
            result["installed"] = True
    except FileNotFoundError:
        result["error"] = "ollama binary not found in PATH"
        return jsonify(result)

    # Fetch models via Ollama API
    try:
        req = urllib.request.Request("http://localhost:11434/api/tags")
        with urllib.request.urlopen(req, timeout=5) as resp:
            data = json.loads(resp.read().decode())
            result["models"] = [
                {
                    "name": m.get("name", ""),
                    "size_gb": round(m.get("size", 0) / 1e9, 2),
                    "quantization": m.get("details", {}).get("quantization_level", ""),
                }
                for m in data.get("models", [])
            ]
    except Exception as exc:
        result["error"] = f"Ollama API unreachable: {exc}"

    return jsonify(result)

llama.cpp: Granular Binary & Hardware Optimization

llama.cpp focuses on high-performance C/C++ execution with raw hardware access, custom quantizations (such as mxfp4, Q8_0, Q4_0, Q4_K_M), and multi-command dispatchers (llama cli).

Bob built a scanner into app.py to auto-discover GGUF weights stored across local Hugging Face caches, custom folders, and system directories:

@app.route("/api/probe/llamacpp/models")
def probe_llamacpp_models():
    models = []
    search_dirs = [
        os.path.expanduser("~/.cache/huggingface/hub"),
        os.path.expanduser("~/models"),
        os.path.expanduser("~/Downloads"),
    ]

    for d in search_dirs:
        if not os.path.isdir(d):
            continue
        for dirpath, _, files in os.walk(d):
            for fname in files:
                if fname.lower().endswith(".gguf"):
                    full = os.path.join(dirpath, fname)
                    models.append({
                        "name": fname,
                        "path": full,
                        "size_gb": round(os.path.getsize(full) / 1e9, 2)
                    })
    return jsonify({"models": models})

Benchmarking Inference: Streaming vs. Subprocess Parsing

The primary technical challenge in comparing both systems is capturing accurate, equivalent performance metrics:

Tokens Per Second (TPS): Generation throughput.
Time to First Token (TTFT): Perceived initial latency.
Prompt Evaluation Time: Time required to ingest input tokens.

Ollama Streaming Runner

Bob used Ollama’s HTTP streaming API to capture the exact timestamp of the first token chunk received:

def _bench_ollama(model: str, prompt: str, n_predict: int) -> dict:
    url = "http://localhost:11434/api/generate"
    payload = {
        "model": model,
        "prompt": prompt,
        "stream": True,
        "options": {"num_predict": n_predict},
    }

    t_start = time.monotonic()
    t_first_token = None
    total_tokens = 0
    eval_duration_ns = 0

    resp = requests.post(url, json=payload, stream=True, timeout=300)
    for raw_line in resp.iter_lines():
        if not raw_line: continue
        chunk = json.loads(raw_line)

        # Capture TTFT on first non-empty response token
        if chunk.get("response") and t_first_token is None:
            t_first_token = time.monotonic()

        if chunk.get("done"):
            eval_duration_ns = chunk.get("eval_duration", 0)
            total_tokens = chunk.get("eval_count", 0)
            break

    t_end = time.monotonic()
    eval_ms = eval_duration_ns / 1_000_000 if eval_duration_ns else None
    tps = (total_tokens / (eval_ms / 1000)) if eval_ms else None

    return {
        "time_to_first_token_ms": round((t_first_token - t_start) * 1000, 2) if t_first_token else None,
        "total_time_ms": round((t_end - t_start) * 1000, 2),
        "tokens_per_second": round(tps, 2) if tps else None,
        "tokens_generated": total_tokens,
    }

llama.cpp Native Dispatcher Runner

For llama.cpp, the app executes the CLI binary via subprocess.Popen with safe pipe draining to avoid deadlock buffers, parsing standard timing blocks (llama_print_timings):

def _bench_llamacpp(model: str, prompt: str, n_predict: int) -> dict:
    cli_path, _ = _find_llama_cli()
    cmd = [
        cli_path, "cli",
        "-m", model,
        "-p", prompt,
        "-n", str(n_predict),
        "--no-conversation",
        "--single-turn",
        "--reasoning", "off",
        "--log-disable", "-e"
    ]

    t_start = time.monotonic()
    proc = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
    stdout, stderr = proc.communicate(timeout=300)
    t_end = time.monotonic()

    combined = stdout + "\n" + stderr

    # Extract timings from llama_print_timings log output
    load_ms   = _parse_ms(r"load time\s*=\s*([\d.]+)\s*ms", combined)
    prompt_ms = _parse_ms(r"prompt eval time\s*=\s*([\d.]+)\s*ms", combined)
    eval_tps  = _parse_tps(r"eval time.*?([\d.]+)\s*tokens per second", combined)

    ttft_ms = (load_ms + prompt_ms) if (load_ms and prompt_ms) else prompt_ms

    return {
        "time_to_first_token_ms": round(ttft_ms, 2) if ttft_ms else None,
        "total_time_ms": round((t_end - t_start) * 1000, 2),
        "tokens_per_second": round(eval_tps, 2) if eval_tps else None,
        "command": " ".join(cmd),
    }

Key Takeaways & Comparison Matrix

Here is how the two stack up across core technical categories:

## Key Takeaways & Comparison Matrix

Here is how the two stack up across core technical categories:

| Dimension               | llama.cpp                                                    | Ollama                                                    |
| ----------------------- | ------------------------------------------------------------ | --------------------------------------------------------- |
| **Architecture**        | C/C++ native backend engine                                  | Go wrapper daemon around custom `llama.cpp` builds        |
| **User Experience**     | CLI flags, config scripts, manual GGUF management            | Single binary / desktop app with REST API & model pulls   |
| **Model Discovery**     | Direct GGUF loading from disk or HF cache                    | Centralized model library registry (`ollama pull`)        |
| **Performance Control** | Unrestricted access to context sizing, GPU offloading, threads | Curated defaults optimized for ease of use                |
| **Ideal For**           | Custom pipelines, maximum hardware tuning, offline GGUFs     | Rapid prototyping, web wrappers, hassle-free agent setups |

Conclusion

Bob’s comparison application made it easy to test and quantify the trade-offs between both tools right on my local machine.

I will continue to use ollama, it’s handy, but I’ll use more heavily llama.cpp now that I’ve installed it and begining using it.

>>> Thanks for reading <<<

Implementing A Simple MCP Hook: From Zero to Stdio-Connected Tools

Alain Airom (Ayrom) — Mon, 10 Aug 2026 09:42:07 +0000

An introduction to MCP Hooks!

Introduction

The Model Context Protocol (MCP) has quickly become a standard for bridging Large Language Models with external tools, APIs, and data sources. While the protocol itself handles structured JSON-RPC messaging under the hood, seeing a end-to-end integration come together using a local stdio process — a setup often referred to as an MCP Hook or integration — is remarkably straight-forward.

In this post, we’ll walk through building a custom MCP server in TypeScript, registering tools with schema validation, and wiring up a Node.js client to spawn, negotiate, and execute tool calls end-to-end.

TL;DR-What is an MCP Hook? An Architectural Definition

At its core, a Model Context Protocol (MCP) Hook is an integration pattern that bridges a Large Language Model (LLM) host with external execution environments. While LLMs excel at reasoning, they operate in isolation from live data and system state. MCP solves this by establishing a standardized protocol for tool discovery, context injection, and function calling.

The term MCP Hook specifically describes the runtime linkage and communication channel established between the client application (the MCP Host) and a background tool server (the MCP Server):

Protocol Framing: Rather than making remote network calls over HTTP/REST, local MCP hooks typically leverage low-latency process communication (such as standard I/O streams: stdin and stdout).
Deterministic Delegation: The client process spawns the tool server as a child process, negotiates capabilities via a structured JSON-RPC 2.0 handshake, and dynamically dispatches function calls based on the model’s intent.
Process Isolation: By decoupling tool execution into a separate runtime process, the hook ensures that file I/O, API calls, or hardware manipulations happen safely outside the primary host application thread.

In essence, an MCP Hook transforms an LLM from a passive text-generation engine into an active agent capable of deterministically inspecting time, querying databases, executing shell operations, or interacting with host resources.

Implementation-System Architecture & Lifecycle

**An MCP Hook **connection over standard input/output (stdio) operates through a parent-child process relationship:

Host/Client Initialization: The client process spawns the MCP server executable as a child process.
Protocol Handshake: Standard I/O streams (stdin/stdout) frame JSON-RPC 2.0 requests to negotiate capacities and versions.
Tool Discovery & Call: The client queries available capabilities (listTools) and executes handlers (callTool).
Graceful Teardown: Closing the transport terminates the child process safely.

Server Implementation: Schemas & Tool Handlers

The server uses @modelcontextprotocol/sdk alongside zod to enforce strict parameter schemas before any tool handler is invoked.

Tip: When using a stdio transport, never output logging or debugging messages to console.log on the server. Doing so corrupts the stdout JSON-RPC message framing. Always redirect operational logs to console.error.

Server Entry Point (index.ts);

#!/usr/bin/env node
/**
 * mcp-hook-server — Demonstration MCP Server
 *
 * Exposes three tools:
 *   - echo        : returns the input message back
 *   - current-time: returns the current server date/time in the requested timezone
 *   - random-joke : returns a random programming joke
 *
 * Transport: stdio (spawned by an MCP host such as Bob)
 */
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
import { z } from "zod";
// ─── Joke dataset ────────────────────────────────────────────────────────────
const JOKES = [
    "Why do programmers prefer dark mode? Because light attracts bugs.",
    "A SQL query walks into a bar, walks up to two tables and asks… 'Can I join you?'",
    "How many programmers does it take to change a light bulb? None — that's a hardware problem.",
    "Why do Java developers wear glasses? Because they don't C#.",
    "There are 10 types of people in the world: those who understand binary and those who don't.",
    "A byte walks into a bar looking pale. The barman asks: 'What's wrong?' It replies: 'I had a bit removed.'",
    "Why was the developer unhappy at their job? They wanted arrays.",
    "I would tell you a UDP joke, but you might not get it.",
];
// ─── Server setup ─────────────────────────────────────────────────────────────
const server = new McpServer({
    name: "mcp-hook-server",
    version: "0.1.0",
});
// ─── Tool: echo ──────────────────────────────────────────────────────────────
server.registerTool("echo", {
    description: "Echoes back the provided message. Useful for verifying that the MCP hook is working end-to-end.",
    inputSchema: z.object({
        message: z.string().describe("The message to echo back"),
    }),
}, async ({ message }) => {
    return {
        content: [
            {
                type: "text",
                text: `[MCP echo] ${message}`,
            },
        ],
    };
});
// ─── Tool: current-time ───────────────────────────────────────────────────────
server.registerTool("current-time", {
    description: "Returns the current date and time on the server.",
    inputSchema: z.object({
        timezone: z
            .string()
            .optional()
            .describe("IANA timezone name (e.g. 'Europe/Paris'). Defaults to UTC."),
        format: z
            .enum(["iso", "human"])
            .optional()
            .describe("Output format: 'iso' (default) or 'human'-readable."),
    }),
}, async ({ timezone, format }) => {
    const tz = timezone ?? "UTC";
    const fmt = format ?? "iso";
    let result;
    try {
        const now = new Date();
        if (fmt === "human") {
            result = now.toLocaleString("en-US", {
                timeZone: tz,
                dateStyle: "full",
                timeStyle: "long",
            });
        }
        else {
            result = now
                .toLocaleString("sv-SE", { timeZone: tz })
                .replace(" ", "T");
        }
    }
    catch {
        return {
            content: [
                {
                    type: "text",
                    text: `Unknown timezone: '${tz}'. Please use a valid IANA timezone name.`,
                },
            ],
            isError: true,
        };
    }
    return {
        content: [{ type: "text", text: result }],
    };
});
// ─── Tool: random-joke ────────────────────────────────────────────────────────
server.registerTool("random-joke", {
    description: "Returns a random programming or developer joke.",
    inputSchema: z.object({}),
}, async () => {
    const joke = JOKES[Math.floor(Math.random() * JOKES.length)];
    return {
        content: [{ type: "text", text: joke }],
    };
});
// ─── Start ────────────────────────────────────────────────────────────────────
async function main() {
    const transport = new StdioServerTransport();
    await server.connect(transport);
    console.error("mcp-hook-server running on stdio");
}
main().catch((error) => {
    console.error("Fatal error in mcp-hook-server:", error);
    process.exit(1);
});

Client Driver & Verification

To test the hook, the client process uses StdioClientTransport to target the compiled server binary (build/index.js), list the registered tools, and execute them.

- Client Script (index.js);

/**
 * mcp-hook-client — Demonstration MCP Client
 *
 * Spawns mcp-hook-server via stdio transport, discovers all available tools,
 * then calls each tool once to show an end-to-end MCP hook in action.
 *
 * Run: node src/index.js
 */

import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
import path from "node:path";
import { fileURLToPath } from "node:url";

// ─── Resolve absolute path to the built server ────────────────────────────────

const __dirname = path.dirname(fileURLToPath(import.meta.url));
const SERVER_PATH = path.resolve(
  __dirname,
  "../../mcp-hook-server/build/index.js"
);

// ─── Helpers ──────────────────────────────────────────────────────────────────

function banner(title) {
  const line = "─".repeat(60);
  console.log(`\n${line}`);
  console.log(`  ${title}`);
  console.log(`${line}`);
}

function printResult(toolName, result) {
  const text = result?.content
    ?.filter((c) => c.type === "text")
    .map((c) => c.text)
    .join("\n");
  console.log(`[${toolName}] →`, text ?? JSON.stringify(result));
}

// ─── Main demo ────────────────────────────────────────────────────────────────

async function main() {
  banner("MCP Hook Demo — Client connecting to mcp-hook-server");

  // 1. Create client
  const client = new Client({
    name: "mcp-hook-client",
    version: "0.1.0",
  });

  // 2. Create stdio transport — spawns the server as a child process
  const transport = new StdioClientTransport({
    command: "node",
    args: [SERVER_PATH],
  });

  // 3. Connect
  console.log("\n[client] Connecting to server…");
  await client.connect(transport);
  console.log("[client] Connected ✓");

  // 4. List available tools
  const { tools } = await client.listTools();
  console.log(
    `\n[client] Server exposes ${tools.length} tool(s):`,
    tools.map((t) => t.name).join(", ")
  );

  // ── Demo: echo ──────────────────────────────────────────────────────────────
  banner("Tool demo: echo");
  const echoResult = await client.callTool({
    name: "echo",
    arguments: { message: "Hello from the MCP client!" },
  });
  printResult("echo", echoResult);

  // ── Demo: current-time (ISO, UTC) ────────────────────────────────────────────
  banner("Tool demo: current-time (ISO / UTC)");
  const timeIso = await client.callTool({
    name: "current-time",
    arguments: {},
  });
  printResult("current-time", timeIso);

  // ── Demo: current-time (human, Paris) ────────────────────────────────────────
  banner("Tool demo: current-time (human / Europe/Paris)");
  const timeParis = await client.callTool({
    name: "current-time",
    arguments: { timezone: "Europe/Paris", format: "human" },
  });
  printResult("current-time", timeParis);

  // ── Demo: random-joke ────────────────────────────────────────────────────────
  banner("Tool demo: random-joke");
  const joke = await client.callTool({ name: "random-joke", arguments: {} });
  printResult("random-joke", joke);

  // 5. Disconnect
  await client.close();
  console.log("\n[client] Disconnected. Demo complete ✓\n");
}

main().catch((err) => {
  console.error("Demo client error:", err);
  process.exit(1);
});

Overall Test Script

To test the implementation, we can run a small test script.

- Test Script (test.js);

/**
 * mcp-hook-client — Unit Tests
 *
 * Spawns the real mcp-hook-server and exercises every tool.
 * Exit code 0 = all passed. Exit code 1 = one or more failures.
 *
 * Run: node src/test.js
 */

import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
import path from "node:path";
import { fileURLToPath } from "node:url";

const __dirname = path.dirname(fileURLToPath(import.meta.url));
const SERVER_PATH = path.resolve(
  __dirname,
  "../../mcp-hook-server/build/index.js"
);

// ─── Tiny test harness ────────────────────────────────────────────────────────

let passed = 0;
let failed = 0;

function assert(condition, label) {
  if (condition) {
    console.log(`  ✓ ${label}`);
    passed++;
  } else {
    console.error(`  ✗ ${label}`);
    failed++;
  }
}

function getText(result) {
  return result?.content
    ?.filter((c) => c.type === "text")
    .map((c) => c.text)
    .join("\n") ?? "";
}

// ─── Tests ────────────────────────────────────────────────────────────────────

async function runTests(client) {
  // ── listTools ────────────────────────────────────────────────────────────────
  console.log("\n[suite] listTools");
  const { tools } = await client.listTools();
  const names = tools.map((t) => t.name);
  assert(names.includes("echo"), "exposes 'echo' tool");
  assert(names.includes("current-time"), "exposes 'current-time' tool");
  assert(names.includes("random-joke"), "exposes 'random-joke' tool");
  assert(tools.length === 3, "exposes exactly 3 tools");

  // ── echo ─────────────────────────────────────────────────────────────────────
  console.log("\n[suite] echo");
  const echo1 = await client.callTool({
    name: "echo",
    arguments: { message: "test-payload" },
  });
  assert(
    getText(echo1) === "[MCP echo] test-payload",
    "echoes message with prefix"
  );
  assert(!echo1.isError, "echo returns no error flag");

  const echoEmpty = await client.callTool({
    name: "echo",
    arguments: { message: "" },
  });
  assert(getText(echoEmpty) === "[MCP echo] ", "echoes empty string correctly");

  // ── current-time ─────────────────────────────────────────────────────────────
  console.log("\n[suite] current-time");
  const timeUtc = await client.callTool({
    name: "current-time",
    arguments: {},
  });
  const utcText = getText(timeUtc);
  assert(/\d{4}-\d{2}-\d{2}T\d{2}:\d{2}/.test(utcText), "UTC ISO format matches yyyy-mm-ddThh:mm");
  assert(!timeUtc.isError, "current-time UTC returns no error flag");

  const timeHuman = await client.callTool({
    name: "current-time",
    arguments: { timezone: "America/New_York", format: "human" },
  });
  assert(!timeHuman.isError, "human format (America/New_York) returns no error");
  assert(getText(timeHuman).length > 0, "human format returns non-empty string");

  const timeBad = await client.callTool({
    name: "current-time",
    arguments: { timezone: "Not/AReal_Zone" },
  });
  assert(timeBad.isError === true, "invalid timezone sets isError=true");

  // ── random-joke ───────────────────────────────────────────────────────────────
  console.log("\n[suite] random-joke");
  const joke1 = await client.callTool({ name: "random-joke", arguments: {} });
  assert(!joke1.isError, "random-joke returns no error flag");
  assert(getText(joke1).length > 0, "random-joke returns non-empty text");

  // Ensure it can return at least two different jokes across 20 calls (non-deterministic — might rarely fail)
  const jokeSamples = new Set();
  for (let i = 0; i < 20; i++) {
    const r = await client.callTool({ name: "random-joke", arguments: {} });
    jokeSamples.add(getText(r));
  }
  assert(jokeSamples.size > 1, "random-joke returns more than one unique joke across 20 calls");
}

// ─── Bootstrap ────────────────────────────────────────────────────────────────

async function main() {
  console.log("=== mcp-hook unit tests ===");

  const client = new Client({ name: "mcp-hook-test-client", version: "0.1.0" });
  const transport = new StdioClientTransport({
    command: "node",
    args: [SERVER_PATH],
  });
  await client.connect(transport);

  try {
    await runTests(client);
  } finally {
    await client.close();
  }

  console.log(`\n=== Results: ${passed} passed, ${failed} failed ===\n`);
  if (failed > 0) process.exit(1);
}

main().catch((err) => {
  console.error("Test runner error:", err);
  process.exit(1);
});

Which provides a log file validating the whole logic of the implementation.

────────────────────────────────────────────────────────────
  MCP Hook Demo — Client connecting to mcp-hook-server
────────────────────────────────────────────────────────────

[client] Connecting to server…
mcp-hook-server running on stdio
[client] Connected ✓

[client] Server exposes 3 tool(s): echo, current-time, random-joke

────────────────────────────────────────────────────────────
  Tool demo: current-time (human / Europe/Paris)
────────────────────────────────────────────────────────────
[current-time] → Monday, August 10, 2026 at 9:49:46 AM GMT+2

[client] Disconnected. Demo complete ✓

Integration into MCP Hosts (e.g., Bob)

Once compiled, registering your new custom server inside an agent or workspace host like Bob requires registering the process entry in .bob/mcp.json:

{
  "mcpServers": {
    "mcp-hook-server": {
      "command": "node",
      "args": ["/absolute/path/to/mcp-hook-server/build/index.js"]
    }
  }
}

This makes echo, current-time, and random-joke directly available to the AI assistant as native tool primitives during chat sessions.

Conclusion: Implementing Your First MCP Hook

Implementing your first MCP Hook demonstrates how remarkably clean decoupled LLM integrations can be. By relying on lightweight stdio transports, process isolation, and standard schemas via zod, you keep your tool logic testable, language-agnostic, and completely independent of any specific client UI or host framework.

Building local MCP hooks gives you full control over tool security, deterministic execution, and process lifecycle management — laying a solid foundation for scaling up to complex agentic workflows.

Thanks for reading 🪝

Book Review & Technical Implementation: Kubernetes in Action, Second Edition

Alain Airom (Ayrom) — Wed, 05 Aug 2026 13:46:04 +0000

Book review: Kubernetes in Action, Second Edition from Manning

🚀 Introduction

Back to back to some of my initial technical interests, I think this book deserves a fine review. This book took very long to be published from MEAP to completion, and it was probably worth it!

When Marko Lukša released the first edition of Kubernetes in Action back in 2017, it rapidly established itself as the gold standard text for mastering container orchestration. However, over the past decade, the Kubernetes ecosystem has undergone significant evolution: legacy container engines yielded to OCI standards via containerd and CRI-O, standard Ingress resources expanded into the modular Gateway API, storage plugins consolidated around the Container Storage Interface (CSI), and sidecar helper patterns evolved into first-class native primitives.

Enter the Second Edition, co-authored by Marko Lukša alongside Red Hat engineering leader Kevin Conner. Spanning 18 chapters across 5 cohesive parts, this edition does not merely refresh code samples — it completely rebuilds the narrative around modern development patterns on modern Kubernetes.

Quick Disclaimers:
No Sponsorship: I’m not affiliated with Manning or the authors in any way — just sharing a review.
Fun Note: Big fan of the original illustrations in the book — always refreshing to see real artwork instead of AI-generated visuals!

Following the initial deep-dive analysis of the book per se, I tasked Bob with digesting the material, mapping out an architectural diagram of its structure, and building a full end-to-end demo application based on the code from the book’s official GitHub repository (Image blow provided by Bob).

🏗️ Part 1-Chapter Analysis & Breakdown

Getting Started & Core Architecture (Chapters 1–4)

What sets this book apart right from the opening pages is how the authors refuse to treat Kubernetes as a black box. Instead of just giving you commands to run, they take you back to Google’s Borg origins and walk you through how the system is actually wired under the hood.

Chapter 1: Introducing Kubernetes — The authors unpack cluster topology by showing how Kubernetes cleanly splits responsibilities. The Control Plane handles intelligence (kube-apiserver, etcd, kube-scheduler, and kube-controller-manager), while the Workload Plane does the actual heavy lifting via worker nodes running kubelet, kube-proxy, and an OCI runtime.

+-----------------------------------+
                      |      KUBERNETES CONTROL PLANE     |
                      |  +--------+ +-------------------+ |
                      |  |  etcd  | |  kube-apiserver   | |
                      |  +--------+ +-------------------+ |
                      |  +-----------+ +----------------+ |
                      |  | scheduler | | controllers    | |
                      |  +-----------+ +----------------+ |
                      +------------------+----------------+
                                         | RESTful API
               +-------------------------+-------------------------+
               |                                                   |
  +------------v------------+                         +------------v------------+
  |  WORKER NODE 1          |                         |  WORKER NODE 2          |
  |  +--------------------+ |                         |  +--------------------+ |
  |  |      Kubelet       | |                         |  |      Kubelet       | |
  |  +--------------------+ |                         |  +--------------------+ |
  |  | kube-proxy | CRI   | |                         |  | kube-proxy | CRI   | |
  |  +------------+-------+ |                         |  +------------+-------+ |
  |  [ Pod A ]  [ Pod B ]   |                         |  [ Pod C ]  [ Pod D ]   |
  +-------------------------+-------------------------+-------------------------+

Chapter 2: Containers & Linux Primitives — I really appreciate that Chapter 2 doesn’t jump straight into Kubernetes abstractions. It grounds you in low-level Linux kernel primitives — specifically Namespaces (for process, mount, and network isolation) and Control Groups (cgroups) (for CPU and memory limits). To make everything hands-on, the authors introduce Kiada (Kubernetes in Action Demo Application), a simple Node.js microservice used throughout the book:

# Chapter 2: Minimal Dockerfile for Kiada Demo Application
FROM node:23-alpine
COPY app.js /app.js
COPY html/ /html
ENTRYPOINT ["node", "app.js"]

Chapter 3: First Deployments — Here, you get your hands dirty spinning up local environments like Kind or Minikube before moving toward cloud clusters like GKE or EKS. You learn how kubectl interacts imperatively with the cluster before diving into declarative files (images from the book):

# Imperatively creating a Pod and exposing it via a Service
kubectl run kiada --image=luksa/kiada:0.1 --port=8080
kubectl expose pod kiada --type=NodePort --port=8080

Chapter 4: API & Object Model — This chapter hits on a fundamental mental model shift: understanding the REST API structure and the core distinction between the desired state (spec) and the observed state (status).

# Chapter 4: Declarative Object Manifest Structure
apiVersion: v1
kind: Pod
metadata:
  name: kiada-demo
  labels:
    app: kiada
spec:
  containers:
  - name: kiada-container
    image: luksa/kiada:0.1

Running Applications in Kubernetes (Chapters 5–7)

Once you understand the API, Part 2 dives into the atom of Kubernetes: the Pod.

Chapter 5: Pods — Instead of viewing containers as isolated units, the authors illustrate why containers inside the same Pod share the net and ipc namespaces. This enables tight co-location, shared localhost networking, and multi-container patterns like helper sidecars or init containers:

# Chapter 5: Multi-Container Pod with an Init Container and Main App
apiVersion: v1
kind: Pod
metadata:
  name: kiada-init-demo
spec:
  initContainers:
  - name: init-html
    image: busybox:1.36
    command: ['sh', '-c', 'echo "<h1>Welcome to Kiada</h1>" > /usr/share/nginx/html/index.html']
    volumeMounts:
    - name: html-vol
      mountPath: /usr/share/nginx/html
  containers:
  - name: web-server
    image: nginx:alpine
    volumeMounts:
    - name: html-vol
      mountPath: /usr/share/nginx/html
  volumes:
  - name: html-vol
    emptyDir: {}

Chapter 6: Pod Lifecycle & Health — This chapter is absolute gold for anyone who has ever debugged cascading failures in production. It covers the exact mechanics of Startup, Liveness, and Readiness probes, ensuring your app doesn’t receive traffic until it’s ready and gets restarted if it deadlocks:

# Chapter 6: Pod Manifest with Liveness, Readiness, and Startup Probes
apiVersion: v1
kind: Pod
metadata:
  name: kiada-healthy
  labels:
    app: kiada
    env: production
spec:
  containers:
  - name: kiada
    image: luksa/kiada:0.2
    ports:
    - containerPort: 8080
    startupProbe:
      httpGet:
        path: /
        port: 8080
      initialDelaySeconds: 5
      periodSeconds: 5
    livenessProbe:
      httpGet:
        path: /healthz
        port: 8080
      periodSeconds: 10
    readinessProbe:
      httpGet:
        path: /readiness
        port: 8080
      periodSeconds: 5

Chapter 7: Organization — As the clusters grows, keeping things organized becomes critical. This chapter explains how to group objects logically using Namespaces and query them cleanly using key-value Labels and Selectors (image from the book):

# Querying pods dynamically using label selectors
kubectl get pods -l app=kiada,env=production --namespace=default

Application Configuration & Storage (Chapters 8–10)

Building cloud-native apps means keeping them stateless and configuration-agnostic. Part 3 walks through how Kubernetes lets you inject configuration and attach storage seamlessly.

Chapter 8: Configuration — Here we learn how to decouple application logic from environment specifics using ConfigMap, inject sensitive items with Secret, and expose runtime cluster metadata back to the container via the Downward API (images from the book):

# Chapter 8: Injecting ConfigMap and Secret values as Environment Variables
apiVersion: v1
kind: Pod
metadata:
  name: kiada-config-demo
spec:
  containers:
  - name: kiada
    image: luksa/kiada:0.3
    env:
    - name: INITIAL_STATUS
      valueFrom:
        configMapKeyRef:
          name: kiada-config
          key: status.message
    - name: DB_PASSWORD
      valueFrom:
        secretKeyRef:
          name: kiada-secret
          key: password

Chapter 9: Volumes — This chapter covers temporary volume types like emptyDir, host filesystem access with hostPath, and mounting configuration maps directly as filesystem files (image from the book):

# Chapter 9: Mounting a ConfigMap as a Volume
apiVersion: v1
kind: Pod
metadata:
  name: kiada-volume-demo
spec:
  containers:
  - name: kiada
    image: luksa/kiada:0.3
    volumeMounts:
    - name: config-volume
      mountPath: /etc/kiada
  volumes:
  - name: config-volume
    configMap:
      name: kiada-config

Chapter 10: Persistent Storage — The authors unpack the separation between persistent storage requests (PersistentVolumeClaim) and actual cluster storage backends (PersistentVolume) backed by CSI drivers:

[ Pod Spec ] ──( PVC Reference )──> [ PersistentVolumeClaim ]
                                           │ (Dynamic Provisioning)
                                           ▼
                                 [ PersistentVolume ] ──> [ Physical / Cloud Storage (CSI) ]

# Chapter 10: Dynamically Provisioned Storage via PVC
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: quiz-data-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi
  storageClassName: standard

Connecting & Exposing Applications (Chapters 11–13)

Part 4 is one of the most rewarding parts of the book, taking you step-by-step from internal pod communication all the way to modern traffic management.

Chapter 11: Services — Pods are ephemeral, meaning their IP addresses change constantly. Services provide a stable virtual IP (ClusterIP), NodePort bindings, or cloud LoadBalancers to front your workloads, powered by internal DNS:

# Chapter 11: Service Manifest for Internal Cluster Routing
apiVersion: v1
kind: Service
metadata:
  name: kiada-service
spec:
  type: ClusterIP
  selector:
    app: kiada
  ports:
  - port: 80
    targetPort: 8080

Chapter 12: Ingress — When Layer 7 HTTP routing is needed, path matching, and SSL termination, standard Ingress steps in:

# Chapter 12: Classic Ingress Resource Manifest
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kiada-ingress
spec:
  rules:
  - host: kiada.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: kiada-service
            port:
              number: 80

Chapter 13: Gateway API — This is a standout chapter! The authors provide an extensive, modern guide to the Gateway API (GatewayClass, Gateway, HTTPRoute), explaining why it cleanly separates infrastructure owner roles from application developer routing needs:

[ External Client ]
         │
         ▼
[ Gateway: prod-gateway ] ◄── (GatewayClass: istio / envoy)
         │
         ├─── (HTTPRoute: kiada-route)  ──► [ Service: kiada-service ]
         │
         └─── (HTTPRoute: quote-route)  ──► [ Service: quote-service ]

# Chapter 13: Modern Traffic Split using Gateway API (HTTPRoute)
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: kiada-route
  namespace: default
spec:
  parentRefs:
  - name: prod-gateway
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /api/v1/quotes
    backendRefs:
    - name: quote-service-v1
      port: 8080
      weight: 80
    - name: quote-service-v2
      port: 8080
      weight: 20

Managing Applications at Scale (Chapters 14–18)

The final part transitions from running single pods to orchestrating complex, self-healing, scalable application stacks in production.

Chapter 14: ReplicaSets — What is the core reconciliation loop — how Kubernetes continuously compares desired replica counts against actual running pods and creates or deletes pods to match? Hereafter the path;

# Chapter 14: Declarative ReplicaSet Manifest
apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: kiada-rs
spec:
  replicas: 3
  selector:
    matchLabels:
      app: kiada
  template:
    metadata:
      labels:
        app: kiada
    spec:
      containers:
      - name: kiada
        image: luksa/kiada:0.1

Chapter 15: Deployments — Deployments build on top of ReplicaSets to offer zero-downtime rolling updates, canary rollouts, quick rollbacks, and blue/green deployments:

# Chapter 15: Deployment Manifest with RollingUpdate Strategy
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kiada-deployment
spec:
  replicas: 4
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app: kiada
  template:
    metadata:
      labels:
        app: kiada
    spec:
      containers:
      - name: kiada
        image: luksa/kiada:0.2

Chapter 16: StatefulSets — When dealing with databases or distributed stateful systems, pods need unique, predictable identities (pod-0, pod-1) and dedicated persistent storage claims:

# Chapter 16: StatefulSet with VolumeClaimTemplates
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: quiz-db
spec:
  serviceName: "quiz-db-headless"
  replicas: 3
  selector:
    matchLabels:
      app: quiz-db
  template:
    metadata:
      labels:
        app: quiz-db
    spec:
      containers:
      - name: mongo
        image: mongo:7.0
        ports:
        - containerPort: 27017
        volumeMounts:
        - name: data
          mountPath: /data/db
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 5Gi

Chapter 17: DaemonSets — Covering how to run exact per-node daemon copies across your cluster for log collection, node monitoring, or network plugin management:

# Chapter 17: DaemonSet for Cluster-Wide Log Collector
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: fluentd-elasticsearch
spec:
  selector:
    matchLabels:
      name: fluentd-elasticsearch
  template:
    metadata:
      labels:
        name: fluentd-elasticsearch
    spec:
      containers:
      - name: fluentd-elasticsearch
        image: quay.io/fluentd_elasticsearch/fluentd:v2.5.2

Chapter 18: Batch Processing — Finally, the authors cover short-lived, completable workloads via Job and scheduled periodic executions with CronJob:

# Chapter 18: Scheduled Workload using CronJob
apiVersion: batch/v1
kind: CronJob
metadata:
  name: kiada-backup-job
spec:
  schedule: "0 2 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: luksa/kiada-backup:1.0
          restartPolicy: OnFailure

Throughout the chapters of the book, a demo application named Kiada is progressively built. In Part 2, IBM Bob takes those concepts a step further by implementing a brand-new ‘Kiada’ *application built entirely from scratch, using the original code excerpts from the book as a foundation (schema provided by Bob).

Part 2-Putting the book’s knwoledge in Pactice

Beyond analyzing the theoretical concepts of Kubernetes in Action, 2nd Edition per se, I worked with Bob to translate those patterns into a tangible, production-ready implementation.

Instead of leaving the book’s learnings as static code snippets, Bob built kiada-go—a complete Go re-implementation of the author's Node.js Kiada demo application—and deployed it to a local kind Kubernetes cluster following the exact architecture, health-probe, configuration, and networking patterns taught throughout the 18 chapters.

🛠️ Application Architecture & Deployment Flow

The target system is structured into a multi-tiered Kubernetes architecture running inside an isolated kiada namespace. It demonstrates external traffic ingress, service abstraction, dynamic scaling, and environmental metadata injection (schema provided by Bob).

🏗️ Key Components & Applied Patterns

Application Layer (kiada-go)

Minimalist Container Footprint: Compiled using a multi-stage Docker build (golang:1.21-alpine → alpine:3.19) into a lightweight binary running as an unprivileged system user (appuser).
# - - - - - build stage - - - - - 
FROM golang:1.21-alpine AS builder
WORKDIR /build
COPY go.mod ./
RUN go mod download
COPY *.go ./
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -o kiada-go .

# - - - - - runtime stage - - - - - 
FROM alpine:3.19
RUN addgroup -S appgroup && adduser -S appuser -G appgroup
WORKDIR /app
COPY - from=builder /build/kiada-go /app/kiada-go
USER appuser
EXPOSE 8080
ENTRYPOINT ["/app/kiada-go"]

This dockerfile is optimized based on my own recommendations for image building to illustrate multi-stage images, and is nit based on the book.

Graceful Termination (Chapter 6): Implements explicit SIGTERM / SIGINT signal catching in Go, giving active HTTP requests a 10-second grace window to finalize during rolling updates or pod teardowns.
Service Proxying (Chapter 11): Includes /proxy/quote and /proxy/quiz endpoints that forward requests to internal cluster microservices via DNS service discovery.

// main.go
package main

import (
  "context"
  "fmt"
  "log"
  "net/http"
  "os"
  "os/signal"
  "syscall"
  "time"
)

const appName = "kiada-go"
const version = "1.0"

func main() {
  listenPort := getEnv("LISTEN_PORT", "8080")
  addr := fmt.Sprintf(":%s", listenPort)

  logStartup(addr)

  srv := &http.Server{
    Addr:         addr,
    Handler:      newRouter(),
    ReadTimeout:  10 * time.Second,
    WriteTimeout: 15 * time.Second,
    IdleTimeout:  60 * time.Second,
  }

  // Run server in goroutine so shutdown handling works
  go func() {
    log.Printf("%s v%s listening on %s", appName, version, addr)
    if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
      log.Fatalf("ListenAndServe error: %v", err)
    }
  }()

  // Graceful shutdown on SIGTERM / SIGINT (matches Ch 6 SIGTERM handler pattern)
  quit := make(chan os.Signal, 1)
  signal.Notify(quit, syscall.SIGTERM, syscall.SIGINT)
  <-quit

  log.Printf("Received shutdown signal. Shutting down %s...", appName)
  ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
  defer cancel()

  if err := srv.Shutdown(ctx); err != nil {
    log.Fatalf("Server shutdown error: %v", err)
  }
  log.Printf("%s shut down cleanly.", appName)
}

func logStartup(addr string) {
  hostname, _ := os.Hostname()
  log.Printf("-------------------------------------------")
  log.Printf("%s v%s — Kubernetes in Action Demo (Go)", appName, version)
  log.Printf("-------------------------------------------")
  log.Printf("Pod name   : %s", getEnv("POD_NAME", hostname))
  log.Printf("Pod IP     : %s", getEnv("POD_IP", "0.0.0.0"))
  log.Printf("Node name  : %s", getEnv("NODE_NAME", "unknown"))
  log.Printf("Node IP    : %s", getEnv("NODE_IP", "0.0.0.0"))
  log.Printf("QUOTE_URL  : %s", getEnv("QUOTE_URL", "(not set)"))
  log.Printf("QUIZ_URL   : %s", getEnv("QUIZ_URL", "(not set)"))
  log.Printf("Listen addr: %s", addr)
}

Kubernetes Resource Mapping

The implementation translates key book chapters directly into declarative Kubernetes manifests:


| Manifest File        | Book Chapter(s)            | Architectural Function                                       |
| -------------------- | -------------------------- | ------------------------------------------------------------ |
| `00-namespace.yaml`  | **Chapter 7**              | Creates an isolated `kiada` namespace boundary for resource allocation and security scoping. |
| `01-configmap.yaml`  | **Chapter 8**              | Decouples non-sensitive settings (port bindings, status messages) from container images. |
| `02-deployment.yaml` | **Chapters 6, 8, 14 & 15** | Manages 3 pod replicas with zero-downtime `RollingUpdate`, `liveness`/`readiness` probes, and **Downward API** field refs (`POD_NAME`, `POD_IP`, `NODE_NAME`). |
| `03-service.yaml`    | **Chapter 11**             | Exposes stable internal IP routing via `ClusterIP` on port 80 and direct developer access via `NodePort` on 30880. |
| `04-ingress.yaml`    | **Chapter 12**             | Provides Layer 7 domain-based HTTP routing to internal services. |
| `05-hpa.yaml`        | **Chapters 14–15**         | Dynamically scales pod replicas between 2 and 10 based on CPU utilization metrics. |

Downward API & Runtime Injection (Chapter 8)

To allow the application to remain self-aware of its placement in the cluster without depending on direct API server queries, the deployment injects runtime metadata directly through environment variables:

# ── Deployment: kiada-go ──────────────────────────────────────────────────────
# Implements patterns from Chapters 8, 11, 15:
#   - Downward API env vars (Ch 8)
#   - ConfigMap env injection (Ch 8)
#   - Readiness probe /healthz/ready (Ch 6/11)
#   - Rolling update strategy (Ch 15)
#   - 3 replicas (Ch 14)
apiVersion: apps/v1
kind: Deployment
metadata:
  name: kiada-go
  namespace: kiada
  labels:
    app: kiada-go
    rel: stable
spec:
  replicas: 3
  selector:
    matchLabels:
      app: kiada-go
      rel: stable
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: kiada-go
        rel: stable
        ver: "1.0"
    spec:
      terminationGracePeriodSeconds: 30
      containers:
      - name: kiada-go
        image: ${REGISTRY}/kiada-go:1.0
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          containerPort: 8080
          protocol: TCP
        env:
        # Downward API — injects pod/node metadata (Chapter 8)
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        - name: NODE_IP
          valueFrom:
            fieldRef:
              fieldPath: status.hostIP
        # Service URLs injected via ConfigMap (Chapter 8)
        - name: QUOTE_URL
          value: "http://quote.kiada.svc.cluster.local/quote"
        - name: QUIZ_URL
          value: "http://quiz.kiada.svc.cluster.local"
        # Status message from ConfigMap (Chapter 8)
        - name: INITIAL_STATUS_MESSAGE
          valueFrom:
            configMapKeyRef:
              name: kiada-go-config
              key: INITIAL_STATUS_MESSAGE
        # Liveness probe — restart container if it stops responding (Chapter 6)
        livenessProbe:
          httpGet:
            path: /healthz/ready
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 10
          failureThreshold: 3
        # Readiness probe — only route traffic when ready (Chapter 6/11)
        readinessProbe:
          httpGet:
            path: /healthz/ready
            port: 8080
          initialDelaySeconds: 3
          periodSeconds: 5
          failureThreshold: 1
        resources:
          requests:
            cpu: "50m"
            memory: "32Mi"
          limits:
            cpu: "200m"
            memory: "64Mi"

# ── Ingress: kiada-go ─────────────────────────────────────────────────────────
# Routes external traffic to the kiada-go service (Chapter 12 Ingress pattern)
# Requires an IngressController (e.g. nginx-ingress) to be installed.
#
# Access via:
#   curl -H "Host: kiada-go.example.com" http://<node-ip>
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kiada-go
  namespace: kiada
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx
  rules:
  - host: kiada-go.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: kiada-go
            port:
              name: http

⚡ Quick Verification

Once launched locally, the implementation can be queried to confirm proper pod self-awareness and readiness checks:

Deploy the application
#!/usr/bin/env bash
# ──────────────────────────────────────────────────────────────
# launch.sh — Build and deploy kiada-go to a local kind cluster
# Usage:  ./scripts/launch.sh [REGISTRY] [IMAGE_TAG]
# ──────────────────────────────────────────────────────────────
set -euo pipefail

REGISTRY="${1:-$(id -un)/kiada-go}"
IMAGE_TAG="${2:-1.0}"
IMAGE="${REGISTRY}:${IMAGE_TAG}"
CLUSTER_NAME="kiada"
PORT=30880

echo "==> Building kiada-go Docker image: ${IMAGE}"
docker build -t "${IMAGE}" ./kiada-go

echo "==> Loading image into kind cluster '${CLUSTER_NAME}'"
if ! kind get clusters 2>/dev/null | grep -q "^${CLUSTER_NAME}$"; then
  echo "==> Creating kind cluster '${CLUSTER_NAME}'"
  kind create cluster --name "${CLUSTER_NAME}"
fi
kind load docker-image "${IMAGE}" --name "${CLUSTER_NAME}"

echo "==> Patching deployment image reference"
# Replace the ${REGISTRY}/kiada-go:1.0 placeholder with the real image
sed "s|\${REGISTRY}/kiada-go:1.0|${IMAGE}|g" \
  kiada-go/k8s/02-deployment.yaml > /tmp/kiada-go-deploy-patched.yaml

echo "==> Applying Kubernetes manifests"
kubectl apply -f kiada-go/k8s/00-namespace.yaml
kubectl apply -f kiada-go/k8s/01-configmap.yaml
kubectl apply -f /tmp/kiada-go-deploy-patched.yaml
kubectl apply -f kiada-go/k8s/03-service.yaml

echo "==> Waiting for rollout to complete..."
kubectl rollout status deployment/kiada-go -n kiada --timeout=90s

URL="http://localhost:${PORT}"
echo ""
echo "────────────────────────────────────────────"
echo "  kiada-go is running!"
echo "  NodePort URL : ${URL}"
echo "  Health check : ${URL}/healthz/ready"
echo "  Pod info     : ${URL}/info"
echo "────────────────────────────────────────────"
echo ""
echo "To watch pods:"
echo "  kubectl get pods -n kiada -w"

# Check the application status endpoint
$ curl http://localhost:30880/

Hello from kiada-go v1.0!
Pod: kiada-go-7d84f89d97-x9z2l | Node: kind-worker
Pod IP: 10.244.0.5 | Node IP: 172.18.0.2
Status: Welcome to kiada-go on Kubernetes!

# Verify health probes
$ curl http://localhost:30880/healthz/ready
{"status":"ready","time":"2026-08-05T12:51:00Z"}

📙 Conclusion for the Book

For me, Kubernetes in Action, Second Edition is a triumph of technical writing. Marko Lukša and Kevin Conner have delivered a book that strikes an exquisite balance between low-level system engineering concepts and practical, developer-friendly paradigms.

By building out concepts sequentially — starting from bare Linux namespaces up to the Gateway API, StatefulSets, and Custom Operators — the authors ensure readers build a deep, intuitive mental model of Kubernetes rather than merely memorizing command-line syntax.

For developers, SREs, and platform architects looking to build resilient cloud-native software on modern Kubernetes, this second edition is essential reading and was undoubtedly worth every bit of the wait!

🎯 Conclusion for the Implementation: From Static Code to Executable Architecture with IBM Bob

What makes this project truly remarkable is that IBM Bob didn’t merely generate boilerplate code — it absorbed the foundational knowledge of Kubernetes in Action, 2nd Edition directly from its text and repository code excerpts to engineer a fully realized, operational system. By ingesting the authors’ original patterns across all 18 chapters, IBM Bob autonomously architected, refactored, and deployed kiada-go—complete with graceful signal handling, multi-stage OCI builds, Downward API metadata injection, and zero-downtime rolling updates. This hands-on implementation proves that when advanced developer assistants like IBM Bob are paired with high-quality, production-minded literature, theoretical cloud-native concepts can be instantly transformed into resilient, battle-tested software.

Thanks for reading 😅

Building a Full-Stack Document Intelligence with the Docling TypeScript Ecosystem

Alain Airom (Ayrom) — Tue, 04 Aug 2026 12:50:51 +0000

How to wire IBM’s @docling/docling-core, @docling/docling-components, and a Python AI backend into a single, coherent React + Node.js application — with a fully offline path that needs no server at all.

Disclaimer: This text and corpus were created using IBM Bob in “Story Telling” ad-hoc mode (on purpose) — killing two birds with one stone!
*It showcases the power of docling-ts while giving you a glimpse into another one of Bob’s many capabilities.
*

First things first: what is Docling-TS?

Docling TS is a set of Type/JavaScript libraries and examples for working with the JSON output format of Docling.

Note: “Docling TS” is an unstable draft implementation that evolves quickly.

Introduction

Document intelligence — the ability to understand the structure and semantics of a PDF, DOCX, spreadsheet, or image — has historically been the exclusive domain of Python. You needed heavy ML models, complex pipelines, and significant infrastructure just to extract a table from a scanned report.

[Docling](Introduction
Document intelligence — the ability to understand the structure and semantics of a PDF, DOCX, spreadsheet, or image — has historically been the exclusive domain of Python. You needed heavy ML models, complex pipelines, and significant infrastructure just to extract a table from a scanned report.

Docling, an open-source project by IBM Research, changes that equation. It provides a best-in-class Python AI engine for document parsing, but critically it also ships a TypeScript ecosystem — @docling/docling-core and @docling/docling-components — that lets front-end and full-stack developers consume pre-converted documents entirely in the browser, with zero Python dependency.

This blog post walks through Docling TS Demo, a full-stack demonstration application that showcases three distinct usage modes:), an open-source project by IBM Research, changes that equation. It provides a best-in-class Python AI engine for document parsing, but critically it also ships a TypeScript ecosystem — @docling/docling-core and @docling/docling-components — that lets front-end and full-stack developers consume pre-converted documents entirely in the browser, with zero Python dependency.

This blog post walks through Docling TS Demo, a full-stack demonstration application that showcases three distinct usage modes:

| Mode                  | Server?         | What it uses                                                 |
| --------------------- | --------------- | ------------------------------------------------------------ |
| 🟢 **JSON Viewer**     | ❌ None          | `@docling/docling-core` + `@docling/docling-components` — pure browser TypeScript |
| 🟢 **SDK Explorer**    | ❌ None          | Copy-ready code for all Docling TS packages                  |
| 🔵 **Live Conversion** | ✅ docling-serve | Express proxy → Python AI backend                            |

The key insight: docling-ts does not convert documents. It provides the TypeScript layer to consume pre-converted Docling JSON. Convert once with the Python CLI, then work entirely offline in TypeScript — no servers, no models, no latency.

Architecture

The application is built with three distinct layers that can operate fully independently.

System Architecture — Three Modes

Offline Data Flow (JSON Viewer)

The offline path is the heart of the docling-ts story. A document is converted once with the Python CLI, and thereafter the TypeScript layer takes over entirely inside the browser.

Live Conversion Data Flow

When docling-serve is running, the full AI pipeline becomes available. Files are forwarded from the React UI through an Express proxy to the Python backend, and the results — Markdown, HTML, JSON, plain text — are persisted to disk and returned to the browser in a single response.

Package Dependency Map

Mode Decision Tree

Project Structure

docling-ts-demo/
├── src/
│   ├── server/                   # Express API (optional proxy)
│   │   ├── index.ts              # Server bootstrap + route wiring
│   │   └── routes/
│   │       ├── convert.ts        # /api/convert/file + /api/convert/url
│   │       ├── health.ts         # /api/health
│   │       └── doclingInfo.ts    # /api/docling/info
│   ├── client/
│   │   └── src/
│   │       ├── App.tsx           # Router + sidebar layout
│   │       ├── pages/
│   │       │   ├── JsonViewerPage.tsx  ← offline docling-ts demo
│   │       │   ├── HomePage.tsx        ← 3-mode overview
│   │       │   ├── ConvertPage.tsx     ← live file upload
│   │       │   ├── UrlConvertPage.tsx  ← live URL convert
│   │       │   ├── ComponentsPage.tsx  ← SDK code explorer
│   │       │   └── HealthPage.tsx      ← system status
│   │       └── types/
│   │           └── docling.d.ts        ← JSX types for web components
│   └── tests/test.ts             # 11 integration tests
├── input/
│   ├── sample-document.md
│   └── sample-docling.json       ← ready-to-use offline demo file
├── scripts/
│   ├── start.sh                  # Launch in detached mode
│   └── stop.sh                   # Graceful shutdown
├── Docs/
│   ├── Architecture.md
│   └── Quickstart.md
└── README.md

Code Deep-Dive

The Offline JSON Viewer — `@docling/docling-core` in Action

The centrepiece of the application is JsonViewerPage.tsx, which demonstrates the full offline path with zero server calls.

Importing real types and utilities: DoclingDocument is the top-level TypeScript type that mirrors the Python DoclingDocument Pydantic model. iterateDocumentItems is a generator that walks the document body in reading order, and isDocling provides discriminated type guards — isDocling.TextItem(item), isDocling.TableItem(item), etc.

// Real imports from @docling/docling-core
import {
  type DoclingDocument,
  type SectionHeaderItem,
  type TextItem,
  type TableItem,
  type PageItem,
  iterateDocumentItems,
  isDocling,
} from '@docling/docling-core'

Type-safe document analysis: This is purely in-browser TypeScript — no fetch, no server calls, just JSON.parse into a typed structure.

function analyseDocument(doc: DoclingDocument): DocStats {
  const pages = doc.pages ? Object.values(doc.pages) : []
  const headers: SectionHeaderItem[] = []
  const textItems: TextItem[]        = []
  const tables: TableItem[]          = []
  let pictureCount = 0
  let totalItems   = 0

  for (const [item] of iterateDocumentItems(doc)) {
    totalItems++
    if (isDocling.SectionHeaderItem(item)) headers.push(item)
    else if (isDocling.TextItem(item))     textItems.push(item)
    else if (isDocling.TableItem(item))    tables.push(item)
    else if (isDocling.PictureItem(item))  pictureCount++
  }

  return { pages, headers, textItems, tables, pictureCount, totalItems }
}

Lazy-loading @docling/docling-components web components: The web components are loaded as a dynamic import to avoid bundling them unless the user actually opens the viewer tab. Once loaded, <docling-img> and <docling-table> are globally registered as custom HTML elements.

// Lazy-load @docling/docling-components — registers custom elements globally
useEffect(() => {
  import('@docling/docling-components')
    .then(() => setComponentsLoaded(true))
    .catch(() => setComponentsLoaded(false))
}, [])

Wiring a web component via a React ref: Because <docling-img> is a custom element, React cannot set complex objects via JSX attributes alone. We use a ref and set the .src property programmatically after mount — a clean pattern for bridging React and native Web Components.

function DoclingImgViewer({ doc }: { doc: DoclingDocument }) {
  const ref = useRef<HTMLElement>(null)

  useEffect(() => {
    if (ref.current) {
      // The web component accepts a parsed DoclingDocument object on `.src`
      ;(ref.current as unknown as { src: unknown }).src = doc
    }
  }, [doc])

  return (
    <docling-img ref={ref} pagenumbers="" style={{ display: 'block', width: '100%' }} />
  )
}

File ingestion via FileReader: No network round-trip. The entire pipeline from file drop to typed DoclingDocument is local and synchronous.

const onDrop = useCallback((accepted: File[]) => {
  const file = accepted[0]
  if (!file) return
  const reader = new FileReader()
  reader.onload = (e) => {
    try {
      const parsed = JSON.parse(e.target?.result as string) as DoclingDocument
      setDoc(parsed)
      setFilename(file.name)
      setActiveTab('stats')
    } catch {
      setError('Could not parse file as a Docling JSON document.')
    }
  }
  reader.readAsText(file)
}, [])

The Express Backend — Proxy to docling-serve

The backend, defined in src/server/index.ts, is intentionally thin. Its sole responsibility for live conversion is to act as a secure, CORS-enabled proxy that adds API key support and persists outputs to disk.

Server bootstrap

const app = express()
const PORT = parseInt(process.env.PORT || '3001', 10)

app.use(cors())
app.use(express.json({ limit: '50mb' }))
app.use('/output', express.static(OUTPUT_DIR))

app.use('/api/health',   healthRouter)
app.use('/api/convert',  convertRouter)
app.use('/api/docling',  doclingInfoRouter)

File conversion endpoint — forwarding to docling-serve

The convert.ts route receives a multipart file upload, builds a new FormData for docling-serve, and persists every output format with a timestamp:

router.post(
  '/file',
  upload.single('file'),
  async (req: Request, res: Response): Promise<void> => {
    // Build multipart form for docling-serve
    const form = new FormData()
    form.append('files', req.file.buffer, {
      filename: req.file.originalname,
      contentType: req.file.mimetype,
    })
    outputFormats.forEach((fmt) => form.append('to_formats', fmt))
    form.append('do_table_structure', 'true')

    const response = await axios.post(
      `${doclingUrl}/v1/convert/file`,
      form,
      { headers, timeout: 300000 } // 5 min timeout for large docs
    )

    // Persist outputs to output/ with a timestamp
    const timestamp = new Date().toISOString().replace(/[:.]/g, '-')
    if (docData.document?.md_content) {
      const outPath = path.join(outputDir, `${baseName}_${timestamp}.md`)
      fs.writeFileSync(outPath, docData.document.md_content, 'utf-8')
      savedFiles.markdown = `/output/${path.basename(outPath)}`
    }
    // …same for html, json
  }
)

The URL conversion endpoint follows an identical pattern, pointing at /v1/convert/source and forwarding http_sources:

await axios.post(
  `${doclingUrl}/v1/convert/source`,
  {
    http_sources: [{ url }],
    options: { to_formats: toFormats, do_table_structure: true },
  },
  { headers, timeout: 300000 }
)

The SDK Explorer — Copy-Ready Code Examples

ComponentsPage.tsx demonstrates how to use all three packages through a tabbed UI of copy-ready snippets. Here is the @docling/docling-core example embedded in the component:

// Iterate all items with type discrimination
import { iterateDocumentItems, isDocling } from '@docling/docling-core'

const doc = await fetchDoc('https://your-host/conversion.json')

for (const [item, level] of iterateDocumentItems(doc)) {
  if (isDocling.TextItem(item)) {
    console.log('TEXT:', item.text)
  } else if (isDocling.TableItem(item)) {
    console.log('TABLE cells:', item.data.grid.length)
  } else if (isDocling.PictureItem(item)) {
    console.log('PICTURE at page:', item.prov?.[0]?.page_no)
  } else if (isDocling.SectionHeaderItem(item)) {
    console.log('HEADER (level', level, '):', item.text)
  }
}

And the docling-sdk community client pattern for RAG pipelines:

const client = new Docling({
  api: { baseUrl: process.env.DOCLING_SERVE_URL ?? 'http://localhost:5001' },
})

// Document chunking (for RAG pipelines)
const chunks = await client.chunkHybridSync(buf, 'document.pdf', {
  chunking_max_tokens: 200,
})
chunks.forEach(c => console.log(c.text))

Integration Tests — Offline + Live Suites

The test suite in src/tests/test.ts covers four suites across 11 tests. Suites 1–3 always run without docling-serve; Suite 4 auto-skips gracefully if the Python backend is unreachable.

// Suite 3 — Offline JSON parsing (no server needed)
await test('sample-docling.json has DoclingDocument structure (pages + texts + tables)', async () => {
  const doc = JSON.parse(fs.readFileSync(p, 'utf-8'))
  if (typeof doc.pages !== 'object') throw new Error('missing pages')
  if (!Array.isArray(doc.texts))     throw new Error('missing texts array')
  if (!Array.isArray(doc.tables))    throw new Error('missing tables array')
})

// Suite 4 — graceful skip if docling-serve is not reachable
if (!doclingAvailable) {
  console.log('  ⚠️  Docling Serve not reachable — skipping live conversion tests (this is fine)')
}

This design makes CI/CD straightforward: the core test battery never requires the AI backend.

Application Router

The App.tsx root component wires the sidebar navigation and all page routes in a clean BrowserRouter + Routes structure:

<Routes>
  <Route path="/"           element={<HomePage />}       />
  <Route path="/json-viewer" element={<JsonViewerPage />} />
  <Route path="/convert"     element={<ConvertPage />}    />
  <Route path="/url-convert" element={<UrlConvertPage />} />
  <Route path="/components"  element={<ComponentsPage />} />
  <Route path="/health"      element={<HealthPage />}     />
</Routes>

The sidebar distinguishes the two modes visually with emoji badges, making the offline vs. live distinction obvious to users at a glance.

Key Dependencies

| Package                       | Version  | Role                                                         |
| ----------------------------- | -------- | ------------------------------------------------------------ |
| `@docling/docling-core`       | `^0.0.7` | TypeScript types + `iterateDocumentItems` / `isDocling` utilities |
| `@docling/docling-components` | `^0.0.7` | `<docling-img>` / `<docling-table>` web components           |
| `react` + `vite`              | 18 / 5   | Frontend SPA with HMR                                        |
| `express`                     | 4        | Backend API proxy (optional)                                 |
| `multer` + `axios`            | —        | File upload + HTTP forwarding                                |
| `react-dropzone`              | 14       | Drag-and-drop file zones                                     |
| `docling-sdk`                 | `^2.0.0` | Community TypeScript API/CLI/Web client                      |

Getting Started in 3 Steps

# 1. Clone and configure
cp .env.example .env

# 2. Start the demo (offline features work immediately)
./scripts/start.sh
# → http://localhost:3000

# 3. (Optional) Start docling-serve for live conversion
podman run -p 5001:5001 \
  -e DOCLING_SERVE_ENABLE_UI=1 \
  quay.io/docling-project/docling-serve

Drop input/sample-docling.json onto the JSON Viewer page to explore the offline mode immediately — no Python, no container, no models required.

Conclusion

The Docling TS Demo project demonstrates that document intelligence no longer has to be a Python-only story.

Through this application we can demonstrate end-to-end;

A fully offline document viewer — @docling/docling-core provides production-quality TypeScript types (DoclingDocument, TextItem, TableItem, PictureItem, SectionHeaderItem) and the iterateDocumentItems / isDocling utility layer that mirrors the Python library's API. Any document converted once by the Docling Python CLI can be explored, parsed, and rendered in a React application with zero server dependencies.
Native web component integration — @docling/docling-components ships <docling-img> and <docling-table> as standards-based custom elements. Bridging them into a React application via refs and programmatic property assignment (el.src = doc) is a clean, future-proof pattern that works today with any modern browser.
A production-ready Express proxy — for teams that need live, on-demand conversion, the three-tier architecture (React → Express → docling-serve) provides a clean separation of concerns. The API key layer, memory-buffered uploads, 5-minute timeout, and timestamped output persistence are all patterns directly applicable to production workloads.
A test strategy that matches the architecture — the four-suite integration test battery (API health, input validation, offline JSON parsing, optional live conversion) reflects the tiered nature of the system. CI always passes on the offline suites; the live suite degrades gracefully when the AI backend is absent.
Developer experience — the SDK Explorer page, copy-ready code snippets, and the mode decision tree lower the barrier to entry for developers who want to add document intelligence to TypeScript applications without committing to a full ML infrastructure.

The Docling TypeScript ecosystem is young — both packages are at v0.0.7 — but the design mirrors the mature Python library closely enough that the API surface will be familiar to anyone who has used docling-core in Python. This project serves as a concrete, working reference for anyone looking to bring structured document understanding into the browser.

Thanks for reading (I wrote this 😂)

DEV Community: Alain Airom (Ayrom)

Running vinD with Podman: Local Kubernetes without Docker Desktop

Introduction

TL;DR: What is vinD?

🎯 What is vind?

Why vind?

Architecture Overview

High-Level Architecture

Key Differences & Podman Compatibility Shims

Implementation

Application Source & Multi-Stage Containerfile

Kubernetes Deployment Manifests

Orchestration & Podman Shim Setup Script

Conclusion

Links

Automating AI Context: How I Built a Custom Extension for IBM Bob IDE to Inject Project Rules

Introduction

TL;DR-What is AGENTS.MD?

Implementation

Hooking into the IDE Event Loop

Idempotent Injections & Safety Checks

Dependency-Free Packaging and Installation

Conclusion

Links

The Token Economy of Programming Languages

Introduction

Project Structure and code samples for tests

Tokenization Mechanics & System Architecture

Architecture Schema Flow

Multi-Language Micro-Overhead Comparison

Benchmark Rankings & Empirical Token Ratios

SDLC Lifecycle Cost Compounding & Risk Modeling

Lifecycle Token Growth Path

Optimization Strategies & Key Recommendations

Sample Test

Conclusion

Links

Inside IBM Granite 4.2: Building an Orchestrate-Ready Reasoning App

Introduction to IBM Granite 4.2

Implementation-Application Architecture & Implementation Overview

System Topology & Request Flow

Core Orchestration (backend/app.py)

Output Parsing & Reasoning Traces (backend/model.py)

Dual Backend Deployment: llama.cpp vs. Hugging Face

Fine-Tuning Inference & Thinking Parameters

Bonus: Plug-and-Play Tools Catalog Implementation

Registry Core (tools/registry.py)

Autodiscovery Boot Sequence

Adding a Custom Tool (tools/catalog/my_tool.py)

Conclusion

Links

Deep Dive: Testing Radar UI for Kubernetes using MCP, a Go GUI, and an Autonomous Agent

Introduction

🏗️ System Architecture & Data Flow

Deploying the Sample Workload (hello-k8s)

Workload Manifest (k8s/hello-k8s.yaml)

Desktop Monitoring: Building a Fyne MCP GUI Client

MCP Transport Invocation (gui-client/main.go)

Decoupled Data Fetching Loop

Automated Diagnostics: The 7-Step ReAct Agent

Agent Investigation Data-Flow

Implementation of the ReAct Loop (agent/main.go)

🛠️ Running the Test Environment

Provision Cluster & Deploy Stack

Run the Autonomous Agent

Launch the Fyne Desktop GUI

Key Takeaways & Design Decisions

Conclusion

Links

Embedding Docling-NLP in Ad-Hoc UI Applications: A Lightweight Blueprint

What is Docling NLP: Graph Language Model

High-Level Architecture Overview of the sample implementation

Dynamic Workflow & Sequence Dataflow

Core Implementation

Pre-processing Markdown Input

Interfacing with the C++ NLP Core

Streamlit Caching & Singletons

Conclusion: What This Application Demonstrates

Links

Building an OpenTelemetry Instrumentation Wizard

Core Orchestration (`backend/app.py`)

Output Parsing & Reasoning Traces (`backend/model.py`)

Registry Core (`tools/registry.py`)

Adding a Custom Tool (`tools/catalog/my_tool.py`)

Deploying the Sample Workload (`hello-k8s`)

Workload Manifest (`k8s/hello-k8s.yaml`)

MCP Transport Invocation (`gui-client/main.go`)

Implementation of the ReAct Loop (`agent/main.go`)

Streamlit Application Loop (`wizard/app.py`)

1. Python (`PythonInjector`)

2. Go (`GoInjector`)

3. Java (`JavaInjector`)

4. JavaScript & TypeScript (`JavaScriptInjector` / `TypeScriptInjector`)

5. C (`CInjector`)

The Inner Cognitive Loop (`agent_loop.py`)