β‘ TL;DR β Ready-to-Deploy Automation Blueprints:
- π Turnkey Production Package (This Guide): Instant Access on Whop | Download on Gumroad (Use code **EARLYBIRD* for -20%)*
- π Free Starter Blueprints (0β¬): n8n AI Memory & Notion/Airtable RAG Sync | Programmatic SEO Engine (Python)
- π MR Studio Lifetime All-Access Vault: Get All Current & Future Workflows (β¬39 One-Time)
If you read the EULAs of modern development tools, you'll notice a disturbing trend: your operating system, your IDE, your AI assistants, and your CLI tools are all harvesting your usage data. They call it "telemetry" for product improvement, but for engineering teams, it's a massive leak of intellectual property and developer behavioral data.
I needed a way to track toolchain usage, IDE adoption, and local productivity metrics for my team without sending a single byte of data to a third-party server.
In this guide, I'll show you how to build a Local Toolchain Analytics & Productivity Engineβa lightweight, privacy-first background daemon written entirely in Python 3.10+.
1. The Bottleneck: Why Commercial Solutions Fail Engineering Teams
There are plenty of time-tracking and developer analytics tools on the market (WakaTime, RescueTime, etc.). However, they all suffer from the same fundamental flaws when applied to strict enterprise environments:
- The Privacy Nightmare: They require constant internet connectivity to stream your active window titles, file names, and branch data to a centralized SaaS.
- Exorbitant Per-Seat Pricing: Paying $10-$20/month per developer just to know how often they use VS Code vs. JetBrains is an unjustifiable expense.
- Fragile Integrations: They rely on brittle IDE plugins rather than OS-level process monitoring, meaning CLI tools and standalone AI clients (like Claude desktop) often go untracked.
We don't need a bloated Electron app. We need a low-overhead, zero-dependency Python daemon that runs silently in the background, aggregates metrics into a local SQLite database, and exports structured JSON payloads for internal BI tools like Grafana or Metabase.
2. The Architecture: A Privacy-First Telemetry Engine
To ensure zero external telemetry and sub-1% CPU overhead, the architecture must be strictly local and asynchronous. Here is the blueprint:
- The Observer: An
asyncioevent loop using thepsutillibrary to poll active processes every 60 seconds. - The Sanitizer: A Python 3.10
match/caserouter that filters out OS noise and categorizes development tools (e.g., categorizingcode,idea, andnvimunderIDE). - The Aggregator: A local SQLite database running with
PRAGMA journal_mode=WALfor lock-free, concurrent batch writes. - The Exporter: A
Pydantic-powered microservice interface that dumps strictly typed JSON payloads for BI ingestion.
3. The Code & Logic
Let's dive into the core engine. We leverage Python 3.10's structural pattern matching and Pydantic for robust data validation.
The Data Models
First, define the schemas. Using Pydantic guarantees that our local database schemas remain perfectly aligned with the JSON exports expected by our BI tools.
from pydantic import BaseModel, Field
from datetime import datetime
from typing import Optional
class ProcessSnapshot(BaseModel):
timestamp: datetime = Field(default_factory=datetime.utcnow)
process_name: str
category: str
duration_seconds: int = 60
metadata: Optional[dict[str, str]] = None
class AggregatedExport(BaseModel):
date: str
total_ide_minutes: int
total_cli_minutes: int
top_tools: list[str]
The Background Observer
This is the heartbeat of the daemon. We use psutil to fetch active processes. To prevent CPU spikes, we don't track every process switch; we sample the active environment at fixed intervals.
import asyncio
import psutil
import sqlite3
# Standardize tool names using Python 3.10+ pattern matching
def categorize_process(proc_name: str) -> str | None:
match proc_name.lower():
case "code" | "idea" | "pycharm" | "nvim":
return "IDE"
case "iterm2" | "alacritty" | "terminal" | "tmux":
return "CLI"
case "docker" | "podman":
return "Infrastructure"
case _:
return None # Ignore OS noise
async def poll_toolchain_usage(db_path: str):
conn = sqlite3.connect(db_path, isolation_level=None)
conn.execute("PRAGMA journal_mode=WAL")
conn.execute("""
CREATE TABLE IF NOT EXISTS tool_usage (
timestamp TEXT,
process_name TEXT,
category TEXT,
duration_seconds INTEGER
)
""")
while True:
active_tools = []
for proc in psutil.process_iter(['name']):
try:
name = proc.info['name']
category = categorize_process(name)
if category:
active_tools.append((name, category))
except (psutil.NoSuchProcess, psutil.AccessDenied):
continue
# Batch write to SQLite
if active_tools:
cursor = conn.cursor()
timestamp = datetime.utcnow().isoformat()
records = [(timestamp, name, cat, 60) for name, cat in active_tools]
cursor.executemany(
"INSERT INTO tool_usage VALUES (?, ?, ?, ?)",
records
)
# Sleep for the polling interval
await asyncio.sleep(60)
Structured JSON Export
Your internal dashboards need clean data. We run an export job that rolls up SQLite data and validates it via Pydantic before dropping it into a pre-configured export directory.
import json
def export_daily_metrics(db_path: str, export_path: str):
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
# Aggregate today's metrics
cursor.execute("""
SELECT category, SUM(duration_seconds) / 60
FROM tool_usage
WHERE date(timestamp) = date('now')
GROUP BY category
""")
results = cursor.fetchall()
payload = {
"date": datetime.utcnow().strftime("%Y-%m-%d"),
"total_ide_minutes": 0,
"total_cli_minutes": 0,
"top_tools": []
}
for category, minutes in results:
if category == "IDE":
payload["total_ide_minutes"] = minutes
elif category == "CLI":
payload["total_cli_minutes"] = minutes
# Validate and export
export_data = AggregatedExport(**payload)
with open(export_path, "w") as f:
json.dump(export_data.dict(), f, indent=2)
4. Deployment & Performance
Running a background script endlessly requires a few production safeguards:
- Process Management: Do not run this inside a raw
tmuxsession. On Linux, wrap the CLI in asystemdservice withRestart=always. On macOS, use alaunchdplist. - Resource Throttling: Ensure
psutilhandlesAccessDeniedexceptions gracefully. System-level processes will reject polling attempts. Catching these prevents the daemon from crashing. - Data Retention: SQLite files can grow indefinitely. Add a daily cleanup routine via an
asyncioscheduled task toDELETE FROM tool_usage WHERE timestamp < datetime('now', '-30 days').
5. Conclusion & Turnkey Package
By keeping telemetry strictly local, you protect your intellectual property while still gaining critical insights into team productivity, IDE migration trends, and AI tool adoption. You can easily take the architecture and code snippets above and expand them into a fully-fledged monitoring tool.
If you want to skip the boilerplate, avoid OS-specific polling edge-cases, and deploy immediately, you can grab the complete, production-ready codebase. The turnkey package includes the full Python 3.10+ CLI tool, systemd/launchd configuration templates, automated test suites, and pre-built Grafana JSON schemas.
Grab the complete pipeline here:
- Instant Access on Whop: Local Toolchain Analytics Engine
- Direct Download on Gumroad: Get the Source Code (Use promo code **EARLYBIRD* for 20% off)*
Top comments (0)