DEV Community

Siong Yuen Cheah
Siong Yuen Cheah

Posted on

Your AI Agent Is Flying Blind. Here's How to Fix It.

tags: mcp, ai, devtools, webdev,

You've spent months building the perfect AI coding assistant. It writes clean TypeScript, refactors legacy code, generates tests. But ask it something simple like "Is my project healthy right now?" and watch it stumble.

The agent fires off a GitHub MCP call. Gets back a wall of JSON. Fires off another. Different schema. By call five, it's lost context. By call ten, you've burned through your context window and still don't know if you should deploy.

This is the dirty secret of AI-assisted development: agents are blind to project health. They can manipulate code but cannot perceive the living system behind it.

I built PulseTel to solve exactly this.


The Problem: Raw APIs vs. Agent-Ready Intelligence

Let's be concrete. Say you want to know if your project is safe to deploy right now. Here's what happens with raw MCP tools:

The Raw GitHub MCP Approach

# Call 1: Check CI status
$ mcp_github_get_workflow_run --repo myapp --run-id 12345
{"id": 12345, "status": "completed", "conclusion": "failure"}

# Call 2: Check for open issues
$ mcp_github_search_issues --repo myapp --query "is:open label:bug"
{"total_count": 47, "items": [...]}

# Call 3: Check PRs
$ mcp_github_list_pull_requests --repo myapp --state open
[{"title": "Fix auth bug"}, {"title": "WIP: refactor"}]

# Call 4: Check dependencies (different tool entirely)
$ mcp_datadog_dependency_metrics --service myapp
{"error": "API key not configured"}

# Call 5: Realize you need deployment status
$ kubectl get pods -n production
error: Unable to connect to the server: Forbidden
Enter fullscreen mode Exit fullscreen mode

Five calls. Three different authentication methods. Inconsistent schemas. No prioritization. The agent has a pile of data but zero insight into what actually matters.

The PulseTel Approach

$ npx pulsetel-cli check

┌─────────────────────────────────────────────────────────────┐
│  PulseTel Project Health Check                              │
│  myapp | main branch | Last checked: 2m ago                  │
├─────────────────────────────────────────────────────────────┤
│  🔴 CRITICAL (1)                                             │
│  ├─ CI/CD: Build failing on main (Node 20 incompatibility)   │
│  │   └─ Action: Pin Node version in .nvmrc                    │
│  │                                                            │
│  🟡 WARNINGS (2)                                             │
│  ├─ Dependencies: 3 high-severity vulnerabilities            │
│  ├─ Health: API latency p95 degraded (420ms → 680ms)         │
│  │                                                            │
│  🟢 HEALTHY (5)                                              │
│  ├─ Deploy: Last successful 3h ago                           │
│  ├─ Issues: 2 open (down from 8 last week)                   │
│  ├─ PRs: 1 ready for review                                  │
│  ├─ Coverage: 84% (stable)                                   │
│  └─ Git: No uncommitted changes on main                      │
└─────────────────────────────────────────────────────────────┘
Enter fullscreen mode Exit fullscreen mode

One command. Eight dimensions of health. Prioritized actions. Exit codes for CI/CD gating.

This is what agent-native telemetry looks like.


How PulseTel Works

PulseTel is an MCP server that speaks "project health." It aggregates 8 check modules:

Module What It Checks Data Sources
CI Workflow runs, flaky tests, build duration GitHub Actions
Deploy Deployment status, rollback candidates Webhooks
Health Endpoint latency, error rates, availability HTTP probes
Dependencies Vulnerabilities, outdated packages npm audit, OSV
Coverage Test coverage trends lcov, Codecov
Issues Bug backlog, critical open issues GitHub
Pull Requests Merge queue, review latency GitHub
Git Commit hygiene, branch divergence Local git

Every check returns a structured response with:

  • status: success | warning | error | critical
  • severity: low | medium | high
  • confidence: 0.0–1.0 based on data freshness
  • actionable: specific recommendation if needed
  • duration_ms: how long the check took

The Killer Feature: pulsetel_recommend

Here's where PulseTel gets interesting. Instead of making the agent sift through check results, you ask for recommendations via the pulsetel_recommend MCP tool:

{
  "recommendations": [
    {
      "rank": 1,
      "severity": "critical",
      "confidence": 0.94,
      "check": "ci",
      "finding": "Build failing on main branch for 47 minutes",
      "impact": "Blocking all deployments, 3 PRs queued",
      "action": "Pin Node.js to 18.19.0 in .nvmrc",
      "estimated_time": "5 minutes"
    },
    {
      "rank": 2,
      "severity": "warning",
      "confidence": 0.87,
      "check": "dependencies",
      "finding": "lodash@4.17.20 has prototype pollution (CVE-2021-23337)",
      "action": "Run 'pulsetel fix --deps' or upgrade to lodash@4.17.21",
      "estimated_time": "2 minutes"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

Rank 1 is what you fix right now. The agent doesn't need to reason about what matters — PulseTel already did.


16 MCP Tools for Agent Integration

PulseTel exposes 15 MCP tools via stdio transport — compatible with Claude Desktop, Cursor, and any MCP-compatible client:

Tool Purpose Speed
pulsetel_check Full health assessment ~10s
pulsetel_quick High-signal subset (skips deps/coverage) ~1-2s
pulsetel_ci CI/CD status only ~2s
pulsetel_health Endpoint health only ~1s
pulsetel_deps Dependency status ~5s
pulsetel_summary One-line status summary <10ms
pulsetel_trends Historical trends ~1s
pulsetel_anomalies Statistical anomalies (2σ detection) ~1s
pulsetel_telemetry OpenTelemetry config status <100ms
pulsetel_recommend Prioritized action list ~1s
pulsetel_status Lightweight health ping <10ms
pulsetel_sentry Sentry error tracking ~2s
pulsetel_guidance Agent reasoning assistance ~500ms
pulsetel_diff Change intelligence (run vs. previous) ~1s
pulsetel_guard Pre/post checks around any command ~15s

Add to Claude Desktop:

{
  "mcpServers": {
    "pulsetel": {
      "command": "npx",
      "args": ["-y", "pulsetel-cli", "mcp-stdio"]
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Then ask: "Should I deploy right now?"


Guard: Drift Detection for Any Command

The pulsetel_guard tool runs pre-check and post-check around any shell command and reports drift:

$ pulsetel guard -- npm install

Running pre-checks...
Running: npm install
Running post-checks...

Drift detected! 3 checks changed (max 100.0% change)

Changed checks:
  - deps.vulnerabilities: before 0, after 2 (+2 new)
  - coverage: before 84%, after 79% (-5%)
  - health.api_latency: before 120ms, after 340ms (+183%)
Enter fullscreen mode Exit fullscreen mode

The agent now knows exactly what a command broke — no need to infer from separate API calls.


Trend Analysis & Anomaly Detection

PulseTel tracks health over time and detects anomalies statistically (2σ from rolling mean):

$ npx pulsetel-cli trends

📊 7-Day Trends

CI Success Rate
████████████████████░░░░ 89% (-4%) ↘ degrading

API Latency p95
████████████████████░░░ 680ms (+260ms) ↗ ANOMALY

Test Coverage
████████████████████░░░ 84% (+1%) → stable

Dependency Vulnerabilities
██░░░░░░░░░░░░░░░░░░░░░ 3 → stable
Enter fullscreen mode Exit fullscreen mode

No arbitrary thresholds — a spike in latency for a high-variance service is treated differently than the same spike for a stable one.


OpenTelemetry Integration

PulseTel v1.2+ exports project health as OTel-compliant telemetry:

  • Traces: Each check run emits a pulsetel.check root span with child spans per module
  • Metrics: Health scores, anomaly counts, vulnerability counters as gauges and counters
  • Logs: Anomaly events, degrading trends as structured log records
# One-shot OTel export
pulsetel check --otel

# Auto-enable in .pulsetel.yml
# otel:
#   enabled: true
#   endpoint: http://localhost:4318
#   protocol: http
Enter fullscreen mode Exit fullscreen mode

Works with OTLP HTTP (port 4318) and file export (.pulsetel/otel/) for offline ingestion.


Sentry Error Tracking

Configure once in .pulsetel.yml:

sentry:
  organization: my-org
  project: my-project
  # token via SENTRY_TOKEN env var
Enter fullscreen mode Exit fullscreen mode

pulsetel_sentry returns:

  • Unresolved issue counts by severity
  • Top 5 issues by frequency with direct links
  • Affected user counts
  • Release attribution

For AI agents: one call instead of navigating the Sentry dashboard.


Security You Can Verify

  • SSRF protection: IPv4/IPv6 blocklists, DNS rebinding prevention
  • No shell injection: All child_process calls use execFileSync (no shell)
  • No token leaks: Secrets never logged, always via env vars
  • Structured exit codes: 0=healthy, 1=critical, 2=warnings, 3=partial

Quick Start

# Fast triage (1-2s)
npx pulsetel-cli quick

# Full check with all modules (~10s)
npx pulsetel-cli check

# Automated dependency fixes
npx pulsetel-cli fix --deps

# Start MCP server for AI agent integration
npx pulsetel-cli mcp-stdio
Enter fullscreen mode Exit fullscreen mode

Configuration is minimal. PulseTel auto-detects Git remote, package.json, .github/workflows.

For custom endpoints, add a .pulsetel.yml:

github:
  repo: owner/repo

health:
  endpoints:
    - name: API
      url: https://api.myapp.com/health

webhooks:
  - url: https://hooks.slack.com/services/T00/B00/XXX
    events: [anomaly, degrading, flaky, critical]
Enter fullscreen mode Exit fullscreen mode

Why I Built This

I was tired of asking Claude "is it safe to deploy?" and watching it make 6 API calls, lose context, and hallucinate a status summary.

AI agents need perception. They need to see the system state, understand what's broken, and know what to fix first. Raw APIs give them eyes but no vision. PulseTel gives them both.

Every response carries a schema contract (schema_version, schema_url) so consumers can validate without guessing. The architecture is modular — add a check module, wire in a new data source. It's built to be extended.


Try It

npx pulsetel-cli check
Enter fullscreen mode Exit fullscreen mode

Or star the repo:

GitHub: github.com/siongyuen/pulsetel
npm: npm install pulsetel-cli
Website: sycheah.com/pulsetel.html

Issues and PRs welcome. MIT licensed. Your AI agent has been flying blind long enough — give it eyes.

Top comments (0)