DEV Community

Qasim Muhammad
Qasim Muhammad

Posted on • Originally published at cli.nylas.com

Track Every Action Your AI Agent Takes — Audit Logs for Claude Code, Copilot, and MCP

#security #ai #devops #tutorial

AI agents can send emails, create calendar events, and read contacts on your behalf. But what did they actually do? Without logging, an agent session is a black box.

Nylas CLI has built-in audit logging that tracks every command Claude Code, GitHub Copilot, and MCP agents execute — with automatic source detection, per-session filtering, and compliance exports.

Why audit AI agents?

If you gave your AI coding agent email access via MCP, it can now send emails, read your inbox, and create calendar events autonomously. You need a record of what happened:

  • Security review — which emails did the agent read during that coding session?
  • Compliance — SOC 2 and HIPAA require evidence of who accessed what, including automated agents
  • Debugging — trace a failed pipeline back to the exact CLI command, grant, and API request
  • Anomaly detection — catch spikes in error rates or commands running outside business hours

Enable audit logging in one command 

nylas audit init --enable
Enter fullscreen mode Exit fullscreen mode

That creates the log directory, writes a default config, and starts recording. Verify with:

nylas audit logs status
Enter fullscreen mode Exit fullscreen mode 
Audit logging: enabled
Log path:      ~/.nylas/audit/
Retention:     90 days
Max size:      50 MB
Format:        json
Enter fullscreen mode Exit fullscreen mode

Every command from this point forward is logged — whether you ran it manually or an AI agent ran it via MCP.

What gets logged

Each command produces a structured JSON entry:

{
  "timestamp": "2026-03-04T14:32:01Z",
  "command": "nylas email send",
  "grant_id": "grant_abc123",
  "invoker": "qasim",
  "source": "claude-code",
  "request_id": "req_xyz789",
  "status": "success",
  "duration_ms": 1240
}
Enter fullscreen mode Exit fullscreen mode

The source field is set automatically — no configuration needed. The CLI detects Claude Code, GitHub Copilot, GitHub Actions, and MCP agents from environment variables. Custom agents can set NYLAS_CLI_SOURCE to identify themselves.

Sensitive data like API keys, tokens, and email bodies are automatically redacted.

Filter by agent

See everything Claude Code did today:

nylas audit logs show --source claude-code --since "2026-03-23"
Enter fullscreen mode Exit fullscreen mode

Review a specific MCP session window:

nylas audit logs show --source mcp --since "2026-03-23T10:00:00" --until "2026-03-23T12:00:00"
Enter fullscreen mode Exit fullscreen mode

Show only errors from CI:

nylas audit logs show --source github-actions --status error --since "2026-03-01"
Enter fullscreen mode Exit fullscreen mode

Pipe to jq for analysis:

nylas audit logs show --json -n 100 | jq '.[] | select(.duration_ms > 5000)'
Enter fullscreen mode Exit fullscreen mode

Summary statistics

Get a high-level view:

nylas audit logs summary --days 30
Enter fullscreen mode Exit fullscreen mode 
Audit Log Summary (last 30 days)
─────────────────────────────────
Total commands:    1,247
Success rate:      98.6%
Unique invokers:   4

Top commands:
  email send         412  (33.0%)
  email list         298  (23.9%)
  calendar list      187  (15.0%)

By source:
  terminal           687  (55.1%)
  claude-code        312  (25.0%)
  github-actions     198  (15.9%)
  mcp                 50  (4.0%)
Enter fullscreen mode Exit fullscreen mode

Export for compliance

CSV for auditors or JSON for SIEM ingestion into Splunk, Datadog, or any system that accepts structured logs:

# CSV for quarterly audit
nylas audit export --format csv --since "2026-01-01" --until "2026-03-31" --output Q1-audit.csv

# JSON for SIEM
nylas audit export --format json --since "2026-03-01" --output march-audit.json
Enter fullscreen mode Exit fullscreen mode

Add to CI/CD

Run nylas audit init --enable --no-prompt early in your pipeline and every subsequent CLI command is logged. Export as a build artifact:

# .github/workflows/sync.yml
steps:
  - name: Enable audit logging
    run: nylas audit init --enable --no-prompt

  - name: Sync calendars
    run: nylas calendar list --grant-id $GRANT_ID

  - name: Export audit log
    run: nylas audit export --format json --output audit.json

  - name: Upload audit artifact
    uses: actions/upload-artifact@v4
    with:
      name: audit-log
      path: audit.json
Enter fullscreen mode Exit fullscreen mode

Configuration 

# Set retention to 180 days
nylas audit config set retention 180

# Set max log size
nylas audit config set max_size 100

# Change log directory
nylas audit config set path /var/log/nylas-audit
Enter fullscreen mode Exit fullscreen mode

FAQ

Does audit logging affect performance?
No. Logs are written asynchronously. Overhead is under 1ms per command.

What agents are detected automatically?
Claude Code, GitHub Copilot, GitHub Actions, and any MCP-based agent. Custom agents set NYLAS_CLI_SOURCE.

Are email bodies logged?
No. Only the command name, flags, grant ID, status, and timing. Sensitive data is redacted automatically.

Full guide with monitoring patterns and weekly health checks: AI Agent Audit Logs with Nylas CLI

Set up email access for your agent first: Give Your AI Coding Agent an Email Address

More guides: cli.nylas.com/guides

Top comments (0)