DEV Community

Qasim Muhammad
Qasim Muhammad

Posted on

Track Every Action Your AI Agent Takes — Audit Logs, Auth Management & Compliance for CLI Automation

#ai #cli #devops #security

Your AI agent just sent 47 emails. Which ones? To whom? At what time? If you can't answer those questions, you have an accountability gap.

The Nylas CLI has built-in audit logging that records every command execution — timestamps, arguments, exit codes, and who invoked it (human or AI agent). Combined with granular auth management, you get full control over what your automation can access and a complete paper trail of what it did.

Why Audit Logging Matters

SOC 2 compliance requires logging access to customer data. AI agent safety requires tracking autonomous actions. CI/CD debugging requires knowing exactly what commands ran. The CLI's audit system handles all three.

Set Up Audit Logging

Initialize

nylas audit init creates the log directory and default config:

nylas audit init --enable
Enter fullscreen mode Exit fullscreen mode

Configure retention (default 90 days) and max file size (default 50 MB) during init, or set them later.

Enable/Disable Logging

nylas audit logs enable starts recording:

nylas audit logs enable
Enter fullscreen mode Exit fullscreen mode

nylas audit logs disable stops recording (existing logs are preserved):

nylas audit logs disable
Enter fullscreen mode Exit fullscreen mode

nylas audit logs status shows whether logging is active:

nylas audit logs status
Enter fullscreen mode Exit fullscreen mode

View Audit Logs

nylas audit logs show displays recorded activity with powerful filters:

# All recent activity
nylas audit logs show

# Filter by date range
nylas audit logs show --since 2025-04-01 --until 2025-04-10

# Filter by command
nylas audit logs show --command "email send"

# Filter by invoker (who ran the command)
nylas audit logs show --invoker claude-code

# Filter by source
nylas audit logs show --source mcp

# Combine filters for targeted investigation
nylas audit logs show --invoker cursor --command "email send" --since 2025-04-08 --json
Enter fullscreen mode Exit fullscreen mode

The --invoker filter distinguishes between terminal (human), claude-code, cursor, and other AI agents. The --source filter separates cli, mcp, and sdk invocations.

Summarize Activity

nylas audit logs summary aggregates stats:

nylas audit logs summary --days 30 --json
Enter fullscreen mode Exit fullscreen mode

Output includes total commands run, error rates, most-used commands, top invokers (AI vs human), and busiest hours.

Export for Compliance

nylas audit export writes logs to JSON or CSV for external analysis:

# JSON export
nylas audit export --format json -o audit-april.json

# CSV for spreadsheet analysis
nylas audit export --format csv --since 2025-04-01 -o audit-april.csv
Enter fullscreen mode Exit fullscreen mode

Configure Audit Settings

nylas audit config show displays current settings:

nylas audit config show
Enter fullscreen mode Exit fullscreen mode

nylas audit config set changes individual settings:

nylas audit config set retention_days 180
nylas audit config set max_size_mb 100
nylas audit config set rotate_daily true
Enter fullscreen mode Exit fullscreen mode

Clear Logs (Destructive)

nylas audit logs clear permanently deletes all entries. Always export first:

nylas audit export -o backup.json
nylas audit logs clear --force
Enter fullscreen mode Exit fullscreen mode

Authentication Management

Audit logging shows you what happened. Auth management controls what can happen.

Configure API Credentials

nylas auth config stores your API key:

nylas auth config --api-key nyl_abc123
Enter fullscreen mode Exit fullscreen mode

Connect Accounts

nylas auth login authenticates via OAuth:

nylas auth login
Enter fullscreen mode Exit fullscreen mode

View Connected Accounts

nylas auth list shows all grants:

nylas auth list --json
Enter fullscreen mode Exit fullscreen mode

Verify Current State

Switch Between Accounts

nylas auth switch changes the active account:

nylas auth switch grant_abc123
Enter fullscreen mode Exit fullscreen mode

Remove Access

Migrate from v2

nylas auth migrate upgrades stored credentials from API v2 to v3.

Nuclear Option

nylas config reset clears everything — API keys, sessions, grants:

nylas config reset --force
Enter fullscreen mode Exit fullscreen mode

Putting It Together: Secure AI Agent Setup 

# 1. Initialize with audit logging
nylas init
nylas audit init --enable

# 2. Connect only the accounts the agent needs
nylas auth login

# 3. Verify least-privilege access
nylas auth scopes

# 4. Let the agent work...

# 5. Review what it did
nylas audit logs show --invoker claude-code --since today
nylas audit logs summary --days 1
Enter fullscreen mode Exit fullscreen mode

Install the Nylas CLI 

brew install nylas/nylas-cli/nylas
Enter fullscreen mode Exit fullscreen mode

See the getting started guide for other install methods.

Full Command Reference

All audit and auth commands with complete flag documentation: Nylas CLI Command Reference.

Top comments (1)

Collapse
 
ali_muwwakkil_a776a21aa9c profile image
Ali Muwwakkil •

One of the key challenges we see is not just tracking actions but ensuring they're part of a compliant workflow. In our experience with enterprise teams, chatgpt agents often get implemented without robust logging, leading to compliance gaps. It's crucial to integrate these logs directly with your existing audit and auth management systems to ensure end-to-end traceability. This doesn't just help with compliance -it enhances team trust in AI outputs. - Ali Muwwakkil (ali-muwwakkil on LinkedIn)