Building an Agentic Authorization Observer with OpenTelemetry SemConv

What we're building

An observability layer that captures every authorization decision your AI agent makes — trust scores, drift detection, and FGA (Fine-Grained Authorization) context — using the proposed OpenTelemetry semantic conventions for agentic authorization.

Prerequisites

• Node.js 18+
• An existing AI agent (or use the one we'll build)
• TracePilot API key (free at tracepilotai.com)
• Basic understanding of OpenTelemetry spans

Step 1: Set up your agent with authorization hooks

First, let's create a simple agent that makes decisions and checks permissions before executing tools.

npm install openai tracepilot-sdk uuid

// agent.ts
import { TracePilot } from 'tracepilot-sdk';
import OpenAI from 'openai';

const tp = new TracePilot(process.env.TRACEPILOT_API_KEY!);
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// Simulated authorization context
interface AuthContext {
  userRole: string;
  resourceType: string;
  action: string;
  trustScore: number;
  driftScore: number;
}

async function checkAuthorization(auth: AuthContext): Promise<{
  allowed: boolean;
  reason: string;
  fgaDecision: string;
}> {
  // In production, this calls your FGA service (Auth0 FGA, OPA, etc.)
  const fgaDecision = auth.userRole === 'admin' ? 'allow' : 'deny';
  const allowed = fgaDecision === 'allow' && auth.trustScore > 0.7;

  return {
    allowed,
    reason: allowed ? 'Authorized' : `Denied: role=${auth.userRole}, trust=${auth.trustScore}`,
    fgaDecision
  };
}

async function runAgent(userInput: string) {
  await tp.startTrace('authorized-agent');

  const authContext: AuthContext = {
    userRole: 'editor',
    resourceType: 'document',
    action: 'delete',
    trustScore: 0.85,
    driftScore: 0.12
  };

  const { result } = await tp.wrapOpenAI(
    () => openai.chat.completions.create({
      model: 'gpt-4o-mini',
      messages: [{ role: 'user', content: userInput }]
    }),
    [{ role: 'user', content: userInput }]
  );

  return result;
}

Step 2: Add authorization observability with SemConv attributes

Now we wrap every authorization check with structured observability. These attributes follow the proposed agent.trust_score, agent.drift_score, and fga.* conventions.

// authorization-observer.ts
import { TracePilot } from 'tracepilot-sdk';

interface AuthorizationSpan {
  trustScore: number;
  driftScore: number;
  fgaType: string;
  fgaDecision: string;
  fgaReason: string;
  resourceType: string;
  action: string;
  userRole: string;
}

export class AuthorizationObserver {
  private tp: TracePilot;

  constructor(tp: TracePilot) {
    this.tp = tp;
  }

  async observeAuthorization(
    authCheck: () => Promise<{ allowed: boolean; reason: string; fgaDecision: string }>,
    context: AuthorizationSpan
  ): Promise<{ allowed: boolean; reason: string }> {
    // Record authorization decision as a span with SemConv attributes
    const { result, spanId } = await this.tp.wrapToolCall(
      'authorization-check',
      authCheck,
      undefined, // parent span — will be linked when called from agent
      1,         // step order
      false      // not destructive
    );

    // Add custom attributes to the span
    // These follow the proposed SemConv naming convention
    await this.tp.addSpanAttributes(spanId, {
      'agent.trust_score': context.trustScore,
      'agent.drift_score': context.driftScore,
      'fga.type': context.fgaType,
      'fga.decision': result.fgaDecision,
      'fga.reason': context.fgaReason || result.reason,
      'fga.resource_type': context.resourceType,
      'fga.action': context.action,
      'fga.subject_role': context.userRole
    });

    return { allowed: result.allowed, reason: result.reason };
  }
}

Step 3: Wire it all together — agent with authorization observability

// authorized-agent.ts
import { TracePilot } from 'tracepilot-sdk';
import OpenAI from 'openai';
import { AuthorizationObserver } from './authorization-observer';

const tp = new TracePilot(process.env.TRACEPILOT_API_KEY!);
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
const authObserver = new AuthorizationObserver(tp);

interface ToolCall {
  name: string;
  args: Record<string, unknown>;
  userRole: string;
  trustScore: number;
  driftScore: number;
}

async function executeToolWithAuthorization(toolCall: ToolCall) {
  // Step 1: Check authorization
  const authResult = await authObserver.observeAuthorization(
    () => checkAuthorization({
      userRole: toolCall.userRole,
      resourceType: toolCall.name,
      action: 'execute',
      trustScore: toolCall.trustScore,
      driftScore: toolCall.driftScore
    }),
    {
      trustScore: toolCall.trustScore,
      driftScore: toolCall.driftScore,
      fgaType: 'role_based',
      fgaDecision: 'pending',
      fgaReason: '',
      resourceType: toolCall.name,
      action: 'execute',
      userRole: toolCall.userRole
    }
  );

  // Step 2: Execute or deny based on authorization
  if (!authResult.allowed) {
    return { error: authResult.reason, authorized: false };
  }

  // Step 3: Execute the tool (wrapped for observability)
  const { result } = await tp.wrapToolCall(
    toolCall.name,
    () => simulateToolExecution(toolCall),
    undefined,
    2,
    true  // destructive — marks with ⚠ in dashboard
  );

  return { data: result, authorized: true };
}

async function simulateToolExecution(toolCall: ToolCall): Promise<string> {
  // Simulate actual tool execution
  return `Executed ${toolCall.name} with args: ${JSON.stringify(toolCall.args)}`;
}

// Usage
async function main() {
  await tp.startTrace('authorized-agent-run');

  const toolCall: ToolCall = {
    name: 'delete-document',
    args: { documentId: 'doc-123' },
    userRole: 'editor',
    trustScore: 0.85,
    driftScore: 0.12
  };

  const result = await executeToolWithAuthorization(toolCall);
  console.log(result);
  // → { data: "Executed delete-document with args: {\"documentId\":\"doc-123\"}", authorized: true }
}

main();

Adding observability with TracePilot

Here's where TracePilot makes debugging authorization decisions obvious. Install the SDK and add one line to see every authorization check in your dashboard.

npm install tracepilot-sdk

The AuthorizationObserver class we built above already uses TracePilot's wrapToolCall and addSpanAttributes methods. Open your TracePilot Dashboard and you'll see:

• Every authorization decision as a separate span with full context
• Trust and drift scores tracked per decision
• FGA attributes (type, decision, reason, resource, action, subject)
• The full execution tree — which agent step triggered which authorization check

To see a failed authorization in action, change the trust score:

typescript
const failedAuth = await executeToolWithAuthorization({
  name: 'delete-document',
  args: { documentId:

---

**Debugging AI agents shouldn't feel like reading The Matrix.** 
Join other engineers who are building reliable autonomous workflows in our community: [TracePilot Discord](https://discord.gg/KzXRAXFM8)