DEV Community

Clay Roach
Clay Roach

Posted on • Originally published at dev.to

Day 11: AI Analyzer Foundation Complete - Model Selection and Evidence Formatting

#ai #analytics #opentelemetry #typescript

Day 11: AI Analyzer Foundation Complete - Model Selection and Evidence Formatting

The Plan: Return to analytics work with protobuf fixes merged

The Reality: "The AI analyzer just became the core differentiator - model selection and evidence formatting are working beautifully!"

Welcome to Day 11 of building an AI-native observability platform in 30 days. With yesterday's protobuf parsing crisis resolved and PR #24 successfully merged to main, today was all about leveraging that solid foundation to make major progress on AI-powered analytics capabilities.

🎯 Major Accomplishments

Foundation Secured: All 91 Tests Passing ✅

Starting the day with confidence - all core tests passing meant the protobuf attribute extraction fixes were solid and ready to support advanced analytics work.

AI Analyzer Evidence Formatting

Enhanced how AI insights are presented to users with structured evidence:

interface AnalysisEvidence {
  confidence: number
  reasoning: string[]
  supportingData: TelemetryPoint[]
  visualizations: ChartConfig[]
  actionableInsights: string[]
}

export const formatAnalysisEvidence = (analysis: AnalysisResult) => {
  return Effect.gen(function*() {
    const evidence = {
      confidence: analysis.confidenceScore,
      reasoning: yield* extractReasoningChain(analysis),
      supportingData: yield* correlateDataPoints(analysis),
      visualizations: yield* generateVisualizationConfigs(analysis),
      actionableInsights: yield* generateActionableRecommendations(analysis)
    }

    return evidence
  })
}
Enter fullscreen mode Exit fullscreen mode

This isn't just displaying AI results - it's building trust through transparent reasoning and actionable recommendations.

Dynamic Model Selection Integration

Implemented intelligent model selection that routes analysis tasks to optimal AI models:

interface ModelSelectionRequest {
  useCase: 'anomaly-detection' | 'dependency-analysis' | 'critical-path'
  dataComplexity: 'simple' | 'moderate' | 'complex'
  latencyRequirement: 'real-time' | 'batch' | 'flexible'
}

export const selectOptimalModel = (request: ModelSelectionRequest) => {
  return Effect.gen(function*() {
    const modelManager = yield* LLMManager
    const analysis = yield* analyzeDataCharacteristics(request)

    // Intelligent routing based on requirements
    if (analysis.requiresLocalProcessing) {
      return yield* modelManager.getLlamaModel()
    } else if (analysis.requiresAdvancedReasoning) {
      return yield* modelManager.getClaudeModel()
    } else {
      return yield* modelManager.getGPTModel()
    }
  })
}
Enter fullscreen mode Exit fullscreen mode

Different AI models excel at different tasks. This routing ensures optimal performance and cost efficiency.

Critical Path Analysis Integration

The system now automatically identifies critical execution paths and bottlenecks:

interface CriticalPathNode {
  spanId: string
  serviceName: string
  operationName: string
  duration: number
  bottleneckScore: number
  dependencies: string[]
}

export const analyzeCriticalPath = (traceData: TraceSpan[]) => {
  return Effect.gen(function*() {
    // Build dependency graph from trace spans
    const graph = yield* buildDependencyGraph(traceData)

    // Calculate critical path using modified Dijkstra's algorithm
    const criticalPath = yield* calculateCriticalPath(graph)

    // Score bottlenecks based on duration and dependency impact
    const bottlenecks = yield* identifyBottlenecks(criticalPath)

    return {
      path: criticalPath,
      bottlenecks,
      optimizationSuggestions: yield* generateOptimizationSuggestions(bottlenecks)
    }
  })
}
Enter fullscreen mode Exit fullscreen mode

This goes beyond simple trace visualization - it's intelligent analysis that identifies what matters most for performance optimization.

🔧 Technical Deep Dive

The Protobuf Foundation Advantage

Yesterday's recursive protobuf extraction fix enabled today's progress. Clean, reliable data extraction means:

// Instead of fighting nested protobuf objects
const messyData = {
  $typeName: "opentelemetry.proto.common.v1.AnyValue",
  stringValue: "frontend"
}

// We get clean, usable values
const cleanData = "frontend"
Enter fullscreen mode Exit fullscreen mode

The AI analyzer can now process telemetry attributes without worrying about data structure variations between JSON and protobuf ingestion paths.

Enhanced Dependency Discovery

Building on clean attribute extraction, dependency discovery became much more reliable:

export const discoverServiceDependencies = (spans: TraceSpan[]) => {
  return Effect.gen(function*() {
    const dependencies = new Map<string, Set<string>>()

    for (const span of spans) {
      const service = span.serviceName // Now guaranteed to be a string!
      const parentService = span.parentSpan?.serviceName

      if (parentService && service !== parentService) {
        if (!dependencies.has(parentService)) {
          dependencies.set(parentService, new Set())
        }
        dependencies.get(parentService)!.add(service)
      }
    }

    return dependencies
  })
}
Enter fullscreen mode Exit fullscreen mode

Clean data enables reliable service mapping - essential for AI-powered insights about system architecture.

UI Status Column Polish

Completed the UI improvement started yesterday - status codes now display cleanly while preserving full semantic information:

const statusDisplayMap = {
  'STATUS_CODE_UNSET': 'UNSET',
  'STATUS_CODE_OK': 'OK', 
  'STATUS_CODE_ERROR': 'ERROR'
} as const

// Clean display with debugging tooltip
<span 
  title={`Full Status: ${fullStatusCode}`}
  className="status-badge"
>
  {statusDisplayMap[status] || status}
</span>
Enter fullscreen mode Exit fullscreen mode

This maintains OpenTelemetry compliance while providing excellent user experience.

🧠 AI-Native Architecture Insights

Why Model Selection Matters

Different AI models have different strengths for observability tasks:

  • GPT-4: Excellent for anomaly detection and general reasoning about system behavior
  • Claude-3: Superior at complex dependency analysis and architectural understanding
  • Llama-2: Optimal for real-time processing and privacy-sensitive environments
  • Auto-routing: Analyzes data characteristics and selects optimal model

This isn't just feature complexity - it's recognizing that AI-native platforms need AI orchestration, not just AI integration.

Evidence-Based Trust Building

Raw AI insights often lack context. Our evidence formatting addresses the trust gap:

  1. Confidence Scoring: Users understand how certain the AI is about its analysis
  2. Reasoning Chains: Explainable AI shows the logical steps taken
  3. Supporting Data: Visual correlation with actual telemetry builds trust
  4. Actionable Insights: Move beyond "what's wrong" to "what to do about it"

This transforms AI from a black box into a transparent reasoning partner.

Critical Path Intelligence

Traditional observability shows you what happened. AI-native observability understands what matters:

  • Identifies the 20% of operations that impact 80% of performance
  • Scores bottlenecks by actual impact, not just duration
  • Provides optimization suggestions based on system architecture understanding
  • Adapts analysis based on service interaction patterns

📊 Development Velocity Impact

Metrics from Today's Work

  • All 91 tests passing: Foundation remains solid while building advanced features
  • Clean data processing: 100% reliable attribute extraction across JSON and protobuf paths
  • AI model routing: Intelligent selection optimizes both performance and cost
  • Evidence quality: Structured insights with confidence scoring and reasoning chains

Integration Success

The OpenTelemetry demo integration continues working flawlessly:

  • 16 services generating telemetry
  • Clean attribute extraction from all service types
  • Reliable dependency discovery across service boundaries
  • AI analysis working with real production-like data

🚀 Tomorrow's Acceleration Opportunities

With the AI analyzer foundation complete, Day 12 can focus on:

Advanced Analytics Implementation

  • Real-time anomaly detection using the evidence formatting system
  • Pattern recognition across service interactions
  • Dashboard generation with AI-powered insights
  • Foundation for self-healing configuration management

UI Integration Polish

  • Model selection interface for users to choose optimal AI routing
  • Evidence visualization components showing reasoning chains
  • Critical path visualization with interactive bottleneck identification
  • Real-time updates as new telemetry flows through the system

💡 AI-Native Development Philosophy

Today reinforced a key insight: AI-native isn't just adding AI features to traditional systems. It's rethinking what the system does when intelligence is built into the foundation.

Traditional observability: "Here's your data, figure out what it means"

AI-native observability: "Here's what your data means, and here's what to do about it"

The evidence formatting, model selection, and critical path analysis work together to create a system that doesn't just collect telemetry - it understands it and helps users make better decisions.

🔮 The Bigger Picture

We're now at Day 11 of 30, with a solid foundation and sophisticated AI capabilities. The platform is evolving from "observability with AI features" to "AI-native observability platform."

Key differentiators emerging:

  • Multi-model orchestration rather than single AI integration
  • Evidence-based insights rather than raw AI outputs
  • Critical path intelligence rather than simple trace visualization
  • Self-improving analysis rather than static dashboard templates

The next 19 days will focus on polishing these capabilities and adding the UI generation and self-healing configuration features that complete the AI-native vision.

Day 11 Status: AI Analyzer Foundation Complete ✅

Next Challenge: Advanced Analytics and UI Polish

Key Learning: AI-native requires orchestration, not just integration

Part of the "30-Day AI-Native Observability Platform" series. Follow along as we build a complete observability platform using AI-assisted development in just 30 days.

Top comments (0)