<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Anand Kannan</title>
    <description>The latest articles on DEV Community by Anand Kannan (@anandindia93).</description>
    <link>https://dev.to/anandindia93</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F667776%2F3a968b03-ac1b-4aed-9a9a-e5979385c591.png</url>
      <title>DEV Community: Anand Kannan</title>
      <link>https://dev.to/anandindia93</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/anandindia93"/>
    <language>en</language>
    <item>
      <title>MCP is the Past, UTCP &amp; Code Mode are the Now: An SRE's Guide to AI Tool Calling</title>
      <dc:creator>Anand Kannan</dc:creator>
      <pubDate>Sat, 31 Oct 2026 10:24:59 +0000</pubDate>
      <link>https://dev.to/anandindia93/mcp-vs-code-mode-for-sres-from-individual-tool-calls-to-intelligent-incident-investigation-17aj</link>
      <guid>https://dev.to/anandindia93/mcp-vs-code-mode-for-sres-from-individual-tool-calls-to-intelligent-incident-investigation-17aj</guid>
      <description>&lt;p&gt;Imagine you're on call. Your application suddenly starts returning HTTP 500 errors.&lt;/p&gt;

&lt;p&gt;You have &lt;strong&gt;Prometheus&lt;/strong&gt; for metrics, &lt;strong&gt;Loki&lt;/strong&gt; for logs, &lt;strong&gt;Kubernetes&lt;/strong&gt; for workload information, and &lt;strong&gt;GitHub&lt;/strong&gt; for deployments.&lt;/p&gt;

&lt;p&gt;You ask an LLM:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;"Why did our application start returning 5xx errors around 10:30 AM?"&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;To answer this accurately, the LLM needs to investigate several things:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Query Prometheus to identify when the error rate increased.&lt;/li&gt;
&lt;li&gt;Check whether request volume or latency changed.&lt;/li&gt;
&lt;li&gt;Search Loki for corresponding application errors around that exact time window.&lt;/li&gt;
&lt;li&gt;Inspect Kubernetes pods and recent restarts.&lt;/li&gt;
&lt;li&gt;Correlate the findings and identify a probable root cause.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This is where three concepts become critical for Site Reliability Engineers: &lt;strong&gt;Model Context Protocol (MCP)&lt;/strong&gt;, &lt;strong&gt;Universal Tool Calling Protocol (UTCP)&lt;/strong&gt;, and &lt;strong&gt;Code Mode&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;They are often conflated in discussions, but they operate at two completely different layers of the stack:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;The Tool Interface Layer (MCP vs. UTCP):&lt;/strong&gt; How an LLM discovers and invokes external operational tools (via intermediate wrapper servers with MCP, or direct native calls via declarative manuals with UTCP).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The Orchestration Layer (Standard Tool Calling vs. Code Mode):&lt;/strong&gt; Whether the LLM invokes tools one by one in a conversational loop, or writes and executes code in a sandbox to orchestrate those tools and filter the data.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Importantly, &lt;strong&gt;Code Mode is an orchestration pattern, not a competing protocol&lt;/strong&gt;—it can run on top of either MCP tools or UTCP manuals underneath.&lt;/p&gt;

&lt;p&gt;Let's explore how these work in practice through a real-world SRE incident investigation, complete with Python implementations for each pattern.&lt;/p&gt;




&lt;h2&gt;
  
  
  1. Understanding MCP (The Standard Proxy Approach)
&lt;/h2&gt;

&lt;p&gt;The &lt;strong&gt;Model Context Protocol (MCP)&lt;/strong&gt; is an open standard that allows LLM applications to connect to external tools and data sources through a consistent client-server interface.&lt;/p&gt;

&lt;p&gt;Without a standard protocol, every integration requires custom glue code. Prometheus, Loki, Kubernetes, and GitHub each expose different APIs and authentication mechanisms. MCP provides a unified way to wrap and expose these capabilities to an LLM.&lt;/p&gt;

&lt;p&gt;For example, an SRE monitoring MCP server could expose tools such as:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;query_prometheus
query_loki
get_pod_status
get_recent_deployments
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The LLM discovers these tools, reads their descriptions and input schemas, and invokes them when needed.&lt;/p&gt;

&lt;h3&gt;
  
  
  How MCP Works
&lt;/h3&gt;



&lt;pre data-lang="mermaid"&gt;&lt;code&gt;flowchart TD
    User([On-Call SRE]) --&amp;gt; LLM[LLM Client]
    LLM --&amp;gt; MCP_Client[MCP Client]
    MCP_Client --&amp;gt;|JSON-RPC| MCP_Server[MCP Server Wrapper]
    MCP_Server --&amp;gt; Prom[(Prometheus API)]
    MCP_Server --&amp;gt; Loki[(Grafana Loki API)]
    MCP_Server --&amp;gt; K8s[(Kubernetes API)]&lt;/code&gt;&lt;/pre&gt;



&lt;p&gt;The MCP server acts as a middleman that handles the integration with the underlying systems. The LLM does not need to know every API endpoint or authentication detail.&lt;/p&gt;

&lt;h3&gt;
  
  
  A Simple Python MCP Server
&lt;/h3&gt;

&lt;p&gt;The following example uses the Python MCP SDK (&lt;code&gt;FastMCP&lt;/code&gt;) to expose a Prometheus query tool.&lt;/p&gt;

&lt;p&gt;Install the SDK and HTTP client:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="s2"&gt;"mcp[cli]"&lt;/span&gt; httpx
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Implementation:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;httpx&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;mcp.server.fastmcp&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;FastMCP&lt;/span&gt;

&lt;span class="n"&gt;mcp&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;FastMCP&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;sre-monitoring&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;PROMETHEUS_URL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;PROMETHEUS_URL&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="n"&gt;PROMETHEUS_TOKEN&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;PROMETHEUS_TOKEN&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;


&lt;span class="nd"&gt;@mcp.tool&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;query_prometheus&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;Execute an instant Prometheus query.&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;

    &lt;span class="n"&gt;headers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Authorization&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Bearer &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;PROMETHEUS_TOKEN&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="n"&gt;httpx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;AsyncClient&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;15&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;PROMETHEUS_URL&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;/api/v1/query&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;params&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;query&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
            &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;

        &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="n"&gt;payload&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;data&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;


&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;__name__&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;__main__&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;mcp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;run&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;transport&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;streamable-http&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You could implement similar tools for Loki, Kubernetes, and GitHub. In production, you must add authentication, authorization, query restrictions, timeouts, rate limits, and response-size limits to avoid exposing unrestricted access to infrastructure APIs.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The SRE Trade-off with MCP:&lt;/strong&gt; While MCP standardizes tool access, it introduces two operational challenges at scale:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Infrastructure Toil:&lt;/strong&gt; You have to deploy, monitor, and maintain intermediate MCP server processes for APIs that already exist.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Sequential Orchestration &amp;amp; Schema Bloat:&lt;/strong&gt; MCP preloads tool schemas into the LLM context and, by default, the LLM calls tools one at a time—pulling huge raw responses back into its context window.&lt;/li&gt;
&lt;/ol&gt;




&lt;h2&gt;
  
  
  2. Enter UTCP (The Direct "Manual" Alternative to MCP)
&lt;/h2&gt;

&lt;p&gt;Before looking at orchestration (Code Mode), let's look at how &lt;strong&gt;Universal Tool Calling Protocol (UTCP)&lt;/strong&gt; addresses the infrastructure and discovery side of MCP.&lt;/p&gt;

&lt;p&gt;Where MCP acts as a &lt;strong&gt;middleman server&lt;/strong&gt; in the data plane, UTCP acts as a &lt;strong&gt;declarative manual&lt;/strong&gt; in the control plane. Its core premise is simple: &lt;em&gt;if your internal systems already have HTTP APIs, OpenAPI specs, or CLIs (&lt;code&gt;kubectl&lt;/code&gt;, &lt;code&gt;promtool&lt;/code&gt;), you shouldn't need to deploy and maintain a wrapper server just so an AI agent can call them.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Instead of running a Python server process like &lt;code&gt;FastMCP&lt;/code&gt;, UTCP uses a static JSON manual (&lt;code&gt;sre_manual.json&lt;/code&gt;) that tells the UTCP Client how to call Prometheus, Loki, or &lt;code&gt;kubectl&lt;/code&gt; directly using native protocols and local environment variables.&lt;/p&gt;

&lt;h3&gt;
  
  
  How UTCP Works
&lt;/h3&gt;



&lt;pre data-lang="mermaid"&gt;&lt;code&gt;flowchart TD
    User([On-Call SRE]) --&amp;gt; LLM[LLM / Agent]
    LLM --&amp;gt; UTCP_Client[UTCP Client]
    UTCP_Client -.-&amp;gt;|Reads Static JSON Spec| Manual[UTCP Manual / OpenAPI]
    UTCP_Client --&amp;gt;|Direct HTTP| Prom[(Prometheus API)]
    UTCP_Client --&amp;gt;|Direct HTTP| Loki[(Grafana Loki API)]
    UTCP_Client --&amp;gt;|Direct CLI / HTTP| K8s[(Kubernetes API / kubectl)]&lt;/code&gt;&lt;/pre&gt;



&lt;h3&gt;
  
  
  Example UTCP Manual (&lt;code&gt;sre_manual.json&lt;/code&gt;)
&lt;/h3&gt;

&lt;p&gt;Instead of writing and hosting the &lt;code&gt;FastMCP&lt;/code&gt; server above, the exact same Prometheus, Loki, and Kubernetes tools in UTCP are defined in a single declarative JSON file:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"manual_version"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"1.0.0"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"utcp_version"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"1.0.0"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"tools"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"query_prometheus"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"description"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Execute a Prometheus range query for metrics."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"tags"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"prometheus"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"metrics"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"error_rate"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"inputs"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"object"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"properties"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"query"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"string"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"description"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"PromQL expression"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"start"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"number"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"description"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Start unix timestamp"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"end"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"number"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"description"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"End unix timestamp"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"required"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"query"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"start"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"end"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"tool_call_template"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"call_template_type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"http"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"http_method"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"GET"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"url"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"${PROMETHEUS_URL}/api/v1/query_range"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"query_params"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"query"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"{query}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"start"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"{start}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"end"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"{end}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"step"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"60s"&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"auth"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"auth_type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"api_key"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"api_key"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Bearer ${PROMETHEUS_TOKEN}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"var_name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Authorization"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"location"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"header"&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"query_loki"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"description"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Query Grafana Loki for application logs in a time window."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"tags"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"loki"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"logs"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"errors"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"inputs"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"object"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"properties"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"query"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"string"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"description"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"LogQL query"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"start"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"number"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"description"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Start unix timestamp"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"end"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"number"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"description"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"End unix timestamp"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"required"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"query"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"start"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"end"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"tool_call_template"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"call_template_type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"http"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"http_method"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"GET"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"url"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"${LOKI_URL}/loki/api/v1/query_range"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"query_params"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"query"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"{query}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"start"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"{start}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"end"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"{end}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"limit"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"2000"&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"get_pod_status"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"description"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Get Kubernetes pod status and restart counts via kubectl CLI."&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"tags"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"kubernetes"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"k8s"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"pods"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"restarts"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"inputs"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"object"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"properties"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"namespace"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"string"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"deployment"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"string"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"required"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"namespace"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"deployment"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"tool_call_template"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"call_template_type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"cli"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"commands"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
            &lt;/span&gt;&lt;span class="nl"&gt;"command"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"kubectl get pods -n {namespace} -l app={deployment} -o json"&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;With UTCP, there is &lt;strong&gt;zero extra server infrastructure&lt;/strong&gt; to deploy or get paged for at 3 AM. Furthermore, the UTCP client includes native progressive discovery (&lt;code&gt;client.search_tools()&lt;/code&gt;), so you don't have to preload all 100 tool schemas into the LLM prompt on Turn 1.&lt;/p&gt;




&lt;h2&gt;
  
  
  3. What is Code Mode?
&lt;/h2&gt;

&lt;p&gt;With a conventional tool-calling approach, the LLM invokes tools individually in a conversational "ping-pong" loop:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Call Prometheus → Read raw result in LLM context.&lt;/li&gt;
&lt;li&gt;Decide to call Loki → Read thousands of log lines in LLM context.&lt;/li&gt;
&lt;li&gt;Decide to inspect Kubernetes → Read pod statuses in LLM context.&lt;/li&gt;
&lt;li&gt;Correlate everything and produce an answer.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;This works for simple checks, but during an incident, the LLM makes multiple sequential round-trips and ingests megabytes of intermediate telemetry data.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Code Mode takes a programmatic approach.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Instead of asking the LLM to orchestrate every individual tool call through conversation turns, Code Mode allows the LLM to &lt;strong&gt;write a small program that performs the orchestration inside an isolated execution runtime&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;For example, the LLM generates Python code that:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Queries Prometheus for the error rate.&lt;/li&gt;
&lt;li&gt;Programmatically identifies the exact timestamp when the error rate crossed the threshold.&lt;/li&gt;
&lt;li&gt;Searches Loki &lt;em&gt;only&lt;/em&gt; for that narrow time window.&lt;/li&gt;
&lt;li&gt;Groups matching errors by type in memory.&lt;/li&gt;
&lt;li&gt;Retrieves relevant Kubernetes pod statuses.&lt;/li&gt;
&lt;li&gt;Returns a compact JSON summary back to the LLM.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  How Code Mode Works
&lt;/h3&gt;



&lt;pre data-lang="mermaid"&gt;&lt;code&gt;flowchart TD
    User([On-Call SRE]) --&amp;gt; LLM[LLM]
    LLM --&amp;gt;|1. Search Needed Tools Only| Discovery[Progressive Tool Discovery]
    Discovery --&amp;gt;|Returns 3 Compact Stubs| LLM
    LLM --&amp;gt;|2. Generates Orchestration Script| Runtime[Isolated Python Execution Runtime]
    subgraph Sandbox [In-Memory Data Processing &amp;amp; Filtering]
        Runtime --&amp;gt;|3a. Direct HTTP| Prom[(Prometheus)]
        Runtime --&amp;gt;|3b. Direct HTTP| Loki[(Loki)]
        Runtime --&amp;gt;|3c. Direct CLI| K8s[(Kubernetes)]
        Prom &amp;amp; Loki &amp;amp; K8s --&amp;gt; Correlate[Process, Filter &amp;amp; Aggregate in Python]
    end
    Correlate --&amp;gt;|4. Return Compact Summary Only| LLM
    LLM --&amp;gt; Final([Final Root Cause Analysis])&lt;/code&gt;&lt;/pre&gt;



&lt;p&gt;&lt;strong&gt;Important architectural distinction:&lt;/strong&gt; Code Mode is an execution pattern, not a replacement for tool definitions. Code Mode can call &lt;strong&gt;MCP tools&lt;/strong&gt; through a bridge, or it can pair with &lt;strong&gt;UTCP&lt;/strong&gt;—where the agent dynamically searches the UTCP manual for only the tools it needs and executes the chained calls directly inside the sandbox.&lt;/p&gt;




&lt;h2&gt;
  
  
  4. Comparing the Approaches: MCP vs. UTCP vs. Code Mode
&lt;/h2&gt;

&lt;p&gt;To keep the mental model clean, here is how traditional tool calling compares to Code Mode, and where MCP and UTCP fit:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Area&lt;/th&gt;
&lt;th&gt;Traditional Tool Calling (MCP)&lt;/th&gt;
&lt;th&gt;Code Mode + MCP&lt;/th&gt;
&lt;th&gt;Code Mode + UTCP&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Main Purpose&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Expose &amp;amp; call individual tools via proxy&lt;/td&gt;
&lt;td&gt;Orchestrate MCP tools using code&lt;/td&gt;
&lt;td&gt;Discover &amp;amp; orchestrate native APIs/CLIs using code&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Infra Footprint&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Requires running MCP wrapper servers&lt;/td&gt;
&lt;td&gt;MCP servers + Code execution sandbox&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;0 wrapper servers&lt;/strong&gt; (JSON manual) + Sandbox&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Upfront Schema Cost&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;High (All tool schemas loaded on Turn 1)&lt;/td&gt;
&lt;td&gt;Medium/High (Unless behind a search bridge)&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;Ultra-Low&lt;/strong&gt; (&lt;code&gt;search_tools&lt;/code&gt; loads only matching stubs)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Execution Flow&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;N sequential LLM round-trips&lt;/td&gt;
&lt;td&gt;1 generated script executes multiple calls&lt;/td&gt;
&lt;td&gt;1 generated script executes direct native calls&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Intermediate Data&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Enters and compounds in LLM context&lt;/td&gt;
&lt;td&gt;Stays inside Python runtime memory&lt;/td&gt;
&lt;td&gt;Stays inside Python runtime memory&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Protocols&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;JSON-RPC over stdio / HTTP / SSE&lt;/td&gt;
&lt;td&gt;JSON-RPC to MCP server&lt;/td&gt;
&lt;td&gt;Native HTTP, gRPC, CLI, WebSockets, OpenAPI&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A useful rule of thumb for SREs:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;MCP&lt;/strong&gt; standardizes tool access via &lt;strong&gt;proxy servers&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;UTCP&lt;/strong&gt; standardizes tool access via &lt;strong&gt;direct manuals&lt;/strong&gt; (zero proxy servers + native &lt;code&gt;search_tools&lt;/code&gt; discovery).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Code Mode&lt;/strong&gt; lets the LLM &lt;strong&gt;compose those tools programmatically&lt;/strong&gt; so raw telemetry never floods your context window.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  5. Real-World SRE Example: Investigating a 5xx Incident (3 Ways)
&lt;/h2&gt;

&lt;p&gt;Let's walk through our 10:30 AM HTTP 500 outage.&lt;/p&gt;

&lt;p&gt;The on-call engineer asks:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;"Investigate why the 5xx error rate increased around 10:30 AM. Check Prometheus, Loki, and Kubernetes. Identify the most likely cause."&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Let's compare how this exact investigation looks across &lt;strong&gt;Approach A (Individual MCP Calls)&lt;/strong&gt;, &lt;strong&gt;Approach B (Code Mode over MCP)&lt;/strong&gt;, and &lt;strong&gt;Approach C (UTCP + Code Mode in Python)&lt;/strong&gt;.&lt;/p&gt;




&lt;h3&gt;
  
  
  Approach A: Individual Tool Calls (The Conversational Loop)
&lt;/h3&gt;

&lt;p&gt;The LLM executes the investigation across four separate conversational turns:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Turn 1 (Prometheus):&lt;/strong&gt; LLM calls &lt;code&gt;query_prometheus&lt;/code&gt;. The raw time-series array (500 data points) is dumped into the LLM context window. The LLM reads the points and sees the &lt;code&gt;5xx&lt;/code&gt; error rate jumped from &lt;code&gt;0.5%&lt;/code&gt; to &lt;code&gt;18%&lt;/code&gt; at &lt;code&gt;10:30 AM&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Turn 2 (Loki):&lt;/strong&gt; LLM calls &lt;code&gt;query_loki&lt;/code&gt;. Loki returns 2,000 matching log lines (~80,000 tokens) directly into the LLM context window.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Turn 3 (Kubernetes):&lt;/strong&gt; LLM calls &lt;code&gt;get_pod_status&lt;/code&gt; and receives full pod manifests (~15,000 tokens) in context, along with the previous 95,000 tokens of conversation history.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Turn 4 (Correlation):&lt;/strong&gt; The LLM reasons across the massive accumulated context history and outputs its conclusion.&lt;/li&gt;
&lt;/ul&gt;




&lt;h3&gt;
  
  
  Approach B: Code Mode with Python (Conceptual Runtime Bridge)
&lt;/h3&gt;

&lt;p&gt;Now imagine the LLM generates Python code and executes it in an isolated runtime. Instead of making 4 conversational turns and pulling 110,000 tokens of raw telemetry into the prompt, the LLM writes a single orchestration function that processes the data in Python memory:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;investigate_5xx_incident&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;start&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;end&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="c1"&gt;# 1. Query the error rate from Prometheus
&lt;/span&gt;    &lt;span class="n"&gt;metrics&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;tools&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;query_prometheus&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;sum(rate(http_requests_total{status=~&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;5..&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;}[5m])) / sum(rate(http_requests_total[5m]))&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;start&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;start&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;end&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;end&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# 2. Identify the first abnormal point in Python memory (not in LLM context!)
&lt;/span&gt;    &lt;span class="n"&gt;first_bad_point&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;find_threshold_crossing&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;metrics&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;threshold&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;0.05&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;first_bad_point&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;status&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;No significant increase detected&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="c1"&gt;# 3. Search Loki ONLY around the abnormal time window (±5 mins)
&lt;/span&gt;    &lt;span class="n"&gt;logs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;tools&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;query_loki&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;{namespace=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;production&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;, app=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;checkout&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;} |= &lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ERROR&lt;/span&gt;&lt;span class="sh"&gt;"'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;start&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;first_bad_point&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;timestamp&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;300&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;end&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;end&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# 4. Group and count 2,000 log lines in Python memory
&lt;/span&gt;    &lt;span class="n"&gt;error_summary&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;summarize_errors&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;logs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# 5. Check Kubernetes pod states
&lt;/span&gt;    &lt;span class="n"&gt;pods&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;tools&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get_pod_status&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;namespace&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;production&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;deployment&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;checkout&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# 6. Return ONLY the distilled findings (~100 tokens) to the LLM
&lt;/span&gt;    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;error_rate&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;first_bad_point&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;log_summary&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;error_summary&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;pod_status&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;summarize_pods&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;pods&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Here, &lt;code&gt;tools&lt;/code&gt; is a controlled Python bridge inside the sandbox. Even if tool schemas stay in the prompt, &lt;strong&gt;none of the 110,000 tokens of intermediate metrics, logs, or pod manifests ever enter the LLM context window&lt;/strong&gt;.&lt;/p&gt;




&lt;h3&gt;
  
  
  Approach C: Full UTCP + Code Mode Implementation in Python
&lt;/h3&gt;

&lt;p&gt;What if we want to eliminate &lt;strong&gt;both&lt;/strong&gt; the MCP wrapper servers &lt;strong&gt;and&lt;/strong&gt; the upfront tool schema bloat in the LLM context window?&lt;/p&gt;

&lt;p&gt;By combining the Python &lt;strong&gt;UTCP SDK&lt;/strong&gt; (&lt;code&gt;pip install utcp utcp-http utcp-cli&lt;/code&gt;) with a &lt;strong&gt;Code Mode execution loop&lt;/strong&gt;, the workflow happens in two clean phases:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Progressive Schema Discovery (&lt;code&gt;search_tools&lt;/code&gt;):&lt;/strong&gt; Instead of preloading 100 tool schemas into the prompt, the agent searches the UTCP manual for &lt;code&gt;"prometheus loki kubernetes"&lt;/code&gt; and loads &lt;em&gt;only&lt;/em&gt; the 3 compact Python call signatures (~400 tokens).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Sandboxed UTCP Execution (&lt;code&gt;execute_code&lt;/code&gt;):&lt;/strong&gt; The LLM writes a Python script using &lt;code&gt;utcp_client.call_tool(...)&lt;/code&gt; that runs inside the sandbox, talks directly to Prometheus (HTTP), Loki (HTTP), and &lt;code&gt;kubectl&lt;/code&gt; (CLI) via &lt;code&gt;sre_manual.json&lt;/code&gt;, filters the data in memory, and returns the compact JSON summary.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Here is the complete, runnable Python implementation showing how UTCP and Code Mode work together:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;asyncio&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;collections&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Counter&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;utcp.utcp_client&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;UtcpClient&lt;/span&gt;


&lt;span class="c1"&gt;# =====================================================================
# STEP 1: Progressive Tool Discovery (Evicts Idle Schemas from Context)
# =====================================================================
&lt;/span&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;discover_sre_tools&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;UtcpClient&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;incident_query&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;
    Instead of loading 100+ MCP tool schemas into the LLM prompt on Turn 1,
    we search the UTCP manual and return ONLY the relevant tool stubs (~300 tokens).
    &lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
    &lt;span class="n"&gt;matched_tools&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;search_tools&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;incident_query&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;limit&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="n"&gt;compact_stubs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;tool&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;matched_tools&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;props&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;list&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;tool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;inputs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;properties&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{}).&lt;/span&gt;&lt;span class="nf"&gt;keys&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;
        &lt;span class="n"&gt;compact_stubs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;- &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;tool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;(&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;, &lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;props&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;): &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;tool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;description&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;compact_stubs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;


&lt;span class="c1"&gt;# =====================================================================
# STEP 2: Code Mode Sandbox Execution (Generated by LLM, Run in Sandbox)
# =====================================================================
&lt;/span&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;llm_generated_code_mode_script&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;UtcpClient&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;start_ts&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;end_ts&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;
    In Code Mode, the LLM generates this Python orchestration block.
    It calls Prometheus, Loki, and kubectl DIRECTLY via the UTCP manual,
    processes 100K+ tokens of raw data in Python RAM, and returns ~100 tokens.
    &lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
    &lt;span class="c1"&gt;# 1. Direct HTTP call to Prometheus via UTCP manual
&lt;/span&gt;    &lt;span class="n"&gt;raw_prom&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;call_tool&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;sre_manual.query_prometheus&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;query&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;sum(rate(http_requests_total{status=~&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;5..&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;}[5m])) / sum(rate(http_requests_total[5m]))&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;start&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;start_ts&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;end&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;end_ts&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;prom_data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;raw_prom&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# 2. Find the exact timestamp where 5xx error rate crossed 5% (0.05)
&lt;/span&gt;    &lt;span class="n"&gt;spike_ts&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;
    &lt;span class="n"&gt;peak_rate&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mf"&gt;0.0&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;series&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;prom_data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;data&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{}).&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;result&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[]):&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;ts&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;val&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;series&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;values&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[]):&lt;/span&gt;
            &lt;span class="n"&gt;rate_val&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;float&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;val&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;rate_val&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mf"&gt;0.05&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;spike_ts&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="n"&gt;spike_ts&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ts&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="n"&gt;peak_rate&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;peak_rate&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;rate_val&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;spike_ts&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;status&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;healthy&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;message&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;No 5xx spike &amp;gt; 5% detected.&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="c1"&gt;# 3. Direct HTTP call to Loki ONLY for the 15-min window around spike_ts
&lt;/span&gt;    &lt;span class="n"&gt;raw_loki&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;call_tool&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;sre_manual.query_loki&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;query&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;{namespace=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;production&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;, app=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;checkout&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;} |= &lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ERROR&lt;/span&gt;&lt;span class="sh"&gt;"'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;start&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;spike_ts&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;300&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;end&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;spike_ts&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;600&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;loki_data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;raw_loki&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# 4. Aggregate 2,000 raw log lines in Python memory (Zero LLM context used!)
&lt;/span&gt;    &lt;span class="n"&gt;error_counts&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Counter&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="n"&gt;total_logs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;stream&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;loki_data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;data&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{}).&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;result&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[]):&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;_&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;log_line&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;stream&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;values&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[]):&lt;/span&gt;
            &lt;span class="n"&gt;total_logs&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;
            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Database connection timeout&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;log_line&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="n"&gt;error_counts&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Database connection timeout&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;
            &lt;span class="k"&gt;elif&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Connection pool exhausted&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;log_line&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="n"&gt;error_counts&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Connection pool exhausted&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;
            &lt;span class="k"&gt;else&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="n"&gt;error_counts&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Other application error&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;

    &lt;span class="c1"&gt;# 5. Direct CLI call to Kubernetes (`kubectl get pods`) via UTCP manual
&lt;/span&gt;    &lt;span class="n"&gt;raw_k8s&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;call_tool&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;sre_manual.get_pod_status&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;namespace&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;production&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;deployment&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;checkout&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;k8s_data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;loads&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;raw_k8s&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="n"&gt;restarting_pods&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;pod&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;k8s_data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;items&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[]):&lt;/span&gt;
        &lt;span class="n"&gt;Statuses&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pod&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;status&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{}).&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;containerStatuses&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[])&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nf"&gt;any&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;restartCount&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;Statuses&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
            &lt;span class="n"&gt;restarting_pods&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;

    &lt;span class="c1"&gt;# 6. Return ONLY the compact distilled summary back to the LLM context!
&lt;/span&gt;    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;incident&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;checkout-5xx-increase&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;spike_timestamp&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;spike_ts&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;peak_error_rate&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;peak_rate&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="si"&gt;:&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;%&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;total_error_logs_analyzed&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;total_logs&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;top_errors&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;error_counts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;most_common&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;pods_with_restarts&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;restarting_pods&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;


&lt;span class="c1"&gt;# =====================================================================
# STEP 3: Main Agent Runner
# =====================================================================
&lt;/span&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;main&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="c1"&gt;# Initialize UTCP Client pointing to our declarative JSON manual (No MCP server!)
&lt;/span&gt;    &lt;span class="n"&gt;client&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;UtcpClient&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;create&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;config&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;manual_call_templates&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
                &lt;span class="p"&gt;{&lt;/span&gt;
                    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;name&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;sre_manual&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;call_template_type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;text&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;file_path&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;./sre_manual.json&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="p"&gt;}&lt;/span&gt;
            &lt;span class="p"&gt;]&lt;/span&gt;
        &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="c1"&gt;# Phase 1: Discover only the tools needed for this incident
&lt;/span&gt;        &lt;span class="n"&gt;stubs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;discover_sre_tools&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prometheus metrics loki logs kubernetes pods&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;=== 1. Compact Tool Stubs Sent to LLM (~120 tokens) ===&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;stubs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

        &lt;span class="c1"&gt;# Phase 2: Execute the LLM-generated Code Mode script inside the runtime
&lt;/span&gt;        &lt;span class="n"&gt;summary&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;llm_generated_code_mode_script&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;start_ts&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1730523000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;end_ts&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1730526600&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="se"&gt;\n&lt;/span&gt;&lt;span class="s"&gt;=== 2. Compact Investigation Output Returned to LLM (~90 tokens) ===&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;summary&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;indent&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;

    &lt;span class="k"&gt;finally&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;


&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;__name__&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;__main__&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;asyncio&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;run&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;main&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Example Investigation Output Returned to the LLM
&lt;/h3&gt;

&lt;p&gt;Instead of ingesting 50+ preloaded tool schemas, 500 metric samples, and 2,000 raw log lines, the LLM receives this compact JSON payload (~90 tokens):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"incident"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"checkout-5xx-increase"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"spike_timestamp"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1730524200&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"peak_error_rate"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"18.0%"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"total_error_logs_analyzed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;2000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"top_errors"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"Database connection timeout"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;1450&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"Connection pool exhausted"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;420&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"Other application error"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;130&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"pods_with_restarts"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The LLM can now explain the findings, separate observed facts from hypotheses, and recommend next steps—without drowning in raw telemetry or maintaining intermediate MCP servers.&lt;/p&gt;




&lt;h2&gt;
  
  
  6. Why Context Windows Matter: The Per-Turn Token Math
&lt;/h2&gt;

&lt;p&gt;A common question engineers ask is: &lt;em&gt;"Does Code Mode only shrink the context window if tool schemas are removed from the prompt?"&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The answer is &lt;strong&gt;no&lt;/strong&gt;—because in SRE observability workflows, &lt;strong&gt;Static Schema Tax&lt;/strong&gt; and &lt;strong&gt;Dynamic Telemetry Payload Tax&lt;/strong&gt; both compete for the context window, and traditional tool calling compounds payloads across every turn.&lt;/p&gt;

&lt;p&gt;Let's count what the LLM actually receives &lt;strong&gt;per turn&lt;/strong&gt; in our 10:30 AM &lt;code&gt;5xx&lt;/code&gt; investigation. Assume:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;50 preloaded tool schemas:&lt;/strong&gt; &lt;code&gt;8,000&lt;/code&gt; tokens&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Base system prompt + user query:&lt;/strong&gt; &lt;code&gt;2,000&lt;/code&gt; tokens&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Raw telemetry responses:&lt;/strong&gt; &lt;code&gt;15,000&lt;/code&gt; tokens (500 Prometheus samples), &lt;code&gt;80,000&lt;/code&gt; tokens (2,000 Loki log lines), and &lt;code&gt;15,000&lt;/code&gt; tokens (Kubernetes pod manifests)&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Per-Turn Breakdown Across All 3 Architectures
&lt;/h3&gt;

&lt;p&gt;Because LLM APIs are stateless, every subsequent turn in traditional tool calling re-sends the schemas &lt;strong&gt;plus&lt;/strong&gt; all prior raw tool responses in the conversation history:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Architecture 1: Traditional MCP Tool Calling (Schemas Preloaded, Sequential Calls)&lt;/strong&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Turn 1 (Call Prometheus):&lt;/strong&gt; &lt;code&gt;8K (schemas) + 2K (prompt)&lt;/code&gt; = &lt;strong&gt;10,000 tokens&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Turn 2 (Read Prom, Call Loki):&lt;/strong&gt; &lt;code&gt;8K + 2K + 15K (Prom raw)&lt;/code&gt; = &lt;strong&gt;25,000 tokens&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Turn 3 (Read Loki, Call K8s):&lt;/strong&gt; &lt;code&gt;8K + 2K + 15K + 80K (Loki raw)&lt;/code&gt; = &lt;strong&gt;105,000 tokens&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Turn 4 (Read K8s, Output RCA):&lt;/strong&gt; &lt;code&gt;8K + 2K + 15K + 80K + 15K (K8s raw)&lt;/code&gt; = &lt;strong&gt;120,000 tokens&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Architecture 2: Code Mode &lt;em&gt;Without&lt;/em&gt; Schema Eviction (Schemas Kept, Payloads Filtered in Sandbox)&lt;/strong&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Turn 1 (Write Python Script):&lt;/strong&gt; &lt;code&gt;8K (schemas) + 2K (prompt)&lt;/code&gt; = &lt;strong&gt;10,000 tokens&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;&lt;em&gt;(Sandbox runs Prom → Loki → K8s in memory and distills 110,000 tokens down to 150 tokens)&lt;/em&gt;&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Turn 2 (Output RCA from Summary):&lt;/strong&gt; &lt;code&gt;8K + 2K + 0.4K (script) + 0.15K (summary)&lt;/code&gt; = &lt;strong&gt;10,550 tokens&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Architecture 3: UTCP + Code Mode &lt;em&gt;With&lt;/em&gt; Schema Eviction (&lt;code&gt;search_tools&lt;/code&gt; + Sandbox)&lt;/strong&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Turn 1 (Search Tools):&lt;/strong&gt; &lt;code&gt;0.4K (meta-tools) + 2K (prompt)&lt;/code&gt; = &lt;strong&gt;2,400 tokens&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Turn 2 (Write Script with 3 Matched Stubs):&lt;/strong&gt; &lt;code&gt;0.4K + 2K + 0.4K (3 stubs)&lt;/code&gt; = &lt;strong&gt;2,800 tokens&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Turn 3 (Output RCA from Summary):&lt;/strong&gt; &lt;code&gt;0.4K + 2K + 0.4K + 0.4K (script) + 0.15K (summary)&lt;/code&gt; = &lt;strong&gt;3,350 tokens&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;1. Standard MCP&lt;/th&gt;
&lt;th&gt;2. Code Mode (Schemas Kept)&lt;/th&gt;
&lt;th&gt;3. UTCP + Code Mode (Schemas Evicted)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Upfront Schema Tokens / Turn&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;8,000&lt;/td&gt;
&lt;td&gt;8,000&lt;/td&gt;
&lt;td&gt;~400 – 800&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Raw Telemetry in LLM Context&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;110,000&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;150&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;150&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;LLM Inference Turns&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;4 turns&lt;/td&gt;
&lt;td&gt;2 turns&lt;/td&gt;
&lt;td&gt;2–3 turns&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Peak Single-Turn Context&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;120,000&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;10,550&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;3,350&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Cumulative Tokens Billed&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;260,000&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;20,550&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;8,550&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Key Takeaway:&lt;/strong&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Evicting schemas&lt;/strong&gt; (via UTCP's &lt;code&gt;search_tools&lt;/code&gt;) solves the &lt;strong&gt;&lt;code&gt;O(N_tools)&lt;/code&gt; static tax&lt;/strong&gt; when your platform scales to dozens or hundreds of tools.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Code Mode execution&lt;/strong&gt; solves the &lt;strong&gt;&lt;code&gt;O(N_steps × Payload_Size)&lt;/code&gt; dynamic tax&lt;/strong&gt; by keeping megabytes of time-series points and log lines inside Python memory rather than compounding them across conversational turns.&lt;/li&gt;
&lt;/ol&gt;




&lt;h2&gt;
  
  
  7. Where Code Mode + UTCP Makes Sense for SRE Teams
&lt;/h2&gt;

&lt;p&gt;Not every operational query needs Code Mode. If you ask &lt;em&gt;"What is the current CPU usage of &lt;code&gt;payment-service&lt;/code&gt;?"&lt;/em&gt;, a single direct tool call is fast and sufficient.&lt;/p&gt;

&lt;p&gt;Code Mode and UTCP shine when the workflow requires &lt;strong&gt;multi-step correlation, conditional branching, or heavy data reduction&lt;/strong&gt;:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;SRE Use Case&lt;/th&gt;
&lt;th&gt;Why UTCP + Code Mode Helps&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Incident Investigation&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Correlates metrics, logs, and K8s state in one execution pass&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Root Cause Analysis (RCA)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Runs fan-out queries across multiple microservices and compares results&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Capacity Planning&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Aggregates 30-day historical metrics and computes statistical trends locally&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Deployment Canary Analysis&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Compares error/latency distributions before and after a GitHub rollout&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Alert Storm Triage&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Groups 50+ firing alerts and extracts common upstream dependencies&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;High-Volume Log Analysis&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Filters and deduplicates thousands of Loki/Elasticsearch lines in memory&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;SLO &amp;amp; Error Budget Burn&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Calculates multi-window burn rates across services deterministically&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  8. Production Safeguards for Python Execution Runtimes
&lt;/h2&gt;

&lt;p&gt;If you introduce a Python execution runtime for Code Mode, &lt;strong&gt;safety and isolation are non-negotiable&lt;/strong&gt;. An LLM-generated script must never have unrestricted access to your production environment.&lt;br&gt;
&lt;/p&gt;

&lt;pre data-lang="mermaid"&gt;&lt;code&gt;flowchart TD
    LLM[LLM Code Generator] --&amp;gt; Validator[AST / Policy Validator]
    Validator --&amp;gt; Sandbox[Isolated Python Sandbox: gVisor / MicroVM]
    Sandbox --&amp;gt; Bridge[Allowlisted Read-Only Tool Bridge]
    Bridge --&amp;gt; Client[UTCP / MCP Client with Scoped Auth]
    Client --&amp;gt; Infra[(Prometheus / Loki / K8s)]&lt;/code&gt;&lt;/pre&gt;



&lt;p&gt;Here is the production checklist every SRE team should enforce:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Strict Sandbox Execution:&lt;/strong&gt; Never run generated code inside your host process using raw &lt;code&gt;exec()&lt;/code&gt;. Use isolated sandboxes (e.g., gVisor, Firecracker microVMs, ephemeral containers, or restricted WebAssembly/Deno sandboxes).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tool Allowlisting:&lt;/strong&gt; Do not give the sandbox arbitrary network access. Expose only an explicit bridge of approved, read-only operational tools (&lt;code&gt;query_prometheus&lt;/code&gt;, &lt;code&gt;query_loki&lt;/code&gt;, &lt;code&gt;get_pod_status&lt;/code&gt;).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Scoped Authentication:&lt;/strong&gt; Use short-lived, read-only service account tokens. With UTCP, credentials (&lt;code&gt;${PROMETHEUS_TOKEN}&lt;/code&gt;) are injected natively at the client execution boundary so secrets are never exposed to the LLM prompt or generated code.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Hard Resource Limits:&lt;/strong&gt; Enforce strict CPU, memory, execution timeout (e.g., 30s max), and payload-size limits on the sandbox.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Query Guardrails:&lt;/strong&gt; Validate PromQL and LogQL queries to block high-cardinality regex scans or unbounded time ranges that could OOM your monitoring stack.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Full Audit Logging:&lt;/strong&gt; Record the prompt, the exact generated code, every underlying tool call, and the user/session identity for post-incident compliance.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Human-in-the-Loop for Mutations:&lt;/strong&gt; Keep investigation 100% read-only. Any state-changing action (pod restarts, rollbacks, scaling) must require explicit human approval.&lt;/li&gt;
&lt;/ol&gt;




&lt;h2&gt;
  
  
  9. A Sensible Adoption Path for SRE Teams
&lt;/h2&gt;

&lt;p&gt;You don't need to over-engineer your stack on Day 1. Follow a phased rollout:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Phase 1: Expose Read-Only Tools (UTCP or MCP):&lt;/strong&gt; Start by exposing Prometheus, Loki, and Kubernetes read-only queries. If you already have clean REST APIs, OpenAPI specs, and CLIs, &lt;strong&gt;UTCP manuals&lt;/strong&gt; let you skip deploying wrapper servers; if you need a centralized proxy boundary, use &lt;strong&gt;MCP&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Phase 2: Validate Basic Workflows:&lt;/strong&gt; Let engineers run simple single-step queries to verify authentication, query guardrails, and accuracy.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Phase 3: Identify Context-Heavy Bottlenecks:&lt;/strong&gt; Spot the workflows where investigations hit token limits, suffer from slow multi-turn latency, or drown in raw logs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Phase 4: Introduce Progressive Discovery &amp;amp; Sandboxed Code Mode:&lt;/strong&gt; Use &lt;code&gt;search_tools&lt;/code&gt; to keep idle tool schemas out of the prompt, and let the LLM generate orchestration scripts that filter telemetry outside the context window.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Phase 5: Measure SRE Impact:&lt;/strong&gt; Track per-turn token reduction, investigation latency (MTTR), number of LLM turns, and root-cause accuracy.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;MCP&lt;/strong&gt;, &lt;strong&gt;UTCP&lt;/strong&gt;, and &lt;strong&gt;Code Mode&lt;/strong&gt; solve distinct pieces of the SRE automation puzzle:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;MCP&lt;/strong&gt; standardizes how LLMs access tools through a consistent &lt;strong&gt;server interface&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;UTCP&lt;/strong&gt; streamlines that access by letting agents discover and call your existing APIs and CLIs &lt;strong&gt;directly via declarative manuals&lt;/strong&gt;, eliminating middleware server toil and upfront schema bloat.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Code Mode&lt;/strong&gt; gives the LLM a safe way to &lt;strong&gt;programmatically orchestrate those tools&lt;/strong&gt;, filter noisy telemetry in memory, and return focused insights.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;An incident investigation is rarely a single query—it is a sequence of hypotheses across metrics, logs, deployments, and cluster state. The future of AI in SRE isn't just giving models access to more tools; it's giving them a lightweight, safe architecture to combine those tools and reason over signal instead of noise.&lt;/p&gt;




&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://www.utcp.io/" rel="noopener noreferrer"&gt;Universal Tool Calling Protocol (UTCP)&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://modelcontextprotocol.io/" rel="noopener noreferrer"&gt;Model Context Protocol (MCP)&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://py.sdk.modelcontextprotocol.io/" rel="noopener noreferrer"&gt;MCP Python SDK&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://prometheus.io/docs/prometheus/latest/querying/api/" rel="noopener noreferrer"&gt;Prometheus HTTP API&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://grafana.com/docs/loki/latest/reference/loki-http-api/" rel="noopener noreferrer"&gt;Grafana Loki HTTP API&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://developers.openai.com/api/docs/guides/conversation-state/" rel="noopener noreferrer"&gt;OpenAI: Conversation State and Context&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>sre</category>
      <category>mcp</category>
      <category>utcp</category>
      <category>codemode</category>
    </item>
    <item>
      <title>RAG vs OKF: What’s the Difference and Where Should SREs Use Them?</title>
      <dc:creator>Anand Kannan</dc:creator>
      <pubDate>Wed, 07 Oct 2026 17:52:17 +0000</pubDate>
      <link>https://dev.to/anandindia93/rag-vs-okf-whats-the-difference-and-where-should-sres-use-them-4b6e</link>
      <guid>https://dev.to/anandindia93/rag-vs-okf-whats-the-difference-and-where-should-sres-use-them-4b6e</guid>
      <description>&lt;h1&gt;
  
  
  RAG vs OKF: What’s the Difference and Where Should SREs Use Them?
&lt;/h1&gt;

&lt;p&gt;If you're working in SRE, you've probably heard a lot about &lt;strong&gt;RAG (Retrieval-Augmented Generation)&lt;/strong&gt; and &lt;strong&gt;OKF (Open Knowledge Framework)&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;At first, they can sound like the same thing:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"Give an AI access to our documentation and let it answer questions."&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;But they're actually solving different problems.&lt;/p&gt;

&lt;p&gt;Let's understand this without the AI buzzwords.&lt;/p&gt;




&lt;h2&gt;
  
  
  The 10-second explanation
&lt;/h2&gt;

&lt;p&gt;Think of an AI system as an SRE engineer.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;RAG says:&lt;/strong&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"Before answering, search our runbooks, documentation and incident reports."&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;&lt;strong&gt;OKF says:&lt;/strong&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"Don't just give me documents. Organize knowledge, relationships and context so an AI can navigate it consistently."&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;So:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;RAG = Retrieve relevant information&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;OKF = Organize knowledge and relationships for AI consumption&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;They aren't competitors. In a good SRE system, you can use both.&lt;/p&gt;




&lt;h1&gt;
  
  
  1. What is RAG?
&lt;/h1&gt;

&lt;p&gt;RAG stands for:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Retrieval-Augmented Generation&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The idea is surprisingly simple.&lt;/p&gt;

&lt;p&gt;Instead of asking an LLM a question and expecting it to know everything, we first retrieve relevant information from our own systems.&lt;/p&gt;

&lt;p&gt;For example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;User:
Why are checkout 500 errors increasing?

        ↓

Search knowledge base

        ↓

Find relevant documents

- checkout-runbook.md
- incident-2026-08.md
- nginx-errors.md

        ↓

LLM

        ↓

Answer
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The AI might respond:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"The most likely cause is database connection exhaustion. The checkout runbook recommends checking the PostgreSQL connection pool and active connections."&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The important part is that the model didn't necessarily know this beforehand.&lt;/p&gt;

&lt;p&gt;We &lt;strong&gt;retrieved the information and gave it to the model&lt;/strong&gt;.&lt;/p&gt;




&lt;h1&gt;
  
  
  2. Why is RAG useful for SRE?
&lt;/h1&gt;

&lt;p&gt;Most SRE organizations already have huge amounts of operational knowledge:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Runbooks&lt;/li&gt;
&lt;li&gt;Incident reports&lt;/li&gt;
&lt;li&gt;Postmortems&lt;/li&gt;
&lt;li&gt;Architecture documents&lt;/li&gt;
&lt;li&gt;Terraform repositories&lt;/li&gt;
&lt;li&gt;Kubernetes manifests&lt;/li&gt;
&lt;li&gt;Jenkins pipelines&lt;/li&gt;
&lt;li&gt;Troubleshooting guides&lt;/li&gt;
&lt;li&gt;Known issues&lt;/li&gt;
&lt;li&gt;Service documentation&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The problem isn't necessarily lack of knowledge.&lt;/p&gt;

&lt;p&gt;The problem is:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Finding the right knowledge at the right time.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Imagine you're paged at 2 AM.&lt;/p&gt;

&lt;p&gt;You see:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;checkout-api
HTTP 500
Error rate: 35%
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Instead of manually searching through 200 runbooks, you ask:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"What should I check for checkout-api 500 errors?"&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;RAG retrieves the relevant documentation and gives you a concise answer.&lt;/p&gt;

&lt;p&gt;That's already extremely useful.&lt;/p&gt;




&lt;h1&gt;
  
  
  3. But RAG has a limitation
&lt;/h1&gt;

&lt;p&gt;Here's where things get interesting.&lt;/p&gt;

&lt;p&gt;Imagine your documentation contains:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;checkout-api.md

Checkout API depends on Payment API.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And another document says:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;payment-api.md

Payment API depends on PostgreSQL.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And another says:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;database-runbook.md

PostgreSQL connection exhaustion can cause payment failures.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A RAG system can retrieve these documents.&lt;/p&gt;

&lt;p&gt;But the relationships are mostly &lt;strong&gt;implicit inside the text&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The AI has to figure out:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Checkout
   ↓
Payment API
   ↓
PostgreSQL
   ↓
Connection pool
   ↓
500 errors
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;OKF becomes interesting here because knowledge can be deliberately structured, linked and made machine-readable instead of being treated as unrelated chunks of text. Practical OKF implementations commonly use Markdown with metadata/frontmatter and links between concepts.&lt;/p&gt;




&lt;h1&gt;
  
  
  4. Think about OKF
&lt;/h1&gt;

&lt;p&gt;Instead of only storing documents, we explicitly represent our operational knowledge.&lt;/p&gt;

&lt;p&gt;For example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Checkout Service
      │
      ├── depends_on → Payment API
      ├── deployed_by → Jenkins
      ├── monitored_by → Prometheus
      └── owned_by → Payments Team

Payment API
      │
      └── depends_on → PostgreSQL

PostgreSQL
      │
      ├── monitored_by → Prometheus
      └── has_runbook → DB-POOL-001
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now the system doesn't just know &lt;strong&gt;what the documents say&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;It knows &lt;strong&gt;how things are connected&lt;/strong&gt;.&lt;/p&gt;




&lt;h1&gt;
  
  
  5. Let's actually build a tiny OKF for SRE
&lt;/h1&gt;

&lt;p&gt;You don't need Kubernetes, a graph database or a huge AI platform to understand the concept.&lt;/p&gt;

&lt;p&gt;Let's build a tiny knowledge base for one imaginary service:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;checkout-api
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Our goal is to answer:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"Why is checkout-api returning 500 errors?"&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  Step 1: Create the knowledge structure
&lt;/h2&gt;

&lt;p&gt;Start with something simple:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;sre-okf/
├── index.md
├── services/
│   ├── checkout-api.md
│   └── payment-api.md
├── infrastructure/
│   └── postgres.md
├── alerts/
│   └── checkout-500.md
└── runbooks/
    └── db-connection-pool.md
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The important thing isn't the exact folder names.&lt;/p&gt;

&lt;p&gt;The important thing is that the knowledge is &lt;strong&gt;structured, predictable and linked&lt;/strong&gt;.&lt;/p&gt;




&lt;h1&gt;
  
  
  Step 2: Create the service definition
&lt;/h1&gt;

&lt;p&gt;Create:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;services/checkout-api.md
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Add:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;service&lt;/span&gt;
&lt;span class="na"&gt;id&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;checkout-api&lt;/span&gt;
&lt;span class="na"&gt;owner&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;payments-team&lt;/span&gt;
&lt;span class="na"&gt;environment&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;production&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then add the actual knowledge:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;&lt;span class="gh"&gt;# Checkout API&lt;/span&gt;

The Checkout API handles customer checkout requests.

&lt;span class="gu"&gt;## Dependencies&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;Payment API&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;&lt;span class="sx"&gt;payment-api.md&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="gu"&gt;## Monitoring&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; Prometheus metric: &lt;span class="sb"&gt;`checkout_http_requests_total`&lt;/span&gt;
&lt;span class="p"&gt;-&lt;/span&gt; Grafana dashboard: Checkout Overview

&lt;span class="gu"&gt;## Alerts&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;Checkout 500 Alert&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;&lt;span class="sx"&gt;../alerts/checkout-500.md&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Notice something important.&lt;/p&gt;

&lt;p&gt;We're not just writing prose.&lt;/p&gt;

&lt;p&gt;We're creating &lt;strong&gt;machine-readable metadata + human-readable content + links&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;This combination is one of the useful patterns seen in practical OKF implementations.&lt;/p&gt;




&lt;h1&gt;
  
  
  Step 3: Define the dependency
&lt;/h1&gt;

&lt;p&gt;Create:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;services/payment-api.md
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;service&lt;/span&gt;
&lt;span class="na"&gt;id&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;payment-api&lt;/span&gt;
&lt;span class="na"&gt;owner&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;payments-team&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;&lt;span class="gh"&gt;# Payment API&lt;/span&gt;

The Payment API processes payment transactions.

&lt;span class="gu"&gt;## Dependencies&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;PostgreSQL&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;&lt;span class="sx"&gt;../infrastructure/postgres.md&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="gu"&gt;## Known Failure Modes&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; Database connection pool exhaustion
&lt;span class="p"&gt;-&lt;/span&gt; Payment provider timeout
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now we have:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Checkout API
     ↓
Payment API
     ↓
PostgreSQL
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h1&gt;
  
  
  Step 4: Define the infrastructure
&lt;/h1&gt;

&lt;p&gt;Create:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;infrastructure/postgres.md
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;infrastructure&lt;/span&gt;
&lt;span class="na"&gt;id&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;postgres-payments&lt;/span&gt;
&lt;span class="na"&gt;technology&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;PostgreSQL&lt;/span&gt;
&lt;span class="na"&gt;environment&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;production&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;&lt;span class="gh"&gt;# PostgreSQL&lt;/span&gt;

Primary database for Payment API.

&lt;span class="gu"&gt;## Used By&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;Payment API&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;&lt;span class="sx"&gt;../services/payment-api.md&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="gu"&gt;## Important Metrics&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; Active connections
&lt;span class="p"&gt;-&lt;/span&gt; Connection pool utilization
&lt;span class="p"&gt;-&lt;/span&gt; Query latency

&lt;span class="gu"&gt;## Runbook&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;Database Connection Pool Runbook&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;&lt;span class="sx"&gt;../runbooks/db-connection-pool.md&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now the relationship exists in both directions.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Payment API
     ↓
PostgreSQL
     ↓
DB Connection Pool Runbook
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h1&gt;
  
  
  Step 5: Define the alert
&lt;/h1&gt;

&lt;p&gt;Create:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;alerts/checkout-500.md
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;alert&lt;/span&gt;
&lt;span class="na"&gt;id&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;checkout-http-500&lt;/span&gt;
&lt;span class="na"&gt;severity&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;critical&lt;/span&gt;
&lt;span class="na"&gt;service&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;checkout-api&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;&lt;span class="gh"&gt;# Checkout HTTP 500&lt;/span&gt;

Triggered when:

&lt;span class="sb"&gt;`checkout_http_5xx_rate &amp;gt; 10%`&lt;/span&gt;

&lt;span class="gu"&gt;## Affected Service&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;Checkout API&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;&lt;span class="sx"&gt;../services/checkout-api.md&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="gu"&gt;## Investigation Path&lt;/span&gt;

Check:
&lt;span class="p"&gt;
1.&lt;/span&gt; Checkout API
&lt;span class="p"&gt;2.&lt;/span&gt; Payment API
&lt;span class="p"&gt;3.&lt;/span&gt; PostgreSQL connections
&lt;span class="p"&gt;4.&lt;/span&gt; Database connection pool

&lt;span class="gu"&gt;## Related Runbook&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;DB Connection Pool Runbook&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;&lt;span class="sx"&gt;../runbooks/db-connection-pool.md&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now an alert isn't just a string.&lt;/p&gt;

&lt;p&gt;It has:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Alert
 ↓
Service
 ↓
Dependency
 ↓
Infrastructure
 ↓
Runbook
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h1&gt;
  
  
  Step 6: Add the runbook
&lt;/h1&gt;

&lt;p&gt;Create:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;runbooks/db-connection-pool.md
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;runbook&lt;/span&gt;
&lt;span class="na"&gt;id&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;db-connection-pool&lt;/span&gt;
&lt;span class="na"&gt;owner&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;payments-team&lt;/span&gt;
&lt;span class="na"&gt;severity&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;critical&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;&lt;span class="gh"&gt;# Database Connection Pool Runbook&lt;/span&gt;

Use this runbook when database connection exhaustion is suspected.

&lt;span class="gu"&gt;## Check&lt;/span&gt;

Prometheus:

&lt;span class="sb"&gt;`pg_stat_activity_count`&lt;/span&gt;

Check connection pool utilization.

&lt;span class="gu"&gt;## Loki&lt;/span&gt;

Search:

&lt;span class="sb"&gt;`"connection pool exhausted"`&lt;/span&gt;

&lt;span class="gu"&gt;## Remediation&lt;/span&gt;
&lt;span class="p"&gt;
1.&lt;/span&gt; Confirm active connections.
&lt;span class="p"&gt;2.&lt;/span&gt; Check for connection leaks.
&lt;span class="p"&gt;3.&lt;/span&gt; Check recent deployments.
&lt;span class="p"&gt;4.&lt;/span&gt; Restart affected workers if approved.
&lt;span class="p"&gt;5.&lt;/span&gt; Escalate to the database team if the issue persists.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now we have a very small operational knowledge system.&lt;/p&gt;




&lt;h1&gt;
  
  
  Step 7: Create an index
&lt;/h1&gt;

&lt;p&gt;Finally, create:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;index.md
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;&lt;span class="gh"&gt;# SRE Knowledge Base&lt;/span&gt;

&lt;span class="gu"&gt;## Services&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;Checkout API&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;&lt;span class="sx"&gt;services/checkout-api.md&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;Payment API&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;&lt;span class="sx"&gt;services/payment-api.md&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="gu"&gt;## Infrastructure&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;PostgreSQL&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;&lt;span class="sx"&gt;infrastructure/postgres.md&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="gu"&gt;## Alerts&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;Checkout 500&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;&lt;span class="sx"&gt;alerts/checkout-500.md&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="gu"&gt;## Runbooks&lt;/span&gt;
&lt;span class="p"&gt;
-&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nv"&gt;DB Connection Pool&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;&lt;span class="sx"&gt;runbooks/db-connection-pool.md&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The index gives an agent a predictable starting point instead of forcing it to blindly search everything.&lt;/p&gt;

&lt;p&gt;This "index + linked knowledge" approach is used by real OKF-style implementations to make knowledge navigable by both humans and agents.&lt;/p&gt;




&lt;h1&gt;
  
  
  6. Now connect an AI agent
&lt;/h1&gt;

&lt;p&gt;At this point, you don't necessarily need a vector database.&lt;/p&gt;

&lt;p&gt;A very simple OKF retrieval flow can be:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;User question
      ↓
Find starting concept
      ↓
Read metadata
      ↓
Follow related links
      ↓
Collect relevant context
      ↓
Send context to LLM
      ↓
Generate answer
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Question:

"Why is checkout returning 500s?"
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The agent might traverse:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;checkout-500.md
      ↓
checkout-api.md
      ↓
payment-api.md
      ↓
postgres.md
      ↓
db-connection-pool.md
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That traversal path is important.&lt;/p&gt;

&lt;p&gt;The AI can explain &lt;strong&gt;why those pieces of knowledge were used&lt;/strong&gt;, rather than simply saying:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"These five chunks looked similar to your question."&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Graph/linked traversal is one of the approaches demonstrated by OKF implementations, where the retrieved context can be represented as a traversal path through related knowledge files.&lt;/p&gt;




&lt;h1&gt;
  
  
  7. Now add your real SRE systems
&lt;/h1&gt;

&lt;p&gt;This is where it becomes useful.&lt;/p&gt;

&lt;p&gt;The OKF itself doesn't have to contain your live metrics.&lt;/p&gt;

&lt;p&gt;Instead, it can tell the AI &lt;strong&gt;where and how to get them&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;For example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;OKF
 │
 ├── Checkout API
 │     ├── Prometheus metrics
 │     └── Grafana dashboard
 │
 ├── Payment API
 │     └── Loki logs
 │
 └── PostgreSQL
       └── Prometheus metrics
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then your agent can call the actual systems:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;OKF
 ↓
Understand architecture
 ↓
Prometheus
 ↓
Loki
 ↓
Jenkins
 ↓
LLM
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now you're moving from a static knowledge base toward an &lt;strong&gt;AI-powered SRE investigation system&lt;/strong&gt;.&lt;/p&gt;




&lt;h1&gt;
  
  
  8. A real incident
&lt;/h1&gt;

&lt;p&gt;Imagine this happens:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;14:21
Checkout 500 rate → 32%
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The AI receives the alert.&lt;/p&gt;

&lt;p&gt;It uses OKF:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Checkout
   ↓
Payment API
   ↓
PostgreSQL
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then it queries Prometheus:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;DB connections → 98%
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then Loki:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;connection pool exhausted
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then Jenkins:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Payment API deployed 4 minutes ago
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And finally the runbook:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;DB connection pool exhaustion
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The AI can produce:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Likely cause:&lt;/strong&gt; Payment API deployment caused database connection-pool exhaustion.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Evidence:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Checkout 500s increased at 14:21.&lt;/li&gt;
&lt;li&gt;Payment API was deployed at 14:17.&lt;/li&gt;
&lt;li&gt;PostgreSQL connection utilization is 98%.&lt;/li&gt;
&lt;li&gt;Loki shows connection-pool exhaustion.&lt;/li&gt;
&lt;li&gt;The affected dependency chain is Checkout → Payment API → PostgreSQL.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Recommended next step:&lt;/strong&gt; Follow &lt;code&gt;DB Connection Pool Runbook&lt;/code&gt;.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;That's a much more useful SRE assistant than a chatbot that simply searches documentation.&lt;/p&gt;




&lt;h1&gt;
  
  
  9. Where RAG fits
&lt;/h1&gt;

&lt;p&gt;Now imagine you have 10,000 historical incident reports.&lt;/p&gt;

&lt;p&gt;You probably don't want to manually create OKF relationships for every sentence.&lt;/p&gt;

&lt;p&gt;This is where RAG becomes useful.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;                 AI SRE Assistant
                        │
             ┌──────────┴──────────┐
             ↓                     ↓
            OKF                    RAG
             │                     │
       Structured              Historical
       knowledge               documents
             │                     │
       "How things             "What happened
        connect"                 before?"
             │                     │
             └──────────┬──────────┘
                        ↓
                  Live systems
              Prometheus / Loki
                        ↓
                       LLM
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;So you might use:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;OKF for stable operational knowledge&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Services
Dependencies
Owners
Architecture
Runbooks
Standards
Definitions
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;RAG for large, changing collections&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Incident reports
Postmortems
Tickets
Slack discussions
Documentation
Historical investigations
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Observability for current state&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Prometheus
Loki
Grafana
Kubernetes
Jenkins
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h1&gt;
  
  
  10. The practical SRE adoption path
&lt;/h1&gt;

&lt;p&gt;Don't try to build everything on day one.&lt;/p&gt;

&lt;h3&gt;
  
  
  Phase 1 — Start small
&lt;/h3&gt;

&lt;p&gt;Create OKF for &lt;strong&gt;5–10 critical services&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Capture:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Service
Owner
Dependencies
Dashboards
Alerts
Runbooks
Repository
Deployment pipeline
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h3&gt;
  
  
  Phase 2 — Add relationships
&lt;/h3&gt;

&lt;p&gt;Start connecting:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Service → Dependency
Service → Team
Service → Alert
Alert → Runbook
Service → Dashboard
Service → Repository
Service → Deployment
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h3&gt;
  
  
  Phase 3 — Connect live systems
&lt;/h3&gt;

&lt;p&gt;Add integrations with:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Prometheus
Loki
Grafana
Kubernetes
Jenkins
Terraform
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h3&gt;
  
  
  Phase 4 — Add RAG
&lt;/h3&gt;

&lt;p&gt;Use RAG for the huge amount of historical and unstructured knowledge:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Postmortems
Incident tickets
Slack
Architecture documents
Historical investigations
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h3&gt;
  
  
  Phase 5 — Let the AI investigate
&lt;/h3&gt;

&lt;p&gt;Eventually:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Alert
  ↓
OKF → Understand the system
  ↓
Prometheus → Check metrics
  ↓
Loki → Check logs
  ↓
Jenkins → Check changes
  ↓
RAG → Find similar incidents
  ↓
Runbook → Find remediation
  ↓
LLM → Explain the incident
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That is where the real value starts appearing.&lt;/p&gt;




&lt;h1&gt;
  
  
  11. RAG vs OKF: the easiest mental model
&lt;/h1&gt;

&lt;p&gt;If you remember only three things:&lt;/p&gt;

&lt;h3&gt;
  
  
  RAG
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;"Find the information."&lt;/strong&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  OKF
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;"Organize the knowledge and its relationships."&lt;/strong&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Observability
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;"Tell me what's happening right now."&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Put all three together and you get something much more interesting:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;An AI that doesn't just search your SRE documentation, but understands your infrastructure, follows the relationships between systems, and uses real-time telemetry to investigate incidents.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;And that's where I think the real opportunity for &lt;strong&gt;AI-powered SRE&lt;/strong&gt; lies.&lt;/p&gt;

</description>
      <category>llm</category>
      <category>okf</category>
      <category>rag</category>
      <category>sre</category>
    </item>
    <item>
      <title>Prompt Caching Explained Like You're Talking to a Smart Human (Not an AI Researcher)</title>
      <dc:creator>Anand Kannan</dc:creator>
      <pubDate>Wed, 20 May 2026 18:28:29 +0000</pubDate>
      <link>https://dev.to/anandindia93/prompt-caching-explained-like-youre-talking-to-a-smart-human-not-an-ai-researcher-26gf</link>
      <guid>https://dev.to/anandindia93/prompt-caching-explained-like-youre-talking-to-a-smart-human-not-an-ai-researcher-26gf</guid>
      <description>&lt;p&gt;If you've started using AI APIs for coding assistants, chatbots, agents, RAG systems, or internal copilots, you've probably heard the term:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;“Prompt Caching”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Most explanations online sound overly technical.&lt;/p&gt;

&lt;p&gt;So let’s explain it in the simplest possible way — while still understanding the &lt;em&gt;real engineering and cost impact&lt;/em&gt; behind it.&lt;/p&gt;




&lt;h1&gt;
  
  
  The Real Problem AI Apps Face
&lt;/h1&gt;

&lt;p&gt;Every time you send a request to an LLM, the model has to process:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;system instructions&lt;/li&gt;
&lt;li&gt;chat history&lt;/li&gt;
&lt;li&gt;codebase context&lt;/li&gt;
&lt;li&gt;documentation&lt;/li&gt;
&lt;li&gt;logs&lt;/li&gt;
&lt;li&gt;user question&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Even if 95% of that information is identical to the previous request.&lt;/p&gt;

&lt;p&gt;That repeated processing is expensive.&lt;/p&gt;

&lt;p&gt;Especially in:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;IDE copilots&lt;/li&gt;
&lt;li&gt;SRE assistants&lt;/li&gt;
&lt;li&gt;AI agents&lt;/li&gt;
&lt;li&gt;RAG pipelines&lt;/li&gt;
&lt;li&gt;enterprise chat systems&lt;/li&gt;
&lt;/ul&gt;




&lt;h1&gt;
  
  
  A Simple Tea Shop Analogy
&lt;/h1&gt;

&lt;p&gt;Imagine you go to a tea shop daily and always say:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;“One ginger tea, less sugar, extra hot.”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The shopkeeper memorizes it after a few days.&lt;/p&gt;

&lt;p&gt;Now you just say:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;“Same tea.”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The shopkeeper doesn’t need the full instruction again.&lt;/p&gt;

&lt;p&gt;That is essentially prompt caching.&lt;/p&gt;




&lt;h1&gt;
  
  
  What Actually Happens Without Prompt Cache
&lt;/h1&gt;

&lt;p&gt;Suppose your AI coding assistant sends this every request:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;System Prompt:
You are a senior SRE assistant.

Repository Structure:
- services/
- monitoring/
- infra/

Guidelines:
- Prefer safe kubectl commands
- Suggest rollback plans
- Follow GitOps practices

Previous Chat:
...
...
...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Size:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;25,000 tokens
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then the user asks:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Why is my Kubernetes pod crashing?
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Another:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;300 tokens
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Total request:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;25,300 tokens
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now imagine the user asks 20 debugging questions.&lt;/p&gt;

&lt;p&gt;Without caching:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;25,300 × 20
= 506,000 tokens processed
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Huge cost.&lt;/p&gt;

&lt;p&gt;Huge latency.&lt;/p&gt;

&lt;p&gt;Huge waste.&lt;/p&gt;




&lt;h1&gt;
  
  
  What Prompt Caching Changes
&lt;/h1&gt;

&lt;p&gt;With prompt caching, the provider notices:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;“This large prefix is identical to the previous request.”
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;So it reuses previously processed context.&lt;/p&gt;

&lt;p&gt;Meaning:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;the model does NOT fully re-process repeated sections&lt;/li&gt;
&lt;li&gt;cached sections become cheaper&lt;/li&gt;
&lt;li&gt;responses become faster&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Now only the new question changes.&lt;/p&gt;




&lt;h1&gt;
  
  
  Realistic Flow
&lt;/h1&gt;

&lt;h2&gt;
  
  
  First Request
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[Large Context]
+ New Question
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Provider:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Processes everything normally
Creates cache
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Second Request
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[Same Large Context]
+ Another Question
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Provider:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Cache hit detected
Reuses processed prefix
Only processes delta efficiently
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h1&gt;
  
  
  The Biggest Misunderstanding
&lt;/h1&gt;

&lt;p&gt;Many people think:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;“Cached tokens are free.”&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;No.&lt;/p&gt;

&lt;p&gt;They are usually:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;discounted&lt;/li&gt;
&lt;li&gt;faster&lt;/li&gt;
&lt;li&gt;partially reused&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;But not free.&lt;/p&gt;

&lt;p&gt;The infrastructure still:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;validates cache&lt;/li&gt;
&lt;li&gt;stores context&lt;/li&gt;
&lt;li&gt;reconstructs attention state&lt;/li&gt;
&lt;/ul&gt;




&lt;h1&gt;
  
  
  Real Cost Impact
&lt;/h1&gt;

&lt;p&gt;Let’s use simplified numbers.&lt;/p&gt;

&lt;h2&gt;
  
  
  Without Cache
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Requests&lt;/th&gt;
&lt;th&gt;Tokens Each&lt;/th&gt;
&lt;th&gt;Total&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;20&lt;/td&gt;
&lt;td&gt;25,300&lt;/td&gt;
&lt;td&gt;506,000&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  With Cache
&lt;/h2&gt;

&lt;p&gt;First request:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;25,300
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Remaining 19 requests:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;large cached prefix reused&lt;/li&gt;
&lt;li&gt;only new tokens heavily charged&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Approx effective active processing:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;25,300
+ (19 × 300)
= 31,000
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Instead of:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;506,000
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That’s not a tiny optimization.&lt;/p&gt;

&lt;p&gt;That is the difference between:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;a scalable AI product&lt;/li&gt;
&lt;li&gt;and an unsustainable one&lt;/li&gt;
&lt;/ul&gt;




&lt;h1&gt;
  
  
  Why Coding Assistants Benefit Massively
&lt;/h1&gt;

&lt;p&gt;Coding tools repeatedly send:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;repo instructions&lt;/li&gt;
&lt;li&gt;architecture&lt;/li&gt;
&lt;li&gt;coding guidelines&lt;/li&gt;
&lt;li&gt;open files&lt;/li&gt;
&lt;li&gt;previous chat history&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Most of this barely changes.&lt;/p&gt;

&lt;p&gt;Without prompt caching:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;token burn becomes enormous&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;With prompt caching:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;only diffs matter&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This is why modern AI IDEs feel dramatically faster than early-generation copilots.&lt;/p&gt;




&lt;h1&gt;
  
  
  The SRE / Observability Example
&lt;/h1&gt;

&lt;p&gt;As an SRE, imagine building an AI incident assistant.&lt;/p&gt;

&lt;p&gt;Every request includes:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;- Kubernetes topology
- Loki queries
- Grafana dashboards
- Service dependencies
- Runbooks
- Deployment metadata
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That alone might be:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;40,000 tokens
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then the engineer asks:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Why are pods restarting in prod?
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Without caching:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;40k+ tokens every query
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;With caching:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Topology cached once
Only incident-specific delta processed
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is where enterprises save:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;money&lt;/li&gt;
&lt;li&gt;latency&lt;/li&gt;
&lt;li&gt;compute&lt;/li&gt;
&lt;li&gt;API throughput&lt;/li&gt;
&lt;/ul&gt;




&lt;h1&gt;
  
  
  Prompt Caching vs Memory
&lt;/h1&gt;

&lt;p&gt;People confuse these two constantly.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prompt Caching
&lt;/h2&gt;

&lt;p&gt;Purpose:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Reduce repeated processing
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Focus:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Efficiency
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Memory
&lt;/h2&gt;

&lt;p&gt;Purpose:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Remember user information across sessions
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Focus:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Personalization
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;They are completely different systems.&lt;/p&gt;




&lt;h1&gt;
  
  
  Important Engineering Detail
&lt;/h1&gt;

&lt;p&gt;Prompt caching usually works best when:&lt;/p&gt;

&lt;p&gt;✅ Prompt prefixes remain stable&lt;/p&gt;

&lt;p&gt;Example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[Same system prompt]
[Same repo instructions]
[Different question]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;p&gt;It works poorly when:&lt;/p&gt;

&lt;p&gt;❌ Entire prompt changes every request&lt;/p&gt;

&lt;p&gt;Example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Completely unrelated documents each time
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h1&gt;
  
  
  Hidden Insight Most Tutorials Miss
&lt;/h1&gt;

&lt;p&gt;Prompt caching becomes exponentially valuable as context windows grow.&lt;/p&gt;

&lt;p&gt;Why?&lt;/p&gt;

&lt;p&gt;Because modern LLM apps now send:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;100k+&lt;/li&gt;
&lt;li&gt;200k+&lt;/li&gt;
&lt;li&gt;even million-token contexts&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Without caching:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;costs explode&lt;/li&gt;
&lt;li&gt;latency becomes painful&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Caching is one of the major reasons large-context AI systems are economically feasible today.&lt;/p&gt;




&lt;h1&gt;
  
  
  Important Reality About GitHub Copilot
&lt;/h1&gt;

&lt;p&gt;Tools like GitHub Copilot likely use:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;internal context reuse&lt;/li&gt;
&lt;li&gt;session optimization&lt;/li&gt;
&lt;li&gt;embedding reuse&lt;/li&gt;
&lt;li&gt;smart prompt assembly&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;But they usually do NOT expose:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;manual cache control&lt;/li&gt;
&lt;li&gt;cache hit visibility&lt;/li&gt;
&lt;li&gt;cache TTL settings&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So users benefit indirectly.&lt;/p&gt;




&lt;h1&gt;
  
  
  Final Thought
&lt;/h1&gt;

&lt;p&gt;Prompt caching sounds like a small optimization feature.&lt;/p&gt;

&lt;p&gt;It isn’t.&lt;/p&gt;

&lt;p&gt;It is one of the foundational techniques making modern AI systems:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;affordable&lt;/li&gt;
&lt;li&gt;fast&lt;/li&gt;
&lt;li&gt;scalable&lt;/li&gt;
&lt;li&gt;production-ready&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Without it, many long-context AI applications would become economically impractical very quickly.&lt;/p&gt;

&lt;p&gt;Especially in:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;copilots&lt;/li&gt;
&lt;li&gt;AI agents&lt;/li&gt;
&lt;li&gt;enterprise assistants&lt;/li&gt;
&lt;li&gt;observability platforms&lt;/li&gt;
&lt;li&gt;multi-turn coding workflows&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The bigger your context becomes, the more prompt caching stops being “nice to have” and becomes essential infrastructure.&lt;/p&gt;




&lt;h1&gt;
  
  
  ai #llm #opensource #programming #devops
&lt;/h1&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>performance</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>🚀 Introducing DevFocus: The Micro-SaaS App for Developers – Join the Waitlist Now! 💻</title>
      <dc:creator>Anand Kannan</dc:creator>
      <pubDate>Mon, 21 Apr 2025 17:02:17 +0000</pubDate>
      <link>https://dev.to/anandindia93/introducing-devfocus-the-micro-saas-app-for-developers-join-the-waitlist-now-3dbl</link>
      <guid>https://dev.to/anandindia93/introducing-devfocus-the-micro-saas-app-for-developers-join-the-waitlist-now-3dbl</guid>
      <description>&lt;h1&gt;
  
  
  🚀 DevFocus – A Micro-SaaS With Max Benefits
&lt;/h1&gt;

&lt;p&gt;Too many tools, too little focus?&lt;/p&gt;

&lt;p&gt;That’s what inspired us — two full-time SREs — to build &lt;strong&gt;DevFocus&lt;/strong&gt;: a tool that pulls your tasks from Slack, GitHub, Jira, Gmail, Outlook, Webex, and more into one unified dashboard.&lt;/p&gt;

&lt;h2&gt;
  
  
  ✨ Why?
&lt;/h2&gt;

&lt;p&gt;In a distributed team, it's hard to communicate your current focus across tools. You’re constantly updating Jira, replying to Slack pings, writing daily standup notes, and syncing in status meetings.&lt;/p&gt;

&lt;p&gt;We asked: &lt;em&gt;What if all that happened automatically?&lt;/em&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  🛠️ Key Features
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;🧠 &lt;strong&gt;Unified Task View&lt;/strong&gt; – Pulls from all major sources&lt;/li&gt;
&lt;li&gt;🔄 &lt;strong&gt;Cross-Tool Visibility&lt;/strong&gt; – Let your team know your focus&lt;/li&gt;
&lt;li&gt;🔒 &lt;strong&gt;Private &amp;amp; Secure&lt;/strong&gt; – OAuth2 + only what you allow&lt;/li&gt;
&lt;li&gt;💬 &lt;strong&gt;Team &amp;amp; Personal View&lt;/strong&gt; – Update-less updates!&lt;/li&gt;
&lt;li&gt;📅 &lt;strong&gt;Auto-detect Deadlines&lt;/strong&gt; – Spot urgency instantly&lt;/li&gt;
&lt;li&gt;📈 &lt;strong&gt;Less cognitive load, more flow time&lt;/strong&gt;
&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  &lt;a href="https://techsei.com/" rel="noopener noreferrer"&gt;🔗 Join the waitlist and be the first to try DevFocus!&lt;/a&gt;
&lt;/h2&gt;




&lt;p&gt;We're actively building this, and feedback from early devs is pure gold. Would love your thoughts or help testing!&lt;/p&gt;




&lt;blockquote&gt;
&lt;p&gt;💬 “A now-page for devs — without the manual work”&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>devops</category>
      <category>productivity</category>
      <category>microsaas</category>
    </item>
  </channel>
</rss>
