Pavan Madduri

Posted on Jul 7

MCP Meets OCI: Teaching AI Agents to See Your GPUs on OKE

#oracle #kubernetes #ai #gpu

How to deploy an MCP server on Oracle Kubernetes Engine that gives AI agents real-time GPU metrics - no Prometheus, no dashboards, just ask.

The Problem: AI Agents Are Blind to Their Own Hardware

Here's a scenario I keep running into. You deploy a vLLM inference server on an OKE GPU node pool. You wire up Claude or Goose as your AI coding assistant. The agent generates code, orchestrates pipelines, and manages deployments, but if you ask it "how's the GPU doing on that inference node?" it has no idea.

The agent can query Kubernetes APIs for pod status, read logs, even check CPU metrics. But GPU utilization, memory pressure, temperature, power draw? Invisible. The most expensive resource in your cluster, and your AI assistant can't see it.

This is the gap that the Model Context Protocol (MCP) fills. MCP is a standard that lets AI agents call tools on external systems, the same way a language model calls a function, but over a protocol that Claude, Cursor, Windsurf, and Goose all understand natively. You register a tool (like "get GPU metrics"), the agent discovers it, and it can call it whenever it needs hardware context.

I built gpu-mcp-server to close this loop. It's an MCP server that reads NVIDIA GPU metrics directly from NVML and exposes them as tools that any MCP-compatible agent can call. No Prometheus pipeline, no PromQL, no dashboard the agent just asks.

What MCP Actually Is

MCP (Model Context Protocol) is an open standard created by Anthropic and adopted by the broader AI tooling ecosystem. Think of it as "USB for AI agents" - a standardized way for agents to connect to data sources and tools.

When an agent connects to an MCP server, it discovers the available tools automatically:

Agent: "What tools do you have?"
MCP Server: "list_gpus, get_gpu_metrics, get_gpu_processes, gpu_summary"
Agent: "OK, calling get_gpu_metrics with index 0..."
MCP Server: { utilization: 87%, memory: 58GB/80GB, temp: 72°C, power: 300W }

The server communicates over stdio (for local use) or HTTP with Server-Sent Events (for remote/Kubernetes deployments). The agent doesn't need to know about NVML, GPU drivers, or device files. It just calls tools and gets structured JSON back.

This is why MCP matters for infrastructure: it makes hardware state a first-class input to AI reasoning. An agent can now say "GPU 0 is at 95% utilization and 72°C. I should check if the inference deployment needs more replicas" without you writing a single monitoring rule.

Architecture on OKE

┌─────────────────────────────────────────────────────────────────┐
│                  OKE Cluster (GPU Node Pool)                    │
│                                                                 │
│  ┌──────────────────┐     ┌──────────────────────────────────┐  │
│  │  AI Agent Pod    │     │  GPU Worker Node (VM.GPU.A10.1)  │  │
│  │  (Claude/Goose)  │     │                                  │  │
│  │                  │ MCP │  ┌────────────────────────────┐  │  │
│  │  "What's the GPU │────▶│  │  gpu-mcp-server            │  │  │
│  │   utilization?"  │     │  │  (DaemonSet pod)           │  │  │
│  │                  │◀────│  │                            │  │  │
│  │  "87% util,      │ JSON│  │  NVML ──→ GPU 0 (A10 24GB) │  │  │
│  │   58GB/80GB mem" │     │  │  NVML ──→ GPU 1 (A10 24GB) │  │  │
│  └──────────────────┘     │  └────────────────────────────┘  │  │
│                           └──────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘

The gpu-mcp-server runs as a DaemonSet one pod per GPU node. Each pod calls NVML directly through Go's cgo bindings to read local GPU hardware metrics. No sidecar collectors, no metric pipelines, no network hops to a central Prometheus. The data comes straight from the GPU driver.

GPU Metrics Exposed as MCP Tools

The server registers four tools:

Tool	What It Returns
`list_gpus`	All GPUs with utilization and memory
`get_gpu_metrics`	Full metrics for one GPU (util, memory, temp, power, PCIe, NVLink)
`get_gpu_processes`	PID-level process attribution
`gpu_summary`	Aggregate stats across all devices

When an agent calls get_gpu_metrics with {"index": 0}, it gets back:

{
  "index": 0,
  "name": "NVIDIA A10",
  "gpu_utilization_percent": 87,
  "memory_used_mib": 18432,
  "memory_total_mib": 24576,
  "temperature_celsius": 68,
  "power_draw_watts": 135,
  "power_limit_watts": 150,
  "pcie_tx_kbps": 524288,
  "pcie_rx_kbps": 262144
}

That's hardware telemetry the agent can reason about. It can correlate GPU memory pressure with inference latency, spot thermal throttling, or detect that a GPU is idle and suggest scaling down all from natural language.

Deploying on OKE

1. Build and Push to OCIR

# Build the binary (requires CGO + NVML headers)
make build

# Build container image
docker build -t iad.ocir.io/mytenancy/infra/gpu-mcp-server:v1.0 .

# Push to OCI Container Registry
docker push iad.ocir.io/mytenancy/infra/gpu-mcp-server:v1.0

2. Deploy as DaemonSet

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: gpu-mcp-server
  namespace: gpu-monitoring
spec:
  selector:
    matchLabels:
      app: gpu-mcp-server
  template:
    metadata:
      labels:
        app: gpu-mcp-server
    spec:
      nodeSelector:
        nvidia.com/gpu.present: "true"
      tolerations:
        - key: nvidia.com/gpu
          operator: Exists
          effect: NoSchedule
      containers:
        - name: mcp-server
          image: iad.ocir.io/mytenancy/infra/gpu-mcp-server:v1.0
          ports:
            - containerPort: 8080
              name: mcp
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop: ["ALL"]
          resources:
            requests:
              cpu: "100m"
              memory: "64Mi"
            limits:
              cpu: "500m"
              memory: "128Mi"
---
apiVersion: v1
kind: Service
metadata:
  name: gpu-mcp-server
  namespace: gpu-monitoring
spec:
  selector:
    app: gpu-mcp-server
  ports:
    - port: 8080
      targetPort: mcp

The DaemonSet targets nodes with NVIDIA GPUs and tolerates the standard GPU taint that OKE applies. Each pod consumes minimal resources under 64MB of memory because it's just reading from the GPU driver. No heavyweight collectors, no metric storage.

3. Connect Your Agent

For Claude Desktop running locally with kubectl port-forward:

kubectl port-forward svc/gpu-mcp-server -n gpu-monitoring 8080:8080

Then in claude_desktop_config.json:

{
  "mcpServers": {
    "oke-gpu": {
      "command": "curl",
      "args": ["-N", "http://localhost:8080/mcp"]
    }
  }
}

For agents running inside the same OKE cluster, they reach the MCP server directly via the ClusterIP service no port-forward needed, no external network exposure.

Why This Matters for Agentic AI Infrastructure

The Agentic AI Foundation (AAIF) a Linux Foundation initiative is building the ecosystem for production AI agent infrastructure. MCP is one of the core protocols they've aligned around. The idea is simple: agents need standard ways to access tools, data, and infrastructure context.

gpu-mcp-server fits this model directly. Instead of building custom integrations for every agent framework, you deploy one MCP server and every MCP-compatible agent gets GPU awareness. Claude, Goose, Cursor, Windsurf. They all speak MCP natively. Ship once, works everywhere.

This is particularly relevant on OCI because Oracle's GPU portfolio is expanding fast. With VM.GPU.A10.1, BM.GPU.A10.4, BM.GPU.H100.8, and the new GB200 shapes, there's real hardware diversity to manage. An AI agent that can query GPU metrics across a mixed fleet A10s for inference, H100s for training and make informed decisions about workload placement is genuinely useful, not just a demo.

OCI Cost Context

GPU observability isn't just about uptime it's about money. On OCI:

Shape	GPUs	On-Demand $/hr	Preemptible $/hr
VM.GPU.A10.1	1 × A10	~$1.52	~$0.46
BM.GPU.A10.4	4 × A10	~$6.08	~$1.82
BM.GPU.H100.8	8 × H100	~$26.00	—

When an agent can see that GPU 0 on a BM.GPU.A10.4 is idle, it can recommend moving the workload to a cheaper VM.GPU.A10.1 or scaling down entirely. That visibility is the first step to GPU FinOps, and MCP makes it queryable in plain language.

Connection to Oracle University Training

This touches several OCI learning paths. The OCI AI Infrastructure courses cover GPU shapes and compute options — the hardware layer the MCP server monitors. The OCI Cloud Native Professional path teaches OKE DaemonSets, OCIR image management, and ClusterIP services, which is exactly how this deploys. If you're studying for the OCI Cloud Native Associate or Professional certifications, building an MCP server on OKE is a strong hands-on project that reinforces container orchestration, service discovery, and node-affinity scheduling.

MCP itself is worth understanding regardless of certifications. As AI agents become standard in enterprise workflows, the engineers who understand how to give agents structured access to infrastructure state will be the ones architecting production systems.

Try It

Provision an OKE cluster with a GPU node pool (even a single VM.GPU.A10.1).
Deploy gpu-mcp-server as a DaemonSet.
Connect Claude Desktop via port-forward.
Ask Claude: "What's the GPU utilization on my OKE cluster?"
Watch it call list_gpus and give you a real answer from real hardware.

Your AI agent should be able to see the infrastructure it runs on. MCP makes that possible. OKE provides the GPU nodes. gpu-mcp-server bridges the gap.

GitHub: github.com/pmady/gpu-mcp-server

Pavan Madduri is a Senior Cloud Platform Engineer at W.W. Grainger, a CNCF Golden Kubestronaut, and CNCF TAG Workloads Foundation Tech Lead. He maintains gpu-mcp-server and keda-gpu-scaler and contributes to CNCF projects including KEDA, Volcano, and Dragonfly. Find him on GitHub.

DEV Community