What we are building
Imagine your company has this problem:
Developer:
"payment-service is returning 503.
Why is it failing and what should I do?"
An ordinary LLM might know general Kubernetes troubleshooting, but it does not automatically know:
our company's payment-service
our runbook
our deployment version
our internal incidents
our current pod status
our current application logs
our company procedures
So we will build this:
USER
|
|
v
"Why is payment-service
returning 503?"
|
v
+-------------------+
| Python AI App |
+-------------------+
/ \
/ \
v v
+---------+ +---------+
| RAG | | MCP |
+---------+ +---------+
| |
| |
v v
Company runbooks Live operational
documentation information
| |
v v
embeddings MCP tools
| |
+-------+--------+
|
v
+---------+
| LLM |
+---------+
|
v
Explanation + action
The most important lesson of the entire lab is:
LLM
=
generates/reasons over text
RAG
=
provides relevant KNOWLEDGE to the LLM
MCP
=
provides standardized access to DATA and CAPABILITIES
DevOps
=
deploys, secures, monitors, scales and operates
all of the above
PART 0 — Before writing code: what exactly is an LLM?
Do this explanation on the whiteboard before opening VS Code.
0.1 Artificial Intelligence
AI is the broad category.
Artificial Intelligence
Machines performing tasks
that appear intelligent
Examples:
recommendations
computer vision
speech recognition
fraud detection
chatbots
autonomous systems
AI is the umbrella.
0.2 Machine Learning
Traditional programming looks like this:
RULES + DATA
|
v
OUTPUT
For example:
if cpu > 90:
print("High CPU")
A human explicitly wrote the rule.
Machine learning changes the idea:
DATA + EXPECTED RESULTS
|
v
TRAINING
|
v
MODEL
Instead of manually writing every rule, the machine learns patterns from examples.
For your students, use this analogy:
Traditional programming:
Engineer writes:
IF CPU > 90
THEN alert
Machine learning:
Give system thousands of examples:
CPU
Memory
Latency
Requests
Failures
and tell it:
NORMAL
ABNORMAL
NORMAL
ABNORMAL
The model learns relationships.
0.3 Deep Learning
Deep learning is a type of machine learning using neural networks with many layers.
You don't need to teach the mathematics first.
Tell students:
Machine Learning
|
v
Deep Learning
|
v
Neural Networks
0.4 What is a model?
This word confuses beginners.
A model is the learned mathematical system produced through training.
Think:
training data
|
v
training process
|
v
MODEL
A deployed model can then receive new input.
That phase is called:
Inference
USER INPUT
|
v
pretrained model
|
v
generated output
In this lab we are not training GPT.
We are doing inference.
That's a critical distinction.
0.5 What does LLM mean?
LLM:
Large
Language
Model
Large
Large amounts of:
parameters
training data
compute
Language
It processes language representations.
Examples:
English
Russian
Python
Java
YAML
Terraform
JSON
Kubernetes manifests
logs
Model
It is a trained mathematical model.
0.6 The most simplified mental model
When you type:
Kubernetes pod is crashing because...
an LLM essentially predicts what text should follow based on learned patterns.
Conceptually:
INPUT
"The Kubernetes pod is..."
|
v
LLM
|
v
possible continuation probabilities
crashing 32%
running 18%
failing 15%
unable 11%
...
|
v
generated output
Real modern LLMs are much more sophisticated than this simplified diagram, but this is the right first mental model.
0.7 What is a token?
The LLM doesn't operate on your sentence exactly as humans see it.
Text is divided into tokens.
For teaching purposes:
"Kubernetes deployment failed"
could conceptually become pieces such as:
Kuber
netes
deployment
failed
Do not tell students one word always equals one token.
It doesn't.
The important idea is:
text
↓
tokens
↓
numbers
↓
model processing
↓
tokens
↓
text
0.8 Why can an LLM hallucinate?
Because the LLM is generating a likely response.
It is not automatically doing:
SELECT *
FROM company_database
WHERE truth = true;
An LLM may produce something that sounds correct but isn't grounded in your company's actual data.
That is one reason RAG exists.
PART 1 — Create the project
We will work on a Mac/Linux terminal.
Open Terminal.
Run:
mkdir ai-devops-rag-mcp
cd ai-devops-rag-mcp
What did we do?
mkdir means:
make directory
We created:
ai-devops-rag-mcp/
Then:
cd ai-devops-rag-mcp
means:
change directory
Now we're inside the project.
Run:
pwd
Expected result on a Mac might look like:
/Users/yourname/ai-devops-rag-mcp
PART 2 — Create a Python virtual environment
Run:
python3 -m venv .venv
Then:
source .venv/bin/activate
Your terminal should change to something similar to:
(.venv) user@macbook ai-devops-rag-mcp %
Why do we need .venv?
Think like a DevOps engineer.
Application A might need:
openai version X
mcp version Y
Application B might require different versions.
Instead of installing everything globally on the laptop:
Mac
├── application A dependencies
├── application B dependencies
├── application C dependencies
we isolate them:
Project
|
+-- .venv
|
+-- openai
+-- mcp
+-- python-dotenv
+-- numpy
This is similar conceptually to dependency isolation you already understand from containers.
PART 3 — Open project in VS Code
Run:
code .
If code isn't configured, open VS Code manually:
VS Code
→ File
→ Open Folder
→ ai-devops-rag-mcp
PART 4 — Create project structure
Create:
ai-devops-rag-mcp/
│
├── .env
├── .gitignore
├── requirements.txt
│
├── knowledge/
│ ├── payment-runbook.txt
│ ├── kubernetes-runbook.txt
│ └── company-architecture.txt
│
├── 01_llm.py
├── 02_embeddings.py
├── 03_rag.py
├── mcp_server.py
└── 05_final_assistant.py
Why these names?
Because I want students to see the evolution:
01_llm.py
↓
basic intelligence
02_embeddings.py
↓
meaning as numbers
03_rag.py
↓
private knowledge
mcp_server.py
↓
external capability
05_final_assistant.py
↓
combine everything
PART 5 — Install dependencies
Open:
requirements.txt
Paste:
openai
python-dotenv
numpy
mcp
Save.
Now terminal:
pip install -r requirements.txt
The official MCP Python SDK can be installed with the mcp package and supports building MCP servers and clients. (GitHub)
PART 6 — API key
Create:
.env
Paste:
OPENAI_API_KEY=YOUR_API_KEY_HERE
Do not commit this.
Create:
.gitignore
Paste:
.env
.venv/
__pycache__/
This is a DevOps/security lesson.
Never do this:
api_key = "sk-real-secret-key"
inside committed source code.
Instead:
source code
+
environment/config
=
running application
Later production architecture might use:
AWS Secrets Manager
Kubernetes Secrets
Vault
GitHub Actions Secrets
OIDC/workload identity
Also remember: an API key alone isn't enough if the API account/project has no available billing quota. An authentication problem and a quota problem are different failures.
PART 7 — Our first LLM call
Create:
01_llm.py
Paste:
from openai import OpenAI
from dotenv import load_dotenv
load_dotenv()
client = OpenAI()
question = "Explain Kubernetes CrashLoopBackOff in simple DevOps language."
response = client.responses.create(
model="gpt-5.6",
input=question
)
print(response.output_text)
OpenAI's current documentation shows the Responses API as the primary pattern for model interaction, including current examples using client.responses.create(...). (OpenAI Platform)
Now run:
python 01_llm.py
You should receive an explanation of CrashLoopBackOff.
The exact wording can differ from run to run.
STOP HERE AND EXPLAIN EVERY LINE
Line 1
from openai import OpenAI
Meaning:
Python, please import the
OpenAIclient class from the OpenAI package.
We installed that package earlier with:
pip install openai
Without the import Python doesn't know what OpenAI means.
Line 2
from dotenv import load_dotenv
This lets Python load values from:
.env
Line 4
load_dotenv()
This reads:
OPENAI_API_KEY=...
and makes it available as an environment variable.
Line 6
client = OpenAI()
This is important.
We create a client object.
Think:
OUR PYTHON PROGRAM
|
| OpenAI client
|
v
OpenAI API
The variable name could technically be:
banana = OpenAI()
and Python wouldn't care.
But we call it:
client
because it represents the API client.
Names are selected by programmers for readability.
Line 8
question = "Explain Kubernetes CrashLoopBackOff in simple DevOps language."
We create a variable named:
question
The value is a string.
A string is text.
Line 10
response = client.responses.create(
Break this into pieces.
client
Our OpenAI connection object.
.responses
We are using the Responses API.
.create()
Create a new model response.
So conceptually:
send request
↓
model processes request
↓
receive response
This line
model="gpt-5.6",
answers:
Which model do we want to use?
Current OpenAI MCP/Responses documentation uses gpt-5.6 in its examples. (OpenAI Platform)
This line
input=question
means:
send the value stored inside variable question
So Python replaces:
input=question
conceptually with:
input="Explain Kubernetes CrashLoopBackOff..."
Finally
print(response.output_text)
response contains the API result.
output_text gives us the generated text.
print() displays it in Terminal.
What students have proven
This is now a functioning LLM application:
Python
|
| request
v
OpenAI API
|
v
LLM
|
| response
v
Python
|
v
Terminal
Ask your students:
Where is the intelligence?
Not here:
print()
Not here:
question =
The trained model provides the learned language capability.
Our code orchestrates access to it.
That separation is extremely important for DevOps engineers.
PART 8 — Demonstrate the LLM's limitation
Now change:
question =
to:
question = """
What is our company's exact procedure for payment-service
when database connection pool utilization exceeds 90%?
"""
Run:
python 01_llm.py
The LLM may provide a reasonable general answer.
But ask:
How could the model know our internal company procedure?
It doesn't automatically have our private runbook.
Here is the problem:
LLM
|
General learned knowledge
|
X
|
private company runbook
This is where RAG enters.
PART 9 — Create private company knowledge
Create:
knowledge/payment-runbook.txt
Paste:
PAYMENT SERVICE PRODUCTION RUNBOOK
Service: payment-service
Team: Payments Platform
Incident: HTTP 503 errors
Common causes:
1. Database connection pool exhaustion.
2. payment-service pods not ready.
3. Upstream authentication-service unavailable.
4. Deployment configuration errors.
Company-specific procedure:
If database connection pool utilization exceeds 90 percent:
1. Check payment-service pod logs.
2. Verify PostgreSQL connectivity.
3. Check active database connections.
4. Scale payment-service from 3 replicas to 6 replicas.
5. Do not restart the PostgreSQL database without approval.
6. Notify the Payments Platform team.
7. Open incident severity SEV-2 if errors continue for more than 10 minutes.
Rollback command:
kubectl rollout undo deployment/payment-service -n banking
Healthy replica count:
3 minimum.
Production namespace:
banking
Now:
knowledge/kubernetes-runbook.txt
Paste:
KUBERNETES TROUBLESHOOTING RUNBOOK
CrashLoopBackOff procedure:
1. Run kubectl get pods -n banking.
2. Run kubectl describe pod POD_NAME -n banking.
3. Run kubectl logs POD_NAME -n banking.
4. Inspect environment variables.
5. Inspect Kubernetes Secrets and ConfigMaps.
6. Check readiness and liveness probes.
7. Review the latest deployment.
8. Roll back only if the newest deployment caused the incident.
ImagePullBackOff procedure:
1. Confirm image name.
2. Confirm image tag.
3. Confirm Amazon ECR image exists.
4. Validate imagePullSecrets when applicable.
5. Validate IAM permissions.
And:
knowledge/company-architecture.txt
Paste:
BANKING PLATFORM ARCHITECTURE
Environment: AWS
Container platform:
Amazon EKS
Production namespace:
banking
Services:
authentication-service
customer-service
account-service
payment-service
notification-service
CI/CD:
GitHub Actions
GitOps:
Argo CD
Container registry:
Amazon ECR
Infrastructure as Code:
Terraform
Monitoring:
Prometheus
Grafana
CloudWatch
Database:
Amazon RDS PostgreSQL
Now the important question:
How do we give these documents to the LLM?
We could put all documents into every prompt.
But imagine:
10 documents
100 documents
10,000 documents
1,000,000 documents
We need retrieval.
PART 10 — What does RAG mean?
RAG:
Retrieval-Augmented Generation
Break the name down.
Retrieval
Find relevant information.
Augmented
Add that information to the model's context.
Generation
The LLM generates an answer.
Therefore:
QUESTION
|
v
RETRIEVE
relevant information
|
v
AUGMENT
the prompt
|
v
GENERATE
answer with LLM
That's RAG.
Very important
RAG does not mean:
train the LLM again
We are not changing the model's weights.
We are doing:
pretrained LLM
+
retrieved context
=
better grounded response
PART 11 — But how can a computer search by meaning?
This introduces:
Embeddings
Suppose our user asks:
"What should I do when payment DB connections are full?"
Our document says:
"If database connection pool utilization exceeds 90 percent..."
Exact words differ.
A simple keyword search may struggle.
But semantically:
DB connections full
and
database connection pool exceeds 90%
are related.
Embeddings represent text as vectors—lists of numbers—which can be compared to estimate semantic relatedness. OpenAI's current documentation shows text-embedding-3-small returning a numeric embedding vector; by default that model produces a 1,536-dimensional vector. (OpenAI Platform)
PART 12 — See an embedding yourself
Create:
02_embeddings.py
Paste:
from openai import OpenAI
from dotenv import load_dotenv
load_dotenv()
client = OpenAI()
text = "Kubernetes pod cannot connect to PostgreSQL"
response = client.embeddings.create(
model="text-embedding-3-small",
input=text
)
vector = response.data[0].embedding
print("Original text:")
print(text)
print("\nFirst 10 numbers:")
print(vector[:10])
print("\nNumber of dimensions:")
print(len(vector))
Run:
python 02_embeddings.py
Expected shape:
Original text:
Kubernetes pod cannot connect to PostgreSQL
First 10 numbers:
[-0.00..., 0.01..., -0.02..., ...]
Number of dimensions:
1536
The exact values will vary.
OpenAI documents the same embedding call pattern and notes that text-embedding-3-small defaults to 1,536 dimensions. (OpenAI Platform)
What happened?
Input:
"Kubernetes pod cannot connect to PostgreSQL"
became something conceptually like:
[
-0.0132,
0.0211,
-0.0043,
...
]
Not random numbers.
They represent learned semantic characteristics in a high-dimensional space.
Why numbers?
Computers can compare numbers efficiently.
Consider simplified 3-dimensional vectors:
"database error"
[0.9, 0.2, 0.7]
and:
"PostgreSQL connection failure"
[0.88, 0.19, 0.72]
They're geometrically close.
But:
"best pizza recipe"
[-0.2, 0.8, -0.5]
would be farther away.
Real embeddings have many more dimensions.
PART 13 — Similarity
We need to compare vectors.
For this lab we will use:
Cosine similarity
Conceptually:
vector A ↘
\ angle
\
vector B --->
Smaller semantic angle → generally higher similarity.
Scores are commonly interpreted comparatively rather than as magical absolute truth.
For the class:
query vs payment runbook 0.82
query vs Kubernetes runbook 0.57
query vs architecture 0.35
Choose highest.
That's retrieval.
PART 14 — Build RAG manually
Now create:
03_rag.py
Paste:
import os
import numpy as np
from openai import OpenAI
from dotenv import load_dotenv
load_dotenv()
client = OpenAI()
def read_documents():
documents = []
for filename in os.listdir("knowledge"):
if filename.endswith(".txt"):
path = os.path.join("knowledge", filename)
with open(path, "r") as file:
text = file.read()
documents.append({
"filename": filename,
"text": text
})
return documents
def create_embedding(text):
response = client.embeddings.create(
model="text-embedding-3-small",
input=text
)
return response.data[0].embedding
def cosine_similarity(vector_a, vector_b):
a = np.array(vector_a)
b = np.array(vector_b)
return np.dot(a, b) / (
np.linalg.norm(a) * np.linalg.norm(b)
)
documents = read_documents()
print(f"Loaded {len(documents)} documents.")
for document in documents:
document["embedding"] = create_embedding(document["text"])
question = """
payment-service returns 503 and database connections
are above 90 percent. What should I do?
"""
question_embedding = create_embedding(question)
for document in documents:
document["score"] = cosine_similarity(
question_embedding,
document["embedding"]
)
documents = sorted(
documents,
key=lambda document: document["score"],
reverse=True
)
best_document = documents[0]
print("\nSimilarity results:")
for document in documents:
print(
document["filename"],
round(document["score"], 4)
)
print("\nBest document:")
print(best_document["filename"])
context = best_document["text"]
prompt = f"""
You are a Senior DevOps Incident Assistant.
Answer the question using ONLY the company context below.
If the answer is not contained in the context,
say that you do not have enough company information.
COMPANY CONTEXT:
{context}
QUESTION:
{question}
"""
response = client.responses.create(
model="gpt-5.6",
input=prompt
)
print("\nAI ANSWER:")
print(response.output_text)
PART 15 — Understand every section of the RAG code
Start here:
import os
os helps Python work with:
files
folders
paths
operating system functionality
Then:
import numpy as np
NumPy gives us numerical operations.
We use it to compare vectors.
np is just a commonly used shorter alias.
This:
def read_documents():
means:
Define a reusable function called
read_documents.
A function is a block of reusable logic.
Think of Terraform:
module "vpc" {
}
You don't want to copy the same infrastructure logic everywhere.
Programming functions serve a similar reuse principle.
Inside:
documents = []
creates an empty Python list.
Think:
documents
[
]
Later:
[
payment-runbook,
kubernetes-runbook,
company-architecture
]
This:
for filename in os.listdir("knowledge"):
means:
Go through every file in the
knowledgefolder.
Like:
ls knowledge
but programmatically.
This:
if filename.endswith(".txt"):
means:
Only process text files.
This:
with open(path, "r") as file:
means:
open this file
in read mode
r = read.
This:
text = file.read()
reads file contents into memory.
Then:
documents.append({
"filename": filename,
"text": text
})
We store:
filename
+
document contents
Example:
{
"filename": "payment-runbook.txt",
"text": "PAYMENT SERVICE PRODUCTION RUNBOOK..."
}
Now the embedding function
def create_embedding(text):
We created our own reusable function.
Input:
text
Output:
embedding vector
Inside:
response = client.embeddings.create(
asks OpenAI's embedding model to convert text into a vector.
The official OpenAI example uses the same API shape with text-embedding-3-small. (OpenAI Platform)
Then:
return response.data[0].embedding
return sends the result back to whoever called the function.
Example:
vector = create_embedding("hello")
becomes conceptually:
vector = [0.02, -0.01, ...]
Now similarity
def cosine_similarity(vector_a, vector_b):
takes two vectors.
Then:
a = np.array(vector_a)
b = np.array(vector_b)
converts regular Python lists to NumPy arrays.
Then:
np.dot(a, b)
calculates their dot product.
And:
np.linalg.norm(a)
calculates vector magnitude.
This implements the cosine similarity formula.
You do not need to teach linear algebra deeply yet.
Senior DevOps students need to understand:
query embedding
|
| compare
v
document embeddings
highest similarity
|
v
most relevant document
PART 16 — Run RAG
Execute:
python 03_rag.py
Expected style of output:
Loaded 3 documents.
Similarity results:
payment-runbook.txt 0.72
company-architecture.txt 0.55
kubernetes-runbook.txt 0.49
Best document:
payment-runbook.txt
Your actual similarity numbers can differ.
Then an answer should include company-specific information such as:
Check payment-service pod logs.
Verify PostgreSQL connectivity.
Check active connections.
Scale payment-service from 3 to 6 replicas.
Do not restart PostgreSQL without approval.
Notify the Payments Platform team.
Open SEV-2 if errors continue over 10 minutes.
Now ask:
Did we retrain GPT?
No.
What happened?
User question
|
v
embedding
|
v
similarity search
|
v
payment-runbook found
|
v
runbook inserted into prompt
|
v
LLM
|
v
grounded answer
That is RAG.
PART 17 — Show students the actual augmented prompt
Before:
response = client.responses.create(
add:
print("\nFINAL PROMPT SENT TO LLM:")
print(prompt)
Run:
python 03_rag.py
Now students can literally see:
You are a Senior DevOps Incident Assistant.
COMPANY CONTEXT:
PAYMENT SERVICE PRODUCTION RUNBOOK
...
QUESTION:
payment-service returns 503...
This is the moment RAG usually clicks.
There is no mysterious magic.
The retrieval system found text and put it into the LLM's context.
PART 18 — Our simplified RAG architecture
What we built:
FILES
|
+-- payment-runbook
+-- kubernetes-runbook
+-- architecture
|
v
embeddings
|
v
vectors in memory
^
|
question → embedding
|
v
cosine similarity
|
v
relevant document
|
v
prompt/context
|
v
LLM
|
v
answer
PART 19 — Why would production RAG be more complicated?
Our lab embeds the same files every execution.
That's intentionally inefficient.
We do it because students need to see the mechanics.
Production might look like:
INGESTION PIPELINE
PDF / Confluence / GitHub / S3
|
v
parsing
|
v
chunking
|
v
embeddings
|
v
vector store
Then separately:
QUERY PIPELINE
user question
|
v
query embedding
|
v
vector search
|
v
top relevant chunks
|
v
prompt
|
v
LLM
PART 20 — What is chunking?
Imagine a 300-page PDF.
We don't necessarily create one vector for the entire book.
Instead:
300-page PDF
|
v
split
|
+-- chunk 1
+-- chunk 2
+-- chunk 3
+-- chunk 4
...
For example:
chunk 1:
payment-service architecture
chunk 2:
database failure procedures
chunk 3:
rollback procedure
chunk 4:
monitoring
Then retrieval can return the specific relevant portion.
Why chunk?
If the user asks:
How do I rollback payment-service?
we would rather retrieve:
rollback section
than feed a huge unrelated document.
PART 21 — What is a vector database?
Our vectors currently live only in Python memory.
Python process
|
v
vectors
When the process stops, our generated vectors disappear.
Production can use a vector-capable retrieval system.
Conceptually:
document
|
embedding
|
v
+----------------+
| vector store |
|----------------|
| vector |
| text |
| metadata |
+----------------+
Possible metadata:
service = payment-service
environment = production
document = runbook
team = payments
version = 4
Then retrieve:
Top 5 most semantically related chunks
OpenAI's embedding documentation explicitly describes saving embedding vectors in a vector database for retrieval/search use cases. (OpenAI Platform)
PART 22 — We still have a problem
RAG can tell us:
company procedure
architecture
documentation
previous knowledge
But suppose the user asks:
How many payment-service replicas are running RIGHT NOW?
Our runbook says:
Healthy replica count: 3 minimum
But that doesn't tell us:
current live replicas
We need external system access.
For example:
kubectl get deployment payment-service -n banking
Or:
kubectl get pods
Or CloudWatch.
Or AWS.
Or GitHub.
This is where MCP becomes useful.
PART 23 — What is MCP?
MCP stands for:
Model Context Protocol
The official MCP project describes it as an open standard for connecting AI applications to external systems—analogous to a standardized connector for AI applications. MCP servers can expose capabilities such as tools and resources to clients. (Model Context Protocol)
Think about USB-C.
Without a standard connector:
Device A → custom cable
Device B → different cable
Device C → different cable
Device D → different cable
With USB-C:
Device A ─┐
Device B ─┼── USB-C
Device C ─┤
Device D ─┘
MCP tries to provide a standardized interface between AI applications and external systems.
Traditional integration problem
Without MCP you might write custom integrations:
AI application
|
+-- custom Kubernetes integration
|
+-- custom GitHub integration
|
+-- custom Jira integration
|
+-- custom AWS integration
|
+-- custom database integration
MCP provides a common protocol model:
AI HOST
|
MCP CLIENT
|
+----------+---------+
| | |
v v v
MCP K8s MCP GitHub MCP AWS
server server server
PART 24 — Important MCP concepts
There are several pieces students need to understand.
MCP HOST
|
MCP CLIENT
|
MCP SERVER
|
TOOLS / RESOURCES
|
external systems
Host
The AI application/environment.
Client
Handles the MCP connection.
Server
Exposes capabilities.
Tool
Something the model/application can invoke.
Official MCP documentation describes tools as server-exposed capabilities that can interact with external systems such as APIs, databases, or computations. (Model Context Protocol)
Examples:
get_pods()
restart_service()
get_logs()
create_ticket()
query_database()
Resource
Data the MCP server exposes as contextual information, such as files, schemas, or application-specific information. (Model Context Protocol)
Examples:
runbook://payment
config://production
schema://database
PART 25 — Build our first MCP server
For safety, we will simulate Kubernetes first.
That is intentional.
Do not give beginner agents permission to delete production resources.
Create:
mcp_server.py
Paste:
from mcp.server import MCPServer
mcp = MCPServer("Banking DevOps MCP")
@mcp.tool()
def get_service_status(service_name: str) -> str:
"""
Return the current simulated Kubernetes status
for a banking service.
"""
services = {
"payment-service": {
"desired_replicas": 3,
"ready_replicas": 2,
"status": "DEGRADED"
},
"authentication-service": {
"desired_replicas": 3,
"ready_replicas": 3,
"status": "HEALTHY"
}
}
service = services.get(service_name)
if service is None:
return f"Service {service_name} was not found."
return (
f"Service: {service_name}\n"
f"Desired replicas: {service['desired_replicas']}\n"
f"Ready replicas: {service['ready_replicas']}\n"
f"Status: {service['status']}"
)
@mcp.tool()
def get_recent_logs(service_name: str) -> str:
"""
Return simulated recent production logs.
"""
if service_name == "payment-service":
return """
2026-08-10T13:02:14Z ERROR database connection pool exhausted
2026-08-10T13:02:15Z ERROR timeout acquiring PostgreSQL connection
2026-08-10T13:02:16Z WARN readiness probe failed
"""
return f"No critical logs found for {service_name}."
@mcp.resource("runbook://payment-service")
def payment_resource() -> str:
"""
Return basic payment-service operational metadata.
"""
return """
Service: payment-service
Namespace: banking
Team: Payments Platform
Environment: production
"""
if __name__ == "__main__":
mcp.run()
The current official Python MCP SDK's v2 examples use MCPServer, and the SDK supports both server tools and resources. (GitHub)
PART 26 — Understand MCP code
This:
from mcp.server import MCPServer
imports the MCP server implementation.
This:
mcp = MCPServer("Banking DevOps MCP")
creates our MCP server.
We named it:
Banking DevOps MCP
The key line
@mcp.tool()
This is a Python decorator.
For beginners, don't go deep into decorator internals.
Explain:
This tells the MCP server: expose the function below as an MCP tool.
Without:
@mcp.tool()
it's just a normal Python function.
With it:
Python function
+
MCP registration
=
MCP tool
Then:
def get_service_status(service_name: str) -> str:
Let's break it apart.
def
Define function.
get_service_status
Function name.
service_name
Input parameter.
: str
Input should be text.
-> str
Function returns text.
Why descriptive function names matter enormously in AI tools
Compare:
def x(a):
with:
def get_service_status(service_name: str):
The second describes what it does.
Tool names and descriptions matter because the AI system needs to understand which capability is appropriate.
This dictionary
services = {
is simply simulated Kubernetes data.
We're pretending:
payment-service
desired = 3
ready = 2
status = DEGRADED
Later we can replace this with:
kubectl
or the Kubernetes Python API.
The MCP interface can stay conceptually similar.
This is powerful:
today
MCP tool
|
simulated dictionary
tomorrow
MCP tool
|
Kubernetes API
The consumer still thinks:
get_service_status
PART 27 — Tool versus resource
We created:
@mcp.tool()
and:
@mcp.resource(...)
What's the difference?
Simplified teaching version:
RESOURCE
≈ information/context
TOOL
≈ callable capability
Example resource:
runbook://payment-service
Example tool:
get_service_status("payment-service")
MCP's specification treats resources as server-exposed contextual data and tools as model-invocable capabilities. (Model Context Protocol)
PART 28 — Why MCP isn't the LLM
Very important.
Our MCP server contains:
get_service_status()
get_recent_logs()
But it does not itself understand natural language like:
Why is checkout broken?
It exposes capabilities.
The LLM provides language interpretation/reasoning.
Think:
LLM
"I think I need logs."
|
v
MCP
"Here is get_recent_logs."
|
v
external system
|
v
log result
|
v
LLM
"I can now explain the incident."
PART 29 — RAG vs MCP
Students MUST be able to answer this.
RAG
Question:
What does our runbook say?
Use RAG.
documents
|
retrieval
|
context
|
LLM
MCP
Question:
What is happening right now?
MCP might access:
Kubernetes
AWS
GitHub
Datadog
database
Jira
through exposed tools/resources.
Memorize this
RAG = KNOW
MCP = CONNECT / ACCESS / ACT
LLM = UNDERSTAND + GENERATE
It's simplified, but extremely useful.
PART 30 — Build final combined assistant
For a classroom lab, I want students to see the mechanics explicitly rather than hiding everything behind an agent framework.
Create:
05_final_assistant.py
Paste:
import os
import numpy as np
from openai import OpenAI
from dotenv import load_dotenv
load_dotenv()
client = OpenAI()
# --------------------------------------------------
# SECTION 1: RAG KNOWLEDGE
# --------------------------------------------------
def read_documents():
documents = []
for filename in os.listdir("knowledge"):
if filename.endswith(".txt"):
path = os.path.join("knowledge", filename)
with open(path, "r") as file:
text = file.read()
documents.append({
"filename": filename,
"text": text
})
return documents
def create_embedding(text):
response = client.embeddings.create(
model="text-embedding-3-small",
input=text
)
return response.data[0].embedding
def cosine_similarity(vector_a, vector_b):
a = np.array(vector_a)
b = np.array(vector_b)
return np.dot(a, b) / (
np.linalg.norm(a) * np.linalg.norm(b)
)
def retrieve_company_knowledge(question):
documents = read_documents()
question_embedding = create_embedding(question)
for document in documents:
document_embedding = create_embedding(
document["text"]
)
document["score"] = cosine_similarity(
question_embedding,
document_embedding
)
documents = sorted(
documents,
key=lambda doc: doc["score"],
reverse=True
)
return documents[0]
# --------------------------------------------------
# SECTION 2: SIMULATED LIVE OPERATIONS
# These are the same capabilities our MCP server exposes.
# --------------------------------------------------
def get_service_status(service_name):
services = {
"payment-service": {
"desired_replicas": 3,
"ready_replicas": 2,
"status": "DEGRADED"
}
}
service = services.get(service_name)
if service is None:
return "Service not found."
return f"""
Service: {service_name}
Desired replicas: {service['desired_replicas']}
Ready replicas: {service['ready_replicas']}
Status: {service['status']}
"""
def get_recent_logs(service_name):
if service_name == "payment-service":
return """
2026-08-10T13:02:14Z ERROR database connection pool exhausted
2026-08-10T13:02:15Z ERROR timeout acquiring PostgreSQL connection
2026-08-10T13:02:16Z WARN readiness probe failed
"""
return "No critical logs found."
# --------------------------------------------------
# SECTION 3: USER QUESTION
# --------------------------------------------------
question = """
payment-service is returning HTTP 503 errors.
Explain:
1. What is happening right now?
2. What is the likely root cause?
3. What does our company runbook tell me to do?
"""
# --------------------------------------------------
# SECTION 4: RAG RETRIEVAL
# --------------------------------------------------
retrieved_document = retrieve_company_knowledge(
question
)
print("\n--- RAG RETRIEVAL ---")
print(
"Retrieved:",
retrieved_document["filename"]
)
print(
"Similarity:",
round(retrieved_document["score"], 4)
)
# --------------------------------------------------
# SECTION 5: OPERATIONAL TOOL DATA
# --------------------------------------------------
service_status = get_service_status(
"payment-service"
)
logs = get_recent_logs(
"payment-service"
)
print("\n--- LIVE TOOL DATA ---")
print(service_status)
print("\n--- LOGS ---")
print(logs)
# --------------------------------------------------
# SECTION 6: AUGMENT THE LLM CONTEXT
# --------------------------------------------------
prompt = f"""
You are a Senior DevOps Incident Assistant.
Your task is to analyze a production incident.
Use the following sources.
SOURCE 1 - COMPANY KNOWLEDGE RETRIEVED BY RAG:
{retrieved_document["text"]}
SOURCE 2 - CURRENT SERVICE STATUS:
{service_status}
SOURCE 3 - CURRENT APPLICATION LOGS:
{logs}
USER QUESTION:
{question}
Respond using this format:
CURRENT STATUS:
Explain what is happening.
LIKELY ROOT CAUSE:
Explain the most likely cause using evidence.
RUNBOOK ACTIONS:
List the company-approved actions.
EVIDENCE:
Explain which supplied information supports your conclusion.
SAFETY:
Do not recommend destructive production actions
that are not explicitly supported by the runbook.
"""
# --------------------------------------------------
# SECTION 7: LLM INFERENCE
# --------------------------------------------------
response = client.responses.create(
model="gpt-5.6",
input=prompt
)
# --------------------------------------------------
# SECTION 8: FINAL RESPONSE
# --------------------------------------------------
print("\n--- AI INCIDENT ANALYSIS ---")
print(response.output_text)
PART 31 — Run final application
Execute:
python 05_final_assistant.py
You should see roughly:
--- RAG RETRIEVAL ---
Retrieved: payment-runbook.txt
Similarity: ...
Then:
--- LIVE TOOL DATA ---
Service: payment-service
Desired replicas: 3
Ready replicas: 2
Status: DEGRADED
Then:
--- LOGS ---
ERROR database connection pool exhausted
ERROR timeout acquiring PostgreSQL connection
WARN readiness probe failed
Then the LLM should conclude something similar to:
CURRENT STATUS:
payment-service is degraded.
Only 2 of 3 replicas are ready.
LIKELY ROOT CAUSE:
The evidence indicates database connection pool
exhaustion and PostgreSQL connection acquisition
timeouts.
RUNBOOK ACTIONS:
1. Check pod logs.
2. Verify PostgreSQL connectivity.
3. Check active database connections.
4. Scale payment-service from 3 to 6 replicas.
5. Do not restart PostgreSQL without approval.
6. Notify the Payments Platform team.
7. Open SEV-2 if errors continue over 10 minutes.
Now the student can see the entire AI architecture.
PART 32 — Exactly what happened?
Walk through it slowly.
User asks:
payment-service is returning 503
Step 1:
question
gets converted into:
embedding vector
Step 2:
We compare it against:
runbook embedding
Kubernetes document embedding
architecture embedding
Step 3:
Highest semantic similarity:
payment-runbook
Step 4:
RAG retrieves:
company procedure
Step 5:
Operational tools provide:
desired replicas = 3
ready replicas = 2
status = degraded
Step 6:
Logs provide:
database connection pool exhausted
Step 7:
We combine:
QUESTION
+
RAG CONTEXT
+
OPERATIONAL CONTEXT
+
INSTRUCTIONS
Step 8:
Send all of that to the LLM.
Step 9:
LLM generates an incident explanation.
PART 33 — Final architecture students should draw
At the end of class, erase everything and ask students to draw this from memory:
USER
|
v
AI APPLICATION
|
+-----------+-----------+
| |
v v
RAG MCP
| |
| |
KNOWLEDGE ACCESS SYSTEM ACCESS
| |
v v
embeddings/vector MCP server
retrieval / | \
| / | \
v v v v
runbooks K8s AWS GitHub
\ \ | /
\ \ | /
+------------------+---+
|
v
PROMPT
|
v
LLM
|
v
RESPONSE
PART 34 — But our final Python file isn't actually invoking MCP yet. Why?
This distinction is pedagogically important.
In mcp_server.py, we built the actual MCP interface.
In 05_final_assistant.py, we call equivalent Python functions directly so students can see:
data enters here
↓
prompt changes here
↓
LLM receives this
If we immediately bury everything behind an MCP client/agent orchestration layer, beginners often learn:
copy framework code
rather than:
understand AI architecture
Once they understand the flow, the production form becomes:
AI application
|
v
MCP client
|
v
MCP server
|
v
get_service_status
OpenAI's Responses API can connect to remote MCP servers using its built-in mcp tool type, while MCP itself standardizes how those servers expose tools/context. (OpenAI Platform)
PART 35 — What real OpenAI + remote MCP looks like
Once an MCP server is available over a supported remote transport and reachable by the API, the architecture can become:
from openai import OpenAI
client = OpenAI()
response = client.responses.create(
model="gpt-5.6",
tools=[
{
"type": "mcp",
"server_label": "banking_devops",
"server_description": "Banking production operations tools",
"server_url": "https://YOUR-MCP-SERVER/mcp",
"require_approval": "always"
}
],
input="""
Check payment-service and explain
why it is unhealthy.
"""
)
print(response.output_text)
This follows the current Responses API pattern for remote MCP servers: an MCP tool configuration includes fields such as a server label, server URL, and approval behavior. (OpenAI Platform)
Notice this:
"require_approval": "always"
For production DevOps, this idea is extremely important.
You don't want:
User:
"Maybe restart everything."
AI:
"Sure."
kubectl delete ...
without governance.
PART 36 — The senior DevOps security lesson
Imagine an MCP tool:
@mcp.tool()
def delete_namespace(namespace):
You have effectively given an AI-accessible system the capability to remove Kubernetes workloads.
The problem is no longer only:
Can the LLM answer correctly?
Now it becomes:
Who can invoke the tool?
Which namespaces?
Read or write?
Production or staging?
Is approval required?
Are calls audited?
Where are credentials stored?
Can prompt injection trigger tools?
Can a compromised MCP server exfiltrate data?
OpenAI's current MCP guidance explicitly supports approval controls for MCP tool calls, and the MCP specification also defines authorization for HTTP-based transports. (OpenAI Platform)
For senior DevOps engineers, this is where the topic becomes very serious.
PART 37 — Read-only MCP first
Start production adoption with:
READ
not:
WRITE
For example:
GOOD FIRST TOOLS
get_pods
get_deployments
get_logs
get_events
get_service_status
get_alarms
get_cpu_metrics
get_argocd_status
get_git_commit
Later:
CONTROLLED WRITE TOOLS
scale_deployment
rollback_deployment
create_incident
restart_pod
update_ticket
And dangerous operations should have strong:
authentication
authorization
least privilege
approval
audit logging
environment boundaries
PART 38 — Replace simulated MCP data with Kubernetes
Later our tool:
def get_service_status(service_name):
could execute:
kubectl get deployment
Conceptually:
import subprocess
def get_service_status(service_name):
command = [
"kubectl",
"get",
"deployment",
service_name,
"-n",
"banking"
]
result = subprocess.run(
command,
capture_output=True,
text=True
)
return result.stdout
Then:
LLM
|
MCP
|
Python
|
kubectl
|
Kubernetes API
|
EKS
But for a first AI lab, simulated data is much safer and easier to understand.
PART 39 — What is the DevOps engineer responsible for?
This is the part I would emphasize most to senior engineers.
The ML team might build:
model behavior
embedding strategy
evaluation
The application team might build:
AI application
API
UI
business logic
But DevOps/Platform/SRE still owns major parts of:
infrastructure
deployment
security
reliability
observability
scalability
cost
CI/CD
secrets
networking
availability
incident response
Your production stack may become:
USER
|
v
Route 53
|
v
ALB
|
v
AI APPLICATION
on EKS
|
+----------+----------+
| |
v v
OpenAI API RAG SERVICE
|
v
vector DB
|
v
S3
+
|
v
MCP SERVICE
|
+---+----+------+
| | |
v v v
EKS AWS GitHub
Then observability:
Prometheus
Grafana
CloudWatch
OpenTelemetry
centralized logs
tracing
PART 40 — New metrics DevOps engineers must care about
Traditional application metrics:
CPU
memory
requests/sec
5xx
latency
pod count
disk
AI introduces additional concerns:
model latency
token usage
API errors
rate limits
retrieval latency
retrieval quality
embedding latency
MCP tool latency
MCP failures
tool-call rate
vector DB latency
cost per request
And one extremely important distinction:
SYSTEM HEALTH
HTTP 200
pod healthy
CPU fine
does NOT necessarily mean
AI QUALITY
correct answer
correct retrieval
correct tool usage
That is one of the biggest conceptual shifts for SRE/DevOps.
PART 41 — A healthy AI application can still be wrong
Imagine:
API: 200 OK
CPU: 20%
Memory: 40%
Pods: 3/3
Latency: 500ms
Everything looks green.
But the chatbot answers:
Production database password is XYZ...
or:
Restart PostgreSQL immediately.
when the runbook explicitly says not to.
Infrastructure is healthy.
AI behavior is bad.
Therefore AI systems need:
traditional monitoring
+
quality evaluation
+
security evaluation
PART 42 — What are evals?
An evaluation is essentially a way to test whether the AI behaves correctly.
Create test questions:
Question:
What do we do when DB pool > 90%?
Expected:
scale payment-service 3 → 6
do not restart PostgreSQL
notify Payments Platform
Another:
Question:
Which namespace hosts payment-service?
Expected:
banking
Another:
Question:
Should PostgreSQL be restarted automatically?
Expected:
No.
Now we can build automated checks.
This starts looking familiar to DevOps engineers:
application tests
+
AI behavior tests
PART 43 — AI CI/CD
Traditional pipeline:
git push
|
v
unit test
|
v
security scan
|
v
Docker build
|
v
ECR
|
v
deploy
AI pipeline:
git push
|
v
unit tests
|
v
RAG tests
|
v
prompt/evaluation tests
|
v
security tests
|
v
Docker build
|
v
ECR
|
v
EKS
|
v
smoke test
This is why senior DevOps engineers need to understand LLM architecture even if they're not ML engineers.
PART 44 — Where Terraform fits
Terraform might create:
VPC
private subnets
EKS
IAM
security groups
load balancers
S3
RDS
Secrets Manager
monitoring
DNS
Potential RAG infrastructure:
vector database
object storage
ingestion workers
queues
Potential MCP infrastructure:
MCP service
IAM roles
network policy
authentication
secrets
audit logs
PART 45 — Where Kubernetes fits
You might deploy:
ai-api
rag-api
document-ingestion-worker
mcp-server
frontend
For example:
EKS
namespace: ai-platform
├── ai-api
├── rag-service
├── ingestion-worker
├── mcp-kubernetes
└── frontend
PART 46 — Why not put the LLM inside Kubernetes?
Important distinction.
If you're using OpenAI's API:
EKS application
|
| HTTPS
v
OpenAI API
|
v
hosted model
Your application is running on EKS.
That does not mean GPT itself is running in your EKS cluster.
Self-hosted models are a separate architecture.
PART 47 — RAG versus fine-tuning
Students will ask this.
Use this simplified answer.
RAG:
change KNOWLEDGE supplied at runtime
Fine-tuning:
adapt MODEL BEHAVIOR/weights through additional training
For changing company documentation frequently:
RAG
usually makes much more conceptual sense than retraining every time a runbook changes.
Example:
Monday:
scale to 6 replicas
Tuesday:
policy changed → scale to 8
With RAG:
update document
re-index
You don't need to retrain the underlying LLM for each documentation update.
PART 48 — MCP versus API
Another common question:
Isn't MCP just an API?
MCP still operates using normal software/networking concepts, but its value is the standardized protocol/interface for AI applications.
Traditional:
Our AI application
|
custom code
|
AWS API
Another integration:
Our AI application
|
different custom code
|
GitHub API
MCP creates a more standardized abstraction:
AI
|
MCP
|
server
|
external system
The official MCP SDK itself describes MCP as somewhat like a web API designed specifically for LLM interactions. (GitHub)
PART 49 — LLM versus agent
Another very important distinction.
LLM:
input
|
model
|
output
An agentic system adds a loop around the model:
USER
|
v
LLM
|
+---- Need tool?
| |
| v
| TOOL
| |
| v
| RESULT
| |
+--------+
|
v
final answer
The LLM is part of the agent system.
They are not synonyms.
PART 50 — The entire system in one sentence
Have every student say:
An LLM generates and reasons over language; RAG retrieves relevant external knowledge and puts it into the model's context; MCP standardizes access to external tools and resources; and DevOps makes the complete AI system secure, reliable, scalable, observable and deployable.
If they can explain that accurately after the lab, the lab worked.
PART 51 — Final classroom challenge
After demonstrating everything, don't let them simply go home.
Give this assignment.
Change the incident from:
payment-service
to:
authentication-service
Create:
knowledge/authentication-runbook.txt
with:
AUTHENTICATION SERVICE RUNBOOK
If users receive HTTP 401 unexpectedly:
1. Check authentication-service logs.
2. Check JWT signing configuration.
3. Validate Secrets Manager secret.
4. Check token expiration configuration.
5. Verify latest deployment.
6. Roll back if incident started immediately after release.
7. Notify Identity Platform team.
Then add MCP status:
"authentication-service": {
"desired_replicas": 3,
"ready_replicas": 3,
"status": "HEALTHY"
}
And logs:
ERROR JWT signing key mismatch
WARN token validation failed
Ask:
Users suddenly receive 401 after today's deployment.
What is happening and what should I do?
The student must explain which parts came from:
LLM
RAG
MCP/tool data
Not just produce an answer.
PART 52 — Questions I would ask students at the end
Use these orally:
Did we train GPT in this lab?
No. We used a pretrained model for inference.What does an embedding do?
Converts text into a numeric vector useful for semantic comparison.Why do we embed both documents and the question?
So they can be compared in the same vector space.What is retrieval?
Finding the most relevant external information for the question.What does RAG stand for?
Retrieval-Augmented Generation.Does RAG modify GPT's model weights?
No.Where is company knowledge stored?
In our external knowledge source, not magically inside the LLM.What does MCP stand for?
Model Context Protocol.Why use MCP?
To standardize how AI applications connect to tools/resources and external systems.RAG versus MCP?
RAG primarily retrieves knowledge; MCP exposes standardized external capabilities/context.Can MCP perform actions?
An MCP server can expose tools that call external systems, subject to the permissions and controls you design. (Model Context Protocol)Why is MCP dangerous for DevOps if designed badly?
Because tools may expose powerful infrastructure operations.Why should production MCP use least privilege?
The AI should have no more capability than necessary.What does an LLM do?
Processes context and generates model output; it is not a company database or Kubernetes API.Why can a perfectly healthy AI pod still produce a bad product experience?
Infrastructure availability and AI answer quality are different dimensions.
PART 53 — The complete mental model
This is the diagram I would put on the final slide or whiteboard:
┌──────────────────┐
│ USER │
└────────┬─────────┘
│
│ natural language
▼
┌──────────────────┐
│ AI APPLICATION │
└────────┬─────────┘
│
┌─────────────┴────────────┐
│ │
▼ ▼
┌──────────────┐ ┌──────────────┐
│ RAG │ │ MCP │
│ │ │ │
│ KNOWLEDGE │ │ CAPABILITY │
└──────┬───────┘ └──────┬───────┘
│ │
┌─────────▼────────┐ ┌────────▼─────────┐
│ Embedding Model │ │ MCP Server │
└─────────┬────────┘ └────────┬─────────┘
│ │
▼ ┌────┼──────────┐
┌──────────────────┐ │ │ │
│ Vector Retrieval │ ▼ ▼ ▼
└────────┬─────────┘ EKS AWS GitHub
│
▼
┌──────────────────┐
│ Relevant Runbook │
└────────┬─────────┘
│
└───────────┐
│
▼
┌───────────────┐
│ CONTEXT │
│ │
│ question │
│ + RAG data │
│ + tool data │
│ + instruction │
└───────┬───────┘
│
▼
┌───────────────┐
│ LLM │
│ INFERENCE │
└───────┬───────┘
│
▼
┌───────────────┐
│ FINAL ANSWER │
└───────────────┘
The single most important takeaway
Do not teach students:
LLM + LangChain + vector DB + MCP + Kubernetes
as five tools they have to memorize.
Teach the problem each component solves:
"We need language intelligence."
↓
LLM
"But the LLM doesn't know our private runbook."
↓
RAG
"But the runbook doesn't know what is happening
in Kubernetes right now."
↓
MCP
"But now we have an AI system touching production."
↓
DEVOPS / PLATFORM / SRE
That's the point where a senior DevOps engineer stops seeing LLMs as “a chatbot API” and starts seeing the complete system architecture.
Top comments (0)