Lab: From LLM RAG MCP

STAGE 1 — LLM
User → LLM → Answer
              ↑
       model knowledge only


STAGE 2 — RAG
User → Retrieval → Internal Runbook
          ↓
         LLM → Answer
               ↑
        private knowledge


STAGE 3 — MCP
User → LLM → MCP Client → MCP Server → DevOps Tool
                                         ↓
                                   live/simulated data

Use this scenario throughout:

We are DevOps engineers supporting payment-service.

And ask:

"Why is payment-service failing?"

---

Part 0 — Create the project

Students create:

mkdir llm-rag-mcp-lab
cd llm-rag-mcp-lab

python3 -m venv venv
source venv/bin/activate

Create:

llm-rag-mcp-lab/
│
├── 01_llm.py
├── 02_rag.py
├── 03_mcp_server.py
├── 04_mcp_client.py
│
├── knowledge/
│   └── production_runbook.txt
│
└── requirements.txt

The important teaching point is that every stage adds one capability.

---

PART 1 — Demonstrate the LLM itself

Before using RAG or MCP, make the LLM answer something it cannot possibly know.

Create:

01_llm.py

For a classroom lab, you can use an API-backed model or a local model. The important experiment is the same.

Conceptually:

question = """
What database does our internal payment-service use,
and what should we check if it returns HTTP 500?
"""

response = llm(question)

print(response)

Ask students before running it

"Will the LLM know our company's payment-service?"

No.

It may answer:

The payment service may use PostgreSQL, MySQL,
or another database.

For HTTP 500 errors, check:
- application logs
- database connectivity
- environment variables
...

It can give a reasonable answer.

But ask:

"Where did PostgreSQL come from?"

It doesn't know.

Maybe PostgreSQL. Maybe MySQL. Maybe DynamoDB.

This demonstrates the first concept:

                  LLM
                   │
         ┌─────────┴─────────┐
         │                   │
      KNOWS              DOESN'T KNOW
         │                   │
   Kubernetes          our passwords
   Terraform           our runbooks
   Python              today's deployment
   AWS concepts        our Jira tickets
                       our production state

Experiment that makes this very obvious

Ask:

What is the database hostname for
JumpToTech payment-service production?

The model cannot reliably know it.

Student conclusion #1

Write this on the board:

LLM = reasoning/generation over the information available in its context and learned model behavior.

It does not automatically know your private environment.

Now tell them:

"We need to give our AI company knowledge."

That introduces RAG naturally.

---

PART 2 — Demonstrate RAG

Now create private information that the LLM has never seen.

Create:

knowledge/production_runbook.txt

Put:

JUMPToTECH BANKING PLATFORM
PRODUCTION RUNBOOK

Application:
payment-service

Environment:
production

Kubernetes Cluster:
banking-prod-eks

AWS Region:
us-east-1

Database:
PostgreSQL

Database Host:
payments-prod.internal

Common Incident:
Payment service returns HTTP 500 when
DB_HOST is configured incorrectly.

Troubleshooting procedure:

1. Check payment-service pod status.
2. Check application logs.
3. Verify DB_HOST.
4. Test connectivity to PostgreSQL.
5. Check Kubernetes Secret payment-db-secret.

Rollback command:

helm rollback payment-service 3

Tell students:

"We invented this information five seconds ago. The LLM was never trained on it."

That's what makes the demonstration powerful.

---

Step 2.1 — Ask the LLM again

Same question:

What database does our payment-service use?

Without providing the document, the LLM still doesn't know.

Now we build retrieval.

---

Step 2.2 — Understand RAG before coding

Take the document:

production_runbook.txt

Split it:

            DOCUMENT
                │
                ▼
             CHUNKS

┌────────────────────────────┐
│ payment-service            │
│ environment: production    │
└────────────────────────────┘

┌────────────────────────────┐
│ Database: PostgreSQL       │
│ Host: payments-prod...     │
└────────────────────────────┘

┌────────────────────────────┐
│ HTTP 500 occurs when       │
│ DB_HOST is incorrect       │
└────────────────────────────┘

Then:

Chunks
   ↓
Embeddings
   ↓
Vectors
   ↓
Vector/Search Store

Student asks:

Why is payment-service returning HTTP 500?

Convert the question to a searchable representation and find the most relevant chunk.

Conceptually:

Question
   ↓
Retrieval
   ↓

Result #1
"HTTP 500 occurs when DB_HOST
is configured incorrectly."

   ↓

Question + Result
   ↓
LLM

Now the model has evidence.

---

Step 2.3 — Make retrieval visible

This is critical for teaching.

Don't just display the final answer.

Your program should print something like:

===============================
USER QUESTION
===============================

Why is payment-service returning HTTP 500?


===============================
RETRIEVED CONTEXT
===============================

Common Incident:
Payment service returns HTTP 500 when
DB_HOST is configured incorrectly.

Troubleshooting:
Verify DB_HOST.
Test PostgreSQL connectivity.


===============================
LLM ANSWER
===============================

According to the production runbook,
a common cause is an incorrectly configured
DB_HOST.

Check DB_HOST and verify connectivity
to PostgreSQL.

Now stop.

Ask students:

"Where did the answer come from?"

Not just the LLM.

Question
   ↓
RETRIEVE
   ↓
AUGMENT
   ↓
GENERATE

That's literally the name:

R = Retrieval
A = Augmented
G = Generation

Student conclusion #2

RAG retrieves relevant information and adds it to the LLM's context before generation.

---

PART 3 — Demonstrate why RAG still isn't enough

Now ask:

How many payment-service pods are currently running?

Your runbook doesn't contain that.

Even with RAG:

RAG
 ↓
Search documents
 ↓
???

Why?

Because this isn't really a static-document question.

You need current system information.

Normally a DevOps engineer might run:

kubectl get pods

This gives us the transition to tools/MCP.

Tell students:

"Reading knowledge is different from interacting with an external system."

---

PART 4 — Demonstrate MCP

For the first class, do not connect production AWS/EKS.

Create a simulated DevOps tool.

Your MCP server exposes something conceptually like:

get_pods()

And it returns:

payment-service-7d8f9    Running
payment-service-2ks91    Running
payment-service-8sa21    CrashLoopBackOff

Architecture:

                   AI APPLICATION
                         │
                         ▼
                       LLM
                         │
                  "I need pod data."
                         │
                         ▼
                    MCP CLIENT
                         │
                  MCP protocol
                         │
                         ▼
                    MCP SERVER
                         │
                    get_pods()
                         │
                         ▼
                 Kubernetes/API

For the classroom, the bottom can initially be simulated.

---

Step 4.1 — Make the MCP server obvious

Have the server expose three safe tools:

get_pods()

get_logs(pod)

get_deployment(service)

For example:

get_pods()

returns:

NAME                        STATUS
payment-service-abc         Running
payment-service-def         Running
payment-service-xyz         CrashLoopBackOff

And:

get_logs("payment-service-xyz")

returns:

ERROR:
connection to payments-prod.internal:5432 failed

FATAL:
database connection refused

---

Step 4.2 — Let the LLM choose the tool

Student asks:

Why is payment-service failing?

Now trace every operation on screen:

USER
│
│ Why is payment-service failing?
│
▼
LLM

"I need current information."

│
▼
MCP TOOL CALL

get_pods()

│
▼

payment-service-xyz
CrashLoopBackOff

│
▼
LLM

"I need logs."

│
▼
MCP TOOL CALL

get_logs("payment-service-xyz")

│
▼

database connection refused

│
▼
LLM

"The payment-service pod is crashing
because it cannot connect to the database."

This is where students usually understand MCP.

The LLM didn't magically execute Kubernetes commands.

A tool was made available through the integration.

---

PART 5 — The best experiment: compare all three

Now put three terminals/windows next to each other.

Window 1 — LLM

Ask:

Why is payment-service failing?

Result:

Possible causes include:
CPU,
memory,
database,
networking,
configuration...

Tell students:

Generic reasoning.

---

Window 2 — RAG

Same question.

Retrieved:

Company Runbook:

HTTP 500 can occur because of incorrect DB_HOST.

Answer:

According to the runbook,
check DB_HOST and PostgreSQL connectivity.

Tell students:

Company knowledge.

---

Window 3 — MCP

Same question.

AI calls:

get_pods()

Result:

CrashLoopBackOff

Then:

get_logs()

Result:

database connection refused

Answer:

The current payment-service pod is failing
because its database connection is refused.

Tell students:

Tool/system information.

---

PART 6 — Then combine RAG + MCP

This is the final "aha" moment.

Student asks:

Payment-service is failing.

Investigate the problem and tell me
how our company runbook says to fix it.

Now both capabilities are required.

                         USER
                          │
                          ▼
                         LLM
                     /          \
                    /            \
                   ▼              ▼
                 MCP             RAG
                  │               │
          Current system       Company
             state             knowledge
                  │               │
                  ▼               ▼
             get_pods()       Runbook
                  │
                  ▼
             get_logs()
                  │
                  └──────┬────────┘
                         ▼
                        LLM
                         │
                         ▼
                    FINAL ANSWER

MCP tells us:

Current pod:
CrashLoopBackOff

Current logs:
database connection refused

RAG tells us:

Company procedure:

1. Verify DB_HOST
2. Test PostgreSQL connectivity
3. Check payment-db-secret
4. If required:
   helm rollback payment-service 3

LLM combines them:

The current payment-service pod is in
CrashLoopBackOff because the database
connection is being refused.

According to the production runbook:

1. Verify DB_HOST.
2. Check payment-db-secret.
3. Test connectivity to PostgreSQL.
4. If the issue was introduced by the latest
   deployment, rollback payment-service.

Now you can ask:

"Which component generated the explanation?"

LLM.

"Which component found our company's troubleshooting procedure?"

RAG.

"Which component let the AI application obtain current pod/log information through exposed tools?"

MCP/tool integration.

That's the distinction you want students to leave class understanding.

---

One important improvement for your class

I would actually make this a real executable lab, not just conceptual Python.

We can build it so students have:

llm-rag-mcp-lab/

01-llm/
   app.py

02-rag/
   app.py
   runbook.txt

03-mcp/
   server.py
   client.py

04-complete-ai-devops/
   app.py

And when they run:

python app.py

they see clearly labeled output:

🤖 LLM
🔎 RAG RETRIEVAL
🔧 MCP TOOL CALL
📄 TOOL RESULT
🤖 FINAL ANSWER