Understanding LLM Lifecycle - For Infra admins - Beginner

Shameer Sh — Tue, 18 Aug 2026 09:28:13 +0000

This LLM life cycle is an easy reading for an Infra admin.

If you've managed server fleets, golden images, GPOs, and patch cycles, you already understand the LLM lifecycle — you just haven't mapped it yet. This guide walks through each stage of an LLM's life using direct infrastructure equivalents.

1. Pretraining — Building the Golden Image

What it is: The base model learns general language understanding from massive datasets. This is expensive, time-consuming, and done once by the model provider (OpenAI, Meta, Cohere, Anthropic).

Now imagine, this is exactly like building a golden Windows Server image — a huge, centralized effort to bake in a broad, general-purpose baseline that many teams will later build on top of. You don't do this yourself, the same way you don't build your own base OS image from source — you pull the vendor-provided baseline and go from there.

Who does this: Big AI labs — not you, not your org.

2. Fine-Tuning — Customizing the Image for Your Environment

What it is: Adapting the general-purpose base model to perform well on a specific task using your own labeled data.

So now this is directly analogous to customizing a golden image for a specific business unit — installing department-specific software, applying tailored GPOs, baking in your org's configuration standards.

Vanilla (Full Fine-Tuning): Retrains most of the model's weights — slow and expensive. Maps to a full reimage: rebuilding the entire image from scratch every time you need a change.
T-Few (Lightweight Fine-Tuning): Touches only a tiny ~0.01% slice of additional layers — fast and cheap. Maps to a hotfix/patch: a targeted config or registry change applied on top of the existing image without touching the base.
3. Prompt Engineering — Runtime Configuration, No Rebuild Needed

**What it is: **Adjusting how you ask the model to behave — instructions, examples, formatting — without touching the model's underlying weights at all.

Infra analogy: This is like adjusting a config file, environment variable, or GPO setting at the client/session level rather than rebuilding or repatching the image. Fast, reversible, zero deployment risk.

Why this matters: Always try this first — the same sysadmin instinct of reaching for the cheap, reversible fix before a heavier one (fine-tuning or full retraining).

4. Evaluation and Iteration — UAT and Health Checks Before Wider Rollout

What it is: Testing the model's output against a validation dataset using metrics like:

Accuracy — how many predictions were right
Loss — how wrong the wrong ones were (error severity, not a pass/fail count)

Based on results, you adjust hyperparameters and retrain if needed.

Infra analogy: This is your UAT/pilot testing phase before a full production rollout, plus ongoing benchmarking — similar to validating a patched image in a staging environment before pushing it fleet-wide, using defined KPIs rather than "it seems to work."

5. Deployment and Ongoing Maintenance — Go-Live and Lifecycle Management

What it is: Deploy the model (or fine-tuned version) for real inference traffic, then monitor performance and collect feedback for future fine-tuning cycles.

Infra analogy: This is your production rollout plus patch/maintenance cycle — deploy, monitor, patch, repeat, exactly like managing any server fleet.

Infrastructure Behind LLMs
Dedicated AI Clusters (Hosting vs. Fine-Tuning): Think resource pools/clusters in vCenter — a fine-tuning cluster needs more GPU "horsepower" (roughly 2 units) than a hosting cluster (1 unit), the same way a build/compile server needs more compute than a server just serving traffic.
Shared GPU Clusters: One base model endpoint plus multiple custom model endpoints sharing the same GPU cluster is exactly like multiple VMs sharing a hypervisor host's resources, or several App Pools sharing an IIS server — efficient because they share a common base and only the deltas differ.
GPU Memory Reload Overhead: Switching between unrelated models carries the same overhead intuition as vMotion vs. cold migration — moving between things that share the same base is cheap; switching to something fundamentally different means a full reload, costing time and resources.
One-Line Summary

Pretraining = build the golden image (someone else does this) → Fine-tuning = customize the image for your org → Prompt Engineering = runtime config tweaks → Evaluation = UAT/benchmarking → Deployment = go-live + ongoing patch/monitor cycle.

Same lifecycle discipline you already run for infrastructure, just applied to a model instead of a server fleet.

RAG Framework with the Infra Lens

Shameer Sh — Tue, 18 Aug 2026 08:56:27 +0000

This is for our Infra Admins who would be more interested in understanding RAG (Retrieval-Augmented Generation) with Infra lens.

RAG Framework — Retriever, Ranker, LLM

Think of this as a three-tier help-desk / ticketing search system:

Retriever — sources relevant information from a large corpus or database using retrieval techniques. This is your search index across your runbooks, KB articles, and past ticket resolutions — like querying your AD/VMware documentation repository for anything that might match the current issue. It casts a wide net, pulling anything plausibly relevant.
Ranker — evaluates and prioritizes what the retriever pulled back, ensuring the LLM gets the most pertinent, high-quality input. This is exactly like triaging search results by relevance before escalating — you don't hand a level-1 tech every KB article that mentions "DNS," you rank by how closely each one matches the actual symptom.
LLM — generates the final human-like response using the ranked, retrieved information plus the original query, aiming for a response that's factually accurate, coherent, and grounded. This is your senior engineer writing up the final resolution note, synthesizing the best-ranked reference material into a clear answer — not inventing one from memory.

RAG Pipeline — the three stages that make this run (Ingestion → Retrieval → Generation): Ingestion is your one-time (or scheduled) documentation import job — documents get chunked, embedded, and indexed into a database, much like you'd batch-import and index a KB into a searchable system. Retrieval is the live query time lookup — a user's question gets matched against that index to pull top-K results. Generation is the final answer synthesis — those top results get handed to the LLM to produce the response.

RAG Techniques — RAG Sequence vs. RAG Token

RAG Sequence: retrieve documents once for the whole query, then generate one cohesive response using all of them together. Like pulling every relevant runbook for an incident up front, then writing one complete resolution report referencing all of them together.
RAG Token: retrieve fresh documents for each part of the response as it's being generated, building the answer incrementally. Like looking something up mid-sentence while writing a report — checking a different reference for each paragraph as you go, rather than gathering everything first. More flexible, but more overhead per response.

Types of RAG — Naive, Advanced, Agentic (this is really a maturity ladder, and it maps almost exactly onto how monitoring/automation tooling matures in an infra team):

Naive RAG — embed the query, grab the top-K matches once, stuff them in the prompt, generate. Simple but brittle if the first search misses. This is like a basic keyword search against your KB with no re-checking — if the search terms don't match well, you get a bad answer with no safety net.
Advanced RAG — adds query rewriting, re-ranking, and better chunking before generation. This is like adding synonym matching, tagging, and a relevance-scoring layer to your ticketing search — you're improving the quality of what gets pulled before anyone acts on it.
Agentic RAG — the LLM itself plans its own retrieval: deciding what to search for, issuing multiple/iterative queries, calling tools, and judging whether it has enough context before answering. This is the leap from a static runbook lookup to a self-driving diagnostic agent — like a monitoring system that doesn't just alert once but iteratively checks multiple data sources (event logs, performance counters, AD replication status) on its own before concluding root cause.

One practical connection if you ever build a "chatbot over your AD/VMware runbooks" idea from earlier, you'd start with Naive RAG to prove it works, then move to Advanced RAG once you notice bad retrievals (mismatched KB articles), and only reach for Agentic RAG if the questions genuinely require multi-step investigation rather than a single lookup — same incremental-rollout instinct you'd apply to any new tooling in production.

DEV Community: Shameer Sh

Understanding LLM Lifecycle - For Infra admins - Beginner

RAG Framework with the Infra Lens