"Generative AI" is on every roadmap, but most introductions either drown you in math or hand-wave with "it predicts the next word." Neither helps you build something or reason about when it's the wrong tool. This is a developer's on-ramp: model families, GPT & transformers, language modeling, the ethics you must handle, and a complete runnable text-gen app. (Full version on the original — link at the end.)
TL;DR
-
Generative models learn to produce data (
P(data)); discriminative models only label it (P(label | input)). "Draw me a cat" vs "is this a cat?" - Four families: autoregressive (LLMs/GPT — text), diffusion (images/video/audio), GANs (fast narrow images), VAEs (latent spaces). For text, autoregressive transformers dominate.
- GPT = a decoder-only transformer trained to predict the next token. Generation is a loop: predict → sample → append → repeat.
- Language modeling is the objective; text generation is the application. Decoding (temperature, top-p) shapes output as much as the model.
- Ethics is not optional — hallucination, bias, copyright, privacy, misuse, and cost are shippable risks. Handle them at the boundary.
1. What "generative" actually means
You already know discriminative models — they draw boundaries between classes ("spam or not spam") and learn P(label | input). Generative models learn the distribution of the data itself — P(data) — so they produce new samples that look like the training data.
Discriminative: input -> label (classify, score, detect)
Generative: prompt -> new data (write, draw, compose, synthesize)
The practical consequence: there's no single "right answer." Given "Once upon a time," there are billions of valid continuations. That's both the power (creativity) and the danger (confident, plausible nonsense).
2. The four families (and a picking guide)
Not all generative AI is an LLM. Knowing the families is how you pick the right tool:
-
Autoregressive (the LLM family) — generate one token at a time, each conditioned on everything before it:
P(x1..xn) = Π P(xi | x<i). GPT, Llama, Claude. Best for: text, code, any sequence. - Diffusion — start with noise and iteratively denoise into a sample. Stable Diffusion, DALL·E 3, Midjourney. Best for: images, video, audio.
- GANs — a generator makes fakes, a discriminator spots them; they improve together. Mostly superseded by diffusion. Best for: fast, narrow image tasks.
- VAEs — compress data into a smooth latent space, then decode. Power the latent space inside diffusion models. Best for: representation learning / compression.
| You want to generate... | Use |
|---|---|
| Text, code, structured output | Autoregressive transformer (LLM) |
| Images, video, audio | Diffusion |
| Fast narrow images (faces, upscaling) | GAN |
| A smooth latent space / compression | VAE |
The rest of this guide focuses on text — where most developers start and can ship the fastest.
3. GPT and transformers, demystified
GPT = Generative Pre-trained Transformer. Generative (produces text), Pre-trained (learned general language from a huge corpus first), Transformer (the architecture underneath). A transformer processes tokens through stacked blocks, each with self-attention + a feed-forward network (with residuals + LayerNorm), × N blocks.
Self-attention — the key idea. For each token the model computes how much to "pay attention to" every other token. Each token emits a Query (what am I looking for?), a Key (what do I offer?), and a Value (what I contribute). Attention weight = how well a query matches each key; output = weighted blend of values:
import torch, math
import torch.nn.functional as F
def attention(Q, K, V): # Q, K, V: (seq_len, d)
scores = Q @ K.T / math.sqrt(Q.size(-1)) # relevance of each token to each other
weights = F.softmax(scores, dim=-1) # turn scores into a distribution
return weights @ V # weighted blend of values
That single operation — match, weight, blend — repeated across heads and layers, is the entire reason transformers work. GPT uses only the decoder half with a causal mask so each token can only see the past — exactly what you need to write text left-to-right.
4. Language modeling & generation
A language model is trained on one task: given the previous tokens, predict the next one (cross-entropy on the shifted sequence). Astonishingly, that single objective over trillions of tokens is enough to produce models that write code, summarize, and answer questions — the capability emerges from scale. Perplexity measures it: roughly "how many tokens is the model choosing between?" (5-20 for a good in-domain model; tens of thousands for random).
Generation is a sampling loop — predict a distribution, pick one token, append, repeat until EOS. The same model produces wildly different text depending on how you sample:
def sample(logits, temperature=0.8, top_p=0.9, top_k=None):
logits = logits / max(temperature, 1e-5) # <1 sharper, >1 wilder
if top_k is not None:
v, _ = torch.topk(logits, top_k)
logits[logits < v[-1]] = float('-inf')
probs = F.softmax(logits, dim=-1)
# top-p (nucleus): smallest set of tokens summing to p, then renormalise and sample
return torch.multinomial(probs, 1).item()
Rules of thumb: factual/code temperature=0.0-0.3 (deterministic, fewer mistakes); chat 0.7, top_p=0.9 (coherent, not robotic); creative 0.9-1.1 (more surprising, more risk). "The model gave a weird answer" is often a decoding problem, not a model problem.
5. Ethics — the part that decides if you can ship
Generative models introduce risks classifiers never had, because they produce content rather than labeling it.
- Hallucination — they predict plausible text, not true text; truth isn't in the loss function. Ground with RAG (answer only from provided context), cite sources, and never use raw output for high-stakes facts without review.
- Bias — models learn their training data's biases. Test across groups; don't use them for consequential decisions about people without audits.
- Copyright — output can reproduce training data; the legal status is unsettled. Keep a human in the loop; check code against licenses.
- Privacy / PII — redact before the prompt leaves your network (the boundary is your responsibility); use enterprise endpoints that don't train on your inputs; don't log raw PII.
- Misuse & cost — rate-limit and authenticate endpoints; use the smallest model that does the job; cache and batch.
import re
PII = {
'email': re.compile(r"\b[\w.+-]+@[\w-]+\.[\w.-]+\b"),
'phone': re.compile(r"\b(?:\+?\d{1,3}[-.\s]?)?\(?\d{3,5}\)?[-.\s]?\d{3,4}[-.\s]?\d{3,4}\b"),
'ssn': re.compile(r"\b\d{3}-\d{2}-\d{4}\b"),
}
def redact_pii(text: str) -> str:
for label, pattern in PII.items():
text = pattern.sub(f'[REDACTED_{label.upper()}]', text)
return text
The ship checklist: PII redacted before leaving your network; high-stakes outputs grounded and/or human-reviewed; smallest model that meets quality; endpoints authenticated + rate-limited; users told they're talking to AI; no raw-PII logs; tested for obvious bias.
6. The project — a real text-generation app
A FastAPI service that generates text with streaming (SSE), PII redaction, a content guardrail, and a Gradio UI — working with either a hosted API (OpenAI/Azure) or a local open-weight model. The request flow: redact PII → content guardrail → rate limit → stream tokens back.
# app/main.py (abridged) — the safe boundary around the model
@app.post('/generate')
async def generate(req: GenRequest, request: Request):
check_rate_limit(request.client.host) # naive in-memory; use Redis in prod
if is_blocked(req.prompt): # content guardrail
raise HTTPException(400, 'Prompt violates usage policy')
safe_prompt = redact_pii(req.prompt) # PII never leaves the process raw
async def event_stream():
async for token in stream_generate(safe_prompt, req.temperature, req.max_tokens):
yield f"data: {json.dumps({'token': token})}\n\n"
yield f"data: {json.dumps({'done': True})}\n\n"
return StreamingResponse(event_stream(), media_type='text/event-stream')
The backend is pluggable — LLM_BACKEND=openai streams from the OpenAI API, LLM_BACKEND=local streams from a local Hugging Face model via TextIteratorStreamer. The Gradio UI reads the SSE stream and renders tokens as they arrive. You get the three things most demos skip: streaming, PII redaction, and a safety guardrail. For production: real moderation (OpenAI moderation / Llama Guard), Redis rate limiting, auth, Langfuse observability, response caching, vLLM, and a golden eval set.
The mental model
Generative AI feels magical, but the model is simple: a model that learned the shape of data well enough to produce new samples of it, run inside a loop, shaped by your decoding choices, and made safe by the guardrails you put around it. Three habits: pick the smallest model that works (capability beyond your need is wasted cost and carbon); treat the boundary as your responsibility (redact, ground, rate-limit, disclose); and measure, don't vibe (build an eval set before tuning prompts or swapping models).
This is a condensed version. The full guide — with every section in depth, perplexity, all the decoding code, the full ethics treatment, and the complete FastAPI + Gradio project (safety, pluggable backend, run instructions) — is on PrepStack:
Top comments (0)