DEV Community: Sujal Suyash

Model Quantization Explained: Shrinking LLMs Without Losing Their Mind

Sujal Suyash — Mon, 10 Aug 2026 17:20:02 +0000

Model Quantization Explained: Shrinking LLMs Without Losing Their Mind

As we all know that when we try to load a 70B parameter model, how much time and space it takes just to load the model. To solve this issue, the concept of quantization was introduced.

This post is my own attempt to understand quantization deeply enough to explain it simply — from what it actually does, to the math behind how it preserves accuracy.

The Core Idea: Precision, Not Parameters

The most common misconception is that quantization means removing parameters from a model. Actually it isn't.
Concretely: quantization converts high-precision parameters (like 32-bit floats) into lower-precision representations (like 8-bit or 4-bit numbers). This reduces:

Memory usage (fewer bits per value)
Inference latency (less data to move, and on supported hardware, faster math)

Floating Point 101: Sign, Exponent, Mantissa

Before we move forward, you need to know what's actually inside a floating-point number. Every float is basically scientific notation in binary:

value = (-1)^sign × 1.mantissa × 2^exponent

Sign — positive or negative
Exponent — how big the number can get (this controls range)
Mantissa (significand) — the precise digits within that range (this controls precision) More mantissa bits → values closer together can be told apart (finer resolution). More exponent bits → much larger or smaller magnitudes can be represented before overflow/underflow.

This is exactly what the "E4M3" / "E5M2" naming in FP8 formats tells you — it's the literal bit split between exponent and mantissa.

Common Quantization Formats

Format	Bits	Exponent	Mantissa	Typical Use
FP16	16	5	10	General inference, good precision, limited range
BF16	16	8	7	Same range as FP32, coarser precision — popular for training
FP8 E4M3	8	4	3	Weights & forward-pass activations (precision matters more)
FP8 E5M2	8	5	2	Gradients in mixed-precision training (range matters more)
NVFP4	4	—	—	Aggressive compression on newer hardware (e.g. Blackwell)

The E4M3 vs E5M2 split exists for a reason: forward-pass weights/activations tend to be well-behaved in magnitude, so the extra mantissa bit in E4M3 helps accuracy. Gradients during training can swing across huge magnitude ranges, so E5M2 trades precision for range to avoid overflowing to infinity or underflowing to zero.

Symmetric vs Asymmetric Quantization

At its core, quantization maps a continuous float range onto a small set of integers using a scale factor (and sometimes a zero-point).

Symmetric Quantization

Used when the value distribution is roughly centered around zero (common for weights). The float range is mapped symmetrically around zero — no offset needed.

scale = max(abs(W)) / (2^(b-1) - 1)

W_quant   = round(W / scale)
W_dequant = W_quant × scale

Where:

W = original full-precision weight tensor
b = target bit-width (e.g., 8 for INT8)
max(abs(W)) = the largest absolute value in the tensor — this is the "MaxAbs" calibration you may have seen referenced Simple, fast, and cheap to compute — but wastes range if the distribution is skewed (e.g., all-positive activations after a ReLU).

Asymmetric Quantization

Used when the distribution is skewed and doesn't center around zero (common for post-activation values). It introduces a zero-point — an offset that shifts the mapped range to better fit the actual data.

scale      = (max(W) - min(W)) / (2^b - 1)
zero_point = round(-min(W) / scale)

W_quant   = round(W / scale) + zero_point
W_dequant = (W_quant - zero_point) × scale

Asymmetric quantization better utilizes the available integer range for skewed distributions, at the cost of a slightly more expensive computation (the extra zero-point term).

How Accuracy Is Preserved

To maintain the model's accuracy despite the loss of numerical precision, several techniques are used:

Quantization-Aware Training (QAT)

QAT simulates quantization during training so the model learns to be robust to the precision loss it will face later.

Fake quantization nodes are inserted into the forward pass. These take the real FP32 weight, round it to what it would look like in low precision, then immediately cast it back to FP32. The weight is still stored in FP32 — only the forward computation "feels" the rounding error.
This injected error flows into the loss function, same as any other forward-pass computation.
The catch: rounding has a derivative of zero almost everywhere, so gradients can't flow through it normally. QAT solves this with the Straight-Through Estimator (STE) — it treats the rounding step as if it were the identity function during backpropagation, letting gradients pass through unchanged.
Over many steps, the optimizer nudges the real weights toward values where quantization rounding hurts the loss the least. Only after training finishes are the weights actually cast down and stored permanently in low precision.

Post-Training Quantization (PTQ) with Calibration

PTQ works on an already-trained model by using a small calibration dataset.

Take a small, representative sample of real inputs (for an LLM: a few hundred text sequences from a general or domain-specific corpus).
Run them through the full-precision model in a normal forward pass (no backpropogation).
At each layer, record the actual distribution of values seen — weights are fixed, but activation ranges can only be known from real data.
Use those observed ranges to compute the scale factor (and zero-point, if asymmetric) for that layer. Representative calibration data matters: if it doesn't resemble real deployment traffic, the resulting scale factors will be miscalibrated — clipping outliers too aggressively, or wasting precision on values that never actually occur.

Two well-known PTQ methods worth naming:

GPTQ — uses calibration data to compute layer-wise Hessian (second-order sensitivity) information and solves for quantized weights that minimize reconstruction error.
AWQ — protects "salient" weight channels identified by activation magnitude, rather than treating all weights equally.

Hybrid / Mixed-Precision Quantization

Not every layer is equally sensitive to precision loss. The hybrid approach quantizes different layers to different bit-widths based on measured sensitivity:

Less sensitive layers → compressed aggressively (e.g., INT4)
Critical layers → kept at higher precision (e.g., FP16 or INT8) to preserve overall performance Sensitivity is often measured using Hessian trace or simple accuracy/perplexity sweeps per layer.

Does the Model Go Back to Its Original Form at Inference?

Not exactly, but there's often a dequantization step involved, and which path is used depends on the hardware:

Dequantize-then-compute: weights sit in memory as INT4/INT8 (saving storage and memory bandwidth), but are upcast back to FP16/BF16 right before the matrix multiply. This saves memory, but the actual math still runs at higher precision — so latency gains are limited.

Native low-precision compute: on hardware with dedicated support (NVIDIA Tensor Cores — Hopper for FP8, Blackwell for NVFP4), the multiply-accumulate operation happens directly in low precision, no upcast required. This is where both memory savings and latency/throughput gains show up, because the silicon itself is doing less work per operation.

So: the stored weights never "revert" — they remain permanently in the lower-precision format. What differs is whether the compute step temporarily upcasts for the math, or the hardware crunches the low-precision numbers directly. Which path gets used depends on the deployment hardware and inference engine (e.g., TensorRT-LLM, vLLM).

Wrapping Up

Quantization isn't one trick — it's a toolbox: pick a format based on range vs precision needs, pick symmetric or asymmetric based on your data's distribution, and pick QAT, PTQ, or a hybrid approach based on how much retraining budget you have. The right combination lets you deploy models at a fraction of the memory and latency cost, with accuracy loss that's often barely measurable.

This post is part of my ongoing series while I learn ML deployment concepts by writing about them. Feedback and corrections welcome!

From Fragment Shaders to Tensor Cores: How CUDA Actually Works ?

Sujal Suyash — Wed, 29 Jul 2026 17:45:55 +0000

Your system has a CPU. It probably also has a GPU. They look similar on a spec sheet - both have "cores," both run at some clock speed, but they were built to solve completely different problems. Understanding why is the key that unlocks everything else about CUDA.

Why does this hardware even exist?

A CPU is latency-optimized. Its job is to finish one complex task as fast as possible, so a large fraction of its transistor budget goes toward things that have nothing to do with raw arithmetic: branch predictors that guess which way an if statement will go before it's even evaluated, out-of-order execution engines that reorder instructions to avoid stalls, and large multi-level caches (L1, L2, sometimes L3) that keep frequently used data close to the core. A CPU is built to handle unpredictable, branching, sequential logic.

A GPU is totally opposite of that of CPU. It assumes the workload is predictable, regular, and mathematically dense. The same instruction applied to enormous amounts of data, like every pixel on a screen or every element in a large matrix. Because of that assumption, a GPU strips away most of the branch prediction and out-of-order machinery a CPU relies on, and instead spends its transistor budget on raw arithmetic units -> thousands of small, simple cores instead of a handful of large, complex ones.

CPUs are optimized to finish one task fast. GPUs are optimized to finish millions of similar tasks at once.

Machine learning workloads like matrix multiplications, convolutions, attention mechanisms are, at the arithmetic level, the same handful of operations repeated across enormous tensors. That's exactly the kind of regular, parallel workload GPUs were built for, which is why they became the default hardware for training and running Neural Networks.

Before CUDA: computing by pretending to render graphics

It's easy to forget that GPUs weren't originally designed to run arbitrary programs at all. Before 2007, a GPU's entire reason for existing was the graphics pipeline: take a scene made of triangles, run a vertex shader on each vertex to figure out where it lands on screen, then run a fragment (pixel) shader on each resulting pixel to decide its final colour, and write that colour to a framebuffer.

Researchers noticed something important that a fragment shader is really just a small program that runs once per pixel, in parallel, across the whole screen. If you could trick the GPU into treating your data as if it were a picture, you could get that same massive parallelism applied to ordinary numerical computation. This workaround became known as GPGPU —> General Purpose computing on Graphics Processing Units and it required contorting real math problems into a form the graphics pipeline would accept:

Input data (say, two matrices you wanted to multiply) had to be encoded as textures, because textures were the only large, structured input the GPU knew how to read.
To actually trigger the computation, you had to render a full-screen quad two triangles covering the entire viewport, a purely as a pretext to force the GPU to invoke your fragment shader once per output element. The "image" being rendered had no visual meaning; it was a disguise to make the hardware execute your math.
The fragment shader, written in a shading language meant for lighting and colour, had to be repurposed for arithmetic — reading inputs via texture lookups instead of normal memory reads, and outputting a numeric result as if it were a pixel colour.

A wildly simplified GLSL fragment shader "computing" a[i] + b[i] by disguising it as pixel colour math looked something like this:

// Old-style GPGPU: addition disguised as a fragment shader
uniform sampler2D textureA; // matrix A, encoded as a texture
uniform sampler2D textureB; // matrix B, encoded as a texture

void main() {
    vec4 a = texture2D(textureA, gl_TexCoord[0].xy);
    vec4 b = texture2D(textureB, gl_TexCoord[0].xy);
    gl_FragColour = a + b; // the "pixel colour" is actually your result
}

The result didn't come back as an array, rather it came back as an image sitting in a framebuffer, which then had to be read back out and reinterpreted as numbers. Control flow was extremely limited, loops and conditionals were weak or unsupported in early shader models, and there was no way to write to an arbitrary memory location only to the specific pixel a shader invocation happened to own.

NVIDIA released Compute Unified Device Architecture (CUDA) in 2007 specifically to remove this barrier, exposing the GPU's parallel execution units through a C-like programming model with real memory reads and writes, real control flow, and no requirement to pretend any of it was a picture. This is where GPU became a general purpose tool.

How a GPU organizes its execution

CUDA uses a model called SIMT — Single Instruction, Multiple Threads. One instruction is issued, and many threads execute it simultaneously, each on its own piece of data. This is organized into a hierarchy that maps directly onto the physical hardware:

Thread is the smallest unit of execution, with its own program counter and private registers.
Warp a group of exactly 32 threads, the actual unit the hardware schedules. All 32 threads in a warp are issued the same instruction on the same clock cycle.
Block up to 1024 threads scheduled onto one Streaming Multiprocessor (SM). Threads in a block can synchronize with __syncthreads() and share fast on-chip memory.
Grid every block needed to run the kernel, spread across all available SMs.

Here's what a minimal CUDA kernel actually looks like — adding two vectors, with one thread handling one element:

__global__ void vectorAdd(float *a, float *b, float *c, int n) {
    int i = blockIdx.x * blockDim.x + threadIdx.x;
    if (i < n) {
        c[i] = a[i] + b[i];
    }
}

Compare that to the GLSL disguise above — real array indexing, real addition, no textures, no framebuffers.

Occupancy and latency hiding

An SM typically holds far more warps resident at once than it can execute in a single cycle. When a warp stalls, most often waiting on a slow global memory reads the SM's scheduler switches to a different warp that's ready to go, and the stalled warp resumes once its data arrives. This is called latency hiding, and it's the main reason GPUs don't need CPU-style caches and speculative execution to stay busy. The fraction of an SM's maximum warp capacity actually kept resident is called occupancy.

Warp divergence

Because all 32 threads in a warp share a single instruction fetch, they must step through the code together:

if (data[i] > threshold) {
    // path A
} else {
    // path B
}

If threads within the same warp evaluate this condition differently, the warp diverges. The hardware masks off the threads taking path B, executes path A for the active threads, flips the mask, executes path B, and reconverges once both paths finish. Work that could have run in parallel now runs sequentially . A simple 2 way branch can roughly half effective throughput for that warp.

Takeaway: uniform, branch-free workloads run efficiently on a GPU. Conditional, data-dependent logic that splits a warp's behaviour is where you lose parallel performance.

The real bottleneck: memory, not compute

A modern GPU can typically perform far more arithmetic operations per second than it can pull bytes from memory per second. In most real workloads, memory bandwidth is the limiting factor. This is formalized in the roofline model: a kernel's achievable performance is capped either by peak compute throughput or by peak memory bandwidth, whichever constraint that kernel's ratio of math-to-bytes hits first.

Efficient CUDA code minimizes round trips to global memory. A common pattern used heavily in matrix multiplication is tiling. Load a chunk of input data from global memory into shared memory once, let every thread in the block reuse that tile for all its arithmetic, and only write the final result back to global memory at the end.

__global__ void matMulTiled(float *A, float *B, float *C, int n) {
    __shared__ float tileA[16][16];
    __shared__ float tileB[16][16];

    int row = blockIdx.y * 16 + threadIdx.y;
    int col = blockIdx.x * 16 + threadIdx.x;
    float sum = 0.0f;

    for (int t = 0; t < n / 16; t++) {
        tileA[threadIdx.y][threadIdx.x] = A[row * n + t * 16 + threadIdx.x];
        tileB[threadIdx.y][threadIdx.x] = B[(t * 16 + threadIdx.y) * n + col];
        __syncthreads();

        for (int k = 0; k < 16; k++)
            sum += tileA[threadIdx.y][k] * tileB[k][threadIdx.x];
        __syncthreads();
    }
    C[row * n + col] = sum;
}

Each value is fetched from global memory once per tile and reused 16 times from shared memory instead of being re-fetched for every multiplication. This is often the single biggest optimization available for compute-heavy kernels like this one.

Memory coalescing

When a warp requests data from global memory, the hardware tries to satisfy that request in as few large, contiguous transactions as possible. This works efficiently if thread 0 requests index 0, thread 1 requests index 1, and so on — coalesced access, where one wide transaction can satisfy the entire warp.

// Coalesced: consecutive threads read consecutive memory
c[i] = a[i] + b[i];              // i = threadIdx.x + blockIdx.x * blockDim.x

// Uncoalesced: large stride scatters each thread's access far apart
c[i] = a[i * stride] + b[i * stride];

If the data is scattered like - a linked list, a fragmented hashmap, or simply a large stride, the hardware can't fetch it in one transaction and instead issues a separate transaction per thread (or small group). This uncoalesced access can cut effective memory bandwidth by an order of magnitude, even though the computation itself hasn't changed — only the layout of the data being read.

Takeaway: the layout of your data in memory can matter as much as the algorithm itself.

From CUDA cores to Tensor Cores

Everything above involves a standard CUDA core which is a simple ALU capable of one scalar operation per clock cycle, per thread. That's already effective for general-purpose parallel work. But deep learning workloads are dominated by one specific operation: matrix multiplication. A fully connected layer is a matrix multiply; a convolution can be reformulated as one; the core operation inside transformer attention is, again, matrix multiplication.

To accelerate this specifically, NVIDIA introduced the Tensor Core with the Volta architecture in 2017.

Where a standard CUDA core computes one scalar operation per cycle, a Tensor Core computes a full mixed-precision multiply-accumulate — D = A × B + C — on small matrix tiles (originally 4×4) in a single clock cycle. Later architectures expanded this substantially: larger effective tile sizes, and support for more numeric precisions — FP16, BF16, TF32, INT8, and structured sparsity, which skips known-zero values to save additional cycles. "Mixed precision" here means the multiplication happens in a lower-precision format for speed, while accumulation happens in a higher-precision format to preserve accuracy.

This is the central hardware reason GPUs shifted from being primarily graphics chips that happened to be useful for AI, to being AI accelerators that also happen to still render graphics.

Putting it together

GPUs trade single-task speed for massive parallel throughput — well suited to the repetitive, regular math in ML workloads.
Before CUDA, that parallelism could only be reached by disguising computation as graphics rendering — data encoded as textures, math run inside fragment shaders, results read back out of a framebuffer.
Threads execute in warps of 32, in lockstep, and SMs hide memory latency by switching between resident warps (occupancy) — branching that splits a warp's execution path (divergence) reduces performance.
Memory bandwidth, not raw compute, is usually the actual bottleneck — structure data so neighbouring threads access neighbouring memory (coalescing), and reuse shared memory via tiling instead of repeatedly hitting global memory.
Tensor Cores exist specifically to accelerate the matrix multiplications neural networks are built from — a core hardware reason large-scale AI training is feasible at all.

Most CUDA optimization techniques you'll encounter fall into one of these five ideas: tiling, shared memory caching, avoiding divergent branches, maximizing occupancy, and choosing the right numeric precision. It's a long way from disguising math as pixel colours to letting a single instruction multiply entire matrix tiles. But the underlying goal never changed: keep thousands of simple cores fed, and keep them all doing the same thing at once.

Flowing vs. Thinking: How Liquid Neural Networks Diverge from LLMs

Sujal Suyash — Mon, 20 Jul 2026 16:41:29 +0000

If you follow the world of Artificial Intelligence, it is easy to assume that scaling up is the only path forward. Large Language Models (LLMs) have dominated the conversation by scaling to hundreds of billions of parameters, acting as massive, discrete reasoning engines.

But not all problems require a massive library of tokens. In the world of robotics, physical sensors, and continuous time, a different architectural philosophy is thriving: Liquid Neural Networks (LNNs).

LLMs process static, discrete symbols, LNNs use highly expressive fluid equations to navigate the chaotic, continuous flow of the physical world. To understand why an LNN can solve complex physics tasks with just a fraction of the compute of an LLM, we have to look at the actual mathematics.

The Common Misconception About "Liquid" Adaptation

People often state that LNNs "adapt" according to the input they receive. In the context of LLMs, we usually think of adaptation as shifting attention weights across discrete tokens. When people hear that LNNs adapt dynamically, a common misconception arises: Do LNNs actually change their weights or biases during the forward pass?

The short answer is no.

Just like an LLM, an LNN's parameters — its input weight matrix ( $U$ ), recurrent weight matrix ( $W$ ), and biases ( $b$ ) — are completely fixed after the training phase is complete.

So, what makes them "liquid"?

The adaptation happens within the hidden state ( $h (t)$ ) and the effective time constant ( $τ (t)$ ) of each individual neuron, which change continuously with time. Unlike LLMs, which are governed by the discrete-time architecture of the Transformer, LNNs are fundamentally built on Ordinary Differential Equations (ODEs).

Inside an LNN, the equation describing a neuron's state looks like this:

\frac{d h ( t )}{d t} = - \frac{h ( t )}{τ ( x ( t ) , h ( t ))} + f (W \cdot h (t) + U \cdot x (t) + b)

Where $f$ is a standard non-linear activation function (like $tanh$ or sigmoid).

Notice the time constant $τ$ . In a traditional continuous-time neural network, $τ$ is a fixed number. In an LNN, $τ$ is a dynamic function of the current input $x (t)$ and the current hidden state $h (t)$ .

In code, the right-hand side of that ODE is just this:

import numpy as np

def dh_dt(h, x, W, U, b, tau_fn, f=np.tanh):
    """
    LNN ODE: dh/dt = -h / tau(x, h) + f(W·h + U·x + b)
    tau_fn(x, h) -> the dynamic time constant function.
    """
    # 1. Compute the dynamic, liquid time constant
    tau = tau_fn(x, h)

    # 2. Compute the continuous rate of change
    return -h / tau + f(W @ h + U @ x + b)

# To update the hidden state over a discrete time step (dt):
def euler_step(h, x, W, U, b, tau_fn, dt):
    return h + dt * dh_dt(h, x, W, U, b, tau_fn)

Therefore, "adaptation" in an LNN doesn't mean the network is rewriting its own code (altering weights) on the fly. Instead, it means the speed and trajectory of the neuron's state change based entirely on the input. If the incoming data suddenly becomes noisy, erratic, or unpredictable, the neuron can mathematically "slow down" its integration to filter out the noise — all without needing new weights.

The Architecture of the Continuous World: Where LNNs Excel

Because of this unique, continuous-time framework, LNNs are uniquely suited for physical, real-world, streaming environments.

Ultra-Low Parameter Footprint & Edge Deployment: While an LLM needs massive server farms, LNNs achieve staggering complexity with very few parameters. Because they require orders of magnitude less memory and power, LNNs can run directly on edge devices, microcontrollers, and drones.
Handling Noisy, Irregular Time-Series Data: Transformers process time discreetly, expecting data in neat, sequential chunks. If you are analyzing continuous motion or gait datasets to detect subtle biomechanical anomalies, LLMs struggle. LNNs thrive here because their underlying ODEs treat time as a continuous flow.
Out-of-Distribution Survival: When an LLM encounters a scenario completely foreign to its training data, it hallucinates. If a drone powered by an LNN is trained in clear skies but suddenly encounters a heavy rainstorm, its liquid time constants automatically adjust to compensate for the erratic sensor noise, allowing it to maintain control.

The Architecture of the Discrete World: Where LLMs Excel

Despite the elegance of LNNs, they are not designed to process semantic knowledge. LLMs remain the undisputed engines of reasoning and massive knowledge retrieval.

Discrete Logic and Language: Language is not a continuous, flowing physical signal; it is composed of discrete symbols. The self-attention mechanism of LLMs allows them to weigh the relationship between every word in a document simultaneously. If you are building a system to parse and classify legal contract clauses, you reach for an LLM. LNNs, as sequential ODE solvers, are fundamentally ill-suited for parsing complex grammar or writing code.
Encyclopedic World Knowledge: Because LLMs boast billions of parameters, they double as incredibly vast, compressed databases of human knowledge. An LNN's hyper-efficient, compact footprint means it possesses virtually no capacity to retain world facts.
Long-Range Static Context: If you feed a massive codebase into a long-context LLM, it can instantly connect a variable on page 1 with a function on page 99. Because LNNs process data sequentially over time, they are susceptible to "forgetting" older static information when forced to read massive blocks of text.

Summary: Different Engines for Different Realities

Feature	Liquid Neural Networks (LNNs)	Large Language Models (LLMs)
Primary Domain	Robotics, autonomous systems, continuous sensors	Text, code, semantic reasoning
Time Paradigm	Continuous (ODEs)	Discrete (Tokens + Attention)
Scale	Extremely small (thousands of parameters)	Massive (billions of parameters)
Adaptation Mechanism	Dynamic, input-dependent time constants ( $τ$ )	Contextual attention weights

Ultimately, we are looking at two brilliant but entirely distinct philosophies of computation. LLMs are built to store, parse, and reason through the accumulated, discrete data of human history. LNNs are built like agile nervous systems to react, filter noise, and adapt to the chaotic, continuous flow of the physical world.

The Data Bottleneck and the Myth of "Recursive Learning" in LLMs

Sujal Suyash — Mon, 20 Jul 2026 10:51:30 +0000

The ultimate competitive moat isn't just algorithmic magic; it's data hoarding and compute infrastructure.

Trying to train a Large Language Model (LLM) from scratch requires a massive amount of computational power, resources, and, most importantly, data. Nowadays, even if we hypothetically consider that we can create a very powerful machine capable of performing large and complex computations, the biggest issue that lies in front of us is data. It is the most vital component responsible for generating a model.

Big MNCs like Google, Anthropic, and OpenAI were able to build their own LLMs from scratch because they had huge funding, years of collected user data, and very advanced, complex machines for training. These resources are generally not available to the general public. It has become a massive business model: they create huge LLMs, and the general public is forced to purchase API keys or subscriptions to get full access to them. The ultimate competitive moat isn't just algorithmic magic; it's data hoarding and compute infrastructure.

But while the economics of these models are clear, there is a common misconception about how they actually operate.

The Myth of Live Recursive Learning

A popular theory is that models like ChatGPT, Claude, and Gemini are based on "Recursive Learning", that they are self-learning machines constantly updating themselves based on the data you provide in a chat, recursively minimizing their loss with every prompt you enter.

In reality, current LLMs do not use a live, continuous recursive learning loop. When you interact with an LLM, its core neural network weights are completely frozen. The "learning" happens in massive, centralized, and highly structured phases long before you ever type a prompt.

Then how LLMs actually learn ?

Instead of live recursive updates, models are trained in distinct and compute heavy stages:

Pre-training: The model is fed trillions of tokens from diverse sources (the web, books, code). Its only objective is to predict the next word (Self-Supervised Learning). This is where it learns grammar, facts, and reasoning patterns. The compute required here is massive, taking thousands of GPUs months to complete.
Supervised Fine-Tuning (SFT): The model is trained on 10k–100k high-quality, human-curated instruction-response pairs. This teaches the model to follow instructions, adopt an "Assistant" persona, and format responses correctly.
Reward Modeling & RLHF (Reinforcement Learning from Human Feedback): Humans rank different model outputs. The AI uses these rankings to train a "Judge" model, which then guides the main model to generate responses that are safer and more aligned with human preferences.

The Danger of True Recursion: Model Collapse

What would happen if an AI actually did train recursively on its own generated outputs? It leads to a critical failure known as Model Collapse.

If an LLM trains iteratively on synthetic data (data generated by itself or other LLMs) without being grounded by fresh, human-generated data, its performance degrades over successive generations. The model begins to forget rare, nuanced information and over-indexes on the most probable, generic tokens. Eventually, this amateurish recursive loop causes the model to generate repetitive, homogenized, and nonsensical outputs.

So where does recursion actually happens ?

While live learning isn't happening in your chat window, Recursive Self-Improvement (RSI) is a massive focus in AI research. AI is increasingly being used to build the next generation of AI, but as a carefully supervised engineering tool rather than an automatic black box.

For instance:

Companies use highly capable models to generate rigorously filtered datasets to train smaller models.
AI agents are used to write the code that optimizes training infrastructure.

Ultimately, the biggest challenge in the field remains exactly what you might suspect: the DATA BOTTLENECK. We are running out of high-quality, human-generated text on the internet to train the next generation of models. That finite supply of human data, combined with the extreme cost of compute, is exactly why proprietary infrastructure remains the ultimate, heavily-guarded moat.