How Does a Data Center Fit Inside Your Laptop?
What an AI model contains — and how it gets smaller.
How Does a Data Center Fit Inside Your Laptop?AI needs enormous computing clusters, expensive chips, and a lot of electricity. At least, that is the picture we usually see.
Then someone opens a laptop and runs an AI model without an internet connection.
Where did all that machinery go?
What comes out of the data center
During training, a language model makes predictions and adjusts billions of numerical values to improve them. These learned values are its parameters, commonly called weights. Together, they shape how it processes a question and generates an answer.
That is what a “20-billion-parameter model” contains: 20 billion learned values. They are not individual facts or ready-made answers. There is no folder marked “French grammar” or a file containing everything the model knows about cooking.
The weights are the main payload your laptop receives, along with the information and software needed to use them. The work of learning those values has already happened.
Training required repeated calculations, adjustments, and extra memory to manage the learning process. Answering your question uses the resulting weights without normally changing them. That is inference.
Think of producing a film and watching it. You do not need the studio to watch the result. An AI model does still calculate each response, so the comparison only goes so far — but it explains why creating a model and using it have very different requirements.
Some models remain too large for a personal computer. Others can fit, especially with a little preparation.
Making the numbers smaller
Numbers can be stored with different levels of precision. Reducing that precision saves space. In AI, this is called quantization.
For 20 billion weights, the arithmetic is straightforward: at 16 bits each, the values occupy about 40 GB. At 8 bits, 20 GB. At 4 bits, 10 GB.
Those are decimal gigabytes for the weight values alone; real formats add overhead and may keep some parts at higher precision. Still, the difference is substantial.
The model retains its learned values in a less precise form. With compatible software, the savings also apply while it runs, unlike a ZIP file that must be unpacked first.
The catch is that changing those values can change the answers. Some models tolerate the reduction well. Push it too far, and quality suffers. Making a model smaller therefore involves testing what it can still do reliably.
A smaller model can learn from a larger one
There is another route: distillation.
A larger model provides outputs or predictions that help train a smaller model. The smaller one develops its own weights, learning to reproduce useful behavior within a tighter budget.
It will not automatically acquire all the teacher’s abilities. But an assistant that summarizes notes and rewrites messages may not need them all.
The two approaches can work together: train a smaller model with help from a larger one, then quantize it for delivery. The model arriving on your device may be several steps removed from the expensive system that helped create it.
Fitting is only half the job
Even a 10 GB model needs more than 10 GB of memory to be useful. Your operating system and applications need room, as do the model’s calculations. Many language models also keep a temporary cache of the conversation’s calculations, which grows as the conversation gets longer.
The hardware then has to move and process that data quickly enough. A model that fits but takes an age to answer is unlikely to become anyone’s favorite assistant.
This is why developers adapt models to particular devices. Memory, speed, software support, and answer quality all have to work together.
You might simply receive an update
Most people will never choose a quantization format or download model weights themselves.
A manufacturer could do that work and deliver the result through a system update. You open your computer and find an assistant that can handle useful tasks locally — even offline. The model can remain proprietary; running on your device does not mean its maker has to publish it for everyone to reuse.
For the user, the result feels like another feature. Behind it sits a chain of training, adaptation, and testing that makes expensive research practical on everyday hardware.
That is why local AI depends on more than faster computers. It also depends on how much useful capability we can fit into them.
The data center has done its work. Your laptop gets to use the result.
Top comments (0)