Why You Should Understand Machine Learning and GPT Technology Before You Start Prompting Your AI

Most of us met large language models through a chat box. You type something, it answers, and it feels like talking to a very well-read colleague. That feeling is useful, and it is also the source of most of the frustration developers run into later. The model is not a colleague. It is a statistical system with specific habits, and once you know those habits, your prompts get better, your agents get more predictable, and your token bill gets smaller.

This article is about those habits.

A quick note on where this came from

You don't need a history lesson, but three moments explain most of what follows.

In 2017, Google researchers published Attention Is All You Need, introducing the transformer architecture. Its key idea, attention, lets a model weigh every token in the input against every other token, rather than reading strictly left to right. Then OpenAI's GPT line (2018 onward) showed that if you train a transformer on a huge amount of text with one simple goal, predicting the next token, it picks up grammar, facts, code, and reasoning patterns as a side effect. Finally, ChatGPT in late 2022 added instruction tuning and reinforcement learning from human feedback, which made these models feel like assistants rather than autocomplete engines.

The takeaway: underneath the polish, it is still next-token prediction. Everything below follows from that.

What the model actually does

An LLM does not look things up. It has no database of facts it consults. During training, its weights were adjusted so that, given a sequence of tokens, it assigns high probability to plausible next tokens. When you prompt it, it generates one token at a time, each chosen based on your prompt plus everything it has generated so far.

Two consequences matter for developers:

1. The model optimizes for plausible, not true. A fluent, confident answer and a correct answer are not the same thing, and the model has no built-in way to tell them apart.
2. The model only knows what is in its weights and in its current context window. Nothing else. No memory of yesterday's chat unless you send it again.

How hallucination comes about

A hallucination is what happens when the most plausible continuation is not a true one. A few common causes:

• Gaps in training data. Ask about an obscure library or a recent API change, and the model has weak signal. It still has to produce something, so it generates what such an answer would typically look like: a function name that sounds right, a parameter that seems reasonable.
• Pattern completion over fact recall. If every fetchUser function in its training data takes an id, the model will happily invent an id parameter for your custom one.
• Pressure from the prompt. If you ask "What does the autoRetry option do?" you have presupposed that the option exists. The model tends to go along with the premise.
• Decoding randomness. Sampling means a low-probability wrong token occasionally gets picked, and every following token builds on it.

Hallucination is not a bug that will be patched out in one release. It is a natural property of the mechanism, which can be reduced but not eliminated. Treat output as a draft to verify, not a source to trust.

A hypothesis: could context stuffing make it worse?

Here is a thought worth exploring. I want to be upfront that this is an informed hunch, not an established finding, so treat it as a hypothesis to test in your own work.

A lot of developers load as much as they can into the prompt: whole files, long documentation dumps, old conversation history, pasted logs. Call it context stuffing. (The more disciplined version of the same activity is often called context engineering: deliberately choosing what the model sees.)

LLMs are trained on an enormous mix of natural languages, programming languages, markup formats, and jargon. That breadth is a strength, but it also means many tokens are ambiguous. The word "table" could be furniture, a database table, or an HTML element. "Class" could be a school class, a CSS class, or an OOP class. An identifier like order could be a purchase, a sort direction, or an instruction. Variable names in one language can resemble words in another. Mixed-language comments, English prose wrapped around Python, JSON, and shell output all sit in one window.

The more heterogeneous material you pour in, the more room there is for the model to latch onto the wrong interpretation of a token or to blend patterns from different domains. Add stale or contradictory snippets and the model has to guess which one you meant. A guess dressed as a confident answer is exactly what a hallucination looks like.

Research gives this hunch some partial support: studies on long-context behavior (for example the "lost in the middle" findings) show that models use information unevenly depending on where it sits in the context, and that irrelevant material can degrade answers. Whether linguistic breadth specifically is a driver is something I can't claim, but the practical advice holds either way: give the model less, cleaner, more relevant context, and say what the terms mean.

Agentic apps: where misunderstanding gets expensive

In a chat, a bad answer costs you a retry. In an agentic app, where the model plans, calls tools, reads results, and loops, a bad step can trigger real actions. Knowing how the model works explains most of the surprises.

Why agents produce undesired output or do things you didn't ask for:

• Vague goals. "Clean up the project" has many plausible readings. The model picks the most statistically common one, which may include deleting files.
• Overlapping or loosely described tools. The model chooses tools by reading their descriptions. If two tools sound similar, or a description is broad, it will call the wrong one or call one unnecessarily.
• Over-permissioned tools. If an agent can write to disk, run shell commands, or hit production APIs, then "plausible next action" includes using those. Capability invites use.
• Helpfulness training. Instruction-tuned models lean toward doing more. Without explicit boundaries, "fix this bug" can turn into "fix this bug and refactor three nearby functions."
• Context drift in long loops. As tool results, errors, and intermediate thoughts pile up, your original instructions get buried. The model follows whatever is most salient, which may now be a stray log line rather than your goal.
• Compounding errors. One hallucinated assumption in step two becomes "fact" in the context for step three. The agent confidently builds on it.
• Untrusted content in context. Text fetched from a web page, an email, or a file can contain instructions, and the model has no reliable way to separate your instructions from data. This is prompt injection, and it is a direct consequence of everything being just tokens in one window.

What helps: state the goal and the boundaries explicitly (including what not to do), give each tool a narrow, distinct purpose and a precise description, grant the minimum permissions needed, require confirmation before destructive actions, keep the original task restated near the end of the context, and validate tool outputs before they re-enter the loop.

How this saves tokens

Tokens are what you pay for, and also what fills your context window. Understanding the mechanics saves them in several ways:

• Models are stateless. Every call resends the whole conversation. In an agent loop of 20 steps, early context gets paid for again and again. Trimming irrelevant material early compounds into big savings.
• Less stuffing, fewer tokens, and often better answers. Sending the relevant function instead of the whole repository costs less and gives the model less to get confused by.
• Fewer retries. A precise prompt that gets it right the first time beats three vague attempts. Every hallucination you have to catch and re-prompt for is paid for twice.
• Fewer runaway loops. An agent with a clear scope and clear stop conditions doesn't wander into unrequested work, and unrequested work is billed work.
• Structure that can be cached. Many providers discount repeated prompt prefixes. Keeping stable instructions at the top and variable content below makes that discount possible.
• Right-sized models and outputs. Knowing that output tokens usually cost more than input tokens, and that smaller models handle simple tasks well, lets you ask for concise formats (like structured JSON) and route easy work to cheaper models.

The bottom line

Prompting is not magic words. It is communicating with a next-token predictor that has no ground truth beyond its training and your context window. Once you internalize that, a lot of "mysterious" behavior stops being mysterious: hallucinations are plausible-sounding gap-filling, agents overreach because you left room to, and your token bill is mostly a record of how much unnecessary material you sent.

You don't need a PhD. Reading about how transformers work, how tokenization splits text, how context windows behave, and how sampling works is a few hours of effort that will improve every prompt you write afterward.

If you've seen mixed-language or messy context cause strange outputs, I'd like to hear about it in the comments. This is exactly the kind of hypothesis that gets better with more real-world examples.