A few months into using ChatGPT and Claude daily for work, I hit a wall. Some days the model felt like magic it would write exactly the function I needed, catch a bug I'd missed, explain a concept better than any textbook. Other days, the same tool would confidently give me garbage. Wrong API names. Made-up library functions. Code that looked right but silently did the wrong thing.
For a while I chalked this up to "the AI being inconsistent." Eventually I realized the inconsistency was mostly on my end. I didn't actually understand what these models were doing under the hood, so I had no framework for knowing when to trust them, when to double-check them, or how to ask for what I actually wanted. Once I started treating prompting as an actual skill instead of "just typing a question," the quality of what I got back changed dramatically.
This is the article I wish someone had handed me back then.
Forget "AI." Think "Autocomplete on Steroids."
The most useful mental shift I made was dropping the sci-fi framing entirely. An LLM is not a mind. It's not reasoning about the world the way you or I do. At its core, it's a system trained on an enormous pile of text to do one job: given everything written so far, predict what token comes next.
A token isn't quite a word — it's more like a chunk of a word. "Understanding" might get split into "under," "stand," and "ing," for example. The model doesn't see letters or meaning directly. It sees these chunks, converts them into number patterns, and repeatedly asks itself "given this pattern so far, what's statistically likely to come next?" It does this one token at a time, feeding its own output back in as it goes.
That's genuinely it. There's no internal database being queried, no fact-checker running in the background, no persistent understanding of your codebase sitting there waiting. Everything the model "knows" is baked into billions of parameters shaped during training, and everything it produces is a very sophisticated guess based on patterns.
Once that clicked for me, a lot of weird behavior stopped being mysterious. Of course it confidently invents a library function that doesn't exist somewhere in its training data, that pattern of "here's a plausible-sounding function name for this task" was common enough to reproduce. It's not lying. It doesn't know the difference between "real" and "plausible." Prompting well is largely about closing that gap.
The Memory Problem Nobody Warns You About
Here's something that tripped me up constantly early on: the model doesn't remember your last conversation unless the product you're using specifically layers memory on top (and even then, it's usually a summarized version, not the full transcript).
Every single time you start a new chat, you're talking to a version of the model with zero knowledge of anything you've discussed before. It doesn't know your codebase, your company's conventions, your past preferences, or that you already told it three times not to use a particular library. You have to re-establish all of that context, every time, unless you're deliberately carrying it forward yourself.
This also applies within a single long conversation. There's a limit the "context window" to how much text the model can actively hold in view at once. Modern models have gotten genuinely good at handling long contexts, but I've still noticed that details buried in the middle of a very long conversation get "forgotten" more easily than things stated near the beginning or end. If something is important, it's worth restating it rather than assuming the model still has it front of mind.
Why the Same Question Gets You Different Answers
If you've ever asked an LLM the exact same question twice and gotten two different answers, that's not a bug. These models generate text probabilistically — at each step, there's a distribution of plausible next tokens, and the model samples from that distribution rather than always picking the single most likely one. This is actually intentional; picking the "safest" token every single time tends to produce dull, repetitive text.
The practical takeaway: don't treat a single response as ground truth, especially for anything factual or high-stakes. If accuracy matters, ask again, ask it to double-check itself, or verify independently. I've caught real mistakes just by asking "are you sure about that?" not because the model has some hidden self-awareness, but because the follow-up prompt gives it another shot at generating a better-calibrated answer.
The Prompting Habits That Actually Moved the Needle for Me
I went through the usual phase of collecting "magic prompt" lists from Twitter threads. Most of it was noise. What actually helped was much less exotic than I expected.
Writing the prompt like I'm briefing a smart contractor who's never seen my project. Not a genius who can read my mind — a competent person who needs the actual context. What's the tech stack? What have I already tried? What does "good" look like here? The lazier my prompt, the lazier and more generic the output.
Giving examples instead of describing what I want in the abstract. This one surprised me with how much of a difference it makes. If I want text reformatted a certain way, showing the model one or two input/output pairs almost always beats three paragraphs explaining the rules. It's the difference between describing a chair to someone versus just pointing at one.
Asking for reasoning before the answer, on anything nontrivial. For simple lookups this doesn't matter. But for debugging, architecture decisions, or anything involving multiple steps, explicitly asking the model to "think through this step by step before giving your final answer" noticeably reduces sloppy mistakes. It's forcing the model to lay out its own logic where it and I can spot errors, instead of jumping straight to a confident-sounding conclusion.
Telling it what NOT to do. I underestimated this for a long time. "Don't use any external dependencies," "avoid marketing language," "don't just restate the question" negative constraints cut out a huge amount of the generic filler these models default to.
Treating it as a back-and-forth, not a vending machine. My first prompt is rarely my best one. I've stopped expecting a perfect answer on the first try and started treating the process more like pairing with someone: get a draft, point out specifically what's off, iterate. This alone probably improved my results more than any specific technique.
Where This Gets Genuinely More Advanced
Once basic prompting stops being the bottleneck, there's a whole layer of stuff worth knowing about, even if you're not building AI products yourself:
Retrieval-augmented generation (RAG) is the fancy term for something intuitive: instead of relying on what the model memorized during training, you fetch relevant documents or data at request time and stuff them into the prompt. This is how a lot of "chat with your docs" tools work, and it's a big part of why some AI products feel much more accurate than raw ChatGPT they're not relying purely on the model's internal knowledge.
Tool use / function calling lets a model call external code hit an API, run a calculation, query a database instead of just generating text and hoping it's right. This is a huge deal in practice because it moves things like arithmetic, live data lookups, and factual retrieval away from "the model's best guess" and toward "an actual deterministic system."
Agents are essentially LLMs given a loop: take an action, observe the result, decide the next action, repeat, usually with some memory of what's happened so far. This is what's behind coding assistants that can browse a codebase, run tests, and fix their own mistakes rather than just producing a single response.
I'd hold off on diving deep into any of these until basic prompting feels comfortable. It's a bit like learning to debug before you learn a framework the fundamentals compound.
The One Thing I'd Tell Past Me
Stop treating the model like it either "knows" or "doesn't know" things, full stop. Treat it like a very well-read collaborator who's fast, tireless, occasionally brilliant, and prone to confidently bluffing when it's out of its depth much like a smart intern who hasn't yet learned to say "I'm not sure." Once you internalize that, you stop being surprised by the failures and start getting a lot more consistent value out of the successes.
If you're just starting to work with these tools seriously, don't expect to be good at this after one afternoon of playing with ChatGPT. It took me actively paying attention to what worked and what didn't over weeks of daily use before it clicked. But it does click and once it does, it's hard to go back to writing vague one-line prompts and hoping for the best.
Top comments (0)