What is an LLM?
A Large Language Model like GPT is a statistical pattern-matching engine designed to predict what text should come after an input sequence. Think of it as autocomplete on steroids 😅.
The model:
- Takes an input sequence of tokens (chunks of text, often parts of words).
- Outputs probability scores for every possible next token.
- Generates responses one token at a time through a process called inference.
Key Tricks in Agentic Programming
Here are the four tricks we use in agentic programming. In fact these are applicable to any agentic workflow:
Context Engineering
The output of an LLM is based entirely on its input (the "context"). Getting this right is everything.
The context gets rebuilt for each call, you remember about LLMs being stateless, right? So when you use GitHub Copilot the extension constructs and sends the context window.
Turn N: [system + tools + history (reasoning, tools called, etc) + user message]
↓
Turn N+1: [system + tools + updated history (reasoning, tools called, etc) + new user message]
I know I am repeating myself, but just to be sure you are getting it: the important distinction is that the model request contains the prior conversation(s) again on the next call, but the new call is not "the previous request duplicated". It is a new request assembled from the conversation state plus the new material.
When the context window gets tight (don't wait for it to be
filled to the brim) you have two options:
- Summarize everything that as been done in that session and open a new empty session.
- Rely on Claude or whatever you're using yo automatically start a compaction when the estimated request exceeds the context limit ( some even have a configured output/buffer allowance).
AGENTS.md
A Markdown file that prepares your agent with project-specific information.
Key Principles:
- Natural Language: write like you're prompting an LLM.
- Concise & Crisp: every word counts (precious context window space 😁).
- Focus on Positives: I always have this nagging feeling that when I have too many not to dos LLMs tend to perform worse. So try to use NOT only in a selected few places.
- Hierarchical: root directory for global rules, subdirectories for specific overrides.
-
Be assertive: direct, commanding language that leaves no room for interpretation or negotiation.
Avoid writing:
# Project Guidelines - It would be nice if you could try to use type hints when you remember to do so. - Maybe consider keeping the code simple if that's possible. - Please try not to add too many comments if you don't need to".Instead write:
# Project Guidelines ## MUST DO - Use type hints for all function signatures. - Keep code simple. No unnecessary abstractions. - Add comments ONLY for complex business logic. - Use uv for all package management.
Here you can find my AGENTS.md, if you like it feel free to use it.
Two Approaches in Agentic Programming
Which Approach is Right? But before answering that keep in mind that it is your job is to deliver code that's proven to work. You CANNOT blame the LLM. Check it works!
Fun story, I asked someone whom uses AI intensively to code about how good LLMs and top models make decisions since I had somewhat bad experience with it. Here is what he said:
I personally find the top models usually make good architecture and design choices. but not always. Perhaps 70% of the time? If you're not mentally prepared for the 30% painful miss, then it's going to be tiresome when it happens. And you might think "Why is everyone so impressed with this?".
| Project | Use |
|---|---|
|
Trust, but Verify |
|
Let It Go |
Trust, but Verify
| TODO List | Description |
|---|---|
| Micromanagement | Approve everything, frequent resets, detailed instructions |
| Plan → Execute | Planning mode for finer control, then execution in phases |
| Spec-Driven | Specify precisely, trust the system, verify at the end |
Let It Go
| TODO List | Description |
|---|---|
| YOLO | No approvals needed, let it run (great for hobby projects) |
| Ralph Loops | Wrap the entire process in bigger loops; run overnight |
| Multi-Agent Swarms | Specialized agents (testing, feedback, managers) working together |
⚠️ Warning
There security concerns with giving LLMs control over what they are creating with YOLO mode. Honestly I do NOT think you need to worry about it ATM. But I felt like mentioning it here.
Quick Reference Card
| Concept | Key Takeaway |
|---|---|
| LLM | Statistical engine predicting next tokens |
| Tokens | Chunks of text (words or parts of words) |
| Context | Everything passed to the LLM (prompt + history + tools) |
| Agent | LLM using tools in a loop to achieve goals |
| agents.md | Markdown file with project rules and standards |
| Compacting | Summarizing history to fit in context window |
| Ralph Loops | Nested loops for long-running tasks |
| YOLO | Zero-permission approach for trusted tasks |
Top comments (0)