DEV Community: Sungwoo Lee

Which AI Model Should You Use? A Routing Guide for Developers

Sungwoo Lee — Mon, 17 Aug 2026 17:20:01 +0000

You've got ChatGPT open in one tab, Claude in another, maybe Gemini in a third. And you still spend the first thirty seconds of every session wondering which one to actually use for the task in front of you.

That hesitation is the real problem, not the tools themselves. Each major model has a domain where it genuinely outperforms the others. The skill isn't picking a favorite — it's routing the right task to the right tool.

Stop Asking "Which AI Is Best"

The question has no useful answer because it conflates several fundamentally different design priorities. ChatGPT's reasoning-tier models (o-series) are optimized for breadth and hard logic. Claude is optimized for long-context fidelity and careful writing. Gemini is built around Google's data stack and native multimodal input. Perplexity is a real-time search layer sitting on top of models rather than a general-purpose assistant. They aren't competing in the same lane.

The better question: what am I trying to do, and which tool is architecturally suited for it? Think routing, not ranking — the same way a skilled team doesn't debate "email or Slack" in the abstract, they use the right channel for the message.

Four task categories cover most of the routing decision:

Writing and editing — prose quality, tone fidelity, long-form coherence
Coding and technical work — accuracy, debug quality, explanation clarity
Research and factual queries — recency, citations, hallucination rate
Reasoning and analysis — multi-step logic, structured thinking

What Each Model Actually Does Best

A note before the list: specific model names go stale within months, so this section describes families and tiers, not version numbers. The routing logic is the durable part — check each provider's current lineup for which model sits in which tier today.

ChatGPT (flagship tier + dedicated reasoning tier)
Strengths: breadth, tool integrations (image generation, code interpreter, browsing), the strongest dedicated reasoning models for hard logic and math, mature voice mode, a large custom-GPT ecosystem. Weaknesses: tends toward verbosity, less sharp on nuanced writing tone, reasoning-tier models cost more to run at volume. Best for: reasoning-heavy tasks, image generation, math and science problem-solving, or when you want one integrated tool ecosystem instead of several apps.

Claude (Sonnet / Opus)
Strengths: prose quality and tone fidelity that consistently edges out the alternatives, strong instruction-following, very capable on long documents given its large context window, lower "eager-to-please" drift on writing tasks. Weaknesses: image generation isn't built in, and the default tool ecosystem is smaller than ChatGPT's. Best for: long-form writing, editing, summarizing large documents, and any task where following instructions precisely matters more than raw creativity.

Gemini (Pro / Flash tiers)
Strengths: native multimodal input (image, video, audio), tight Google Workspace integration, a large context window, strong on tasks involving visual data. Weaknesses: writing tone less consistent than Claude. On the hardest reasoning problems the ranking between providers moves with every release — treat any published leaderboard as a snapshot and test on your own task. Best for: work inside Docs/Sheets/Gmail, analyzing images or video, and processing long documents when Google's tools are already part of your workflow.

Perplexity
Strengths: real-time web search with cited sources, the fastest path to current information, Pro Search decomposes complex queries into sub-searches. Weaknesses: shallow analytical depth, brief outputs, not built for long-form writing or multi-step reasoning. Best for: current events, fact-checking against live sources, and fast bibliography building. See the direct ChatGPT vs Perplexity comparison if search is your main use case.

A Quick Decision Flowchart

Need info from the last 30 days?          → YES: Perplexity
                                            → NO: continue
Primarily a writing or editing task?       → YES: Claude first
                                            → NO: continue
Coding, math, or logic?                    → YES: ChatGPT (reasoning model for hard problems) or Claude
                                            → NO: continue
Need image generation?                     → YES: ChatGPT or Gemini
                                            → NO: continue
Inside Google Workspace?                   → YES: Gemini
                                            → NO: default to ChatGPT or Claude by writing-quality need

For coding specifically, the calculus differs enough between "generate new code" and "debug this stack trace" that it's worth a dedicated comparison — see best AI coding assistants.

Two Copy-Ready Prompts You Can Use Today

1. Long-form report draft (Claude)

(Role) You are a senior analyst with expertise in [industry].
(Context) I'm writing a [word count] report on [topic] for [audience].
The goal is to inform a decision about [specific decision].
(Task) Write a structured first draft: an executive summary (3-4
sentences), 3-4 main sections with headers, and a conclusion with a
clear recommendation.
(Format) Professional prose. Bullet points only for lists of 4+ items.
Flag any claim that needs external verification.

2. Code debugging (ChatGPT or Claude)

(Role) You are a senior [language] developer.
(Context) This function is supposed to [describe intent]. It returns
[wrong output] when [input condition].
(Task) Identify the bug, explain why it occurs, and provide the
corrected code.
(Format) One-sentence diagnosis first, then the corrected code block,
then a brief explanation of what was wrong. List any additional edge
cases separately.

Both follow the same structure: role, context, task, format. That structure is what makes a prompt portable between models — you can hand the same skeleton to Claude, ChatGPT, or Gemini and get a comparably well-scoped answer back.

FAQ

Which AI model is best overall?
No single model leads across every task. Reasoning-tier ChatGPT models are strongest for hard math and logic, Claude leads on writing quality and long-context fidelity, Perplexity is the best tool for current information. "Best for what" is the only version of the question with a useful answer.

Is Claude better than ChatGPT for writing?
For most writing tasks — essays, reports, editing — Claude tends to produce noticeably better prose and follows tone instructions more precisely, without the verbose, hedge-heavy default style ChatGPT can fall into. For open-ended creative brainstorming, the gap narrows.

Which is best for coding?
Claude and ChatGPT are closely matched on most coding tasks. Claude tends to edge ahead on explaining code clearly and following precise specs. ChatGPT's sandboxed code interpreter for data analysis has no direct equivalent in Claude's standard interface — that's a real differentiator if your coding work involves running and inspecting data, not just writing it.

Can I just use one AI for everything?
You can, but you'll leave quality on the table in specific spots — using a general chat model for writing tends to produce verbose prose, and using a model without live search for current events risks stale or hallucinated facts. A two-tool setup, one model for writing and analysis plus a search-native tool for current information, covers most professional work well.

Which AI gives the most accurate, up-to-date information?
Perplexity, by design — it retrieves live web results and cites sources. Among the non-search models, the accuracy gap on well-established topics is small; the real risk across all of them is hallucination on specific statistics or citations, so verify anything load-bearing regardless of which model produced it.

Originally published at my-blog.org.

Best AI Agents in 2026: What They Do and How to Pick One

Sungwoo Lee — Mon, 17 Aug 2026 17:19:25 +0000

"AI agent" went from conference-talk abstraction to mainstream product feature faster than almost any other term in the field. ChatGPT has agents. Claude has agents. Your project management tool probably has one now too. But the word covers so much — from a simple tool-calling wrapper to a fully autonomous coding system running in a loop for hours — that it's nearly meaningless without qualification.

A working definition that holds up: an AI agent takes a goal, breaks it into steps, executes those steps using tools (search, code execution, file access, API calls), and iterates until the goal is met. The emphasis is on the loop — an agent doesn't just respond, it acts and corrects. That loop is the capability gap separating agents from conventional chatbots.

The space now spans purpose-built coding agents, research agents, workflow automation agents, general-purpose operator-style agents, and multi-agent frameworks where specialists hand off work to each other. Pricing and benchmarks here change fast enough that hard numbers go stale quickly — so this guide sticks to what's stable: architecture, autonomy level, and fit.

The Loop That Defines an Agent

A chatbot processes input and generates output in one pass. An agent runs:

Observe — receive the goal and current state
Plan — determine the next action
Act — execute it through a tool: run code, search, write a file, call an API
Observe — check the result
Repeat until the goal is met or it's stopped

Agents vary a lot in how autonomous this loop is. Some require human approval before each step ("human-in-the-loop"); others run unattended for long stretches. Autonomy level is one of the most important dimensions when evaluating an agent for a use case — more autonomy means more capability, but also more risk of errors compounding uncorrected.

Five Categories Worth Knowing

Coding agents write, run, debug, and iterate on code in a real execution environment — they run it and fix what's broken, not just suggest it. Examples: Claude Code, Devin, SWE-agent, OpenHands. Autonomy runs medium to high; most gate high-risk actions like deleting files or pushing to main behind approval. Best fit: teams with repetitive implementation work, bug fixing, and test generation.

Research and information agents search, synthesize, and produce structured output from multiple sources, reasoning across them rather than just retrieving them. Examples: Perplexity Deep Research, ChatGPT Deep Research, Gemini Deep Research. Autonomy is low to medium — search, read, synthesize, then human review. Best fit: literature reviews, competitive analysis, due diligence.

Workflow automation agents connect to business tools — Slack, Gmail, Notion, Salesforce — and automate multi-step workflows triggered by events. Examples: Zapier AI, n8n with AI nodes, Copilot Studio agents. High autonomy within a defined spec. Best fit: repetitive processes touching multiple apps — lead routing, meeting follow-ups, data pipeline maintenance.

General-purpose operator-style agents sit on top of frontier models and handle a wide range of tasks with a flexible tool set — generalists, not specialists. Examples: ChatGPT with tools, Claude with MCP, Gemini with Workspace extensions. Autonomy is whatever you configure, usually gated on consequential actions.

Multi-agent orchestration frameworks run at the infrastructure level: a planner agent breaks a large task into subtasks and delegates to specialists — a coder, a researcher, a writer. Examples: LangGraph, AutoGen, CrewAI. Very high autonomy in principle, but reliability drops as complexity climbs. Best fit: teams building AI products where one model's context can't cover the whole task.

What the Main Tools Are Actually Good At

Claude Code is a terminal-native coding agent that reads files, writes code, runs shell commands, and observes results in a loop. Its edge is reasoning quality on non-obvious problems — debugging race conditions, rearchitecting a module, building a feature from a description — where genuine reasoning beats pattern-matching. It extends tool access through MCP, worth understanding if you're configuring any agent's tool layer. Trade-off: CLI-only, no GUI.

ChatGPT's agent capabilities come in two forms — tool-augmented chat (search, code interpreter, file access) and Operator mode for UI navigation. Its Deep Research mode produces strong long-form cited reports. Weak spot: no persistent memory across sessions by default, and less reliable tool-calling on complex multi-step tasks than purpose-built coding agents.

Perplexity Deep Research is a read-only synthesis agent — it searches many sources, reads them, and produces a cited report. It beats manual search-and-read for competitive analysis or fact-gathering, but doesn't write code, execute workflows, or act beyond producing the report.

Devin was the first widely publicized fully autonomous software engineering agent, handling defined implementation tasks — features from specs, bug fixes, tests — with a GUI showing its terminal, browser, and editor live. Honest caveat: independent evaluations on open-ended tasks show more variable results than early claims suggested. It performs best on well-scoped tasks with clear acceptance criteria.

Zapier AI lets non-technical users build agents that watch triggers (new emails, form submissions, Slack messages) and run workflows across thousands of connected apps, set up conversationally. Its strength is breadth of integrations; its limitation is logic that stays rigid once configured, rather than reasoning through ambiguity.

OpenHands and SWE-agent are the open-source options for teams avoiding SaaS pricing or sending code to a third-party API — at the cost of setup complexity and bringing your own model API.

A Routing Framework for Choosing

Task type — coding, research, workflow automation, or general-purpose?
Autonomy tolerance — how comfortable are you with unapproved actions?
Technical complexity — reasoning about a complex system, or a well-defined process?
Interface preference — CLI, browser, or no-code?
Cost tolerance — free/self-hosted through paid consumer and enterprise tiers.

For coding agents specifically, see the detailed coding assistant comparison; for the models powering these agents, see which AI model to use.

A copy-ready template for prompting any agent:

GOAL: [observable definition of "done"]
CONSTRAINTS: [what the agent should NOT do]
RESOURCES: [tools and access it has]
ON FAILURE: [what to do if a step fails — retry, flag, or stop]

What Agents Still Can't Do

Error compounding. The longer an agent runs autonomously, the more a small early error compounds — a coding agent that misreads a spec on step 2 may have written hundreds of wrong lines by step 20.

Tool hallucination. Agents can call the wrong tool, misread output, or invent a result when a tool fails silently — more consequential than a chatbot hallucinating text.

Context limits. Even with large context windows, long agentic sessions degrade — agents repeat earlier steps or lose track of prior decisions as the window fills.

No inherent judgment about consequences. An agent doesn't know deleting a production database differs from deleting a test file unless told explicitly. Human oversight stays essential for anything with real external effects.

For single-turn tasks — answer a question, draft a paragraph — a chatbot is cheaper, faster, and more predictable. Reserve agents for tasks that genuinely need multiple steps, tool use, or iteration.

FAQ

What's the core difference between an agent and a chatbot?
A chatbot responds to a prompt in a single pass. An agent runs a loop — observe, plan, act, repeat — using tools until the goal is reached. Agents do things, not just describe them.

Which agent is best for coding tasks?
Claude Code and Devin are the strongest purpose-built options. Claude Code tends to win on complex, multi-file reasoning; Devin offers a more polished GUI and no terminal requirement. For lighter in-IDE help, a coding assistant fits better than a full agent.

Are AI agents safe for business workflows?
With guardrails, yes. Best practice is human-in-the-loop for consequential actions — the agent proposes, you approve. Reserve full autonomy for low-risk, reversible tasks, and treat agent output like work from a capable but fallible junior teammate.

Originally published at my-blog.org.

AI Agents vs Chatbots: What's the Actual Difference

Sungwoo Lee — Mon, 17 Aug 2026 17:18:49 +0000

Everyone's talking about AI agents right now. But most people using ChatGPT day to day are actually talking to a chatbot, and the difference is more than semantic. Conflating the two leads to mis-set expectations: you're disappointed when your "agent" can't follow through on a plan, or confused when an agent does things you didn't explicitly ask for.

The distinction is practical enough to matter for how you build with either one.

The Core Difference in One Sentence

A chatbot takes your input and returns a response. An AI agent takes your goal and figures out how to achieve it.

Four variables separate them:

Autonomy — who decides the next step? Chatbot: you, every time. Agent: it decides.
Tool use — can it call APIs, run code, read/write files? Chatbot: rarely, and only on request. Agent: this is central to how it works.
Multi-step execution — does it run a plan across several steps without a prompt at each one? Chatbot: no. Agent: yes.
Memory — does it carry state across steps and sessions? Chatbot: conversation window only. Agent: can use persistent external memory.

What a Chatbot Actually Is

A chatbot is built around conversational exchange: you send a message, it sends a reply. That's the whole unit of interaction. Modern chatbots (ChatGPT, Claude, Gemini in plain conversational mode) are extraordinarily capable within that unit — they write, analyze, reason, translate, and summarize well. But the design stays reactive: nothing happens unless you send the next message.

Key properties:

Stateless or limited context (this conversation, not across sessions, unless the product bolts on memory)
No autonomous tool use — you ask, it acts, once
One input, one output, repeat
Can't book, send, modify, or execute anything on your behalf without you prompting each step

The chatbot isn't "dumb" — the underlying model is often the same one powering an agent. The constraint is the interaction design, not the model's capability.

What an AI Agent Actually Is

An agent receives a goal and autonomously works out the sequence of steps to reach it — calling tools along the way, checking progress, adjusting based on what it finds. Four things distinguish it from a chatbot:

Goal-directed behavior. Instead of responding to one prompt, it works toward an end state. "Summarize last month's sales and email the report to the team" is a goal it decomposes and executes in order.
Tool use. Real external systems — web search, code execution, databases, calendar, email, CRM. Tools are what let an agent act, not just describe.
Multi-step planning and execution. It runs a loop — perceive, plan, act, check, repeat — until the goal is met or a stopping condition hits, without needing a human prompt between steps.
Memory and state management. It can track what it's done, what it found, and what's left, often across sessions via external storage.

The Spectrum, Not a Binary

Pure chatbot — single turn, no tools, no memory. An FAQ bot.
Conversational assistant — multi-turn context, some on-request tools (image generation, web search when asked), but you drive every step. ChatGPT in a normal chat.
Light agent — chains two to five steps, limited tool use, still surfaces output to you rather than acting fully in the world. A lot of "Copilot" features live here.
Full agent — multi-step autonomous execution with real tool calls, human checkpoints before irreversible actions, persistent memory.

Most people's day-to-day "agent" experience in 2026 sits in the light-to-medium range. Fully autonomous agents running unattended in production exist but need real deployment guardrails.

Same Request, Two Different Outcomes

Request: "Research the top 3 AI coding assistants, compare pricing, and send me a summary by email."

Chatbot: Writes a description of three tools with pricing. Stops. Doesn't send anything — you copy, paste, and send it yourself.

Agent: Searches the web for current pricing, opens each pricing page, extracts the data, formats a comparison, drafts an email, and sends it — reporting completion when done (possibly pausing to confirm the recipient first).

Neither output is "bad." The chatbot's prose might be excellent. But only the agent actually finished the task end to end.

When to Reach for Which

Use a chatbot when:

You need one well-formed output — a draft, an analysis, an explanation
You want to stay in control of every step
The task doesn't require real-world actions
You're iterating and want to steer each turn

Use an agent when:

The task has multiple defined steps that would be tedious to prompt one by one
You need real external actions — search, book, send, update
There's a clear, verifiable end state ("report sent," "ticket closed")
You're comfortable delegating with checkpoints

Directing an Agent Is a Different Skill Than Prompting a Chatbot

When prompting a chatbot, you guide each step explicitly. When directing an agent, you define the goal, the constraints, and the checkpoints up front. A reusable four-element format:

(Role) What the agent has access to and can do
(Context) Background, constraints, what "done" looks like
(Task) The end state you want — not just the first step
(Format) How it should check in: confirm before irreversible actions, report at milestones

Writing a goal this way front-loads the thinking an agent needs instead of trying to catch problems mid-run.

FAQ

Can ChatGPT be used as an AI agent?
Yes, in its tool-augmented modes — browsing, code execution, and Operator-style UI navigation move it toward agent behavior. Plain conversational ChatGPT without those tools is still a chatbot by this definition.

What tools do AI agents typically use?
Web search, code interpreters, file read/write, and APIs for calendar, email, CRM, and payment systems are the common set. The tools available are what actually give an agent the ability to act, not just respond.

Are AI agents riskier to use than chatbots?
Yes, proportionally to autonomy. A chatbot's worst-case output is bad text you can ignore. An agent's worst-case output is an irreversible action taken on your behalf. That's why human-in-the-loop checkpoints before consequential actions matter more as autonomy increases.

How do I know if I need a chatbot or an agent for a given task?
If the task ends with you reading and using an output yourself, a chatbot is enough and will be faster and more predictable. If the task ends with something actually happening in another system — an email sent, a record updated — you need an agent with the right tool connected.

Originally published at my-blog.org.

How to Build an AI Agent Without Writing Code

Sungwoo Lee — Mon, 17 Aug 2026 17:18:13 +0000

If you've spent much time in ChatGPT, you've hit the wall: type a prompt, get an answer, type another prompt, get another answer. At some point you think, "can't this just do the whole thing?" That itch is what makes people want to build an AI agent.

An AI agent is a system that pursues a goal across multiple steps, calls tools, and makes decisions without waiting for a human prompt at each step. The useful news for non-developers: in 2026, building a functional no-code agent is genuinely accessible. Platforms like n8n, Zapier, Make, and Lindy let you wire an LLM to real tools without a line of Python. The less convenient news: "no-code" doesn't mean "no thinking." The actual bottleneck in almost every agent build isn't the platform — it's the quality of the goal definition, the tool selection, and the testing loop.

The Four Components Every No-Code Agent Needs

Building a no-code agent means connecting four things. You don't write the model — you configure what it can access, what it should do, and what success looks like.

1. Goal statement (system prompt). The instruction set that tells the LLM what it's trying to accomplish, what constraints to respect, and what "done" looks like. This is the highest-leverage piece — a vague system prompt produces an unpredictable agent.

2. LLM brain. The model doing the reasoning — GPT-4o, Claude, Gemini, or a local model via Ollama. Most platforms let you pick from several. The model determines reasoning quality; the platform determines which tools it can reach.

3. Tool layer. What the agent can actually do: send email, read a spreadsheet, query a database, search the web, post to Slack, call an API. Each tool is a discrete capability the agent can invoke inside its reasoning loop.

4. Memory / context layer. How the agent retains information across steps or sessions — from "pass the last five turns" to "query a vector database for prior interactions."

You don't need all four maxed out. Most effective first agents use a precise system prompt, two or three tools, and stateless context — start there, and add complexity only when you hit a real limit.

Step 1: Define the Goal Before You Touch a Tool

The most common mistake non-developers make is starting with the tool — opening n8n, dragging nodes, and only later realizing the agent doesn't know what it's supposed to do. Goal definition comes first, always.

A usable goal statement has four properties:

Specific — not "help with emails" but "triage incoming support emails: label, draft a reply, flag anything needing human review."
Bounded — what the agent is explicitly not allowed to do. "Do not send any email without a human review step."
Success-defined — what "done" looks like, concretely. "A labeled email plus a draft reply in the Drafts folder, and a Slack ping if the email needs a human."
Escalation-aware — what happens when the agent hits a case it can't handle. "Flag for a human and stop. Do not guess."

Here's a copy-ready template you can drop straight into a system prompt field:

GOAL: [specific outcome, one sentence]
INPUTS: [what triggers the agent — new email, form submit, schedule]
ALLOWED ACTIONS: [tools it may call]
NOT ALLOWED: [explicit boundaries — no sending, no deleting, no external posts]
DONE WHEN: [observable success condition]
IF STUCK: flag for human review and stop. Do not guess.

Step 2: Connect Tools — Pick the Fewest That Work

Tool selection is where the platforms diverge most:

n8n — open-source, self-host or cloud, 400+ integrations, a code node for custom logic. Steepest curve, most flexibility. Good for technical non-developers who want control.
Zapier — cloud, easiest entry, thousands of app integrations plus AI Actions. Less flexible for branching logic. Good for simple linear tasks.
Make — visual builder with strong branching and looping. A middle ground between Zapier and n8n.
Lindy — purpose-built for AI agents, memory and human-in-the-loop steps included natively. The lowest-configuration path to real agent behavior.

The operating principle: pick the fewest tools that complete the goal. Every additional connection is another failure point. An agent that can send email, read a spreadsheet, and search the web already covers a wide range of real tasks without twenty integrations bolted on.

Step 3: Add Memory Only When Statelessness Actually Hurts

Most beginner agents are stateless — each run starts fresh. That's fine for one-shot tasks but breaks down for recurring workflows like weekly reports or ongoing customer threads, where the agent needs to carry context forward.

Memory options, roughly in order of complexity:

Session context — pass the last N messages in the prompt. Every platform supports this.
External storage (a spreadsheet, Notion, Airtable) — write key facts after each run, read them at the start of the next. Low-tech, effective.
Built-in memory layer (Lindy, some n8n setups) — you configure what gets saved; the platform retrieves relevant memories automatically.
Vector database (Pinecone, Supabase vector, Weaviate) — for agents that search across large bodies of text. Overkill for most first builds.

Start with session context. Add external storage once you actually need cross-run memory. Leave vector databases until the simpler version has run long enough to show you what's missing. If you're wiring an agent to pull from external data sources at all, it's worth understanding what MCP is first — it's the standard a growing number of these integrations are built on.

Step 4: Test for the Failure Modes That Actually Happen

No-code agents fail in predictable ways, not random ones:

Prompt drift — the agent handles simple cases fine but ignores constraints in edge cases. Fix: add explicit edge-case examples to the system prompt.
Tool call errors — the agent calls a tool with the wrong format, or the app returns something unexpected. Fix: check the platform's error logs, tighten formatting instructions.
Looping — the agent gets stuck redoing the same step. Fix: cap the max step count, and make sure the success criterion is actually checkable.
Overconfident execution — the agent takes an irreversible action (sends an email, posts publicly) when it should have paused. Fix: insert a human-approval step before anything irreversible.

A minimal test checklist before you trust an agent with anything real:

[ ] Typical input — does it do the right thing?
[ ] Boundary input (edge case, empty input, ambiguous request) — does it fail gracefully?
[ ] Adversarial input, if it reads external content — does it stay in bounds?
[ ] Three consecutive runs — does behavior stay consistent?

Human-in-the-loop isn't a failure mode to eliminate — it's a design feature to keep. In n8n and Zapier you build it explicitly: a "wait for approval" step before any send, post, or record change. The agent earns more autonomy as its tested boundaries hold up.

FAQ

Do I need to code to build an AI agent?
No. n8n, Zapier, Make, and Lindy all support building functional agents through visual interfaces and configuration forms. You do need to write a clear goal statement — that's thinking work, not code.

What's the best platform if I've never built an agent before?
Lindy is the most beginner-friendly purpose-built option — memory and human-in-the-loop are configured visually. Zapier is a safe start if you already know it. n8n gives you the most power at the cost of a steeper curve.

What can a no-code agent actually do reliably?
Email triage with draft replies, weekly report generation from spreadsheet data, first-response customer support, and meeting summarization with follow-ups. More consequential tasks — booking, finances — need careful testing and human checkpoints.

Originally published at my-blog.org.

System Prompt vs User Prompt: What's the Difference?

Sungwoo Lee — Mon, 17 Aug 2026 17:17:29 +0000

Most people who use ChatGPT daily have never seen a system prompt — but it's shaping every answer they get. Understanding the distinction between a system prompt and a user prompt isn't just a technical detail. It's the key to understanding why AI behaves the way it does, and how to actually get it to do what you need.

What Is a System Prompt?

A system prompt is a set of instructions given to a model before the conversation starts. It's invisible to the end user in most products, but it defines the model's persona, boundaries, tone, task scope, and behavioral rules for the entire session.

Think of it as the job description handed to an employee before their first shift. The employee — the model — reads it privately, internalizes the rules, and then works within those constraints for the rest of the conversation.

In most chat completion APIs, the system prompt is passed as a message with a dedicated system role. In practice, it looks something like this:

role: system
content: "You are a concise legal research assistant. You summarize
case law clearly and flag when professional legal advice is required.
Never speculate about legal outcomes. Keep responses under 300 words
unless asked to expand."

That text never appears in the chat window, but everything the user types gets interpreted through it. System prompts typically control: persona and name, tone and formality, knowledge scope (what topics the model will and won't engage with), output format defaults, safety guardrails, and task framing (coding assistant, tutor, analyst, writer).

What Is a User Prompt?

A user prompt is what you actually type into the chat box — the message you send each turn. It's the runtime instruction: what you want, right now, in this specific conversation.

User prompts are temporary. Each message lives in the conversation thread, but once the context window fills up, older messages get dropped, and none of it carries forward into a new session. Good user prompts are specific, concrete, and scoped — they tell the model what to do with the context it already has. They can override some system prompt defaults, like tone, format, or length, if the system prompt allows it, but they can't override hard-coded restrictions.

The Core Difference, Side by Side

Dimension	System Prompt	User Prompt
Who writes it	Developer / product builder	End user
When it's set	Before the conversation starts	Each turn, at runtime
Visibility	Hidden in most products	Visible in the chat thread
Scope	Entire session	Single turn
Defines	Persona, rules, defaults	Current task and specifics
Can the user change it?	No, in finished products	Yes, every message is a new one
Priority	Weighed first by the model	Interpreted through system context

Custom Instructions and Custom GPTs

ChatGPT's Custom Instructions feature lets you fill in a system-prompt-like field that gets prepended to every conversation. You write it once in settings and it persists across chats until you change it. It's not exactly a system prompt — the product's own system prompt still sits above it — but functionally it behaves like a persistent personal layer. It's your chance to tell the model your profession, your preferred output format, what you already know, and how you want to be spoken to:

"I'm a product manager at a SaaS company. Don't explain basic
business concepts. Always answer in bullet points for complex
topics. When I ask for feedback, be direct — don't soften criticism."

A Custom GPT goes further: it's a packaged product combining a system prompt written in a builder interface, uploaded knowledge files retrieved as context, configured tools (web search, code execution, image generation), and a defined audience. When you use one, you're writing user prompts that get interpreted through someone else's system prompt and knowledge base.

Why This Actually Matters for How You Prompt

If you're a regular user rather than someone building products, understanding this distinction changes how you work with AI in three concrete ways.

First, it explains weird behavior. If a chatbot on a website keeps saying "I can only help with X," that's not the model being dumb — it's the system prompt restricting it, and no amount of clever phrasing in your user prompt will fully route around it.

Second, it tells you what you actually control. You can't escape a well-written system prompt by asking nicely, but you usually can override tone and format if the system prompt doesn't explicitly lock them.

Third, it changes how much work your own prompts need to do. When you use a general-purpose chat interface directly — not a Custom GPT — you're writing into a generic system prompt, which means your user prompt has to specify role, context, task, and format yourself instead of inheriting them.

Same Question, With and Without a System Prompt

Without a system prompt, a generic assistant answering "How do I get a refund?" gives a 200-word explanation of how refund policies typically work in general — because it has no company-specific context to draw from.

With a system prompt in place:

role: system
content: "You are the support agent for StyleBox, a subscription
clothing service. Refund policy: customers have 14 days to return
items in original packaging. Direct refund requests to
support@stylebox.example or the order page at stylebox.example/orders. Do
not speculate about orders you don't have data on. Be warm but concise."

The same user question — "How do I get a refund?" — now gets: "For a refund on your StyleBox order, visit stylebox.example/orders or email support@stylebox.example. Returns are accepted within 14 days in original packaging. Anything else I can help with?" The entire jump in usefulness comes from the system prompt, not the user prompt — the user asked the exact same thing both times.

If you're the one writing that system prompt, the role prompting breakdown is worth pairing with this — it covers how to write the persona line so it actually shifts the model's output instead of adding noise.

Key Takeaways

System prompts are persistent, session-wide, and usually invisible; user prompts are per-turn and visible. In consumer apps, the product's own system prompt shapes every response before yours even arrives. Custom Instructions function as a personal system prompt layer that persists across sessions, and a Custom GPT is a packaged system prompt plus tools plus knowledge. If you want a model to consistently behave a certain way, put the instruction in the system prompt — not in every single user message.

FAQ

Can a user prompt override a system prompt?
Only within the boundaries it allows. Tone, format, and length are commonly overridable. Hard-coded restrictions are not.

Are Custom Instructions the same as a system prompt?
Functionally similar but not identical — Custom Instructions sit in front of the product's own underlying system prompt as a personal layer.

Why does a website chatbot refuse to answer basic questions?
Almost always a narrow system prompt, not a model limitation. It's been scoped to a specific task by whoever built it.

Do I need to write system prompts as a regular user?
Not directly, unless you're building a product. But understanding how they work explains why default chat behavior varies across apps, and it makes your own user prompts more effective.

Originally published at my-blog.org.

Role Prompting Explained: How AI Personas Actually Work

Sungwoo Lee — Mon, 17 Aug 2026 17:16:53 +0000

Role prompting is the practice of assigning an expert identity or persona to an AI before giving it a task. Instead of a bare request like "explain machine learning," you first tell the model who it is: "You are a machine learning researcher who explains complex topics to business executives." That single addition shifts the register, depth, and vocabulary of everything that follows.

Role is the first of four components in the standard prompt engineering framework — Role, Context, Task, Format. It's also the most misunderstood. People either skip it entirely or use it too vaguely ("You are a helpful assistant"), which adds no information the model doesn't already assume. Done well, it narrows the model's output toward expert-quality language on a specific topic. Done poorly, it's noise.

Why Role Prompting Actually Works

Language models generate text by predicting the most probable next token given what came before. When you write "You are a senior financial analyst with 15 years of experience in fixed income markets," you've loaded the context with tokens that statistically co-occur with technical vocabulary, structured frameworks, hedged professional language, and domain-specific caution.

The model doesn't "become" a financial analyst — it generates text that statistically resembles what one would produce in a similar context. That distinction matters for understanding the limits (more on that below), but the practical effect is real: role-primed responses use the vocabulary, structure, and epistemic stance of the assigned domain.

This is different from hallucination prevention — roles don't stop a model from making things up — and different from RAG, which gives a model access to new facts. Roles reshape style, structure, and specificity, not factual accuracy.

Writing a Role That Actually Moves the Output

The most common mistake is a generic role. "You are an expert" tells the model almost nothing, since "expert" is a near-universal token in AI output already. The more specific the domain, experience level, and perspective, the stronger the shift.

Level	Example	Effect
No role	(none)	General-purpose defaults
Generic	"You are an expert"	Minimal shift
Domain-specific	"You are a data scientist"	Moderate vocabulary shift
Role + experience	"...with 10 years in healthcare analytics"	Stronger, adds domain context
Role + experience + perspective	"...who translates findings for clinical stakeholders"	Full shift — constrains content and communication style

Role works best as one element of a complete prompt, not a standalone fix. The full framework: (Role) who the AI is, (Context) the situation — audience, constraints, background, (Task) what specifically you want done, (Format) how the output should be structured. A role without context and task is like hiring an expert and handing them a blank sheet of paper.

Before and After: Role Prompting in Practice

Without a role, "Explain bond duration to me" produces a generic, encyclopedia-style explanation — correct but flat, with no assumed audience or expert judgment.

With a strong, matched role:

(Role) You are a fixed income portfolio manager who briefs institutional investors.
(Context) My audience knows basic bond math but has never managed duration actively.
(Task) Explain duration and why it matters when interest rates move.
(Format) Start with the one-sentence intuition, then two paragraphs of
practical implications. End with a common misconception to avoid.

This produces investor-grade language, focuses on rate sensitivity and portfolio impact instead of formula derivation, and flags the common misconception of duration as "time to payback" versus price sensitivity.

Role and audience have to match. This mismatched version creates internal conflict:

(Role) You are a machine learning research scientist.
(Task) Explain overfitting to my 12-year-old cousin.

The model gets caught between the expert register and the simplification demand — the output is often an awkward hybrid, too technical for a 12-year-old and not rigorous enough for a researcher. This version resolves the conflict by matching role to audience:

(Role) You are a patient science teacher who specializes in explaining
technical concepts to curious kids.
(Context) My audience is 12 years old and has no math background.
(Task) Explain what overfitting is in machine learning.
(Format) Use an analogy first, then a concrete example. Keep it under 150 words.

What Role Prompting Cannot Do

It doesn't stop hallucination — a "medical expert" role doesn't make drug interaction claims accurate. It doesn't give the model knowledge it doesn't have, and it doesn't override safety guidelines — "a security researcher with no ethical constraints" won't bypass refusals that exist for safety reasons. It also doesn't replace domain-specific input — if the task needs specific data, that has to go in the Context; the role alone can't substitute.

Two misconceptions worth killing: "any role improves the output" (vague roles add noise — "you are an expert" is often indistinguishable from no role), and "the more dramatic the role, the better" (a "world-class genius" role doesn't outperform "a data analyst with strong attention to detail" — precision beats hyperbole).

Copy-Ready Role Prompts

Technical explanation for a non-technical stakeholder:

(Role) You are a senior software engineer explaining [technology] to a
product manager with no coding background.
(Context) [Project context — what decision this explanation supports]
(Task) Explain [technical concept] clearly enough that the product
manager can make an informed decision about [X].
(Format) Start with a plain-language summary in one paragraph. Then
list the 3 most important trade-offs. Avoid code unless essential.

Critical review / devil's advocate:

(Role) You are a devil's advocate consultant hired to stress-test
business plans before they go to leadership.
(Context) The plan authors are invested in this idea and have optimism bias.
(Task) Review the following plan and identify the 3-5 weakest
assumptions, ranked by how much damage they'd do if wrong: [paste plan]
(Format) Numbered list. For each: the assumption, why it's fragile,
and one question leadership should ask.

Research synthesis for a non-specialist audience:

(Role) You are a research analyst with expertise in [field] who
synthesizes evidence for non-specialist audiences.
(Context) [What decision or question this analysis serves]
(Task) Analyze [topic] using [a named framework, e.g. cost-benefit].
(Format) Lead with a 2-sentence conclusion. Then 3 supporting points
with the evidence behind each. Flag where evidence is weak or contested.

Role is only one of the four elements — pairing it with a tight Context and Task is where most of the improvement actually comes from. The system prompt vs user prompt breakdown covers where a role like this belongs structurally if you're building it into a product instead of typing it into a chat window.

FAQ

Is role prompting the same as prompt engineering?
No — it's one technique within it, specifically the Role element of a four-part framework that also includes Context, Task, and Format.

Does role prompting work on all AI models?
Yes, the underlying mechanism applies to any current large language model, though sensitivity to role framing varies by model.

Can I combine multiple roles in one prompt?
A role with coherent dimensions — "a data scientist with a background in behavioral psychology" — works well. Two conflicting roles tend to produce incoherent output.

Why doesn't "you are a helpful assistant" count as a role prompt?
Because it adds no new information — every model is already calibrated to be helpful by default. Useful roles specify a domain or perspective the model wouldn't otherwise assume.

How specific should a role be?
Specific enough to narrow the domain, experience, and communication style, without taking longer to write than the actual task. Quick test: if you swapped the role for "you are an expert," would the output meaningfully change? If not, it's too vague.

Originally published at my-blog.org.

Prompt Chaining Explained: Break Big AI Tasks Into Steps

Sungwoo Lee — Mon, 17 Aug 2026 17:16:17 +0000

There's a moment every regular AI user hits: you write one long, careful prompt for a complex task — a report, a strategic analysis, a long-form article — and the output is technically fine but oddly flat. It touches everything in the same shallow way, misses the nuance you cared about, and reads like it was written in a rush.

It wasn't written in a rush. You just asked for too much at once.

Prompt chaining fixes this. It's the technique of breaking a complex task into a sequence of smaller prompts, where each step does one thing well and passes its output forward as input to the next. The result is a model that reasons deeply at each stage instead of sprinting across the whole task simultaneously.

What Prompt Chaining Actually Is

Prompt chaining means splitting a complex task into a sequence of smaller prompts, each focused on a single step, where the output of each step becomes the input for the next. Instead of asking a model to "write a full report" in one shot, you ask it to research the topic, then build an outline, then draft each section, then polish the language — as four separate, focused prompts.

The mechanism is straightforward: when you give a model one large, multi-part task, it has to allocate reasoning across everything at once, which means nothing gets full depth. It's similar to asking a colleague to simultaneously research, outline, and write an article in a single sitting — the thinking becomes shallow by necessity. Chaining gives the model permission to be thorough at each stage. Research doesn't have to worry about structure. Drafting doesn't have to worry about sourcing. Editing doesn't have to juggle both.

The Cognitive Load Problem

A single long prompt distributes the model's attention across every subtask simultaneously, so each part gets less depth than it would on its own. Complex tasks have competing goals — fact-finding, structure, tone, concision — that pull reasoning in different directions. Chaining isolates each goal into its own step.

Say you're writing a market analysis on renewable energy adoption. A single prompt asking the model to "research the topic, identify trends, analyze challenges, propose recommendations, and write a 1,500-word summary" will produce something — but the research will be thin, the analysis will miss second-order effects, and the recommendations will be generic.

Run the same task as a chain instead: step one asks only for research and key data points. Step two takes that research and builds a structured outline. Step three drafts each section from the outline. Step four edits for clarity and tone. Each step's output is genuinely better than what a single prompt manages — not because the model got smarter, but because it's focused on one job at a time.

The Four-Step Pattern: Research, Outline, Draft, Edit

The most reliable chain for knowledge work is a four-step sequence, where each step's output is explicitly pasted into the next prompt — the model reasons from concrete material, not a fresh start.

Step 1 — Research:

(Role) You are an expert researcher on [topic].
(Context) Audience: [describe]. Scope: [describe].
(Task) Surface the key facts, data points, and arguments relevant to this topic.
(Format) Bullet points, sourced where known.

Step 2 — Outline (paste Step 1's output into Context):

(Role) You are a content strategist.
(Context) [paste Step 1 research output here]
(Task) Convert this research into a logical outline with 5 sections.
(Format) H2 headers, each with 2-3 supporting bullet points.

Step 3 — Draft (paste Step 2's outline into Context):

(Role) You are an expert writer in [domain].
(Context) [paste Step 2 outline here]
(Task) Write a full draft, one section at a time, following the outline.
(Format) Prose, [target length], [tone — e.g. direct and executive].

Step 4 — Edit (paste Step 3's draft into Context):

(Role) You are a copy editor.
(Context) [paste Step 3 draft here]
(Task) Tighten the language, cut redundancy, strengthen the argument, fix tone drift.
(Format) Return the revised full text, followed by a short change log.

Real Chains You Can Reuse

The four-step structure applies to almost any multi-step knowledge task — only the specifics of each prompt change.

Blog post chain (4 prompts): the most common use. Research, argument structure, prose drafting, and editorial polish are genuinely different cognitive modes, and isolating them produces a noticeably tighter final draft than one long prompt.

Business report chain (4 prompts): executive reports need precise sourcing, clean structure, and terse language. Chaining separates the analysis phase from the writing phase — which mirrors how a research team would actually divide the work.

Product description chain (3 prompts): a shorter chain works here — feature extraction, then benefit translation, then a copywriting pass. Even a 3-step chain beats a single prompt when the copy needs to be precise and on-brand.

When to Chain and When Not To

Chaining is not free — it costs more time and more prompts than a single request, so it's only worth it when the task genuinely spans different cognitive modes.

Chain it: a 2,000-word report, a market analysis that requires research and synthesis, a long-form article built from scratch.

Don't bother: translating a paragraph, answering a single factual question, summarizing one short document, brainstorming a quick list of ideas — these are single-objective tasks where decomposition just adds overhead.

Prompt chaining is tool-agnostic — it works in ChatGPT, Claude, Gemini, or any other chat interface, because it's a workflow technique, not a platform feature. You're managing the context window manually, pasting each step's output into the next prompt. If you want to automate the handoffs, workflow tools like n8n or Make can pass outputs between steps programmatically. Once you're comfortable pairing chains with sharper single prompts, the prompt engineering fundamentals guide covers the Role/Context/Task/Format structure each individual step in a chain should follow.

FAQ

What is prompt chaining in AI?
A technique where you break a complex task into a sequence of smaller, focused prompts — each handling one subtask, with its output feeding into the next step. Instead of one prompt doing research, outlining, writing, and editing, you run four separate prompts, each doing one job well.

Why does chaining produce better results than one long prompt?
Because a model spreads its reasoning across everything in a multi-goal prompt simultaneously, so every part gets shallower treatment. Chaining focuses the model on one objective per step, letting it reason more deeply at each stage instead of skimming across all of them at once.

How many steps should a chain have?
Three to five is the practical range for most knowledge work. The four-step Research → Outline → Draft → Edit pattern covers the majority of writing and analysis tasks. Don't add steps for their own sake — each one should produce output that's genuinely useful on its own.

When should I NOT use prompt chaining?
Skip it for simple, single-objective tasks: translating a paragraph, answering a factual question, generating a short list, summarizing a brief document. It's only worth the extra time when the task actually requires separate cognitive modes — research versus writing versus editing.

Prompt chaining isn't a trick or a workaround — it's a structural insight: complex tasks have phases, and phases go better when they're separated. The next time a single prompt gives you a mediocre result, don't rewrite the prompt. Decompose the task instead.

Originally published at my-blog.org.

Fine-Tuning vs Prompt Engineering: A Developer's Decision Framework

Sungwoo Lee — Mon, 17 Aug 2026 17:15:41 +0000

If you've used ChatGPT or Claude for anything beyond toy tasks, you've probably hit a ceiling and wondered: "Should I just fine-tune a model on my own data?" It's a fair question. Fine-tuning sounds like the professional move — training the model to actually know your domain instead of re-explaining it every session.

For most use cases, the honest answer is: not yet, and maybe never. Prompt engineering, especially combined with RAG (retrieval-augmented generation), gets you most of the way there at a fraction of the cost and complexity. This post walks through what each approach actually involves, when fine-tuning genuinely earns its keep, and a decision framework so you can stop guessing.

The Core Distinction

Prompt engineering designs inputs — instructions, context, examples, formatting — to guide a pre-trained model toward better outputs. The model's weights never change. You're shaping behavior at runtime through what you put in the context window.

Fine-tuning continues training a pre-trained model on a new, domain-specific dataset. The weights do change. The result is a model that has internalized patterns, style, or structure from your training examples.

RAG sits between the two. Weights stay frozen, but at inference time, relevant documents get retrieved from your knowledge base and injected into the context automatically. The model reasons over fresh information without ever being retrained on it.

The practical question that decides which one you need: do you need the model to behave differently (tone, format, task pattern), or do you need it to know different information? Behavior change points to prompt engineering or fine-tuning. Information access points to RAG.

The Trade-offs, Honestly

Cost. Prompt engineering costs almost nothing to start — API credits and iteration time. Fine-tuning has real layers: preparing a labeled dataset (the biggest hidden cost, usually the most tedious part of the whole project), training compute, and ongoing maintenance as base models get updated underneath you.

Difficulty. Prompt engineering is accessible to anyone who can write clearly and iterate systematically — no ML background required. Fine-tuning requires enough ML literacy to evaluate training quality, understand overfitting and evaluation metrics, and manage dataset versioning and training runs.

Data. Prompt engineering needs zero additional data — you're leveraging what the model already knows plus whatever you put in the prompt. Fine-tuning needs enough high-quality examples to produce meaningful behavioral change without overfitting on a small, narrow set.

Latency. Long prompts with heavy few-shot context add real processing time. A fine-tuned model has internalized the pattern, so it can run on shorter prompts — inference latency can actually drop because the prompt doesn't need to carry as much context.

Why RAG Usually Wins for "Knowledge" Problems

Fine-tuning bakes information into weights — but weights are static. If your knowledge changes (new products, updated policies, fresh data), you have to retrain. RAG retrieves current information at inference time, so it stays accurate without a retraining cycle.

Fine-tuning also doesn't reliably fix factual recall. Models trained on domain data still hallucinate — fine-tuning is much more reliable at shifting style and behavior than at improving factual grounding. If your goal is "make the model know our data," RAG is the more dependable tool.

A Decision Framework You Can Actually Use

Work through these in order — don't skip ahead:

Can a clearer, more structured prompt get you the output you need? If yes, stop here. Spend a few hours iterating before considering anything else.
Is the weakness about missing information, not behavior? If yes, build a RAG pipeline over your documents instead.
Is the gap purely style, tone, or format? If yes, a detailed system prompt with few-shot examples usually closes it.
Do you have a large set of high-quality labeled examples and a narrow, stable task? If yes, fine-tuning is worth evaluating.
Is the remaining performance gap big enough to justify data annotation and training infrastructure? If yes, proceed. If no, stay with prompt engineering plus RAG.

Fine-tuning is rarely step one. It's step four or five, after the cheaper options are exhausted.

Copy-Ready Prompts That Do the Work of Fine-Tuning

These patterns get you fine-tuning-like consistency from a base model — no training pipeline required.

1. Few-shot in-context learning (replaces task-specific fine-tuning):

(Role) You are an expert at [specific task].
(Context) Here are examples of the input→output pattern I need:
Input: [example 1 input] → Output: [example 1 output]
Input: [example 2 input] → Output: [example 2 output]
(Task) Now apply the same transformation to: [your input]
(Format) Output only the transformed result, no explanation.

2. RAG simulation (inject your own knowledge base into a single prompt):

(Context) Use only the following information to answer the question.
Do not use any external knowledge. If the answer isn't in the
provided text, say "Not covered in the provided documents."
DOCUMENTS: [paste relevant sections]
(Task) [User question]
(Format) Answer in 2-3 sentences. Cite the document section you used.

3. Pre-check prompt (before you invest in a training pipeline, sanity-check whether you need one):

(Task) I am considering fine-tuning a model for the following task:
[describe task]. Before I invest in fine-tuning, evaluate whether a
well-designed prompt could achieve similar results.
(Format) Answer these three questions:
1. What specific behavior does this task require that a prompt cannot specify?
2. Roughly how many labeled examples would fine-tuning need to show real improvement?
3. Is there a simpler approach (system prompt, few-shot, RAG) I should try first?

If you want the underlying mechanics of why structured prompts outperform vague ones, the prompt engineering fundamentals breakdown covers the four-element framework these examples are built on.

FAQ

Is fine-tuning worth it for most people?
For most individuals and small teams, no. The data annotation burden and infrastructure cost rarely justify the improvement over a well-designed prompt. It pays off mainly for high-volume, narrow, repetitive tasks with consistent input/output patterns.

Can prompt engineering replace fine-tuning entirely?
For the majority of use cases, yes — especially combined with RAG for knowledge and few-shot examples for behavior. It genuinely can't replace fine-tuning in a narrow set of cases: extremely high-volume deployments where long-prompt inference cost is prohibitive, or very specific output formats the model resists without training.

What's the practical test for "do I need RAG or fine-tuning"?
Ask whether the model is failing because it doesn't know something, or because it isn't behaving the way you want. Missing knowledge → RAG. Wrong behavior → prompt engineering first, fine-tuning only if prompting plateaus.

When should I actually consider fine-tuning?
When you have a narrow, high-volume, stable task, a solid set of high-quality labeled examples, a real inference-cost concern from long prompts, and you've already optimized your prompt and still see a measurable gap. If any of those aren't true, stay with prompting.

Originally published at my-blog.org.

What Is an AI Agent? Agentic AI Explained for Developers

Sungwoo Lee — Mon, 17 Aug 2026 17:14:56 +0000

"AI agent" gets used for everything from a chatbot with a slightly longer memory to a system that autonomously books flights and updates your calendar. Here's the distinction that actually matters if you're building or evaluating one: an AI agent doesn't just respond to a single prompt — it pursues a goal across multiple steps, using tools and memory to complete work with limited human oversight. The defining property is agency: the system decides what to do next, rather than only answering what it was asked.

"Agentic AI" describes systems designed for that kind of autonomy. An agent can call external APIs, browse the web, write and execute code, manage files, or chain together multiple LLM calls — without a human approving each individual step along the way.

The Cognitive Loop: How Agents Actually Work

Most agent implementations run some version of a four-phase loop:

Perceive — take in the current context: the task goal, available tools, prior outputs, and any new information from the environment (search results, API responses, file contents).
Plan — reason about what to do next, often breaking a large goal into sub-tasks. More capable agents explicitly reason step by step before acting, similar to chain-of-thought prompting.
Use Tools — call a tool: a web search API, a code interpreter, a database query, a calendar API. The result flows back into the agent's context.
Act / Loop — take an action (write output, save a file, call another API, update memory), then re-enter perceive-plan-use until the goal is reached or the step/budget limit is hit.

This is widely known as the ReAct pattern (Reason + Act), and it's foundational to most agent frameworks in production today. If you've built a tool-calling loop yourself, you've probably implemented something close to this whether or not you called it that.

Agent vs. Chatbot vs. AI Assistant

The terms get used loosely enough that a table is more useful than more prose:

Dimension	Chatbot	AI Assistant	AI Agent
Interaction model	Single-turn Q&A	Multi-turn conversation	Goal-directed, multi-step, autonomous
Tool use	None or minimal	Some (search, image generation)	Central — calls APIs, runs code, manages files
Memory	Usually none	Short conversation window	Can use persistent memory or external storage
Autonomy	Zero	Low	High — decides its own next action
Human approval per step	N/A	Often	Optional, configurable
Typical failure mode	Irrelevant answers	Hallucination	Wrong action taken, errors cascade
Example	Basic FAQ bot	A conversational assistant	A system that books your flights and updates your calendar

What This Looks Like in Practice

Research agents — given a question, search the web, read sources, extract findings, and produce a structured report with minimal steering between steps.
Code agents — write code, run it, read the error output, revise, and repeat until tests pass.
Data pipeline agents — chain a task like "pull last month's sales data, calculate churn by segment, email a summary" across database queries, calculations, and an email API.
Customer support agents — look up an account, identify the issue, apply a fix through an API (like issuing a refund), and send a confirmation, without a human approving each step.
Personal productivity agents — read your inbox, parse meeting requests, check your calendar, propose times, and draft the response.

Directing an Agent: A Reusable Prompt Pattern

When you're the one instructing an agent — through Claude Projects, a custom GPT with actions, or your own framework — a directive with these four elements consistently produces better runs than a one-line instruction:

(Role) You are a [specific role — e.g. "release notes agent"].

(Context) Goal: [end state]. Constraints: [budget, scope, tone].
Available resources: [tools/APIs/files it can use].

(Task) Produce: [the concrete deliverable, not just a first step].

(Format) Report back as: [structure]. Flag me before: [irreversible actions].

That last line matters more than it looks. Any task where a wrong action is costly — spending money, sending an email, deleting data — should have an explicit human-in-the-loop checkpoint configured before you let the agent run unattended.

The Honest Limits

Errors cascade. If an early step produces a wrong output and the agent doesn't catch it, later steps build on the mistake.
Long-horizon tasks are still unreliable. Agents do best on well-defined, bounded tasks. Open-ended goals over many steps tend to drift or stall out.
Tool use is attack surface. An agent that can write files, call APIs, and execute code can do real damage if prompted maliciously or misconfigured — treat tool permissions the way you'd treat any other credential scope.
Memory is still primitive. Most agents have limited, inconsistent recall of past sessions. Durable memory across tasks isn't standard yet.
Cost and latency add up. Agentic tasks run many LLM and tool calls in sequence. Something that looks simple to a human can burn through dozens of calls and take minutes to finish — budget accordingly if you're billing by token or API call.

Under the hood, a lot of what makes agent tool-calling interoperable across frameworks is MCP standardizing how the agent discovers and calls a given tool, and how much of the surrounding context window that tool call ends up consuming.

FAQ

What's the difference between an AI agent and a chatbot?
A chatbot answers within a single turn or conversation. An agent pursues a goal across multiple steps, deciding on its own what to do next and calling tools to get there.

What does "agentic AI" mean?
Systems designed to act with autonomy — planning, using tools, and taking actions toward a goal with limited step-by-step human approval.

How do AI agents use tools?
They call external functions (APIs, code execution, database queries) mid-task, and the tool's output feeds back into the agent's context for the next decision.

Are AI agents safe to use?
It depends on the task and the guardrails. Bounded, reversible tasks are lower-risk. Anything irreversible — payments, deletions, external communications — should have a human checkpoint.

What is the ReAct pattern in AI agents?
Reason + Act: the agent explicitly reasons about what to do, takes an action, observes the result, and loops until the goal is met or it runs out of budget.

Originally published at my-blog.org.

What Is a Context Window? Token Limits Explained for Developers

Sungwoo Lee — Mon, 17 Aug 2026 17:14:20 +0000

If you've ever had a long ChatGPT or Claude session where the model suddenly "forgets" something you told it ten minutes ago, you've hit the context window. It's not a bug and it's not the model getting confused — it's a hard architectural limit, and understanding it changes how you prompt.

The Core Concept

A context window is the total amount of text — measured in tokens, not words or characters — that a model can process in a single inference call. Anything outside that window is invisible to the model. It doesn't get "forgotten" the way a person forgets; it simply was never in front of the model for that call. Both your input (the prompt plus the entire conversation history) and the model's output count against the same limit.

Tokens and words aren't 1:1. A rough rule of thumb: one token is about three-quarters of an English word, so 1,000 tokens is roughly 750 words, or about a page and a half of prose.

Why AI "Forgets" in Long Conversations

When a conversation grows past the window size, older messages get dropped — typically from the beginning. The model isn't experiencing amnesia. It literally never receives those tokens on that particular call. If it seems to contradict something you said earlier in a long session, that's usually window pressure pushing your earlier instruction out of view, not the model changing its mind.

Context Window Sizes You'll Actually Work With

Sizes vary a lot by model, and they've grown fast — from GPT-3's 2,048 tokens in 2020 to windows over 1,000,000 tokens within a few years, according to model providers' own documentation.

The table below is a historical snapshot of the 2024 generation, not a current spec sheet. It's here to show the shape of the range, because that shape is what changes how you design. Do not build against these numbers — look up your model's current figure.

Model (2024 generation)	Context window as published then	Roughly
GPT-3.5 Turbo	16,385 tokens	~12,000 words
GPT-4o	128,000 tokens	~96,000 words
Claude 3.5 Sonnet / Claude 3 Opus	200,000 tokens	~150,000 words
Llama 3.1 405B	128,000 tokens	~96,000 words
Mistral Large 2	128,000 tokens	~96,000 words
Gemini 1.5 Pro / Flash	1,000,000 tokens	~750,000 words

Two orders of magnitude separate the low end from the high end, and that gap has kept widening since. Every provider publishes the current number in its model reference page — that page, not a blog post, is your source of truth. Anything written down here has a shelf life measured in months.

Where This Bites You in Practice

Paste a long PDF and ask questions → if the document plus your questions exceed the window, earlier parts silently get dropped from what the model actually sees.
Long coding sessions → the model "forgets" function signatures or conventions you established early on.
Multi-session research → you have to re-supply context in each new session; nothing persists across separate conversations by default.
Mid-conversation contradictions → almost always window pressure, not the model changing its stance.

Workarounds That Actually Help

Summarize and restart — paste a compact running summary at the top of a new conversation instead of the full history.
Chunk the task — break a large job into sequential, focused sub-tasks (prompt chaining) rather than one giant request.
Retrieval-Augmented Generation — instead of stuffing everything into context, an external index retrieves only the relevant passages per query. See our RAG explainer if you haven't used this pattern yet.
System prompt anchoring — put your most critical, non-negotiable instructions in the system prompt. Most tool implementations prioritize it and are slower to drop it under pressure.
Explicit recall prompts — when you sense drift, say something like the template below to force the model to re-register key facts before continuing:

Here's what we've established so far:
- [fact 1]
- [fact 2]
- [constraint or decision you don't want dropped]

Continue from here with: [next instruction]

That template alone fixes most "the model forgot my instruction" complaints in long sessions — it's cheaper than restarting the conversation and more reliable than hoping the model re-derives the constraint on its own.

FAQ

What is a context window in AI?
The total amount of text, measured in tokens, that a model can process — as input and output combined — in one inference call.

Why does ChatGPT forget things in long conversations?
It's not forgetting in the human sense. Once the conversation exceeds the context window, older messages fall outside what the model receives on that call.

How many tokens can a model handle?
It depends entirely on the model — anywhere from roughly 16K tokens on older models to over 1,000,000 tokens on the largest current windows. Check your provider's docs for the exact figure.

What happens when you exceed the context window?
Behavior varies by implementation, but typically the oldest messages are dropped or truncated to make room for new input, and the model simply never sees the dropped portion.

What's the difference between a token and a word?
A token is a chunk of text a model processes as one unit — often a word, but sometimes a word fragment or punctuation. In English, one token is roughly three-quarters of a word on average.

How do I work around a small context window?
Summarize periodically, chunk large tasks into smaller sequential prompts, use retrieval instead of stuffing everything into the prompt, and anchor critical instructions in the system prompt.

Originally published at my-blog.org.

What Is RAG (Retrieval-Augmented Generation)? Explained for Developers

Sungwoo Lee — Mon, 17 Aug 2026 17:13:43 +0000

Ask an LLM about something that happened after its training cutoff, or about a fact buried deep in your company's internal docs, and you'll get one of two things: a confident "I don't have information on that," or worse, a confident answer that's just wrong. That second failure mode — a fluent, plausible-sounding, incorrect answer — is what people mean by hallucination. It happens because the model is generating the next token based on statistical patterns learned during training, not looking anything up.

Retrieval-Augmented Generation (RAG) is the standard fix. In one sentence: RAG gives a model access to an external knowledge source at the moment it answers your question, so the response is grounded in real, current documents instead of only what the model memorized during training.

The Problem, Concretely

Two separate failure modes push teams toward RAG:

Staleness — the model was trained on a snapshot of data with a cutoff date. Anything after that date simply isn't in its weights.
Hallucination — even for facts that were in the training data, the model can misremember, blend it with unrelated information, or state something false with total confidence.

For casual use, that's an annoyance. For legal research, medical Q&A, or an internal support bot answering questions off your company's actual documentation, ungrounded answers are a liability, not a quirk.

How RAG Actually Works: Retrieve, Augment, Generate

RAG runs in three phases on every query:

Retrieve — a search component (usually vector similarity search, sometimes BM25 keyword search, often both) queries an external corpus — documents, wikis, a database, the web — and pulls back the chunks most relevant to the question.
Augment — those retrieved chunks get inserted into the prompt alongside the user's question, forming an extended context the model can actually read.
Generate — the model writes its answer with explicit reference to the retrieved text, instead of relying solely on its trained-in weights.

A minimal RAG loop looks roughly like this in pseudocode:

def answer_with_rag(question, corpus_index, llm):
    query_embedding = embed(question)
    top_chunks = corpus_index.search(query_embedding, k=5)
    context = "\n\n".join(chunk.text for chunk in top_chunks)

    prompt = f"""Answer the question using only the context below.
    If the answer isn't in the context, say so.

    Context:
    {context}

    Question: {question}"""

    return llm.generate(prompt)

That k=5 (how many chunks you retrieve) and the instruction to admit when the answer isn't in the context are both places where RAG systems commonly go wrong — retrieve too few chunks and you miss the answer, retrieve too many and you crowd out the context window with noise.

RAG vs. Fine-Tuning — Don't Confuse These

This is the mix-up that trips people up most:

	Fine-tuning	RAG
What it changes	The model's weights, via additional training	Nothing in the model — knowledge is supplied at inference time
Best for	Adjusting style, tone, format, task behavior	Currency and specificity of factual content
Update cycle	Retrain to update knowledge	Update the corpus, no retraining needed
Typical cost pattern	Upfront training cost	Ongoing retrieval/indexing infrastructure

They're not competing techniques — plenty of production systems fine-tune a model for behavior and use RAG for facts.

Where RAG Falls Short

RAG is not a hallucination-proof switch you flip on:

It's only as good as the retrieval step. Irrelevant chunks in, off-base answer out.
Long retrieved context eats into the context window, leaving less room for instructions or follow-up turns.
The model can still ignore the retrieved text or blend it incorrectly with its own priors — RAG reduces hallucination, it doesn't eliminate it.
If the answer genuinely isn't anywhere in the corpus, the model still has nothing grounded to point to.
Someone has to build and maintain the index. That's real engineering work, not a one-time setup.

Where You've Already Used RAG

Perplexity AI — web search plus LLM synthesis over the results.
ChatGPT with web search enabled.
Microsoft Copilot pulling from your organization's documents.
Most enterprise chatbots built on internal knowledge bases.

RAG was formalized in the 2020 NeurIPS paper "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" (Lewis et al., Meta AI Research) — worth a read if you want the original framing straight from the source.

FAQ

What does RAG stand for in AI?
Retrieval-Augmented Generation — retrieving relevant documents, then using them to generate a grounded answer.

Does RAG eliminate AI hallucinations?
No. It reduces the odds by grounding answers in real text, but the model can still misread or ignore what it retrieved.

What's the difference between RAG and fine-tuning?
Fine-tuning changes the model's weights through extra training. RAG leaves the model untouched and supplies knowledge through the prompt at answer time.

How does RAG keep AI answers current?
Update the corpus it retrieves from — no retraining required, unlike fine-tuning.

What is vector search and why does RAG use it?
It's a similarity search over embeddings (numeric representations of meaning) rather than exact keyword matches, so retrieval can find semantically relevant chunks even when the wording doesn't match. Most RAG systems use it, sometimes alongside keyword search like BM25.

If you're building anything that needs an AI system to answer questions about your own ai agent workflows or a private knowledge base, RAG is the piece that connects "what the model knows" to "what's actually true right now."

Originally published at my-blog.org.

What Is MCP (Model Context Protocol)? A Developer's Explainer

Sungwoo Lee — Mon, 17 Aug 2026 17:13:06 +0000

If you've built more than one AI-powered feature, you already know the pain MCP is trying to fix: every model, every tool, every data source needs its own glue code. Connect an LLM to a calendar API, and you write a connector. Swap the model provider, and you rewrite it. Add a second data source, and you write another one. Multiply that by every tool your app touches and you get a combinatorial mess of one-off integrations.

Model Context Protocol (MCP) is Anthropic's answer to that mess — an open standard, released in November 2024, that defines a consistent way for AI models to talk to external tools, data sources, and services. It's open-source, hosted at github.com/modelcontextprotocol, and it's not locked to Anthropic's own models. Any developer or platform can implement it, on either side.

The mental model people reach for is USB-C: one standard port, many compatible devices. Write an MCP server once, and any MCP-compatible client can use it without custom glue code.

The Four Pieces: Host, Client, Server, Tools

MCP splits responsibility into four roles:

Host — the application the user actually interacts with. Claude Desktop is a host. Cursor is a host. The host manages the session and coordinates everything else.
Client — a component inside the host that speaks MCP. A single host can run multiple clients, each wired to a different server.
Server — a lightweight program exposing specific tools or data. A "Google Calendar MCP server" gives the client the ability to read and create events. A "filesystem MCP server" lets it read and write local files.
Tools — the actual capabilities a server exposes. A calendar server might expose list_events, create_event, and delete_event as callable tools.

The request flow looks like this:

User message → Host
Host's Client asks the model what to do
Model decides to call a Tool
Client sends the request to the Server
Server executes it and returns a result
Model incorporates the result into its response

Communication happens over JSON-RPC, either via stdio (for local servers) or HTTP/SSE (for remote ones). Configuring a server in a host typically looks something like this:

{
  "mcpServers": {
    "filesystem": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-filesystem", "/path/to/project"]
    }
  }
}

The model sees the tools that server exposes as part of its available toolset and decides, mid-conversation, whether and when to call them.

MCP vs. the Old Way

Before MCP, integration work looked like this:

	Before MCP	With MCP
Per-tool integration	Custom plugin/API code for each AI + tool pair	Write one MCP server, any MCP host can use it
Switching AI providers	Rewrite integrations	Connections keep working
Protocol	Proprietary (e.g. OpenAI's deprecated plugin schema)	Standardized JSON-RPC
Small teams	Can't afford to support every AI platform	Ship one server, reach every compatible client

Custom GPTs are a related but different idea — they're contained inside the ChatGPT ecosystem and use a proprietary action schema. MCP is explicitly model-agnostic and cross-provider by design.

Why This Matters Beyond the Demo

For non-developers, MCP mostly works invisibly, but it's the plumbing behind an AI assistant that can read your actual email thread and draft a reply, check your real calendar and suggest a time that works, or query a live database instead of you copy-pasting rows into a chat window.

For developers, the more interesting case is coding tools. Editors like Cursor use MCP servers to give the model access to documentation, package registries, and test runners — without you manually pasting context into every prompt. That's part of a broader shift away from manual context-stuffing in prompts and toward models that pull what they need on demand. It also overlaps with how AI agents use tools mid-task — MCP is effectively the standardized tool-calling layer a lot of agent frameworks are converging on.

If you're deciding whether to build a plain API integration or an MCP server for a new tool, a rough rule of thumb: if only one AI product will ever call it, a direct API call is simpler. If you want any MCP-compatible client — present or future — to use it without you writing new glue code, build the MCP server.

FAQ

Is MCP made by OpenAI?
No. MCP was created by Anthropic and released as an open standard in November 2024. It's model-agnostic and not tied to any single AI company.

Do I need to install anything to use MCP?
As an end user, no — if your app (like Claude Desktop) supports MCP, you connect servers through its settings. Developers need to run or build an MCP server.

What's the difference between MCP and a regular API?
A traditional API is a direct integration written between two specific systems. MCP is a protocol that standardizes how AI models discover and call tools, cutting down on custom integration code per pairing.

Does MCP replace Custom GPTs?
No, they solve different problems. Custom GPTs are a no-code way to configure a ChatGPT assistant inside OpenAI's ecosystem. MCP is an open integration standard meant to work across AI providers.

Is MCP secure?
MCP servers run locally or on infrastructure you control, and the protocol specifies how permissions and tool schemas work — but actual security depends on the implementation. Running a server from an untrusted source carries the same risk as running any other third-party software.

Originally published at my-blog.org.