Sanya

Posted on Aug 17

From Attention to Agency: The Progressive Evolution of AI-Assisted Programming

#ai #llm #machinelearning #programming

From Attention to Agency: The Progressive Evolution of AI-Assisted Programming

A Developer's Field Guide to the AI Stack in 2026

Introduction: Why This Stack Exists in This Order

If you've been following the AI tooling space over the past few years, you've probably noticed something strange: the concepts arrive in waves, but they don't arrive randomly. Transformer architectures gave us the foundation. Pre-training gave us raw capability. Post-training gave us alignment. ChatGPT gave us a product. And then, almost inevitably, we got agents.

But here's what most articles miss — each layer of this stack didn't just appear. It emerged because the layer below it hit a wall. And understanding that chain of causation is what separates developers who use AI tools from developers who understand them.

This article traces that chain: from the math that started it all, through the engineering choices that made it practical, to the agentic architectures that are reshaping what "programming" even means. We'll cover every concept in your toolkit — Transformer, Pre-training, Post-training, ChatGPT, React, Agentic AI, AI Agents, Harness, Tools, Hooks, Permission, Skills, Compact, Memory, Sub-agents, and MCP — and show how they fit into a coherent progressive evolution.

1. The Foundation: Transformer & Attention — "Attention Is All You Need"

In 2017, a team at Google published a paper with a provocatively simple title: "Attention Is All You Need" [1]. The paper introduced the Transformer architecture, which replaced recurrent neural networks (RNNs) with a mechanism called Self-Attention.

Why Attention Changed Everything

RNNs processed sequences step by step — reading token 1, then token 2, then token 3. This made them painfully slow for long sequences and caused them to "forget" early tokens by the time they reached the end.

Self-Attention solves this differently. Instead of processing sequentially, it lets every token in a sequence "look at" every other token simultaneously and compute a weighted relevance score. This is the attention mechanism — the model learns which parts of the input matter most relative to each other, regardless of distance.

The key insight: parallelization. Transformers can process entire sequences at once, making them dramatically faster to train and capable of capturing long-range dependencies that RNNs struggled with.

The Architecture Basics

A Transformer consists of:

Encoder: Reads the input and builds a representation
Decoder: Generates output one token at a time, attending to both the input and previously generated tokens
Multi-Head Attention: Runs multiple attention mechanisms in parallel, allowing the model to capture different types of relationships simultaneously
Feed-Forward Layers: Process the attended representations
Positional Encodings: Inject sequence order information since attention itself has no notion of position

Modern LLMs like GPT-4, Claude, and Llama are mostly decoder-only Transformers (following the GPT architecture from OpenAI's 2018 paper). The encoder is often omitted because generation tasks (writing code, answering questions) are inherently sequential.

The Scaling Revolution

What made Transformers truly revolutionary wasn't just the architecture — it was what happened when you scaled them. The 2020 scaling laws paper from OpenAI [2] showed that model performance follows a smooth power law with respect to compute, data, and parameters. More of everything meant better results, predictably.

This is the engine that drove everything that followed: the race to larger models, the emergence of emergent capabilities, and ultimately, the ability to write coherent code.

Key takeaway: The Transformer is the engine. Attention is the fuel. Without this foundation, none of the rest of this stack exists.

2. Pre-Training: Building the World Model

Once you have a Transformer, you need to teach it something. That's what pre-training does.

Language Modeling as the Task

The core pre-training objective is deceptively simple: predict the next token. Feed the model a sequence of text, mask the last token, and ask it to predict what comes next. Repeat on billions of tokens.

This sounds trivial, but it's extraordinarily powerful. To predict the next token well, the model must implicitly learn:

Grammar and syntax
World knowledge and facts
Reasoning patterns
Coding concepts and structure
Cultural and contextual nuances

The Data Stack

Pre-training data is typically a massive corpus mixing:

Web text (Common Crawl, The Pile, etc.)
Books and papers (BooksCorpus, ArXiv, etc.)
Code (GitHub repositories — this is why LLMs can code)
Conversational data (Reddit, forums, etc.)

The quality and diversity of this data directly determines what the model can do. Code-specific pre-training (training on large amounts of programming code) is a major reason why models like GPT-4, Claude, and Codex developed strong coding abilities.

What's Missing After Pre-Training

Here's the critical point: a pre-trained model is not yet useful. It can predict text, but it doesn't know how to behave. It might complete a sentence rudely, helpfully, incorrectly, or dangerously — with equal probability. The model has "knowledge" but no "judgment."

This is why pre-training alone isn't enough, and it's exactly why post-training exists.

Key takeaway: Pre-training gives the model knowledge and capability. It's a powerful but undirected force — like having a vast encyclopedia memorized but no common sense about when to share what.

3. Post-Training: Turning Knowledge into Behavior

Post-training is the umbrella term for the techniques that take a raw pre-trained model and make it actually useful and safe. This is where the magic really happens.

Stage 1: Supervised Fine-Tuning (SFT)

The first step is often Supervised Fine-Tuning — training the model on high-quality examples of desired behavior. Human annotators write or curate prompt-response pairs that demonstrate good answers.

For code tasks, this might mean:

A prompt asking to write a React component → a well-structured, commented, working component
A prompt about debugging → a methodical explanation with the right answer

SFT teaches the model what good responses look like in specific domains. It's relatively simple and efficient, but it has a ceiling: the model can only be as good as the examples it sees.

Stage 2: RLHF and Beyond

To push past that ceiling, most modern models use Reinforcement Learning from Human Feedback (RLHF). The process:

Generate multiple responses to the same prompt
Have human raters rank them from best to worst
Train a reward model that predicts human preferences
Use the reward model to further fine-tune the base model

This is what makes models like ChatGPT feel natural and helpful — they learned not just what to say, but how to say it in ways that humans prefer.

More recent approaches like DPO (Direct Preference Optimization) simplify this by directly optimizing against preference data without needing a separate reward model.

Post-Training for Code Specifically

Code models undergo specialized post-training:

Code-specific SFT on high-quality open-source repositories
Instruction tuning focused on coding tasks (debug, explain, refactor, test)
Tool-use training so models learn to call functions and use external resources

The result is a model that doesn't just complete code — it engages with code tasks the way a thoughtful developer would.

Key takeaway: Pre-training is education. Post-training is etiquette training. The model knows things; post-training teaches it when and how to share that knowledge appropriately.

4. ChatGPT: The Product That Changed Everything

Everything before this point was invisible infrastructure. Then OpenAI shipped ChatGPT in November 2022, and AI became visible.

Why ChatGPT Was a Tipping Point

ChatGPT didn't introduce new technology — it packaged existing technology into a product that was:

Instantly accessible (no code, no API, just conversation)
Broadly capable (coding, writing, reasoning, analysis)
Free to try (reducing the barrier to zero)

The developer community's reaction was immediate. Within weeks, developers were integrating GPT into their tools. Within months, every software company had an "AI strategy."

The Coding Breakthrough

For developers specifically, ChatGPT (and its API) opened the door to:

Code generation from natural language descriptions
Debugging assistance by pasting error messages
Documentation generation and explanation
Code review and optimization suggestions

But ChatGPT as a chat interface had real limits for coding work: no file system access, no terminal, no persistent context across sessions. It was a brilliant assistant for thinking through problems, but not yet a capable agent that could act in your codebase.

Key takeaway: ChatGPT proved that LLMs were genuinely useful. But it was a starting point — the foundation on which agentic tooling would later be built.

5. React: AI-Assisted UI Development

The term React in this context goes beyond the JavaScript library — it describes a pattern of AI-assisted development where the AI reactively assists the developer rather than driving the interaction.

The React Pattern in AI Coding Tools

Modern AI coding assistants (Claude Code, GitHub Copilot, Cursor) embody the React pattern:

Developer writes code → AI observes and reacts
AI offers suggestions → Developer reviews and accepts or rejects
Developer asks questions → AI responds with context
Errors occur → AI proposes fixes

This is a human-in-the-loop model where AI augments human intent rather than replacing it. The human remains the conductor; the AI is a powerful instrument.

Claude Code and the React Implementation

Anthropic's Claude Code exemplifies the React pattern. It:

Runs as a CLI tool that operates within your project directory
Reads your files, understands your codebase structure
Responds to natural language instructions from the developer
Executes commands, edits files, and runs tests — but always with the developer able to review and intervene
Provides a compact output style where it summarizes changes rather than regenerating entire files

The React pattern is fundamental because it acknowledges a crucial truth: for complex, high-stakes development work, human judgment is irreplaceable. AI accelerates and assists, but doesn't autonomously decide.

Key takeaway: The React pattern keeps humans in control while letting AI handle the mechanical work. It's the practical intersection of AI capability and human oversight.

6. Agentic AI & AI Agents: When AI Takes Initiative

This is where the stack takes its most significant leap. Agentic AI refers to AI systems that can autonomously plan, reason, and act toward goals — not just respond to prompts.

What Makes an AI "Agentic"?

An AI agent is characterized by:

Autonomy: It can take actions without continuous human input
Goal-oriented behavior: It plans a sequence of steps to achieve an objective
Tool use: It can call external tools, APIs, or functions to interact with the world
Memory: It maintains state across interactions and can learn from feedback
Self-correction: It can evaluate its own outputs and adjust its approach

The Agent Loop

The canonical agent loop looks like this:

Observe → Think → Plan → Act → Evaluate → Repeat

This is often implemented with a ReAct (Reasoning + Acting) pattern, where the model interleaves natural language reasoning with tool-calling actions.

Real Examples in 2026

The agentic AI space has exploded:

Claude Code (Anthropic): CLI agent for software development
DeepSeek Harness (DeepSeek, open-sourced August 2026): Agent harness with Standard, Code (PTC), Minimal, and Creator modes [3]
OpenWorker (Andrew Ng): AI coworker that completes tasks end-to-end
Devin (Cognition): Autonomous software engineer
Codex CLI (OpenAI): Command-line coding agent

The Permission Problem

As agents become more capable, the question of permission becomes critical. What should an agent be allowed to do autonomously? The spectrum ranges from:

Read-only: Can analyze and suggest, but not modify
Tool-use limited: Can call specific approved tools
File-system access: Can read and write files in specific directories
Full autonomy: Can execute commands, push to git, deploy code

Modern agent frameworks like Claude Code implement granular permission systems where developers can scope what the agent can access and do. This is both a security concern and a practical necessity — you want your agent helpful, but not dangerously so.

Key takeaway: Agentic AI shifts the paradigm from "AI answers questions" to "AI solves problems." The challenge is building agents that are helpful, safe, and reliable — which requires solving permission, memory, and tool integration problems.

7. Harness: The Agent Framework

An agent harness is the software framework that orchestrates an AI agent's behavior — its tools, memory, reasoning patterns, and permissions. Think of it as the operating system for an AI agent.

What a Harness Provides

A well-designed harness handles:

Tool registration and execution: Defining what tools the agent can use and how to call them
Context management: Feeding the right information to the agent at the right time
Loop control: Managing the agent's reasoning-act-evaluate cycle
Permission enforcement: Enforcing what the agent is and isn't allowed to do
State persistence: Maintaining memory and context across sessions

DeepSeek Harness: A Case Study

DeepSeek's Harness (open-sourced under MIT in August 2026) provides four operating modes [3]:

Standard Mode: Balanced for general tasks
Code (PTC) Mode: "Pure Training-free Collaboration" — designed for coding without requiring extensive agent training
Minimal Mode: Lightweight, for constrained environments
Creator Mode: Optimized for generative and creative tasks

The plugin architecture in DeepSeek Harness is noteworthy — every capability is a plugin, making the system extensible without modifying core logic.

Claude Code's Harness Philosophy

Claude Code takes a different approach, optimizing for developer ergonomics:

Inline tool definitions via CLAUDE.md and AGENTS.md
Skills system for extensible capabilities
Sub-agent orchestration for parallel task execution
Hooks for lifecycle events (pre-command, post-command, etc.)

Key takeaway: The harness is the architecture that turns a language model into an agent. Good harnesses are opinionated about what's allowed and provide clean abstractions for extensibility.

8. Tools: The Agent's Hands

Tools are the interface between an AI agent and the external world. Without tools, an agent is just a very sophisticated text generator.

Types of Tools

File System Tools

Read, write, edit, delete files
Execute shell commands
Search within files

Search & Retrieval Tools

Web search for up-to-date information
Code search within a repository
Vector search for semantic retrieval

API Tools

HTTP requests to external services
Database queries
Cloud service integrations

Development Tools

Git operations (commit, push, branch)
Build and test runners
Linters and formatters

Tool Definition: The Schema Problem

Every tool needs a schema — a machine-readable description of what the tool does, what inputs it expects, and what outputs it produces. This is harder than it sounds:

Too little detail → agent uses tool incorrectly
Too much detail → agent gets confused by irrelevant parameters
Ambiguous naming → agent picks the wrong tool

This is where MCP (Model Context Protocol) comes in — it's a standardized way to define and discover tools.

Compact Tool Output

A key challenge is compactness — tool outputs (especially file system reads) can be enormous and quickly fill up the context window. Effective agents use strategies like:

Truncation: Only reading relevant portions of files
Summarization: Compressing tool outputs before feeding them back
Selective reading: Only accessing files that are actually needed
Delta-based approaches: Operating only on changed files (as in tools like Scrut, a Python linter that only reviews changed code)

Key takeaway: Tools are how agents act. Designing good tool schemas — concise, accurate, and appropriately scoped — is one of the most practical skills in agent engineering.

9. Hooks: The Agent's Reflexes

Hooks are lifecycle callbacks that run at specific points in an agent's execution. They enable developers to intercept, validate, or modify agent behavior without changing the agent's core logic.

Common Hook Points

pre_think    → Before the agent starts reasoning
post_think   → After reasoning, before acting
pre_tool     → Before a tool is called
post_tool    → After a tool returns
pre_response → Before the agent's response is delivered
post_response→ After the response is delivered
on_error     → When an error occurs

Practical Uses

Compliance & Safety

Check if a proposed command is destructive (rm -rf, database drops)
Validate that file changes don't violate project policies
Block operations that touch sensitive directories

Quality Gates

Run linters before accepting code changes
Enforce test coverage thresholds
Verify formatting standards

Observability

Log all agent actions for audit trails
Track which tools are used most
Measure agent decision quality over time

Custom Routing

Route certain requests to specialized sub-agents
Switch models based on task complexity
Apply different permission sets per task type

Hooks vs. Hard Constraints

Hooks are powerful because they're composable — you can add, remove, and chain them without breaking the agent. But they work best for soft constraints. For hard safety guarantees, you need permission systems (see below) that can't be bypassed by a hook.

Key takeaway: Hooks are the nervous system of an agent framework. They let you inject custom logic at precisely the right moments — making agents behave intelligently without monolithic rewrites.

10. Permission: The Security Layer

If hooks are the agent's reflexes, permissions are its immune system. Permissions define what an agent cannot do, regardless of what it's asked to do.

Permission Models in Practice

Path-Based Permissions

Whitelist specific directories the agent can access
Block access to .env files, credentials, private keys
Scope the agent to a specific project or repository

Action-Based Permissions

Allow file reads but block file deletes
Allow git commits but block force pushes
Allow API reads but block destructive writes

Capability-Based Permissions

Enable/disable specific tools entirely
Require human approval for high-risk operations
Time-box agent sessions

The Principle of Least Privilege

The best practice is to grant only the permissions needed for the specific task:

A code-review agent → read-only access, no file modifications
A refactoring agent → file modifications within a specific directory
A deployment agent → access to CI/CD systems but not source code

This is the same principle that applies to human access control, and for good reason: agents, like humans, make mistakes. Permissions are the last line of defense.

Claude Code's Permission Model

Claude Code implements a thoughtful permission model:

Files are read and written within the project directory by default
Shell commands run in the project context
Dangerous operations (network calls, system modifications) can be explicitly scoped
The --verbose flag exposes what the agent is doing, enabling human oversight

Key takeaway: Permissions are not about distrust — they're about building reliable systems. An agent with appropriate permissions is an agent that can be trusted to be helpful.

11. Skills: Extensible Agent Capabilities

Skills are packaged capabilities that extend what an agent can do. Think of them as plugins or skill packs that add domain-specific expertise.

What Skills Contain

A skill typically includes:

Instructions (SKILL.md): How to use the skill, when to use it, and what it does
Tool definitions: New tools specific to the skill
Prompts and templates: Pre-written prompts for common tasks
Examples: Few-shot examples demonstrating desired behavior
Constraints: Rules specific to this skill's domain

Skill Examples

Code Analysis Skills

Security vulnerability detection
Performance profiling
Test coverage analysis

Domain Skills

Database schema design
API design review
Accessibility auditing

Integration Skills

Cloud provider specifics (AWS, GCP, Azure)
CI/CD pipeline configuration
Container orchestration (Kubernetes)

The Skill Discovery Problem

As the number of skills grows, skill discovery becomes important. The agent needs to know which skill applies to a given task. This is typically solved through:

Skill metadata (description, tags, triggers)
Semantic matching against task descriptions
Explicit invocation by the developer

Clawhub.ai and similar registries are emerging as the ecosystem for sharing and discovering agent skills.

Key takeaway: Skills are how you specialize a general-purpose agent into an expert in your specific domain. A well-designed skill is self-contained, well-documented, and composable with other skills.

12. Compact: Efficiency in Context

Compact refers to the practice of keeping the agent's context lean and efficient. It's not a feature — it's a discipline.

The Context Window Problem

Every LLM has a finite context window — the total amount of text it can "see" at once (measured in tokens). Modern models offer 128K to 200K+ tokens, which sounds like a lot but fills up quickly:

A medium-sized codebase → tens of thousands of tokens
A long conversation history → tens of thousands more
Tool outputs and error messages → thousands more

Research from Liu et al. ("Lost in the Middle") shows that models perform 20-30% worse when relevant information is buried in the middle of a long context [4]. The model literally "forgets" things in the middle.

Compactness Strategies

Context Engineering (as it was named in 2026) involves:

Selective file reading: Only read files that are relevant to the current task
Delta-based operations: Work on what changed, not what exists
Summarization: Compress large files or outputs into concise summaries
Chunking: Break large files into relevant sections
Memory offloading: Store information outside the context window in a retrieval system

The Context Window Is a Cache

A useful mental model: the context window is a cache, not a memory. It's fast to access but limited in size. Long-term information should be stored in a proper memory system, not kept in context [5].

Key takeaway: Compactness is not about being stingy — it's about being effective. The best agents are not the ones with the most context; they're the ones with the right context.

13. Memory: The Agent's Long-Term Knowledge

This is where most AI agents fail. Memory is the system that allows an agent to persist information across sessions, learn from past interactions, and maintain continuity.

Why "Memory" Is a Misnomer

Most AI agents don't actually have memory — they have context windows. Each session starts fresh. The "memory" is just whatever you put in the prompt this time.

True memory requires:

Persistence: Information survives session boundaries
Retrieval: Information can be found when relevant
Relevance filtering: Not everything needs to be remembered
Learning: Patterns should inform future behavior

Memory Architecture Patterns

Explicit Memory Files

Store key decisions, project conventions, and user preferences in structured files
Example: CLAUDE.md, AGENTS.md in Claude Code
Pros: Simple, transparent, version-controllable
Cons: Requires manual organization

Vector Memory

Store conversation chunks and code snippets as embeddings
Retrieve semantically similar content when relevant
Pros: Automatic, captures nuance
Cons: Can return irrelevant results, expensive to index

Semantic Memory Systems

Extract structured facts from interactions
Maintain a "knowledge graph" of what the agent knows about the project
Pros: Precise retrieval, reasoning-capable
Cons: Complex to build and maintain

The Learning vs. Storing Problem

The most insightful framing comes from the "Learn, Don't Store" methodology: most agents today remember everything and learn nothing. A better approach is to actively extract patterns and principles rather than storing raw transcripts.

Example:

❌ Bad memory: "User asked me to refactor component X on March 15"
✅ Good memory: "Project convention: always use functional components with hooks"

Key takeaway: Memory is the hardest unsolved problem in AI agents. The difference between a tool that remembers and one that truly learns is whether it can extract actionable patterns from experience.

14. Sub-Agents: Parallel Problem Solvers

When a task is too complex for a single agent, sub-agents (also called delegates or child agents) divide the work.

Sub-Agent Patterns

Fanout Pattern

A parent agent breaks a task into independent subtasks
Multiple sub-agents work on them in parallel
Parent synthesizes results
Best for: parallel research, multi-file refactoring, simultaneous analysis

Pipeline Pattern

Tasks are strictly sequential
Each sub-agent's output feeds into the next
Best for: multi-step transformations (code → test → review → deploy)

Hierarchical Pattern

A supervisor agent delegates to specialist agents
Specialist agents may further delegate
Best for: complex projects with distinct domains (frontend, backend, DevOps)

Sub-Agent Architecture: Context Explosion

The biggest challenge with sub-agents is context explosion. Each sub-agent needs relevant context to do its job, but providing too much context:

Slows down individual agents
Risks information leakage between agents
Increases cost exponentially

Effective sub-agent design uses context isolation — each sub-agent gets only the information it needs, not the full project context.

Delegation Runtime

A delegation runtime handles the mechanics of sub-agent management: spawning agents, routing tasks, collecting results, handling errors, and managing concurrency. Claude Code's architecture, for example, allows sub-agents to be spawned with isolated contexts or forked contexts depending on whether they need the parent's conversation history.

Key takeaway: Sub-agents are about leverage — one human can orchestrate many specialized agents. The skill is knowing when to split work and how to keep each sub-agent focused.

15. MCP: The Model Context Protocol

MCP (Model Context Protocol) is the emerging standard for connecting AI models to external tools and data sources. Developed by Anthropic and now adopted broadly, MCP provides a standardized interface for tool discovery, invocation, and data exchange.

Why MCP Was Needed

Before MCP, every agent framework defined tools differently:

Different schemas, different formats, different discovery mechanisms
Tools couldn't be shared between frameworks
Developers had to reimplement tool integrations for each platform

MCP solves this by providing:

A standard protocol for tool definition and invocation
A discovery mechanism so agents can find available tools
A schema format for describing tool inputs and outputs
A transport layer for communication (typically stdio or HTTP)

MCP Architecture

┌─────────────┐     MCP      ┌──────────────┐
│  AI Agent   │◄────────────►│ MCP Host     │
└─────────────┘              │ (your app)   │
                             └──────┬───────┘
                                    │ MCP
                              ┌─────▼─────┐
                              │ MCP Server│
                              │ (plugin)  │
                              └─────┬─────┘
                                    │ native
                              ┌─────▼─────┐
                              │File/DB/API│
                              └───────────┘

The MCP Host (your application) runs MCP Servers (plugins) that connect to actual resources (files, databases, APIs). The AI agent talks to the host via MCP, without needing to know how the underlying systems work.

MCP in Practice

Common MCP servers:

Filesystem: Read/write/search files
Git: Git operations and repository analysis
Database: SQL queries and schema inspection
Search: Web search and documentation lookup
Slack/Discord: Messaging and notifications

The power of MCP is that it's composable — you can run multiple MCP servers simultaneously, giving your agent access to a diverse set of capabilities through a unified interface.

Key takeaway: MCP is the USB-C of AI tooling. It standardizes how agents connect to the outside world, making the entire ecosystem more interoperable and extensible.

16. The Full Picture: How It All Connects

Here's the complete evolutionary chain:

Transformer + Attention
        ↓
  Pre-Training (raw knowledge)
        ↓
  Post-Training (alignment, behavior)
        ↓
  ChatGPT (proof of concept, consumer product)
        ↓
  React Pattern (human-in-the-loop assistance)
        ↓
  Agentic AI (autonomous goal pursuit)
        ↓
  ┌─────────────────────────────────┐
  │  Harness (orchestration layer)  │
  │  ├── Tools (capabilities)       │
  │  ├── Hooks (lifecycle events)   │
  │  ├── Permission (security)      │
  │  └── Skills (extensibility)     │
  ├─────────────────────────────────┤
  │  Compact (context efficiency)   │
  │  Memory (persistence & learning)│
  │  Sub-Agents (parallelization)   │
  │  MCP (standardized integration) │
  └─────────────────────────────────┘

Each layer exists because the layer above it hit a limitation. You can't have useful agents without good harnesses. You can't have effective harnesses without well-defined tools. You can't have reliable tools without compact, well-managed context. And you can't have any of this without the Transformer foundation.

17. Where This Is Going

The trajectory is clear:

Foundation models will keep improving — better reasoning, longer contexts, lower costs
Agents will become more reliable — better error recovery, stronger safety guarantees, clearer boundaries
Memory will get smarter — from storing everything to actively learning patterns
MCP will become ubiquitous — the standard for tool integration across frameworks
The human role will shift — from writing code to orchestrating agents and validating outputs

The developers who thrive in this new paradigm won't be the ones who resist AI tooling. They'll be the ones who understand the stack deeply enough to build on it, extend it, and debug it when it breaks.

References

[1] Vaswani, A., et al. "Attention Is All You Need." NeurIPS 2017. https://arxiv.org/abs/1706.03762

[2] Kaplan, J., et al. "Scaling Laws for Neural Language Models." arXiv 2020. https://arxiv.org/abs/2001.08361

[3] Rohit Raj. "DeepSeek Harness vs Claude Code vs Codex CLI: The v0.1 Developer Preview, Honestly — 2026." DEV Community. https://dev.to/rohit_raj_8c7902b7d37cf21/deepseek-harness-vs-claude-code-vs-codex-cli-the-v01-developer-preview-honestly-2026-433e

[4] Liu, N. F., et al. "Lost in the Middle: How Language Models Use Long Contexts." arXiv 2024. https://arxiv.org/abs/2407.01073

[5] Loop & Retry. "The context window is a cache, not a memory." DEV Community. https://dev.to/loopandretry/the-context-window-is-a-cache-not-a-memory-29f8

This article is written for developers who want to understand the full AI-assisted programming stack, not just the parts they already use. Bookmark it, share it, and come back when you need to understand how the next piece fits in.