AI Engineering for Backend Developers: From LLM Fundamentals to Production AI Agents

LEVEL 1 — Fundamentals

1. LLM — Large Language Model

• An LLM is a model trained on huge amounts of text/code to predict and generate the next token.
• Examples: Claude, GPT, Gemini, Llama.
• As a backend developer, you interact with an LLM through an API rather than building the model yourself.
• Real life: Build an API endpoint /generate-summary that sends customer/order data to an LLM and returns a summary.

2. Model

• A model is the actual trained neural network that performs a task.
• Different models have different capabilities, costs, latency, context sizes, and reasoning abilities.
• You choose a model based on your application's requirements.
• Real life: Use a cheaper/faster model for classification and a stronger model for complex code generation.

3. Token

• A token is a chunk of text processed by an LLM; it may be a word, part of a word, punctuation, etc.
• Input and output are generally measured in tokens.
• Tokens directly affect cost, latency, and context-window usage.
• Real life: Before sending a huge database result to an LLM, truncate/summarize it so you don't waste thousands of tokens.

4. Context

• Context is the information you provide to the model for producing its current response.
• It can contain instructions, user messages, previous conversation, retrieved documents, tool results, etc.
• The model doesn't automatically know your application's database or state.
• Real life: Send customer_id, recent orders, and your business rules as context when asking the LLM to handle a support request.

5. Prompt

• A prompt is the instruction/input given to the model.
• Good prompts specify the task, constraints, available information, and expected output.
• Backend applications often construct prompts dynamically rather than hardcoding one user message.
• Real life: Your service creates a prompt like: "Analyze this order and return JSON containing risk, reason, and recommended action."

6. Context Window

• The context window is the maximum amount of information a model can process in one request.
• It includes things such as system instructions, conversation history, retrieved documents, tool results, and the current request.
• Large context ≠ unlimited context; you still need to manage what enters the request.
• Real life: For a chatbot with thousands of messages, keep recent messages + summarized history instead of sending the entire conversation every time.

---

LEVEL 2 — AI Application

7. Tool

• A tool gives an LLM the ability to interact with something outside itself.
• Examples: database query, REST API, calculator, search, email service, filesystem.
• The LLM decides when/why to use the tool; your backend actually executes it.
• Real life: Give your support agent a get_order(order_id) tool so it can retrieve real order information.

8. Function Calling

• Function calling is the structured mechanism through which an LLM requests your application to execute a function.
• The model produces structured arguments such as {"order_id": 12345} rather than arbitrary text.
• Your backend validates the arguments, executes the function, and sends the result back to the model.
• Real life: LLM → getOrder(12345) → MySQL → order data → LLM → response.

9. Agent

• An agent is an LLM-powered system that can reason about a goal and take actions using tools.
• Instead of simply answering, it can decide: search → inspect → call API → analyze → act.
• Your backend provides the tools, permissions, state, and execution environment.
• Real life: Build an incident agent that reads logs, checks service health, queries metrics, identifies the likely failure, and proposes remediation.

10. Agent Loop

• The agent loop is the repeated cycle: observe → reason → choose tool → execute → observe result → repeat.
• The loop continues until the task is completed or a safety/iteration limit is reached.
• This is essentially the runtime behind many coding and autonomous agents.
• Real life: User request → LLM → tool call → result → LLM → another tool → result → final answer.

11. Memory

• Memory allows an AI application to retain information beyond the current model call.
• Short-term memory can be conversation history; long-term memory can live in Redis, PostgreSQL, or a vector database.
• The LLM itself isn't necessarily your application's persistent memory.
• Real life: Store user preferences such as "prefers concise responses" and retrieve them when generating future responses.

12. RAG — Retrieval-Augmented Generation

• RAG means retrieve relevant information first, then give it to the LLM.
• Your application searches a knowledge source and puts the relevant results into the model's context.
• This lets an LLM answer using your private/current data without retraining the model.
• Real life: Question → search company documentation → retrieve relevant sections → LLM → answer.

---

LEVEL 3 — Claude / Coding Agents

13. Claude Code

• Claude Code is Anthropic's coding agent that can inspect a repository, modify files, run commands, and use development tools.
• It is different from simply asking Claude to generate a code snippet.
• It operates more like an engineer working inside your development environment.
• Real life: Give it a Go microservice repository and ask it to implement an API, run tests, investigate failures, and modify the relevant files.

14. CLAUDE.md

• CLAUDE.md is a project instruction/context file used by Claude Code.
• You can describe architecture, coding conventions, commands, testing requirements, and important project rules.
• It acts like persistent instructions for the coding agent working in that repository.
• Real life: Tell Claude: "Use Go 1.24, run make test, don't modify generated files, use repository error-handling conventions."

15. Skills

• A skill is a reusable capability/instruction package that teaches an agent how to perform a particular type of task.
• Instead of repeatedly giving the same detailed instructions, you package the workflow once.
• Skills can encode domain knowledge, procedures, templates, and tool usage.
• Real life: Create a database-migration skill that tells your coding agent how to create migrations, update models, test rollback, and validate compatibility.
• Think of it as "a reusable playbook for an agent."

16. Hooks

• Hooks are automated actions triggered at specific points in an agent/tool workflow.
• They are useful when you want deterministic behavior rather than asking the LLM to remember a rule.
• For example, run a formatter after code modification or security checks before committing.
• Real life: Agent edits Go file → hook runs gofmt → hook runs tests/linter → result returned to agent.

17. Subagents

• A subagent is a separate agent invoked to handle a smaller specialized task.
• The main agent can delegate work instead of doing everything itself.
• This is useful for parallelism and separation of responsibilities.
• Real life: Main agent asks one subagent to inspect database changes, another to review API design, and another to write tests.

18. Permissions

• Permissions define what an agent is allowed to access or execute.
• This is critical because an agent may have access to your filesystem, terminal, cloud resources, databases, etc.
• Never give an autonomous agent unrestricted production access by default.
• Real life: Allow an agent to read source code and run tests, but require approval before executing DROP TABLE, deploying infrastructure, or modifying production.

---

LEVEL 4 — Integration

19. MCP — Model Context Protocol

• MCP is a standardized protocol for connecting AI applications to external tools, resources, and prompts.
• Instead of building a custom integration for every AI client, you expose capabilities through an MCP server.
• Think of MCP as a standard interface between an AI client and external systems.
• Real life: Build an MCP server exposing your company's Jira, GitHub, database, or internal APIs to an AI coding agent.

20. MCP Client

• The MCP client is the application/agent that connects to MCP servers.
• It discovers what capabilities the server provides and can invoke them.
• Claude Code, for example, can act as an MCP client.
• Real life: Your coding agent connects to a Jira MCP server and asks it to retrieve the requirements for ticket PROJ-123.

21. MCP Server

• An MCP server exposes capabilities to an MCP client using the MCP protocol.
• It can expose tools, resources, and prompts.
• The server is essentially an adapter around your existing backend systems.
• Real life: Build company-mcp-server that internally calls your Order API, Customer API, and Inventory API.

22. Resources

• MCP resources expose information/data that an AI client can read.
• They are generally about providing context/data, rather than performing an action.
• Examples include files, documentation, schemas, database information, or configuration.
• Real life: Expose your API documentation or database schema as MCP resources so an agent can understand your system.

23. Tools

• MCP tools are executable capabilities that an AI client can invoke.
• Examples: getOrder, searchCustomer, createTicket, queryDatabase.
• Your backend should validate inputs and enforce authorization because the model's decision isn't a security boundary.
• Real life: Expose getOrder(order_id) through MCP so Claude can investigate a customer's order.

24. Prompts

• MCP prompts are reusable prompt templates exposed by an MCP server.
• They standardize how an AI client should perform a particular workflow.
• This separates reusable AI instructions from the application/client itself.
• Real life: Expose a debug-production-error prompt that guides the agent through logs → metrics → traces → recent deployments → root-cause analysis.

---

LEVEL 5 — Production AI

25. Embeddings

• An embedding converts text/data into a numerical vector representing its semantic meaning.
• Similar concepts produce vectors that are close together in vector space.
• Embeddings are heavily used for semantic search and RAG.
• Real life: Convert 100,000 company documents into embeddings so "How do refunds work?" can retrieve documents containing completely different wording.

26. Vector Database

• A vector database stores embeddings and efficiently finds vectors similar to a query vector.
• Examples include Pinecone, Weaviate, Milvus, Qdrant, and PostgreSQL with pgvector.
• It is commonly one component of a RAG pipeline.
• Real life: Document → embedding → vector DB; then question → embedding → similarity search → relevant documents → LLM.

27. Guardrails

• Guardrails prevent an AI system from doing things it shouldn't.
• They can enforce input/output schemas, prevent unsafe actions, filter sensitive data, restrict tools, and validate model responses.
• Treat guardrails as application security, not merely prompt engineering.
• Real life: Your support agent can refund an order only if order.status == DELIVERED and refund amount ≤ allowed limit.

28. Observability

• AI observability means monitoring what your AI system is actually doing.
• Track latency, token usage, cost, model responses, tool calls, failures, retrieval quality, and traces.
• Traditional backend metrics are still important, but AI systems need additional visibility into model/tool behavior.
• Real life: Trace API → LLM → MCP tool → database → LLM to determine why an agent took 15 seconds and cost $0.08.

29. Evaluation

• Evaluation measures whether your AI application actually works.
• Instead of testing only whether an API returns 200, evaluate correctness, relevance, hallucination, tool selection, retrieval quality, and consistency.
• Build a fixed dataset of representative questions and expected behavior.
• Real life: Every time you change your prompt/model, run 500 customer-support questions and compare the new results against your baseline.

30. Human-in-the-Loop

• Human-in-the-loop means a human approves or intervenes in important AI decisions.
• You should especially use it for irreversible, expensive, sensitive, or high-risk actions.
• The AI can automate analysis while humans retain control over critical actions.
• Real life: Agent detects a production issue and prepares a fix, but requires engineer approval before deploying it.

---

The entire thing as a Backend Developer

The most important part is seeing how these concepts connect:

                    USER
                      │
                      ▼
                  Your API
                      │
                      ▼
                    LLM
                      │
             ┌────────┴────────┐
             │                 │
          RAG/Search         Agent
             │                 │
       Vector Database      Agent Loop
                               │
                    ┌──────────┼──────────┐
                    ▼          ▼          ▼
                  Tool       MCP Tool   Subagent
                    │          │
                    ▼          ▼
                 Backend     MCP Server
                   APIs         │
                    │           ▼
                    └──────► Databases
                              Services

And in production, you wrap the whole thing with:

                 ┌─────────────────────┐
                 │      Guardrails     │
                 └──────────┬──────────┘
                            │
User → API → Agent → Tools/MCP → Backend
                    │
                    ├── Memory
                    ├── RAG
                    ├── Observability
                    ├── Evaluation
                    └── Human Approval

The mental model I recommend