DEV Community: Deepak Patil

The Orchestrator in Agentic Systems

Deepak Patil — Sat, 08 Aug 2026 09:20:28 +0000

A multi-agent system without an orchestrator is just a collection of agents. Each one is capable, but none of them coordinated. They might all be excellent at their individual jobs - searching the web, writing code, calling APIs - but without something deciding what gets done, in what order, by whom, and what to do when a result comes back wrong, the system does not behave like a system. It behaves like a group project with no project manager.

The orchestrator is the project manager. Its job is not to do the work. Its job is to make sure the work gets done - and that is a harder, more subtle problem than it sounds.

What an orchestrator is responsible for

An orchestrator does four things, and only these four things:

1. Decompose the goal. Turn a high-level objective into a concrete set of subtasks. This is a planning problem, not an execution problem. The orchestrator decides what needs to happen, not how to do it.

2. Route tasks to the right workers. Match each subtask to an agent capable of doing it. This requires knowing what tools and capabilities each worker has - not in detail, but well enough to delegate correctly.

3. Manage state across the workflow. As workers return results, the orchestrator decides what those results mean for the remaining plan. Sometimes a result changes the plan entirely. Sometimes it confirms the next step. The orchestrator holds the full picture.

4. Synthesise the final output. Worker outputs are partial. The orchestrator assembles them into a coherent response and decides when the goal has been met.

Notice what is absent: the orchestrator does not call APIs, does not run code, does not search the web. It reasons about work and routes it. The moment an orchestrator starts executing, it loses the focus that makes it good at coordination.

Building one from scratch

Here is a minimal orchestrator in Python. It plans upfront, delegates to type workers, and synthesizes results:

import json

def orchestrator(goal: str, workers: dict) -> str:
    # Step 1: plan
    plan_prompt = f"""
    Goal: {goal}
    Available workers: {list(workers.keys())}

    Return a JSON array of steps: [{{"task": "...", "worker": "..."}}]
    Return JSON only, no explanation.
    """
    plan = json.loads(llm(plan_prompt))

    # Step 2: execute each step, collect results
    results = []
    for step in plan:
        worker = workers[step["worker"]]
        result = worker.run(step["task"])
        results.append({"task": step["task"], "result": result})

    # Step 3: synthesise
    synthesis_prompt = f"""
    Original goal: {goal}
    Worker results: {json.dumps(results, indent=2)}

    Synthesise a final answer from these results.
    """
    return llm(synthesis_prompt)

Three things in this snippet are worth pulling apart.

The orchestrator calls llm() twice - once to plan, once to synthesize - but never once to execute. Execution is entirely delegated. If you find yourself adding a tool call directly inside the orchestrator loop, stop and ask whether a worker should own that instead.

The plan is a first-class object - a list of typed steps, not an implicit chain of thought. This means you can inspect it, log it, replay it, and re-plan from any point when something goes wrong.

Results are kept verbatim before synthesis. The orchestrator does not summarise early. It gives the synthesis step the full picture, letting the model decide what is relevant. Premature summarization is where context gets lost.

Re-planning when reality diverges

A fixed plan fails the moment a worker returns an unexpected result. A real orchestrator needs to decide: Does this change the remaining plan?

def orchestrator_with_replan(goal: str, workers: dict) -> str:
    plan = initial_plan(goal, workers)
    completed = []

    while plan:
        step = plan.pop(0)
        result = workers[step["worker"]].run(step["task"])
        completed.append({"task": step["task"], "result": result})

        # check if the result warrants re-planning
        replan_prompt = f"""
        Remaining plan: {plan}
        Latest result: {result}

        Does this result change what should happen next?
        If yes, return a revised plan as JSON. If no, return null.
        """
        revision = llm(replan_prompt)
        if revision and revision != "null":
            plan = json.loads(revision)

    return synthesise(goal, completed)

This is the plan-and-execute pattern from the last post made concrete. The orchestrator works from a checklist but checks after each result whether the checklist still makes sense. The loop terminates when the plan is empty, not when a fixed number of steps has run.

What frameworks add

Writing an orchestrator from scratch gives you control and understanding, but production use cases introduce problems the snippet above does not handle: state that needs to persist across restarts, workflows that need branching and loops, and debugging when something goes wrong three levels deep. This is where the frameworks come in.

LangGraph

LangGraph represents a workflow as a directed graph - nodes are agents or functions, edges are transitions between them, and a centralized StateGraph holds shared state across the whole run.

from langgraph.graph import StateGraph, END
from typing import TypedDict

class WorkflowState(TypedDict):
    goal: str
    plan: list
    results: dict
    final_answer: str

graph = StateGraph(WorkflowState)

graph.add_node("planner", planner_node)
graph.add_node("researcher", researcher_node)
graph.add_node("coder", coder_node)
graph.add_node("synthesiser", synthesiser_node)

graph.add_conditional_edges(
    "planner",
    route_to_worker,           # function that reads state and picks next node
    {"research": "researcher", "code": "coder"}
)

graph.add_edge("researcher", "synthesiser")
graph.add_edge("coder", "synthesiser")
graph.add_edge("synthesiser", END)

graph.set_entry_point("planner")
app = graph.compile()

The graph-based model earns its complexity because it makes branching and cycles explicit. The conditional edge above is not hidden inside a prompt - it is code. You can read the graph and know exactly what routes exist. LangGraph also ships with checkpointers that persist state to disk or a database, so a workflow that crashes halfway through can resume from the last checkpoint rather than starting over.

The honest tradeoff: a simple workflow that would take 40 lines in plain Python takes closer to 120 in LangGraph. You pay in boilerplate. You get auditability, resumability, and explicit control flow in return.

AutoGen

AutoGen takes a different approach. Instead of a graph, it models orchestration as message-passing between agents. A GroupChat manager decides which agent speaks next based on context, and agents broadcast their replies so everyone shares the same conversation history.

from autogen import AssistantAgent, UserProxyAgent, GroupChat, GroupChatManager

planner = AssistantAgent("planner", system_message="Break goals into tasks.")
researcher = AssistantAgent("researcher", system_message="Search and retrieve facts.")
coder = AssistantAgent("coder", system_message="Write and run Python code.")
critic = AssistantAgent("critic", system_message="Review outputs for errors.")

group_chat = GroupChat(
    agents=[planner, researcher, coder, critic],
    messages=[],
    max_round=12
)

manager = GroupChatManager(groupchat=group_chat)
planner.initiate_chat(manager, message="Build a data pipeline for X.")

The GroupChatManager is the orchestrator here. AutoGen v0.4 rebuilt this around an actor model where each agent runs independently and communicates through typed messages - a cleaner design for truly concurrent workflows.

The full-context broadcast is AutoGen's key architectural bet: every agent sees the whole conversation, which means specialist agents can catch problems earlier in the chain. The cost is that the context window fills faster in long workflows.

CrewAI

CrewAI uses a role-driven model. You define a crew of agents with named roles, assign tasks, and let the framework handle delegation. Configuration-first rather than code-first - most of the setup lives in YAML:

from crewai import Agent, Task, Crew, Process

researcher = Agent(role="Researcher", goal="Find accurate information",
                   backstory="Expert at web research", tools=[search_tool])
writer = Agent(role="Writer", goal="Produce clear summaries",
               backstory="Experienced technical writer")

research_task = Task(description="Research topic X", agent=researcher)
write_task = Task(description="Write a summary of the research", agent=writer)

crew = Crew(
    agents=[researcher, writer],
    tasks=[research_task, write_task],
    process=Process.sequential
)

crew.kickoff()

Process.hierarchical switches to an orchestrator model where a manager agent routes tasks dynamically rather than following a fixed sequence. CrewAI is the fastest path from idea to running prototype, but teams consistently report hitting its ceiling 6–12 months in when workflows grow beyond sequential or simple hierarchical patterns - at which point a migration to LangGraph tends to follow.

The most honest guidance: start from scratch to understand the pattern, then adopt a framework when you need what it specifically offers - not because frameworks are the default. LangGraph is the current production default for teams that need long-running, resumable workflows with explicit control flow. CrewAI is the right choice when you need something working this week and can accept the ceiling. AutoGen is strongest when agents genuinely need to debate and revise rather than execute a fixed plan.

The orchestrator's failure modes

The orchestrator is the single point of failure in a multi-agent system, which means its failure modes are expensive.

Over-delegation. The orchestrator sends a task to a worker that is underspecified. The worker returns garbage. The orchestrator synthesises the garbage. Build task schemas - typed descriptions of what a worker expects - and validate them before dispatch, not after.

Plan rigidity. An upfront plan that doesn't re-evaluate when results diverge will execute confidently toward a wrong answer. Build in a replan check after any result that introduces new information.

Silent worker failures. Workers that fail quietly return None or an empty string, which the orchestrator may synthesise as if it were real output. Workers should fail loudly with typed errors that the orchestrator can inspect and route around.

Context overload. The orchestrator accumulates results from every worker. In a long workflow, its context fills with raw worker outputs. Pass summaries, not raw transcripts, unless the raw content is genuinely needed for synthesis.

The orchestrator's value is its deliberate ignorance of the details. It does not know how to search the web, run code, or call an API. It knows what needs doing and who should do it. That separation - reasoning about work versus doing work - is what allows multi-agent systems to scale beyond what any single agent could manage alone.

The loop is still the heartbeat. The orchestrator is the brain that decides how many loops to start, what they should do, and when to stop.

Single-Agent vs Multi-Agent System

Deepak Patil — Sat, 01 Aug 2026 18:33:43 +0000

When one loop is enough - and when you need a team

The previous post ended with a loop: reason, act, observe, repeat. That loop, running inside a single agent, can handle a surprising range of tasks. But spend enough time building agents and you'll hit a wall. The task is too long for one context window. Two subtasks need to run at the same time. One part of the problem requires a specialist that would be noise everywhere else. When you hit that wall, you have a choice: push harder on the single agent or split the work across multiple agents. Knowing which to reach for - and why - is what this post is about.

What a single agent actually is

Before comparing, it's worth being precise. A single agent is one model, one context window, and one tool set, running one loop. Everything it has learned about the task lives in that growing history of thoughts, actions, and observations. Its memory is its context.

That constraint is the source of both its simplicity and its limits.

Where a single agent excels:

Tasks that fit comfortably in one context window
Linear workflows where each step depends on the last
Problems where keeping a shared mental model matters more than speed
Prototyping - one loop is trivial to trace and debug.

Where it breaks down:

Long tasks that exceed the context limit midrun
Tasks with independent subtasks that could be parallelised
Workflows that need different capabilities in different phases
Any situation where one agent failing silently kills the whole job

What a multi-agent system adds

A multi-agent system is two or more agents coordinating to complete a task. Coordination can mean many things - one agent spawning others, agents running in parallel and reporting back, and a pipeline where each agent's output is the next agent's input. The common thread is that no single agent owns the whole task.

This unlocks three things:

Parallelism. If you need to research five companies simultaneously, a single agent does them sequentially. A multi-agent system spawns five subagents and gets all five results at once. For I/O-bound tasks - anything involving search, API calls, or file reads - this is the primary win.

Specialisation. A coding agent prompted and tooled for writing Python is better at writing Python than a generalist agent. A multi-agent system lets you route subtasks to agents built for them - a planner, a researcher, a coder, a critic — each with the right system prompt and the right tools.

Context management. Each subagent gets a fresh, focused context. Instead of one agent accumulating 50,000 tokens of noise from early steps, a subagent receives only what it needs to do its job. The orchestrator summarises and routes; the workers stay sharp.

The orchestrator–worker pattern

The most common multi-agent architecture is a two-level hierarchy: one orchestrator and one or more workers.

The orchestrator receives the goal, makes a plan, and delegates subtasks. It does not do the work itself - it decides what the work is and who does it. Workers receive a scoped task, run their own loop, and return a result. They don't know about each other or the overall goal.

def orchestrator(goal, worker_agents):
    plan = llm(f"Break this goal into subtasks: {goal}")
    # plan = [{"task": "...", "agent": "researcher"}, ...]

    results = {}
    for step in plan:
        worker = worker_agents[step["agent"]]
        results[step["task"]] = worker.run(step["task"])

    final_answer = llm(f"Synthesise these results: {results}")
    return final_answer

Each worker.run() is the same agent loop from the previous post - reason, act, observe, repeat - but scoped to a single subtask. The orchestrator never picks up a tool itself; it reads a plan and routes. The workers never see the big picture; they just solve their piece.

This separation matters. It keeps orchestrator context lean (plans and results, not tool noise) and keeps worker context focused (one task, right tools, nothing else).

Parallelising with subagents

When subtasks are independent, run workers concurrently. In Python, asyncio or ThreadPoolExecutor are the usual choices:

from concurrent.futures import ThreadPoolExecutor

def run_parallel(tasks, worker):
    with ThreadPoolExecutor(max_workers=len(tasks)) as pool:
        futures = {pool.submit(worker.run, task): task for task in tasks}
        return {task: f.result() for f, task in
                [(v, k) for k, v in futures.items()]}

Two things to watch. First, rate limits: five agents hitting the same API simultaneously will hit quotas faster than one agent doing it sequentially. Build in back-off logic at the worker level, not just the orchestrator level. Second, result merging: parallel results arrive out of order and may conflict. The orchestrator's synthesis step needs to handle gaps and contradictions, not assume clean, uniform output.

When multi-agent is the wrong answer

Multi-agent systems have real costs, and reaching for them too early is one of the most common mistakes in agent engineering.

Complexity compounds. A single agent failing is easy to diagnose - you read the trace. Multiple agents failing is a distributed systems problem. Which agent failed? Did the orchestrator misparse the result? Did the worker receive a bad task? Every hop is a new failure surface.

Latency can get worse, not better. Parallelism helps I/O-bound tasks. For CPU-bound or model-call-bound work, the overhead of spawning agents, merging results, and making extra orchestration calls can exceed what a single focused agent would have taken.

Context hand-offs lose information. When the orchestrator summarises a worker's result to pass to the next, it makes decisions about what to keep. Those decisions are lossy. A single agent that ran the whole task never had to summarise itself.

The honest default: start with a single agent. Add a second agent when you have a concrete bottleneck - context overflow, a parallelism win you can measure, or a capability mismatch you cannot solve with a better prompt and better tools.

A practical decision framework

Situation	Reach for
Task fits in context, linear flow	Single agent
Task exceeds context mid-run	Orchestrator + summarising subagents
Independent subtasks, I/O-bound	Parallel workers
Phases need different tool sets	Specialist workers per phase
Need a second opinion on outputs	Critic agent in the loop
Prototyping or debugging	Single agent - always start here

A single agent is not a stepping stone to multi-agent. It is the right architecture for a large class of tasks and the one you should default to until you have a specific reason to split work. Multi-agent systems earn their complexity when tasks genuinely exceed what one context window can hold, when parallelism produces a real speed win, or when specialist agents meaningfully outperform a generalist on a defined subtask.

The loop does not change. What changes are, who owns which part of it, and how do the pieces report back?

Agent Memory & Context Engineering

Deepak Patil — Tue, 28 Jul 2026 18:41:11 +0000

How agents remember - and why deciding what to forget is the real skill

An agent that starts every step with a blank mind cannot really pursue a goal. It would reintroduce itself to you on every message, forget what it just tried, and repeat the same mistake forever. Memory is what turns a stateless model into something that accumulates - that knows who you are, what it has already done, and what it learned last Tuesday. This post is about how that works and, more importantly, about the discipline of deciding what an agent should remember at all.

The context window is not memory.

The first thing to unlearn: a model’s context window is not its memory. The context window is working memory - RAM, not a hard drive. It is finite, it is reset on every request, and every token in it costs money and dilutes the model’s attention. Stuffing an entire conversation history and knowledge base into the prompt does not scale, and past a point it actively hurts - the model loses the important signal in a sea of stale detail. Real memory lives outside the window and is selectively loaded into it when needed.

Four kinds of memory

Borrowing loosely from cognitive science, agent memory is usually split into four types, and good systems use all of them:

Short-term/working memory - the current conversation and the agent’s recent thoughts and observations. Lives in the context window.
Long-term episodic memory - a record of what happened: past conversations, decisions, and the outcomes of previous tasks.
Long-term semantic memory - facts and knowledge: who the user is, domain information, documents. This is what retrieval-augmented generation pulls from.
Procedural memory - how to do things: learned skills, tool-use patterns, and reusable strategies.

Short-term memory: the rolling buffer

The simplest memory is just keeping recent turns in the prompt. The problem is that conversations outgrow the window, so the standard move is to keep the last few turns verbatim and summarise the older ones into a compact note:

def build_context(history, window=6):
    recent = history[-window:]
    older = history[:-window]
        if older:
            summary = llm(f"Summarize this conversation so far:\n{older}")
            return [f"Summary so far: {summary}", *recent]
    return recent

This keeps the prompt bounded while preserving the gist of what came before. It is crude, but it is the backbone of almost every chat agent in production.

Long-term memory: embeddings and vector search

To remember across sessions, an agent writes information to an external store and retrieves it later by meaning rather than exact keywords. The mechanism is embeddings: each piece of text is converted into a high-dimensional vector, stored in a vector database, and later retrieved by finding the vectors closest to the current query. This is what lets an agent recall a relevant fact even when you phrase your question completely differently from how the fact was stored.

# Write a memory
vec = embed("User prefers window seats and vegetarian meals.")
store.add(id="pref-1", vector=vec, text="User prefers window seats and vegetarian meals.")

# Later, retrieve by meaning
query_vec = embed("book me a flight to Delhi")
hits = store.search(query_vec, top_k=3)
context = "\n".join(h.text for h in hits)   # injected into the prompt

That retrieval step is what gives an agent its long memory without bloating the context window: you store everything but load only the handful of memories relevant to the moment.

Why vector search alone isn’t enough

Vector similarity is powerful but blunt, and it is worth knowing its limits before you lean on it. Similarity relies on relevance - not recency, not authority, not workflow state. It will happily surface an outdated preference the user changed yesterday or a fact from a draft that was later overruled, simply because the words are close. It does not understand which memory is current, which is authoritative, or where you are in a multi-step task. Production memory systems layer on recency weighting, source ranking, and explicit state to compensate. Memory in 2026 is treated as a real engineering discipline with measurable trade-offs, not a database you bolt on and forget.

Context engineering: the real skill

All of this rolls up into the discipline people increasingly call context engineering: deliberately deciding what goes into the context window on every single step. Prompt engineering asks How do I word the instruction? Context engineering asks What information should the model see right now, and what should I leave out? It is closer to managing a tight working-memory budget than to writing clever prompts.

A recent and influential pattern is Agentic Context Engineering (ACE), which treats context as something the agent actively curates through a three-role loop: a Generator produces an attempt, a Reflector evaluates it and flags what was missing or wrong, and a Curator distils the lesson into a growing “playbook” that improves future context. Reported results show meaningful accuracy gains on agent benchmarks without retraining the underlying model - the improvement comes entirely from feeding it better context. That is the whole thesis of context engineering in one experiment.

Practical patterns to start with

Summarise old turns, keeping recent ones verbatim - bounded prompt, preserved gist.
Retrieve, don’t dump - pull only the top few relevant memories per step instead of the whole store.
Tag memories with time and source so you can prefer recent, authoritative information over stale matches.
Write back what matters - after a task, save the durable facts and lessons, not the entire transcript.

The takeaway

Memory is what separates a chatbot from an agent that genuinely accumulates competence over time. But the headline isn’t “store everything” - it’s the opposite. The skill is curation: deciding, on every step, the smallest set of information that lets the model act well and leaving the rest in long-term storage until it’s needed. Treat context as critical infrastructure, and your agents get sharper, cheaper, and more reliable all at once.

Next in the series: One agent can only do so much. We’ll look at multi-agent systems - orchestration, the A2A protocol, and how to split a goal across a team of specialised agents without the whole thing collapsing into chaos.

Tools, Function Calling & MCP

Deepak Patil — Thu, 23 Jul 2026 16:29:21 +0000

How agents reach into the real world - and how to do it without creating a security nightmare

A language model on its own is a brain in a jar. It can reason beautifully about the weather and tell you nothing about whether it is actually raining outside, because it has no senses and no hands. Tools are the senses and hands. They are the single feature that turns a model that can talk about booking a flight into an agent that can actually book one. In the last post we saw the agent loop; this post is about the Act step inside that loop.

What a tool actually is

Strip away the jargon and a tool is two things: a function, and a description the model can read. The function is ordinary code - a database query, an HTTP call, a shell command. The description is a schema that tells the model the tool’s name, what it does, and what arguments it takes. The model never sees your code; it only sees the schema and decides, in the moment, whether this tool is the right one for the goal in front of it.

def get_weather(city: str) -> dict:
    """Return current conditions for a city."""
    return weather_api.current(city)

# The schema is what the MODEL sees:
{
    "name": "get_weather",
    "description": "Get current weather conditions for a city.",
    "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string", "description": "City name, e.g. 'Mumbai'"}},
        "required": ["city"]
    }
}

The one rule that keeps you safe: decide vs. execute

Here is the most important sentence in this entire post. The model decides; your code executes. When a model “calls a tool,” it does not run anything - it returns a structured request that names a function and its arguments. Your application receives that request, decides whether it is allowed, runs the real function, and feeds the result back into the loop.

The LLM should decide what to do, but never be the thing that does it. Keep a layer of your own code between the model’s intent and any real-world effect.

That gap between intent and execution is where you put validation, permission checks, rate limits, and human approval for anything dangerous. Collapse that gap - let the model run code directly - and you have handed an unpredictable system the keys to your infrastructure.

# The full decide -> execute -> observe cycle

response = client.chat(messages=history, tools=schemas)
for call in response.tool_calls:
    if not is_allowed(call.name, call.args):     # YOUR guardrail
        result = "Error: not permitted."
    else:
        result = REGISTRYcall.name  # YOUR code runs it
    history.append({"role": "tool", "name": call.name, "content": str(result)})
    final = client.chat(messages=history)            # model uses the results

Designing tools the model can actually use

An agent is only as capable as its tools are well-designed, and most agent failures trace back to a badly described or badly scoped tool rather than a weak model. A few principles go a long way:

Name and describe for the model, not for you. cancel_order(order_id) with a one-line description of when to use it beats a clever internal name the model has to guess about.
Keep each tool narrow. One tool, one job. A single do_everything tool with a mode flag forces the model to reason about your implementation instead of the task.
Validate inputs and fail informatively. Return “Error: city ‘Xyz’ not found, did you mean a valid city name?” rather than a stack trace. The model reads that error and corrects itself on the next loop.
Prefer idempotency. Agents retry. A tool that charges a card twice when called twice is a liability; design so repeats are safe.

The M×N problem, and why MCP exists

Once tools work, a scaling problem appears. If you have M agent applications and N systems you want to connect (Slack, GitHub, your database, Google Drive), the naive world requires M×N custom integrations - every app re-implementing a connector to every system. The Model Context Protocol (MCP), introduced by Anthropic and now broadly adopted, collapses that into M+N. MCP is an open standard - often described as “USB-C for AI tools” - that defines a common way for a model to discover and call tools exposed by any compliant server.

The division of labour in the 2026 agent stack is clean: MCP standardizes how agents talk to tools and data; the Agent-to-Agent (A2A) protocol standardizes how agents talk to each other. Build a tool once as an MCP server and any MCP-aware client - LangGraph, CrewAI, ADK, a desktop assistant - can use it without bespoke glue.

# A minimal MCP server exposing one tool (Python SDK)

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("weather-server")

@mcp.tool()
def get_weather(city: str) -> dict:
    """Get current weather conditions or a city."""
    return weather_api.current(city)

if __name__ == "__main__":
    mcp.run()   # any MCP client can now discover and call get_weather

The payoff is an ecosystem effect: the catalogue of ready-made MCP servers keeps growing, so connecting an agent to a new system increasingly means pointing it at an existing server rather than writing an integration.

Tools are your attack surface

Every tool you give an agent is also a door someone could walk through. A tool that runs shell commands or sends money is exactly as dangerous in an agent’s hands as in an attacker’s, and prompt injection - hostile instructions hidden in a web page or document the agent reads - can trick an agent into misusing the tools it has. Treat tool access as a security boundary, not a convenience:

Least privilege. Give each agent the narrowest set of tools its job requires - nothing more.
Never auto-execute irreversible actions. Payments, deletions, and trades should require explicit human confirmation, not an agent’s say-so.
Validate and sanitize all tool inputs, exactly as you would for any untrusted user, because in effect they are

Tools are what make an agent useful, and MCP is making them composable across the whole ecosystem. But the discipline that matters most is the gap you keep between the model’s decision and the real action - that gap is where your validation, permissions, and human oversight live. Get the tool design and that boundary right and you have an agent that is both genuinely capable and safe to put in front of real users.

Inside The Agent Loop

Deepak Patil — Tue, 21 Jul 2026 16:26:15 +0000

How an agent actually thinks and acts: ReAct, tool calling, and the loop that powers every agent

In the first post, we described an agentic system as software that can plan, reason, use tools, and execute multi-step workflows. This sentence hides the most important idea in the whole field. What is actually happening when an agent “thinks”? The honest answer is surprisingly simple, and once you see it, every agent framework on the market stops looking like magic and starts looking like a single design pattern repeated with variations.

The pattern is a loop. A plain language model answers your prompt once and stops. An agent wraps that same model in a loop (its output is fed as its next input and so on) that keeps running - reasoning, taking an action, observing the result, and reasoning again - until the goal is met. That loop is the heartbeat of every agent you will ever build.

From one-shot answers to a loop

A chatbot is a function: text in, text out, done. The model never finds out whether its answer was any good. An agent breaks that single shot into a cycle. After each step it gets feedback from the real world and uses that feedback to decide what to do next. This is the difference between a student who writes an exam answer and walks away and a mechanic who turns a bolt, checks whether the engine starts, and adjusts.

The most influential way to structure that cycle is the ReAct pattern, introduced by Yao et al. in 2022. ReAct stands for Reason + Act, and it interleaves three things on every turn:

Thought - a short chain-of-thought where the model reflects on the situation and decides what to do.
Action - a concrete step, usually calling a tool or an API.
Observation - the result that action produced, fed straight back into the model.

By forcing the model to reason in the open and ground each decision in a real observation, ReAct reduces hallucination: the model cannot quietly invent a fact when the next line of the transcript is the actual output of a search it just ran.

A traced example

Suppose the goal is "What was the closing price of the stock the CEO of Tesla most recently tweeted about?” A ReAct trace might look like this:

Thought: I need to find Elon Musk's most recent tweet about a stock.

Action: search_tweets(user="elonmusk", topic="stock")

Observation: "Just bought more $X ..." (posted 2h ago)

Thought: The ticker is X. I need today's closing price.

Action: get_quote(ticker="X")

Observation: { "close": 41.80, "currency": "USD" }

Thought: I now have the answer.

Action: finish("The closing price was $41.80.")

Notice that no single model call could have answered this. The agent had to act, look at what came back, and only then decide its next move. That is the entire value proposition of the loop.

Building the loop from scratch

Stripped of any framework, an agent loop is about a dozen lines. The model returns a structured decision; your code executes it and feeds the result back:

def run_agent(goal, tools, max_steps=8):
    history = [system_prompt(goal, tools)]
    for _ in range(max_steps):
        decision = llm(history) # model reasons + picks an action
        if decision.action == "finish":
            return decision.answer
        tool = tools[decision.action]
        result = tool(**decision.args)   # YOUR code runs the tool
        history.append(decision)
        history.append(f"Observation: {result}")
    return "Stopped: step budget exhausted."

Three details in that snippet matter more than they look. First, the loop has a hard step budget (max_steps). Without it, a confused agent will happily call tools forever and burn through your API bill. Second, the model never executes anything - it only names an action and arguments; your code decides whether and how to run it. Third, every observation is appended to history, so the model’s context grows with what it has learned.

ReAct text vs. native tool calling

There are two ways to get that decision out of the model. The original ReAct approach asks the model to write its thoughts and actions as text, which you then parse. Modern models offer native tool calling (also called function calling): you hand the model a set of typed function schemas, and it returns a structured JSON object naming the function and its arguments - no fragile text parsing required.

Native tool calling: the model returns structured intent, not prose.

tools = [{
        "name": "get_quote",
        "description": "Get the latest market quote for a ticker symbol.",
        "parameters": {
            "type": "object",
            "properties": {"ticker": {"type": "string"}},
            "required": ["ticker"]
        }
}]

response = client.chat(messages=history, tools=tools)
call = response.tool_calls[0]    # {name: "get_quote", args: {ticker: "X"}}

The trade-off is real and worth understanding. Text-based ReAct is transparent and model-agnostic - you can read the reasoning, and it works on any model, even ones without a tool-calling API. Native tool calling is more efficient and reliable - structured output, fewer round trips, and the model can request several tools at once - but the reasoning behind a call is less visible. In practice most production agents use native tool calling and recover transparency through tracing and logging.

When one loop isn’t enough: plan-and-execute

Pure ReAct decides its next step one observation at a time, which is flexible but can wander on long tasks. A common evolution is plan-and-execute: the agent first drafts a full plan and then executes the steps, replanning only when reality diverges. Separating slow, expensive planning from fast tactical execution tends to handle complex multi-step workflows more efficiently than reacting from scratch every turn. You can think of ReAct as improvising and plan-and-execute as working from a checklist you’re allowed to revise.

Where loops go wrong

The loop is powerful precisely because it is open-ended, and that is also where the failure modes live. Agents get stuck repeating the same failing action, loop until the budget runs out, or call an expensive tool dozens of times. The defenses are unglamorous but essential: a hard step limit, detection of repeated identical actions, a cost ceiling, and clear tool error messages so the model can recover rather than retry blindly. Treat every observation as something the model might misread, and design your tools to fail loudly and informatively.

The takeaway

Every agent - from a customer-support bot to GitHub Copilot’s agent mode - is, at its core, this loop: “reason, act, observe, repeat, until done." "Frameworks add memory, multiple agents, and orchestration on top, but they are all variations on the same heartbeat. Once you can write the loop yourself, you understand agents from the inside, and every framework becomes a convenience rather than a mystery.

Next in the series, the Act step deserves its own post. We’ll go deep on tools, function calling, and the Model Context Protocol - how agents actually reach into the world without becoming a security liability.

How to Efficiently Run Meta LLaMA on a MacBook Air M1 with Limited RAM

Deepak Patil — Fri, 06 Dec 2024 15:16:20 +0000

Running advanced AI models like Meta’s LLaMA on a MacBook might seem ambitious specifically when you have M1 with 8 GB of RAM, But with the right steps, you can start building AI apps locally on your Mac easily. Thanks to Apple’s processor architecture and efficient libraries like Llama.cpp, you can unlock the power of large language models right from your lightweight laptop.

Let's get you started with the MacBook Air M1 for running these models efficiently.

Step 1:

Download the entire model from Meta's official site by providing your information & details for usage.

Install the necessary packages as mentioned in the readme file.

Run the below command to run the model

torchrun \
  --nproc_per_node=$NGPUS \
  llama_models/scripts/example_chat_completion.py $CHECKPOINT_DIR \
  --model_parallel_size $NGPUS

Definitely, this is not going to work ☹️. To solve this issue we will be following two methods below

first, we will be using llama.cpp which provides lightweight C++ implementation for running models on various hardware.
We will use quantization to reduce the model size so that we will be able to run it easily.

Step 1 (This one will work):

Install llama.cpp using brew.

brew install llama.cpp

Step 2:

Now let's quantize the llama model. To perform this we have a very good space on HuggingFace called GGUF-My-Repo. Follow the below link to go to space.

GGUF-MY-REPO

On this space login with your HuggingFace credentials. Then select the model repository that you want to quantize. For llama, you need to get access to the repo. Select the checkbox for 'Create a private repo under your username'. If you are a beginner then leave other options to default and proceed.

Once the process is done you will have a quantized model created in your private repository.

Step 3:

Clone the HuggingFace repo just created on your Mac. Run the model using the command below

llama-cli -m GGUF_MODEL_FILE_NAME -n 1024 -ngl 1 -c 512 --prompt PROMT cnv

example

llama-cli -m meta-llama-3.1-8b-q4_k_m.gguf -n 1024 -ngl 1 -c 512 --prompt "Hello" -cnv

You can also run directly from the repository name.

llama-cli --hf-repo Deepcodr/llama_sample_chat-Q4_K_M-GGUF --hf-file llama_sample_chat-q4_k_m.gguf -p "The meaning to life and the universe is"

Tip: If you are a beginner avoid using base models if you don't want to get some gibberish or random responses. Instead use instruct models such as chat, or text. etc. You can find some already quantized models here

How to upload react build files to S3 from GitHub Actions

Deepak Patil — Tue, 08 Oct 2024 07:25:07 +0000

Every time you push new changes to your React app, it’s live on the web within minutes—no manual uploads, no repetitive commands, just pure automation magic. With GitHub Actions and Amazon S3, you can make it happen but there is a catch, Very few prebuilt scripts are available for uploading react build files to S3 from GitHub actions. In this post, Let's see how to connect the dots between your React project and AWS S3 using GitHub Actions, turning every code push into an automatic deployment.

To begin with we need to create a script that will iterate over all files in the build folder and then use aws-sdk to upload all files to the S3 bucket configured with static website hosting. First we will import required packages to JS script.

const S3 = require('aws-sdk/clients/s3');
const path =require('path');
const fs = require('fs');
const klawSync = require('klaw-sync');
const {lookup} = require('mime-types')

Then we will initialize the S3 client and get build folder path which will be generated from github action with npm run build

var awsoptions = {
    accessKeyId : process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey : process.env.AWS_ACCESS_KEY
}

const buildFolderPath = path.join(__dirname,'build')

var s3 = new S3(awsoptions);

Then we will use klaw-sync to get file path for each file and then upload that file to the S3 bucket using upload() function.

const filePaths = klawSync(buildFolderPath,{
    nodir : true
});

filePaths.map((filePath)=>{
    const fileContent = fs.createReadStream(filePath.path);
    const bucketPath = path.join('',path.relative(buildFolderPath,filePath.path));


    config ={
        Bucket : 'YOUR_BUCKET_NAME',
        Body : fileContent,
        Key : bucketPath,
        ContentType : lookup(filePath.path) || "text/plain"
    }

    upload(config);
})

function upload(config){
    return new Promise((resolve) => {
        s3.upload(config, (err, data) => {
          if (err) console.error(err);
          console.log(`uploaded - ${data.Key}`);
          console.log(`located - ${data.Location}`);
          resolve(data.Location);
        });
    });
}

Thats it place this script file in react project directory where package.json is present. Run this script from your github action workflow after building the react project. Once called it will upload all files from build folder to bucket.

Example job steps for building & uploading files

    steps :
      - uses : actions/checkout@v4
      - uses : actions/setup-node@v4
        with :
          node-version : 20
      - name : Install node modules
        run : npm install
      - name : Build the project for production
        run : npm run build
      - name : Upload Files to S3
        run : node s3upload.js

With GitHub Actions and S3 working together, deploying your React static site is no longer a tedious task but an automated part of your workflow. This setup not only saves time but also ensures that every time your static files get pushed to the bucket while developer can focus on development.

What are AI accelerators or NPUs?

Deepak Patil — Thu, 12 Sep 2024 14:46:50 +0000

With the increasing popularity of AI software & technologies, there is a need for specialized hardware units to provide high performance to AI services. Traditional processors like CPUs and GPUs often struggle with the massive computational load required by advanced machine learning models. Neural Processing Units (NPUs), often known as AI accelerators or DL(Deep Learning) Accelerators, are the answer to this problem. NPUs are specialized hardware designed to boost AI tasks by optimizing performance and energy efficiency. These purpose-built processors are revolutionizing AI application performance and data processing.
NPUs perform various complex computational operations required by AI efficiently compared to GPUs which are more power-consuming and come at cost. NPUs have dedicated hardware circuits for training models on Trillions of parameters. Currently, GPUs are gaining popularity due to their performance in training the models. But upcoming NPUs are disrupting this trend providing much more performance than GPUs.NPUs implement in-memory computing architecture allowing them to replicate human brain-like working.

source : AWS

These devices can help build the most complex neural networks and vision applications. Implementing the circuit for performing three types of computations (scalar, vector, tensor) makes them the elite hardware for next-generation computers. Companies like Google, Intel, and AWS have already begun constructing their NPUs dedicated for computing devices and cloud infrastructures.AWS Inferentia & Trainium are the NPUs powering modern AI workloads in Cloud infrastructure.
With the research still going on to develop more efficient NPUs it is interesting to see what next things will come to the AI evolution.

How to install Nginx on Mac

Deepak Patil — Wed, 04 Sep 2024 19:25:32 +0000

Nginx is a widely used web server, valued for its speed and reliability in serving web content. Installing Nginx on macOS allows you to leverage its local development or testing capabilities. In this post, let's understand the steps to install Nginx smoothly on your Mac, making it easy to manage and serve your web projects.

Although you can install Nginx from the source code on MacOS, it is a bit complex. So, HomeBrew is the best option for installing Nginx.

Steps to install Nginx on MacOS

Install Homebrew Package Manager

You can install Homebrew from official website or Just enter below command to install.

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Install Nginx

Use the homebrew command below to install nginx.

brew install nginx

Start & Use Nginx

Start the Nginx service using brew.

brew services start nginx

Now go to http://localhost:8080 and you should see the Nginx welcome page.
Below are some commands you can use for managing nginx service.

Stop

brew services stop nginx

Restart

brew services restart nginx

Status

brew services status nginx

By default nginx on Mac uses 8080 port. You can change nginx.conf and update the port number to 80 so that it will serve directly on localhost.

With Nginx now running on your macOS, you're ready to explore and deploy web apps with nginx.

Using S3 Bucket to host React Application

Deepak Patil — Mon, 26 Aug 2024 18:15:25 +0000

It becomes confusing when it comes to deploying a React application in production. Because there are several ways we can deploy a React application in production. But considering a static single-page website hosting, there is one popular way: AWS S3.

Hosting the React application in S3 not only provides a simple yet highly available deployment option but also brings other features to the table.S3 provides versioning, logging, and security features to make the experience more seamless. Integrating it with CloudFront makes it highly available, cacheable, and faster. S3 also allows CORS configuration, which can help to control access to the resources. Along with this bucket, it can be attached to a custom domain name configured with Route 53 or DNS, making a complete, full-fledged web application.

Considering these features, S3 becomes the best option for deploying React in production. It only takes a few steps to deploy a React application to S3.

Create a S3 bucket in AWS

Create an S3 bucket as per the required configuration and storage class. Make sure to allow public access to all objects and remove any other blocking access options.

Generate a Production Build of React application

Run the below command in the project root to generate a production build.

npm run build

this will create a build folder in the project directory with all static files.

Upload the build folder to S3 Bucket

Upload the contents of the build folder to the bucket along with the static folder.

Add a policy to a bucket for getObject

Edit the bucket policy to provide public access to get objects in the bucket. The policy will look like below.

{
  "Id": "Policy1729235341904",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1729235340511",
      "Action": [
        "s3:GetObject"
      ],
      "Effect": "Allow",
      "Resource": "YOUR-S3-BUCKET-ARN/*",
      "Principal": "*"
    }
  ]
}

Enable Static hosting for S3 Bucket

In the properties of the bucket enable static website hosting. Specify index and error document as index.html (Other names if differ). Save the changes.

Test the running website using the link

Copy the generated link and paste it into the browser to view your website running. Add a custom domain if required.

Thats it! In just a few clicks the React site is live using S3.

Which is the best stack for the web today? MERN vs MEAN vs Spring Boot

Deepak Patil — Mon, 12 Aug 2024 13:37:15 +0000

Many devs nowadays have a common question in their mind, which web frameworks and stacks are best to make a career? Well the question itself is unanswerable looking at the circumstances today. However, I will try to provide the best answer based on my experience and current market trends.

Let's consider the three most used and popular stacks in the market today which are MERN, MEAN and Spring. All three of these stacks come with unique functionalities and applications fulfilling different use cases. If we look at MERN Or MEAN, They are more likely to be used for complete dynamic web application development while on the other hand, spring boot is suited for microservices and loosely coupled REST API-based enterprise applications.

MERN and MEAN use javascript which makes it easy to write but still allows us to use it with modern features like NoSQL datasets, Caching etc. Similarly, Spring boot uses Java which is a language that is part of most of the legacy applications. Spring Boot makes a powerful framework that allows us to develop loosely coupled and serverless architectures. Looking at other features all stacks allow support for REST APIs, Scalability, and Fault tolerance by integrating with the cloud.

key features of MEAN, MERN, and Spring Boot:

Aspect	MEAN Stack	MERN Stack	Spring Boot
Database	MongoDB (NoSQL)	MongoDB (NoSQL)	SQL/NoSQL Databases
Server-Side Framework	Express.js	Express.js	Spring Framework
Development Paradigm	Full-Stack JavaScript, Single Page Application (SPA)	Full-Stack JavaScript, Component-Based Architecture	Enterprise-Level Application, Microservices Architecture
Auto-Configuration	No	No	Yes
Embedded Servers	No	No	Yes (e.g., Tomcat, Jetty, Undertow)
Microservices Support	Limited, typically not the primary focus	Limited, typically not the primary focus	Strong support for microservices
Production-Ready Features	Depends on configuration	Depends on configuration	Yes, includes metrics, health checks, etc.
Security	Depends on additional libraries	Depends on additional libraries	Built-in with Spring Security
Scalability	MongoDB provides scalability for data	MongoDB provides scalability for data	High scalability, especially in microservices architecture
Ecosystem	Angular ecosystem for front-end	React ecosystem for front-end	Comprehensive Spring ecosystem, including Spring Cloud, Spring Data, etc.
Development Speed	Fast due to full-stack JavaScript and easy to understand	Fast due to full-stack JavaScript and easy to understand	Fast with Spring Boot’s auto-configuration and Spring Initializr but little bit complicated

So deciding which is better is not possible. But according to me for learning a framework as a novice, MERN or MEAN stacks are the best. Because it will give a proper understanding of how web applications or REST APIs work. On another hand, if you are already well familiar with working with Java-based applications and technologies like servlets, hibernate etc. It is better to advance yourself with the power of Spring Boot.

Whichever framework you choose I prefer to dive deeper into it unless you feel lost in it and then you start exploring endless possibilities with it. Once you gain so much knowledge about a particular technology the question of the best stack to make career? becomes trivial.

v0.dev :) Future of Web UI. Will UI and Frontend Devs exist in future?

Deepak Patil — Fri, 26 Jul 2024 13:05:52 +0000

Building and designing UIs from UI/UX tools like Figma and XD with languages like HTML, CSS, and JS is way too far for today's GenAI Web Development. From these tools to website builders like WordPress and Wix, and from these platforms to AI website builders, is the next step in the world of web development. v0.dev by Vercel is the revolutionary platform that's turning the web development community on its head. With its innovative approach to design and automation, v0.dev raises the important question of whether frontend and UI developers will soon be extinct?

In modern web development scenarios, implementing and leveraging the powers of GenAI is a powerful way of automating the entire web development cycle. v0.dev implements a prompt-based UI creation tool based on Shadcn UI components and layouts. It is capable of generating UI within seconds with an elegant combination of components.

Have a look at what it has generated within just a few clicks and prompts.

Generate a UI for product catalog of ecom website

It will take a lot of time for an experienced developer to develop such a page as compared to v0. Not only this, it also provides theming and customization options to make it more beautiful and brandy. We can copy the whole code for the UI in JSX or HTML. Just some configuration for the backend and the full-fledged site will be ready within a few minutes. And how can we forget Vercel? Using which site can be deployed and made live within another few clicks.

Thus, using v0.dev and Vercel makes it a deadly combination for building web applications. Well, will it kill the jobs of web developers or UI designers is still a question. But the radical evolution in the field of web development with AI is an interesting thing to watch and observe what more it will bring to the table.