DEV Community: Adi

[Boost]

Adi — Tue, 01 Jul 2025 12:48:03 +0000

Adi

Jun 29 '25

a loop is all you need: building conversation ai agents

#ai #tutorial #startup #howto

Comments 1

10 min read

coding = writing text: approach it like an essay

Adi — Sun, 29 Jun 2025 17:27:21 +0000

Every person who writes code believes it is clean. What a fallacy!

I used to think the same until my colleague Manuel asked me to review his Pull Request (PR) for a project we were working on. As I reviewed his code, I realized that his code is so much cleaner than mine.

What made his code cleaner? Uff, I wouldn't be able to tell, but it felt better. It is like reading your own university essay and then reading one by Paul Graham or Malcom Gladwell - you just know there's a difference.

At first, I thought it was just my lack of experience with JavaScript/TypeScript (as a data scientist I mostly wrote Python before). But in reality, it was just me not paying enough attention to and not recognizing all patterns of clean code.

Manuel recommended me a book by Robert C. Martin and I bought it immediately. The book is called The Clean Code: A Handbook of Agile Software Craftsmanship. It's an extensive and detailed book that you should not read as a novel from start to finish within a day or two. Instead, it serves as a handbook - read a chapter, reflect on it, and ideally apply it in your work. While the code examples are in Java, most of the lessons are universal to other languages.

In this blog post I won't be summarizing or paraphrasing the entire book. First, honestly I don't think I internalized the entire wisdom written there to be able to share it effectively. Second, I encourage everyone to read the book and draw their own conclusions from original source. This advice applies to any topic - always go to the original source. Finally, the book has so many valuable insights that a single blog post can't capture them all.

Instead, in this blog, I'll share the first key takeaway I gained after reading and applying some of its principles.

write code like you're writing an essay, not just a script

Let me give one example of code and how to rewrite it if you think about the code as an essay.

code example

Suppose that we have a pipeline or a workflow that executes a set of checks. First it runs external_check. If this external check returns APPROVED, then it proceeds with the second step where the status is determined based on metrics.

Here is some dummy implementation of this workflow:

from enum import Enum
from some_file import BaseWorkflow
from typing import Any, Dict, Optional

class WorkflowStatus(Enum):
  CREATED = "CREATED"
  APPROVED = "APPROVED"
  REJECTED = "REJECTED"
  PENDING = "PENDING"

class CheckOutcome(Enum):
  CREATED = "CREATED"
  PASSED = "PASSED"
  FAILED = "FAILED"
  PENDING = "PENDING"

class Workflow(BaseWorkflow)
  def __init__(self, status):
    self.status = status

  async def external_check(self) -> Dict[str, Any]
    # some operation, returns { outcome: CheckOutcome }
    pass

  async def fetch_metrics(self) -> Dict[str, Any]:
     # Does some operations
     # trigger data processing if it was not triggered
     # convert preprocessed data to metrics
     # returns metrics
     pass

  async def run(self) -> WorkflowStatus:
    check = await external_check()
    status = get_status(outcome=check["outcome"])

    if status != WorkflowStatus.APPROVED:
      self.status = status
      return self.status

    if status == WorkflowStatus.CREATED:
      await fetch_metrics()
      self.status = WorkflowStatus.PENDING
    else:
      metrics = await fetch_metrics()
      self.status = get_status(metrics=metrics)
      return self.status

  @staticmethod
  def get_status(
    outcome: Optional[CheckOutcome] = None,
    metrics: Optional[Dict[str, Any]] = None
  ) -> WorkflowStatus:
    status = WorkflowStatus.PENDING

    if outcome is not None:
       # Logic based on outcome
       pass

    if metrics is not None:
       # Logic based on metrics

    return status

improvement #1: write from top to bottom

The main method in this class is .run() which is how this workflow will be used in other codebase. Imagine you see somewhere in the codebase workflow.run() and you want to know what it does.

You scroll down to the middle of the file to find .run() implementation. It references three other methods: .external_check(), fetch_metrics() and get_status(). So, you scroll up a bit to check implementation of the first two methods. Where is get_status()? Ahh you scroll down again, this time to the bottom of the file, to find its implementation.

Quite a lot of scrolling to follow the code. Imagine reading an essay where the introduction is halfway down the page, the main arguments are at start, and the resolution is in the bottom. We can structure our code better than that.

How about reorganizing the code so that as you follow it, you only need to scroll down? This way, everything flows naturally, just like reading a well-structured document.

class Workflow(BaseWorkflow)
  def __init__(self, status):
    self.status = status

  async def run(self) -> WorkflowStatus:
    # calls external_check()
    # calls fetch_metrics()
    # calls get_status()
    pass

  async def external_check(self) -> Dict[str, Any]
    pass

  async def fetch_metrics(self) -> Dict[str, Any]:
     pass

  @staticmethod
  def get_status(
    outcome: Optional[CheckOutcome] = None,
    metrics: Optional[Dict[str, Any]] = None
  ) -> WorkflowStatus:
    pass

improvement #2: group into chapters

The code inside .run() handles several tasks at varying levels of abstraction: it calls an external check, fetches metrics, and derives the status from both. However, at its core, it’s a two-step process. First, it performs an external_check (Step 1), and only if the first step is APPROVED does it proceed with the rest of the code (Step 2). If this were a book, these would be two distinct chapters: one for each step.

So, let's break .run() into two separate methods: execute_first_step() and execute_second_step(). This division allows us to enhance each step with more complex logic without altering the original .run() implementation.

class Workflow(BaseWorkflow)
  def __init__(self, status):
    self.status = status

  async def run(self) -> WorkflowStatus:
    first_step_status = await self.execute_first_step()

    if first_step_status != WorkflowStatus.APPROVED:
      self.status = first_step_status
      return self.status

    second_step_status = await self.execute_second_step()
    self.status = second_step_status
    return self.status

  async def execute_first_step(self) -> WorkflowStatus:
    check = await external_check()
    status = get_status(outcome=check["outcome"])
    return status

  async def execute_second_step(self) -> WorkflowStatus:
    if self.status == WorkflowStatus.CREATED:
      await fetch_metrics()
      self.status = WorkflowStatus.PENDING
    else:
      metrics = await fetch_metrics()
      status = get_status(metrics=metrics)
      return status

  async def external_check(self) -> Dict[str, Any]
    pass

  async def fetch_metrics(self) -> Dict[str, Any]:
     pass

  @staticmethod
  def get_status(
    outcome: Optional[CheckOutcome] = None,
    metrics: Optional[Dict[str, Any]] = None
  ) -> WorkflowStatus:
    pass

improvement #3: have intentional naming

Now let's take a closer look at the other methods: get_status() and fetch_metrics().

The get_status() method's name suggests it returns a status, which is clear and straightforward. However, we notice that get_status() takes in two unrelated arguments: outcome and metrics. To understand what this method does, how it uses each argument, and what arguments it actually accepts, you'll need to dive into its implementation—especially if you plan to use it elsewhere.

To make the code more intuitive, let's improve the naming and split it into two methods: get_status_from_outcome() and get_status_from_metrics(). This not only simplifies the functions and ensures each does only one thing, but also makes it clear what each method accepts and returns, just from their names:

get_status_from_outcome: accepts outcome and returns status
get_status_from_metrics: accepts metrics and returns status

Otherwise, instead of creating get_status_from_outcome, we could also consider get_status_from_check which would allow us to pass the entire check variable directly, rather than just check['outcome'].

Now let's take a closer look at execute_second_step().

This method uses fetch_metrics. From the name, you'd expect it to simply fetch metrics from somewhere. When the current status isn't CREATED, the method is used exactly that way: it fetches metrics, stores them in a metrics variable, and then the status is determined from these metrics.

However, when the status is CREATED, the function is called without saving its output. This suggests that fetch_metrics is doing something more than just fetching metrics, contrary to what the name implies.

To address this confusion, we need to dive into the implementation of fetch_metrics. Upon inspection, we find that it actually performs two tasks:

It triggers a data pre-processing job if it hasn't been triggered already.
It fetches the pre-processed data and converts it into metrics.

The solution is to refactor fetch_metrics so that the data pre-processing logic is separated into its own function, e.g. trigger_data_preprocessing(). This will make the code more understandable and prevent any misconceptions about what fetch_metrics is doing.

Ultimately, the names of functions and variables should reflect their purpose and intention clearly. By choosing meaningful names, you're essentially translating your plan—"first, I will trigger data processing, then I will fetch metrics, and finally, I will derive the status from these metrics"—into the code itself. The specific details, such as how processing or fetching is implemented, become secondary and abstracted away.

To conclude, the core plan remains clear and easy to follow, ensuring that anyone reading the code understands the overall structure and flow at a glance, much like skimming a book by chapters and then diving into each chapter by sections.

Does it look better? What else can be improved?

a loop is all you need: building conversation ai agents

Adi — Sun, 29 Jun 2025 16:52:44 +0000

After months of building AI agents, I've come to a counterintuitive conclusion: those fancy agent frameworks everyone seems to be using? You probably don't need them.

Let me explain how I got here.

essential complexity is not in tech

A few months ago, I joined an AI startup as founding engineer.

Coming from years of doing data science, I thought I knew what I was getting into. I'd worked with libraries like LangChain, CrewAI, AutoGen, Pydantic AI and taught these agent frameworks at AI engineering bootcamps. I had notebooks full of multi-agent systems, chain-of-thought prompting experiments, and RAG pipelines.

Instead, I found myself reading "Conversations with Things: UX Design for Chat and Voice" by Diana Deibel and Rebecca Evanhoe, published in 2021, long before ChatGPT and modern AI wave.

The book opened my eyes to something I'd been missing: building conversational AI agents isn't primarily a technical challenge. It's a conversation design challenge. And conversations are messy, cultural, and deeply human.

turn-taking is difficult

The first thing that hit me was how bad we developers are at teaching our systems the most basic human skill: knowing when to let the AI speak and when to wait.

In AI there's this concept called "turn-taking" - the invisible dance of who speaks when. Humans are masters at this. We pick up on tiny cues: a slight intake of breath, a change in posture, the way someone's voice drops at the end of a thought. We know instinctively when it's our turn.

But when you're building an agent, you have to explicitly program these decisions: has the user finished their thought? Should the system respond now or wait for more input?

the messaging problem

Modern messaging systems introduces its own weird dynamics. You know that typing indicator - "Adilet is typing..."? That little social contract that says "hold on, I'm not done." But AI agents interact through APIs. They can't see when someone's typing, and they don't trigger the typing indicator themselves. So you get situations like:

User: Hi I have a question
User: It's about my order  
User: The one from last week
AI: [Already responding to the first message]

Some users write novels in single messages. Others send thoughts like morse code - tap, tap, tap. After weeks of experimentation, I still don't have a perfect solution. Do you wait a few seconds after each message to see if more are coming? Do you analyze message length patterns? Do you look for linguistic cues that suggest completion?

cultural turn-taking

I noticed something fascinating from my own multilingual background. Growing up in Kazakhstan, but then studying and working abroad, I've experienced how differently cultures handle conversational pauses.

In Spanish conversations, people overlap constantly - it's not rude, it's engaged. In Kazakhstan, those longer pauses aren't awkward: they're respectful. Now imagine programming a system to handle both styles. A 3-second pause might signal "I'm done" in one culture and "I'm thinking" in another.

The best solution so far I've found combines multiple strategies:

Looking for linguistic completion markers
Waiting fixed amount of time before replying
When it inevitably responds too early, to handle interruptions gracefully

This focus on handling mistakes gracefully brings me to the most important lesson in building conversations AI agents.

the art of failing gracefully

"Conversations with Things" introduced me to the concept of "repair" or what I call recovery. Human conversations go off the rails constantly, but we fix them together almost unconsciously. We clarify, we backtrack, we laugh it off.

Your AI will fail. Not occasionally, but regularly. The question isn't how to prevent failures; it's how to recover from them.

Let me give you a theoretical example. Imagine an AI agent for a food delivery service. A customer says they ordered "the usual" but the system has no record of previous orders (for whatever reason). The agent could crash and burn with "I don't understand 'the usual'" or it could recover gracefully: "I'd love to get you your usual! Could you remind me what that is? I want to make sure I get your order exactly right."

The difference? One leaves the customer frustrated (and makes AI agent seem stupid), the other makes them feel heard while smoothly getting the information needed.

💡 People are surprisingly forgiving of AI mistakes if you handle the recovery well.

This isn't just about having a few canned "sorry" responses. It's about building recovery into every interaction:

Acknowledgment: Don't pretend the error didn't happen
Empathy: Show you understand the inconvenience
Humor (when appropriate): A little self-deprecation goes a long way
Action: Actually fix the problem or offer an alternative

Once you've mastered recovery, you need to think about who exactly is doing the recovering - and that's where personality comes in.

well-defined personality isn't optional

Building a conversational AI agent is not just merely using LLMs to generate correct responses. We all can notice the difference when we interact with Amazon Alexa, Apple Siri, ChatGPT, Yandex Alice (or even Claude Chat / Mistral LeChat – anyone?), but often we cannot explicitly tell what actual differences are. Yet, we all perceive that these assistants have different personalities, which are accurately constructed by developers and designers that make us, humans, feel in an unique way.

People infer personality from the set of things: word choices, voice (sound) and behavior (more about this in the next section). Here's something that surprised me: the personality you give your AI agent fundamentally changes how users interact with it. This isn't about making your bot "fun" or "quirky." It's about creating consistent, predictable interactions that users can adapt to.

This is called "accommodation" or "mirroring" - how people naturally adjust their communication style to match their conversation partner. I noticed users actually start mirroring the agent's communication style. Formal agents get formal responses. Friendly agents get friendly responses. It's like watching a dance where one partner subtly leads and the other follows.

behavior and intents are the key

The trick isn't writing vague instructions like "be friendly and professional." It's defining specific behaviors for specific situations, and from these behaviors emerge the intents your system needs to handle.

But first, what are intents? In conversational AI, an intent represents what the user wants to accomplish or what action the system should take. Instead of trying to handle infinite variations of user input, you define specific set of intents that capture the core actions your agent can perform.

from vague prompts to specific intents

Here's what many developers do wrong. They write prompts like:

You are a helpful customer support agent. Be empathetic and professional.

This is too vague. What does "empathetic" mean in practice? Instead, define specific behaviors that create empathy.
For example, when you want the agent to be "empathetic," what you actually want is the behavior of "always acknowledge the customer's emotion before providing solutions."

This behavior naturally leads to creating intents like

class AcknowledgeEmotion(BaseModel):
    """Recognize and validate customer feelings"""
    emotion_type: Literal["frustrated", "confused", "happy", "angry"]
    intensity: Literal["mild", "moderate", "severe"]

class RequestClarification(BaseModel):
    """Ask for more details when the issue isn't clear"""
    unclear_aspect: str
    suggested_questions: list[str]

These intents emerged from defining how the agent should behave in emotional situations. The desired behavior drives the technical implementation, not the other way around.

conditional intents

Intents can also be conditionally available based on context. For a customer support agent that handles both free and paid users, we can add intent of RedirectToHuman for paid users and CannotRedirectToHuman intent for free users.

if user.account_type == "paid":
    intents.append(RedirectToHuman)
else:
    intents.append(CannotRedirectToHuman)

This prevents the agent from promising human support to someone who hasn't bought anything yet, while still letting it explain why it can't help with certain requests.

useful intents to consider

Besides adding intents based on condition and context, there are some intents that you might consider adding always.

One useful intent might be to handle off-topics:

class RedirectOffTopic(BaseModel):
    """Handle conversations that drift from support issues"""
    topic_type: Literal["personal_chat", "competitor_questions", "philosophical"]
    redirect_strategy: Literal["gentle_reminder", "humor", "firm_boundary"]

This came from the behavior of maintaining professional boundaries while keeping customers engaged. The humor strategy is particularly effective - when someone's clearly testing the system with "What's the meaning of life?", a playful "That's above my pay grade! But I'm great at tracking orders 😊" works much better than a robotic rejection "I cannot talk about this, I can only talk about support" from LLM guardrails system.

Another useful intent might be needed to handle cases where agent doesn't know or cannot provide the answer. This intent will minimize hallucinations better than adding phrases like "Never invent information that is not in your context" to your prompt.

Now that we understand intents and behaviors, let's see how to implement them without complex frameworks.

understanding agents from first principles

Here's where I probably lose half of you: after all this work, I built everything from scratch using just Python and the OpenAI SDK. No LangChain, no CrewAI, no new fancy frameworks.

I've been heavily influenced by Anthropic's definition of agents and HumanLayer's 12-factor agent principles. Both essentially say the same thing: agents are just models using tools in a loop.

Anthropic's definition:

env = Environment()
tools = Tools(env)
system_prompt = "Goals, constraints, and how to act"

while True:
    action = llm.run(system_prompt + env.state)
    env.state = tools.run(action)

HumanLayer's definition:

initial_event = {"message": "..."}
context = [initial_event]
while True:
    next_step = await llm.determine_next_step(context)
    context.append(next_step)

    if next_step.intent == "done":
        return next_step.final_answer

    result = await execute_step(next_step)
    context.append(result)

It is the same pattern: observe context, decide action, execute, update state, repeat. That's it. Everything else is implementation details.

back to basics

There's something beautifully ironic about this journey. When I taught Python for data science at bootcamps, one of the first concepts was loops:

for item in items:
    process(item)

Simple, right? But then when you get to data analysis (pandas, numpy), you spend weeks learning to avoid loops. "Never iterate over DataFrame rows!" we'd say. "Use vectorized operations!"

Now with AI agents, we're back to the most basic pattern:

while not done:
    observe()
    think()
    act()

Sometimes the oldest patterns are the best patterns. There's a certain poetry to it - the most advanced AI technology we have, operating on the most fundamental programming construct.

why not frameworks?

When you use a framework to build an agent, you're adding layers of abstraction over this simple loop. These frameworks are built for maximum flexibility - they need to handle every possible use case from chatbots to autonomous research agents.
But when you only have a specific use case and specific needs, such as:

Fine-grained monitoring of every decision
Precise conversation flow control
Custom error handling for conversation-specific edge cases
Integration with your specific communication channels and your particular tech stack

... those abstractions become obstacles. You spend more time fighting the framework than solving actual problem.

According to HumanLayer's research, most companies building production agents end up rolling their own. After trying both approaches, I understand why. The complexity isn't in the agent loop - it's in your conversation design, intent and error handling, and integration needs. Frameworks can't abstract those away.

customer support agent using first principles

We have been talking about the customer support agent as an example previously.

From the first principles and conversation design ideas, here is the draft implementation:

continue = True
while continue:
    intents = llm.determine_intents(context)
    context.append(intents)

    action = process_intents(intents)

    if action.is_final = True:
        continue = False

    result = await execute_action(action)
    context.append(result)

example flows

Let's walk through two scenarios to see how this works:

Simple FAQ example - "How to cancel my subscription?"

In customer support there is always a set of questions that users ask the most, called FAQs. We can preload the FAQs into LLM prompt, so that agent can respond immediately.

Agent determines intent: ReplyToUser
Processes this intent which results in action of sending message. Action of sending message is final, so the loop will end after execution
Executes the action by sending this message via API
Updates context (conversational history) with this message

Complex example - "When my subscription renews?":

Agent determines intent: FetchUserSubscriptionData
Executes fetch for subscription data
Updates context with user's renewal date
Loop continues with context now containing user data
Agent determines intent: ReplyToUser
Generates and sends personalized response with exact date
Marks action as final, loop ends

The beauty is that more complex scenarios just mean more loop iterations, not fundamentally different code.

context is everything

The final piece in the conversational AI agent development is context engineering.

This term, coined by Dex at HumanLayer, has gained traction recently with endorsements from Shopify CEO Tobi Lütke and even Andrej Karpathy.

The same user message means completely different things depending on context.

User: "Yes".
Without context, this is meaningless. With context:

After "Would you like to see our menu?" → User wants to browse options
After "Is this your current address?" → User is confirming information
After "Should I cancel your order?" → User wants to cancel

Consider the following scenario where context engineering is more than just prompt engineering:

Someone from the customer support team tries to call the user about the issue, but the call fails. We can log that event and add it to our customer support agent's context.
Then our customer support agent sees this within own context and naturally acknowledges it with proper intent: "Hey, it seems like we tried calling you earlier but couldn't reach you. Is now a better time to call?".
After user confirmation, the human from the customer support team calls again and all conversation is transcribed and saved.
If we pass this transcript to customer support agent's context, then the agent can say things like "As you discussed on the phone yesterday with ..."

Crafting the context like that makes the experience feel cohesive.

key takeaways

I'm still early in this journey. Every day brings new edge cases, new failures to recover from, new patterns to recognize. But here is my learnings so far:

Conversation design > technical complexity: Understanding turn-taking, cultural differences, and recovery strategies matters more than the latest framework
Recovery > perfection: Users forgive mistakes if you handle them gracefully. Build recovery into every interaction.
Context > features: A simple agent with rich context beats a complex one without it
Simple loops > complex frameworks: Most production agents are just while loops with good intent handling
Behavior drives implementation: Define specific behaviors first, then derive the technical intents from them

The future of conversational AI isn't in complex frameworks or ever-larger models. It's in understanding the fundamentals of conversation, context, and human interaction. It's in building systems that fail gracefully, adapt to their users, and remember that at the end of the day, they're having a conversation with a human who just wants to be understood.
And sometimes, that's as simple as a while loop that knows when to listen.

These ideas are based on personal experience building production agents. All examples are illustrative, and any views are my own—not those of my employer. If you're building conversational AI, I'd love to hear your thoughts. Are you team framework or team from-scratch?