First Steps in AI Engineering: Improving the simple chat bot

Intro

Following up on my last post where we built that simple CLI chatbot from scratch, I spent a bit more time looking over the code and realized a few things. Building something for the first time is great, but going back and refining it is where the real learning happens.

I noticed a couple of areas where I could optimize how we handle the chat history, dig a little deeper into what the Anthropic SDK actually gives back to us, and, you know, actually handle errors instead of just hoping for the best. Refactoring is just part of the process. Let's look at some quick tweaks and improvements to make this little AI agent a bit more robust!

We don't need a starting "hello" from the assistant

That seems to also be wasting input, in that we add that to the history. We can just keep the starting messages list containing the first user prompt, no biggy.

Exploring the API response in more depth

The api sdk for anthropic has some cool stuff in there, I didn't get a chance to show it off, but I will show an example of what they are here:

    for block in response.content:
        if block.type == "text":
            messages.append({
                "role": "assistant",
                "content": block.text
            })
        if block.type == "thinking":
            print(f"[assistant] {block.thinking}")

Just like what we saw the raw body in the first part of this blog, we can see that we also have the thinking block here displayed. We can use that to our advantage to show that the assistant is thinking, it can also be used later on to analyse how it got to a certain decision. I can imagine organisations wanting to audit it's decisions and perform evaluations on it's thought process, to help further refine it.

Other common properties that exist in that body are model, input_tokens and output_tokens , and plenty more. The point is that we can use properties like these at some point to help us point in the right direction when it comes to building an AI app with a particular purpose.

Handling exceptions and guarding against empty inputs

One thing I fail to appreciate in every programming language is that the way errors are handled look different but mostly remain the same style. In this case with Python and using the Anthropic SDK is no different. In any programming language, it is the norm to try and familiarise myself with the type of errors that would come up from an SDK for a specific action.

And this is what we're doing at the moment, while looking at .venv\Lib\site-packages\anthropic\_exceptions.py. The structure of most of these errors that we may be interacting with is this. It shows the errors mentioned with where they inherit from.

AnthropicError
 └─ APIError
     ├─ APIConnectionError        (network unreachable)
     │    └─ APITimeoutError      (request timed out)
     └─ APIStatusError            (server responded with 4xx/5xx)
          ├─ AuthenticationError      401 — bad/revoked key
          ├─ RateLimitError           429 — sending too fast
          ├─ ServiceUnavailableError  503 — Anthropic overloaded
          ├─ OverloadedError          529 — Anthropic overloaded
          ├─ BadRequestError          400 — malformed reques
          └─ InternalServerError      5xx — Anthropic's fault

All of these can be imported from the package, and they can be used in the standard try, catch blocks, to catch a specific error that would otherwise be caught in general by the run time. I tested this with the AuthenticationError, and imported it like so: from anthropic import AuthenticationError I stuck the catch o the outer loop of the turn based code block, to make sure that it A) doesn't propagate any further upwards and expose some vulnerable stack trace details, but B) close enough for the exception to not aggregate into a larger one.

We can also save ourself the headache of creating another catch block for empty strings going into the sdk by simply ensuring that there are no empty strings that go in:

question = user_question()
if not question.strip():
    print("(empty input — try again)")
    continue 

Conclusion

And that wraps up this round of tweaks! We managed to trim some unnecessary fat from the message history, unlocked a peek into the LLM's actual thought process, and put a safety net in place with proper exception handling.

It's funny how just a few small changes can make a simple script feel way more like a real, production-ready application. Digging into the SDK like this really opens up the possibilities for what we can build next, especially when it comes to tracking token usage or auditing how the AI makes decisions. I'll definitely be keeping these patterns in mind for future AI projects. Catch you in the next one, happy coding! 🙌🏻