Why I Built a Local AI Assistant Instead of Yet Another Chatbot

Jiwan — Tue, 04 Aug 2026 12:41:11 +0000

When I decided to build a generative AI service, one question came to mind almost immediately:

“Do we really need another chatbot?”

General-purpose AI systems like ChatGPT can already do an incredible number of things.

They can answer questions, write content, generate code, analyze documents, and help with countless everyday tasks.

Simply connecting an LLM API to a chat interface didn't feel like enough of a reason to build another AI product.

So I started asking a different question:

What if, instead of trying to know everything about the world, an AI system were designed to understand one city really well?

That question became the starting point for Hey Daejeon, a local AI project built around the city of Daejeon, South Korea.

What General-Purpose AI Often Misses

General-purpose AI knows a lot.

But the information we need in everyday life is often surprisingly local.

Which local support programs are currently accepting applications?

What administrative changes are happening in a particular neighborhood?

Where can I go this weekend?

Which local organization can help with a specific problem?

The challenge is that this information rarely exists in one clean database.

It is scattered across municipal websites, public datasets, institutional announcements, news articles, and countless web pages.

And much of it changes constantly.

That led me to think about the problem differently.

Maybe the problem isn't that AI isn't intelligent enough. Maybe it isn't local enough.

Instead of trying to give an AI system even more general knowledge, I wanted to explore what would happen if it became better at working with the information and context of a specific region.

The region I chose was Daejeon.

What If AI Were Designed Around a City?

The original idea behind Hey Daejeon was relatively simple:

Use data generated in Daejeon for the people and communities of Daejeon.

But simply adding something like “You are an expert on Daejeon” to a system prompt obviously wasn't enough.

Different questions require different kinds of information.

Some questions can be answered with the general knowledge of an LLM.

Others require current information from the web.

Some require local datasets.

And some require combining multiple sources.

So instead of designing Hey Daejeon around one massive prompt, I started building it as a system that could choose different paths between the user, data sources, and the LLM.

At a high level, the flow looks something like this:

User Query → Intent Analysis → Routing → Data/Search → LLM → Post-processing & Guardrails → Response

At first, this looked fairly straightforward.

But once I started implementing it, this became one of the most interesting parts of the entire project.

Calling the LLM Wasn't the Hard Part

Calling an LLM API is relatively easy.

The difficult questions appeared around it.

Does this query require search?

Should the system use local data?

Can this request go directly to the general-purpose LLM?

What happens when the retrieved information isn't good enough?

How do I detect when the router makes the wrong decision?

How do I know whether the system is reliable enough to deploy?

As the service grew, these questions kept multiplying.

And gradually, orchestration became one of the central parts of Hey Daejeon.

The system needed to understand the request, select an appropriate processing path, retrieve the necessary information, pass the right context to the model, and handle the result.

While building this, I started experimenting with components such as heuristic routing, LLM-based routing, guardrails, and internal evaluation.

One thing became increasingly clear:

As an AI system becomes more complex, adding new capabilities isn't always the hardest problem.

Reducing wrong decisions often matters more.

An AI Product Is More Than Its Model

Building Hey Daejeon also changed the way I think about AI products.

When I first became interested in generative AI, I naturally focused heavily on models.

Which model is better?

How many parameters does it have?

How well does it perform on benchmarks?

Those questions still matter.

But when I started building an actual service, a different set of problems kept appearing.

How should user requests be classified?

Which data source should be selected?

When should the system search?

What happens when the request is routed incorrectly?

How should system quality be measured?

What should be tested before deployment?

Eventually, I realized that I wasn't just building an application that uses an LLM.

I was building an AI system in which the LLM is only one component.

That distinction turned out to be much more important than I expected.

It's Not a Perfect System

Hey Daejeon is far from finished.

The routing can be improved.

The evaluation methodology can become more rigorous.

The coverage of local data is still limited.

And real users will almost certainly discover problems that I haven't encountered yet.

But that's also what makes the project interesting to me.

I didn't start with a perfect architecture.

Instead, the system evolved through a repeated cycle:

Find a problem → Form a hypothesis → Build → Test → Fail → Improve

That process has gradually taught me what it actually takes to turn LLMs into a working AI service.

And many of the most useful lessons came from the parts that didn't work the first time.

Local AI as an Experiment

For now, I think of Hey Daejeon as an experiment.

Does AI designed around a specific region still make sense as general-purpose AI becomes increasingly powerful?

What happens when local data is combined with general-purpose LLMs?

Can a city itself become a useful boundary for an AI service?

And could AI become more than something that simply answers questions about a region?

Could it become an interface that helps turn data generated within a region back into something useful for that region?

These are the questions I want to explore through Hey Daejeon.

What I'll Be Writing About Next

In this first post, I wanted to focus more on why I built Hey Daejeon than exactly how I built it.

In future posts, I plan to go deeper into the engineering behind the project:

the overall system architecture, how queries are routed, heuristic routing versus LLM-based routing, guardrail design, internal evaluation and pre-deployment testing, and the technical debt and failures I encountered while building the service.

Rather than only showing the finished product, I want to document the decisions behind it:

what I built, why I designed it that way, what failed, and how I tried to improve it.

Hey Daejeon started with a simple question:

What would happen if an AI system were designed to understand one city really well instead of trying to understand the entire world?

I'm still finding out.

DEV Community: Jiwan