ANALYTICS_QUERY Isn't a State: Plugging an LLM Into a WhatsApp FSM

SokoFlow AI Build Log, Week 3 of 6: intent vs state, a routing policy I didn't plan for, and the latency number I owed you from Week 2

The spec said: add the ANALYTICS_QUERY intent to the existing FSM. One line.

I spent days on it. The code wasn't hard. I just couldn't tell whether ANALYTICS_QUERY was an intent, a state, or something else, and every wrong answer would have broken a different part of a system that already worked.

The answer turned out to be "mostly neither", and finding it is what made the rest of the week simple.

TL;DR

• Week 3 connected last week's loop (LLMClient, ToolRegistry, ToolExecutor, plus the Orchestrator) to the existing webhook and FSM, and wired the remaining four tools.
• ANALYTICS_QUERY is handled in IDLE's intent dispatch, not registered as a SessionState. Nothing ever waits in an "analytics" position. The one exception is product disambiguation, which is a real parked state.
• I kept Intent.UNKNOWN and added a routing policy above the resolver. It leans toward sending messages to the LLM and lets the model say "no tool".
• Follow-ups like "and yesterday?" are context, not state: I store the last validated tool call and let the model propose a new one.
• Closing Week 2's gap: end-to-end p50 of 1.20 s and p95 of 2.27 s on 7 requests. That's a diagnostic baseline, not a production number.

Where we are

SokoFlow is a headless WhatsApp ERP for Kenyan SMEs. I've been building it for about five months. There's no app and no UI to learn; for a shopkeeper, the conversation is the interface. Phase 3 adds one AI feature: natural-language shop analytics. A shopkeeper asks "which products are running low?" or "how does this week compare to last week?", and an LLM, using function calling against SokoFlow's existing services, figures out which query to run.

Last week I built the core function-calling loop:

• LLMClient: the provider gateway and boundary.
• ToolRegistry: the source of truth for every tool SokoFlow exposes to the AI layer.
• ToolExecutor: the execution boundary that runs a suggested tool call with trusted context.

This week the job was to connect all of it to the real conversation, so that a shopkeeper's message can enter through the webhook and flow through the existing finite state machine (FSM).

Tracing the existing logic

First wall: where do I plug this in without rebuilding everything? The spec's Week 3 target read:

Add the ANALYTICS_QUERY intent into the existing FSM/webhook. Handle session context so a follow-up question ("and yesterday?") can reuse prior context. Test via your existing GUI/CLI simulators.

It wasn't as simple as adding an intent, so I first had to retrace how the current system handles a message. Every incoming message enters through the webhook and is enqueued to the conversation_task queue, where a worker picks it up and runs this sequence:

1. Retrieve the session from Redis, or create one if none exists.
2. Process the message by calling fsm_engine.process_message().
   • The engine has a handler map: SessionState → a handler that produces an FSMResult.
   • Handlers handle transitions and call application services for CRUD and business logic.
3. Save the updated session back to Redis. An atomic Lua script uses CAS (compare-and-swap) against the expected old state, which prevents race conditions.
4. Render the reply with render_message(...), using the shop's locale and the message key.
5. Send it through the configured medium, mock or WhatsApp.

ANALYTICS_QUERY: intent or state?

The spec calls it an intent, and that confused me while I was looking for where to integrate it. Making it a SessionState would put it on the same level as our existing FSM states, like ADD_PRODUCT_PRICE or CHECK_STOCK, and wire it into the handler map:

self._handler_map: dict[
    SessionState, Callable[[UserSession, str], Awaitable[FSMResult]]
] = {
    SessionState.ADD_PRODUCT_NAME: self.add_product_flow.handle_name,
    SessionState.ADD_PRODUCT_PRICE: self.add_product_flow.handle_price,
}

# Get the correct handler based on state. Default to IDLE.
handler = self._handler_map.get(session.state, self._handle_idle)

But that approach didn't fit, and it forced me back to the difference between the two ideas:

• Intent is what the user wants, classified from a message.
• SessionState is where the conversation is parked, which decides how the next message is interpreted.

An entry in that handler map means "the session is waiting here, and this handler receives the next message." Analytics never waits. A question arrives in IDLE, gets fully processed, and the session is IDLE again. An ANALYTICS_QUERY entry in the map would be misleading, because nothing ever parks there.

So analytics belongs in the intent-dispatch step that IDLE already does. That first clarification made everything after it simpler.

UNKNOWN vs ANALYTICS_QUERY

SokoFlow has one special intent. When the resolver can't confidently classify a message, it produces Intent.UNKNOWN, and the system re-prompts the user with the available options.

That's exactly what happens today if a shopkeeper types something like "How much did I sell last week". The regex-based resolver can't match a message that free-form to an intent.

Now that analytics routes unclassified messages to the LLM, why keep UNKNOWN at all? Why not delete it, or keep it and send everything in it to the model? I decided to keep it. UNKNOWN means "no regex matched", which is a fact about the resolver. "Therefore send it to the LLM" is a routing policy, and policy belongs above the resolver. My reasons for keeping them separate:

• Control (and, a little, cost): otherwise every greeting, typo or stray emoji becomes a model call. As you'll see below, cost turned out to be the weaker half of this argument.
• The model has its own "unknown": the LLM can return no tool call at all ("hi", "what's the weather"). That's the real analytics-versus-not distinction, and it needs a defined outcome that maps to the help reply.

Notice that I just introduced a routing policy. That's a component I'll come back to.

Ambiguity broke my "one-shot" assumption

I was settled on analytics being purely one-shot. Then ambiguity contradicted that.

SokoFlow already has one disambiguation branch. Users write in natural language, so we have to map their words to something the backend understands. A shopkeeper checking stock simply says "Milk". But a shop can realistically have Milk 250ml and Milk 500ml, which are different products with different prices. So the system runs a database-backed fuzzy match and asks the user which one they meant.

That's a genuine parked state: the flow pauses, goes back to the user, and continues when they reply. So that case belongs in the handler map, as a state for picking from candidates.

The overall idea:

• Happy path: stateless. IDLE → analytics → IDLE.
• Disambiguation branch: one real state. It's entered when the outcome says "needs selection", with candidates from the fuzzy match. The cardinality logic stays in domain services.

The interception point

With that settled, I knew where the new flow plugs in. The updated flow:

1. Retrieve the session.
2. The FSM engine processes the message:
   1. The resolver tries regex first, and in IDLE falls back to analytics.
   2. The analytics handler passes the message and the last tool call to the Orchestrator.
   3. The Orchestrator coordinates LLMClient and ToolExecutor and returns a typed outcome.
   4. The ResponseFormatter produces FSMResult(message_key, params), and the handler also records the new last-call context.
3. CAS-save the session.
4. render_message renders the text in the shop's locale.
5. The message sender sends it.

Not a lot changed, and that was the beauty of it. But it did involve a few new components.

The adapter, or: two things called "handler"

In step 2 I mentioned an "analytics handler", and for a while that name confused me, because my mental picture of a handler was the handler map keyed by SessionState.

Analytics is different. The IDLE handler calls the analytics handler directly, gets an FSMResult back, and the session stays IDLE. Nothing needs the map, because no future message needs special interpretation. So the analytics handler is simply a collaborator of the IDLE handler, the way a service is, much like the existing stock_lookup_flow. It has two kinds of entry point:

• One called by intent dispatch (the query itself). It's never registered in the map.
• One registered in the map for the parked disambiguation state.

So "handler" in the handler map and "handler" in the analytics design were two different roles sharing a name.

The ResponseFormatter

This one felt odd at first. SokoFlow already has render_message for i18n, so I assumed that was enough. Why add a component?

• render_message is a template lookup: a key plus flat params in, a locale-specific string out. It's deterministic and knows nothing about analytics.
• ResponseFormatter bridges the gap between variable-shaped domain data and that flat template world. It picks the message_key (summary, top-N list, trend, or empty result) and builds the params: currency and percentage formatting, date display, truncating a long list to fit a WhatsApp message, and so on.

This is also where I locked in a decision I flagged back in Week 1. In most tool-calling tutorials, once the business logic runs, the result goes back to the LLM to write a friendly reply. I'm deliberately not doing that. The LLM stays out of the path that produces numbers. That means no hallucinated figures, no wording that drifts between locales, and no second model call adding latency and cost.

The boundary: the LLM routes and extracts arguments; application code owns everything the shopkeeper reads.

Control, ownership and boundaries

By now there were many components, and it helped to see how they relate and how control flows. I designed them as nested layers, where each layer only knows the one directly below it:

Worker
 └─ FSM engine
     └─ Analytics handler        (knows: session, FSMResult, Orchestrator, Formatter)
         └─ Orchestrator         (knows: LLMClient, ToolExecutor)
             ├─ LLMClient        (knows: the model provider)
             └─ ToolExecutor     (knows: the tool registry, domain services)

• The handler controls the Orchestrator and the Formatter, calling them in sequence.
• The Orchestrator controls LLMClient and ToolExecutor, and is the only component that knows both exist.
• LLMClient and ToolExecutor are peers and never call each other. LLMClient has no idea tools get executed, and ToolExecutor has no idea an LLM chose the call. That's what makes each swappable and testable alone, and it's why last week's boundary contracts mattered.

Control flows down and results flow back up. No component calls upward, and none skips a layer. That separation paid off during implementation, especially in unit tests, because every component was isolated.

I introduced the Orchestrator last week, and it was the missing piece that stopped me from measuring a full end-to-end loop. This week it comes to life.

What happens inside one Orchestrator call

This is the part my earlier diagram hid. The Orchestrator runs a small sequence and owns every decision in it:

1. Ask LLMClient to turn the question (plus prior context) into a tool call.
2. Inspect the response. Is it a valid tool call, an invalid tool name, or no tool call at all?
3. If invalid, apply the bounded retry. If valid, hand it to ToolExecutor.
4. Inspect what ToolExecutor returns: success data, a validation failure, or a domain signal like "ambiguous product, here are the candidates".
5. Package everything into one typed outcome and return it.

The routing policy I hadn't planned for

I didn't foresee this component, and it turned out to be one of the most important parts of the integration.

The distinction is about which layer decided. The resolver says UNKNOWN. The policy then decides whether to escalate that message to analytics. So where does the policy live?

Option A: route inside _handle_idle(). When the resolver returns UNKNOWN, the FSM invokes the analytics handler instead of immediately raising InvalidInputError.

Option B: route after process_message() returns. The worker sees MessageKey.UNKNOWN_INTENT and decides whether to invoke analytics.

I chose Option A, because routing is a decision about what a message means, and that already belongs to intent dispatch in IDLE. Option B was worse than it looks. By the time process_message() returns UNKNOWN_INTENT, the FSM has already treated the message as an error:

• the error counter may have been incremented,
• the session context may have been mutated,
• an FSMResult carrying the "didn't understand" key has already been built.

For the worker to then invoke analytics, it would have to undo or override all of that. It would also need a second session update (the last-call context) after the first, which means either two CAS saves or a worker editing another component's result. Both break the uniform pipeline (retrieve, process, save, render, send), and the worker would start carrying domain knowledge about when "unknown" really means "analytics".

What belongs in the policy

One thing worried me. The policy looked like it needed to act as a classifier, to make sure routing decisions were accurate. But deciding whether a message is an analytics question is an intent-classification problem, which is the exact hard thing I was trying to hand to the LLM.

It doesn't have to be. The policy only decides whether a message is allowed and worth sending to the LLM, which is a much easier question.

I also resisted the tempting keyword prefilter: "message contains sales, stock, today…". Shopkeepers write in English, and often mix in Swahili and Sheng. A keyword list would silently reject legitimate questions, in ways that are hard to notice.

Why an imperfect policy is fine

The two possible mistakes have very different costs:

Mistake	Consequence	Severity
Escalate a non-analytics message	One model call. The LLM returns no tool call, the user gets the normal unknown reply, and the counter increments.	Low, and the outcome is the same one they'd have had anyway
Fail to escalate a real analytics question	The user gets "I didn't understand" for a question we could have answered	Higher, and it erodes trust in the feature

So the policy should lean toward escalating and let the LLM reject. The cost argument for gating is also weak here: at roughly $0.00005 per call with the model I picked in Week 1, a wasted call is almost free. The real reasons for having a policy are rate limiting, availability, and keeping the feature switchable.

Session context and multi-turn follow-ups

At this point there's a solid entry point and a routing policy. Next came multi-turn follow-ups. Minutes after asking "How much did I sell today?", the shopkeeper types: "And yesterday?"

Without context, that's impossible to answer. I took inspiration from the MVP's ambiguous-product logic, which stores candidates in the session context and uses them when the user replies with their choice. This problem is similar, and the framing that came out of it is: follow-up memory is context, not state. "And yesterday?" arrives with the session in IDLE, gets escalated by the routing policy because fresh context exists, and the model resolves it against the previous request.

The decisions:

• Store the last structured call (tool name and validated arguments) in its own context slot, with a timestamp. With a 600 second TTL to ensure the context stays fresh.
• It's not raw chat history and it's not result data. It's small, bounded, already validated and language-agnostic, and the resulting tool call can be asserted in tests.
• The model resolves the ellipsis by proposing a new call, typically the same tool with a changed period or product. That new call goes through normal validation.

Disambiguation is a real parked state. When the outcome is NeedsSelection, the session parks in an analytics product-selection state, registered in the handler map with the same shape as the existing stock-lookup selection state:

• The session stores the pending validated call and the candidate list.
• When the user chooses, the selection handler completes the pending call and executes it directly. It does not go back to the model, because the model's job is done and re-asking adds latency, cost and interpretation risk.
• After completion, the same last-call context update as the happy path runs, so follow-ups work after a disambiguated query too.

Wiring the rest of the tools, and the latency number I owed

Last week I wired the first three tools (get_sales_summary, get_stock_level, get_top_products) through the executor but not into the WhatsApp flow. This week I wired the remaining four: get_low_stock_items, get_product_price, get_sales_trend and get_slow_moving_items.

Measuring latency

I also came back to Week 2's latency target, which I'd postponed until the WhatsApp integration existed. Instead of benchmarking a pile of temporary workarounds, I built a repeatable benchmark around the integrated request path.

It uses the real Groq API (openai/gpt-oss-20b), the application's actual tool executor and services, and a controlled test environment. It measures the time from entering the conversation handler to producing a formatted reply at the outgoing-message boundary, excluding actual WhatsApp delivery.

In my first run, all seven analytics queries succeeded, with no timeouts and no incorrect tool selections. The numbers:

• p50: 1.20 s
• p95: 2.27 s
• Slowest request: about 2.55 s

That closes the Week 2 requirement to measure and record p95. But seven requests are enough for an initial diagnostic baseline, not a statistically reliable estimate of production performance. The numbers also reflect my current test conditions, including a fairly limited internet connection. I'm treating this as a baseline to guide further testing, not a production latency figure.

What's next

We're past the halfway point. Week 4 is robustness and guardrails: fallbacks for unresolved queries, per-shop rate limiting, and cost-cap enforcement. Then the interaction logger, so failures are diagnosable and the system is more observable.

Read more: the 4-month MVP build log

Check out the repo: SokoFlow on GitHub