Build Smarter AI Agents: Use Deterministic Code Before Calling an LLM

There is a tempting assumption when building AI-powered applications: if an LLM can make a decision, why not ask it to make that decision every time?

It sounds reasonable. The model is already part of the application, it can interpret context, and it can handle a wide range of situations.

But there is a catch.

Not every problem needs reasoning. Some problems just need code.

If an AI agent repeatedly solves the same problem, it may be time to turn that solution into a deterministic function. Doing so can reduce inference costs, improve response times, and make the system more predictable.

This is a small architectural decision with potentially significant benefits.

The problem: asking an LLM to solve the same thing repeatedly

Imagine an AI agent that investigates failed CI/CD pipelines.

Whenever a build fails, the agent:

1. Reads the pipeline logs.
2. Interprets the error messages.
3. Investigates the likely cause.
4. Recommends what to do next.

This is a useful application of an LLM. Build logs can be long, unfamiliar, and sometimes ambiguous. The agent may need to connect several clues before identifying the root cause.

Now imagine that the same known error appears repeatedly:

error NU1301: Unable to load the service index for source.

After investigating the issue, the agent determines that the package feed is temporarily unavailable and recommends retrying package restore.

The next time the same failure occurs, should the application ask the LLM to investigate everything again?

What about the hundredth time?

Probably not. Once the pattern is understood, we can handle it with ordinary application code.

Turn a known decision into deterministic code

For example, we could introduce a small C# function that recognizes known build errors:

public string AnalyzeBuildFailure(string logs)
{
    if (logs.Contains("NU1301", StringComparison.OrdinalIgnoreCase))
    {
        return "NuGet feed may be unavailable. Check connectivity and retry package restore.";
    }

    if (logs.Contains("CS0246", StringComparison.OrdinalIgnoreCase))
    {
        return "A type or namespace could not be found. Check using directives and project references.";
    }

    if (logs.Contains("401 Unauthorized", StringComparison.OrdinalIgnoreCase))
    {
        return "Authentication failed. Check the credentials or token used by the pipeline.";
    }

    return "Unknown failure. Further investigation is required.";
}

This is a simplified example; a production implementation should account for log structure, error context, false matches, and the possibility that one error message has multiple causes.

The important idea is that known patterns can be handled without an LLM call.

Instead of sending every failure to a model, the application can first check whether it recognizes the issue. If it does, it follows a tested path. If it does not, it can escalate the problem to the LLM.

That gives us a useful division of responsibility:

• Code handles known patterns and explicit rules.
• The LLM handles unfamiliar situations, ambiguity, and reasoning.

What do we mean by deterministic code?

Deterministic code produces the same result for the same input, assuming the same relevant state and execution environment.

For example:

public decimal CalculateTotal(
    decimal subtotal,
    decimal taxRate)
{
    return subtotal + (subtotal * taxRate);
}

Given the same values, this function returns the same result. There is no need for an LLM to interpret the request or generate a calculation.

Compare that with asking an LLM to calculate an invoice total. The model may produce the correct answer, but it introduces unnecessary uncertainty into a task that should be handled by precise, testable business logic.

This distinction matters:

Use deterministic code when...	Use an LLM when...
The rules are known and explicit	The rules need to be inferred from context
The result must be precise and repeatable	The input is ambiguous or unstructured
The task involves calculations or validation	The task involves interpretation or summarization
The same decision is made frequently	The situation is unfamiliar or requires flexible reasoning

An LLM is not a replacement for every function in an application. It is a tool for the parts of a problem where language understanding and flexible reasoning add value.

Two principles for designing efficient AI systems

There are two practical principles worth keeping in mind.

1. If you can reliably express a decision as rules, use code

Consider a small business invoicing application.

Some features might include:

• Calculating invoice totals.
• Applying a predefined discount.
• Validating required customer information.
• Checking whether an invoice is overdue.
• Updating an invoice's payment status.

These are ordinary business rules. They belong in application code, supported by unit tests.

Other features may benefit from an LLM:

• Understanding a request such as, "Show me the invoices that are still outstanding from last month."
• Summarizing a customer's message.
• Turning an informal instruction into a structured draft invoice.
• Explaining an unfamiliar error in plain language.

The goal is not to make the application less intelligent. It is to use the right tool for each responsibility.

2. If an LLM keeps solving the same problem, consider turning the solution into code

Suppose an agent repeatedly investigates API failures and discovers that a particular class of failures is caused by expired access tokens.

Initially, the LLM may be useful because it needs to examine logs and understand the situation.

But once the pattern is confirmed, checking token expiration is straightforward:

public bool IsTokenExpired(DateTime expiresAtUtc)
{
    return expiresAtUtc <= DateTime.UtcNow;
}

The application can now perform this check directly.

The same idea applies to repeated decisions in many domains:

• Recognizing known CI/CD failures.
• Validating API request structures.
• Applying business rules.
• Checking authorization conditions.
• Detecting predefined data-quality issues.
• Routing requests based on known categories.

A repeated LLM decision is an opportunity to ask: Can we capture what we have learned as a reliable software rule?

A hybrid architecture: deterministic code plus LLM reasoning

A practical AI agent does not need to choose between traditional code and an LLM. It can use both.

Consider a simplified workflow for our CI failure investigation agent:

                 CI Pipeline Failure
                          |
                          v
                Read and collect logs
                          |
                          v
                 Check known patterns
                          |
                +---------+---------+
                |                   |
             Recognized          Unknown
                |                   |
                v                   v
          Deterministic         Send context
             handler             to LLM
                |                   |
                v                   v
          Apply known fix       Investigate
          or recommendation    and reason
                |                   |
                +---------+---------+
                          |
                          v
                  Validate the result
                          |
                          v
                 Report the outcome

The deterministic path handles situations the system already understands. The LLM path handles cases that need further investigation.

There is an important engineering detail here: recognizing a known pattern is not the same as proving its cause.

For example, an error code may have multiple possible causes. A rule-based handler should only take an automated action when the evidence and risk level justify it. Otherwise, it should provide a safe recommendation or escalate to the LLM or a human.

This is why validation, tests, observability, and clear fallback behavior remain essential.

AI-assisted development makes this approach easier

There is another interesting part of this idea: AI coding tools can help us create the deterministic code that reduces future LLM usage.

For example, a developer can ask an AI coding assistant to:

1. Review recurring agent decisions or known failure cases.
2. Identify decisions that can be expressed as explicit rules.
3. Propose a small function or service to handle them.
4. Generate unit tests, including edge cases.
5. Refactor the agent to call the new function before invoking the LLM.
6. Help compare behavior before and after the change.

The generated implementation still needs to be reviewed and tested. An AI-generated function is not automatically correct just because it is deterministic.

But AI can reduce the effort required to turn a discovered pattern into maintainable, tested software.

This creates an interesting feedback loop:

1. The LLM investigates an unfamiliar problem.
2. Engineers verify the diagnosis.
3. A recurring pattern is identified.
4. The pattern is implemented as deterministic code.
5. Future occurrences use the code path.
6. The LLM is reserved for genuinely new or ambiguous cases.

Over time, the system can handle a larger share of familiar situations without inference.

Does this mean future AI agents will use fewer LLM calls?

Potentially, yes.

A more capable agent does not necessarily need to make more model calls. In a well-designed system, repeated reasoning can gradually become reusable software.

That does not mean every LLM interaction should be replaced with a rule. Real-world inputs change, edge cases appear, and some decisions remain inherently ambiguous. The system still needs a way to handle cases that do not match known patterns.

The objective is not to eliminate LLM calls. It is to make each call worthwhile.

A useful way to think about the architecture is:

• Use deterministic code for precision, repeatability, and known business logic.
• Use LLMs for interpretation, ambiguity, and unfamiliar problems.
• Use tests and validation to make sure both paths behave safely.
• Keep a fallback path for cases the deterministic logic cannot confidently handle.

Final thoughts

One of the biggest opportunities in AI-assisted development may not be adding AI to more parts of an application. It may be using AI to build better software around the places where AI is genuinely needed.

When an LLM solves a problem once, it provides an answer.

When we understand that solution, verify it, and turn it into tested code, we create something reusable.

That code can run repeatedly, predictably, and without another inference call.

The best AI architecture may not be the one that calls an LLM for everything. It may be the one that knows exactly when not to.

And perhaps one of the most valuable uses of AI is helping us write the code that makes future AI calls unnecessary.