When Should You Use Reinforcement Learning Instead of Rules?

dennyi konyak — Wed, 23 Sep 2026 06:26:47 +0000

Reinforcement learning is attractive for problems involving repeated decisions, changing environments, and limited resources.

But a complicated decision problem is not automatically an RL problem.

While working on hospital resource allocation, I kept coming back to a simpler question:

What would justify using reinforcement learning instead of a rule-based policy?

My conclusion was simple:

Start with rules. Consider RL only when current decisions meaningfully affect future states, those delayed effects can be measured, and RL can outperform a strong baseline without making important outcomes worse.

When Rules Are Enough

Rule-based systems work well when decisions are mostly determined by clear constraints and priorities.

For example, allocating a hospital resource may depend on:

clinical eligibility,
urgency,
resource availability,
safe waiting time,
available alternatives,
operational constraints.

If these factors produce a stable decision, adding RL may only introduce more complexity.

Rules also have practical advantages: they are easier to inspect, audit, debug, and explain.

And some things should remain rules regardless of the model.

An RL policy should not have to learn that an unavailable resource cannot be allocated or that a hard safety constraint should not be violated. Those constraints should define which actions are allowed before the policy makes a decision.

When RL Becomes Interesting

The stronger case for RL appears when a decision changes what happens next.

Suppose allocating a ventilator now means it will not be available for another emergency later.

Waiting for a predicted release may preserve capacity, but waiting also carries risk.

Using an alternative resource may solve the current request while creating pressure somewhere else.

Now the problem is not simply:

Who should receive the resource?

It becomes:

What decision now leads to the best sequence of outcomes later?

That is the kind of sequential decision problem RL is designed for.

A useful test is:

If I change the current action, does it significantly change the future states and decisions?

If not, RL may be unnecessary.

If yes, it becomes worth investigating.

RL Still Needs a Strong Baseline

One mistake is comparing RL against an intentionally simple rule.

If an RL agent beats “first come, first served,” that does not prove much if a real system would already consider urgency, waiting time, alternatives, or expected resource releases.

The baseline should represent a policy you could realistically deploy.

Only then does beating it mean something.

It is also useful to compare both approaches against an exact solution on small scenarios when possible.

If the rule-based system is already close to optimal, there may simply not be enough room for RL to justify its additional complexity.

Better Reward Does Not Always Mean Better Policy

This was one of the most important lessons from my experiments.

A Q-learning policy could improve its average reward while individual outcome categories became less convincing.

One type of failure decreased while another increased.

The metric improved, but the underlying system had not necessarily improved in the same way.

This is a fundamental RL problem:

The model optimizes the objective you give it, not necessarily the outcome you intended.

So evaluation cannot stop at reward.

You also need to inspect failure rates, waiting times, worst-case behaviour, subgroup outcomes, constraint violations, and the overall state of the system.

A policy that produces a higher reward while moving failures from one category to another has not necessarily solved the problem.

So When Should You Use RL?

I would seriously consider RL when:

decisions happen repeatedly,
current actions affect future states,
delayed consequences can be observed,
complete decision-to-outcome trajectories are available,
a strong rule-based baseline already exists,
the policy can be evaluated safely,
hard constraints remain outside the learned policy,
RL improves real system outcomes, not only its reward,
the improvement holds on unseen scenarios.

Otherwise, rules may still be the better engineering choice.

The question should not be:

Can this problem be formulated as reinforcement learning?

Many problems can.

The better question is:

What does reinforcement learning give us that a simpler rule-based system does not?

If that advantage cannot be clearly measured, start with the rules.

If it can—and the improvement survives strong evaluation—then RL may have earned its place.