When Should Software Refuse to Continue?

A program reaches a state it did not expect. It can still execute instructions. Nothing has crashed yet. Should it continue?

I don't think there is a single answer.

If software knows it is making an assumption, silently guessing what the user intended is questionable. For a reversible operation, continuing may be reasonable if the uncertainty and its cause are clearly shown. Irreversible actions deserve a different threshold. Sometimes confirmation makes sense; sometimes the operation should simply be rejected.

That logic changes once the system itself becomes unreliable. Memory corruption, a security compromise, or an undefined internal state creates a different problem. A system in an undefined state cannot confidently decide that an action is reversible, because the mechanism making that judgment may itself be affected. Returning to a known-good state is preferable whenever possible. Otherwise, stopping may prevent corrupted execution from travelling further.

But stopping is not automatically the safe option either. In safety-critical systems such as avionics or medical equipment, suddenly removing a function can itself create danger. The design may need redundancy, isolation, degraded operation, failover, or another predefined recovery path that keeps essential functions available rather than simply terminating everything.

Asking the user isn't an escape from every decision. Waiting for an answer can leave an application blocked or preserve a dangerous state. Some decisions must happen before a user could reasonably intervene. Even an application that wants to continue exists under a larger authority; if the operating system terminates it to protect the machine, it cannot simply override that decision.

So I don't see "always continue" or "always stop" as useful absolute rules. Predicting the next state is itself a probability problem.

The harder engineering question is deciding what we are protecting—the current work, the data, the application, the wider system, or continued operation itself—and whether the system is still trustworthy enough to make that choice.