DEV Community

Alex Chen
Alex Chen

Posted on

Learn Outage Fallback With a Tiny Python State Machine

Suppose a task times out after you send it to a model. Is it safe to send it again? The surprising beginner answer is “not yet,” because timeout describes what the client observed, not what the server completed.

A real timeline makes the lesson concrete. OpenAI reports that one July 25 incident began at 09:17:49 UTC, reached mitigation monitoring at 10:02:52, and resolved at 11:08:36. Another incident started at 11:35:24. At research time, the second was identified with elevated errors and mitigation in progress; the status site showed Partial System Degradation. These facts do not tell us a root cause, global reach, exact number of people affected, or how the later event finally ended.

We can model the safe choices without calling any API.

Prerequisites and states

You need Python 3.11 or newer. The complete example below is unexecuted, so the output is expected output rather than a test result.

from dataclasses import dataclass
from enum import Enum, auto

class State(Enum):
    LOCAL = auto()
    SENT = auto()
    UNKNOWN = auto()
    PAUSED = auto()
    SWITCH_READY = auto()
    DONE = auto()

@dataclass
class Task:
    state: State = State.LOCAL
    provider: str = 'primary'
    request_id: str | None = None

    def send(self, request_id: str):
        if self.state not in {State.LOCAL, State.SWITCH_READY}:
            raise ValueError('task cannot be sent from this state')
        self.request_id = request_id
        self.state = State.SENT

    def timeout(self):
        if self.state is not State.SENT:
            raise ValueError('only a sent task can time out')
        self.state = State.UNKNOWN

    def reconcile_missing(self):
        if self.state is not State.UNKNOWN:
            raise ValueError('reconcile an unknown attempt first')
        self.state = State.PAUSED

    def choose_fallback(self, provider: str):
        if self.state is not State.PAUSED:
            raise ValueError('pause before switching')
        self.provider, self.request_id = provider, None
        self.state = State.SWITCH_READY
Enter fullscreen mode Exit fullscreen mode

Try this driver:

job = Task()
job.send('attempt-1')
job.timeout()
job.reconcile_missing()
job.choose_fallback('alternate')
print(job.state.name, job.provider)
Enter fullscreen mode Exit fullscreen mode

Expected output:

SWITCH_READY alternate
Enter fullscreen mode Exit fullscreen mode

The important intermediate state is UNKNOWN. Jumping directly from timeout to fallback can duplicate work. reconcile_missing() is deliberately named to remind us that a real application needs evidence—such as a provider lookup or an operator decision—before replacing the attempt.

One error input

Run this after constructing a fresh task:

Task().choose_fallback('alternate')
Enter fullscreen mode Exit fullscreen mode

Expected result: ValueError: pause before switching. If it does not fail, the state machine allows an unreviewed provider change.

A second provider creates semantic risk, not just a new endpoint. Models may interpret the same prompt differently, support different tools, or produce incompatible structured data. A learning extension is to add VALIDATING and require three provider-neutral fixtures before DONE.

Common mistakes include treating status-page color as a command to retry, storing only the current state instead of transitions, and assuming a different model preserves meaning. Persist each transition with its request ID in a real program.

Two ways to inspect a larger system

After understanding the toy machine, learners can evaluate the overseas MonkeyCode hosted option. The page currently shows “Start free.” Its official README describes managed server-side cloud environments with build, test, and preview plus integrated models. It is currently free to start; model/server quotas, regions, and uptime or SLA details should be verified in the console because these terms may change.

The exact official MonkeyCode repository is AGPL-3.0 open source. At main commit 18baaf54937a65a7d47f1f9d83dd808777aa6cea, the README describes built-in management for development environments, models, tasks, and requirements. I see the repository as source material for inspection and a possible self-hosting exit option, not a guarantee against outages. Hosted MonkeyCode reliability was not tested for this article.

A useful exercise is to map the toy states to repository concepts, then try one non-sensitive hosted task and note every assumption. Keep learning claims separate from operational claims.

Disclosure: I'm a MonkeyCode user sharing my own experience, not affiliated with the project.

AI assistance disclosure: This article was drafted with AI assistance and reviewed against the cited primary sources.

Top comments (0)