DEV Community

Robin
Robin

Posted on

Treat Per-Task Model Switching as a Concurrency Protocol

#distributedsystems #ai #architecture #reliability

Changing the model for a running AI task is not a settings update. It is a distributed operation:

read current task -> prepare credentials/config -> request restart -> receive result -> persist active model
Enter fullscreen mode Exit fullscreen mode

If two switches overlap, completion order can differ from request order. The system needs a rule for which intent wins.

The concrete case

At commit c58bcd4, MonkeyCode records model-switch attempts with from/to model IDs, request ID, load-session flag, success, message, session ID, and timestamps in TaskModelSwitch.

The reviewed task use case creates a switch record, asks taskflow to restart with the target model configuration, and completes the switch record and task model based on the response. The accompanying tests cover success and failure paths.

From this source review, I could not establish an explicit compare-and-swap generation or a per-task serialization contract around overlapping requests. That does not prove an exploitable race: serialization may exist elsewhere in the deployment or taskflow boundary. It means concurrency semantics deserve an explicit test and contract.

Why last completion is unstable

Assume request A selects model A, then request B selects model B:

time ->
A: request ---- restart ---------------- complete
B:        request -- restart -- complete
Enter fullscreen mode Exit fullscreen mode

If each successful completion writes its model, B applies first and late A overwrites it. Reverse network timing and the result changes.

The companion simulator makes that order dependence visible:

export function naiveCompletionOrder(completions) {
  let model = "initial";
  for (const completion of completions) {
    if (completion.success) model = completion.model;
  }
  return model;
}
Enter fullscreen mode Exit fullscreen mode

[A, B] ends on B. [B, A] ends on A. The caller's latest intent is not part of the rule.

Add a monotonic generation

Assign a generation while accepting each request:

A -> generation 41
B -> generation 42
Enter fullscreen mode Exit fullscreen mode

Completion may update active state only when its generation equals the task's current requested generation:

UPDATE tasks
SET active_model_id = :model,
    applied_generation = :generation
WHERE id = :task_id
  AND requested_generation = :generation;
Enter fullscreen mode Exit fullscreen mode

Zero updated rows means the operation was superseded. Keep its history, but do not apply stale state.

The minimal GenerationSwitch class in the companion artifact implements that rule. Run:

node test-model-switch.mjs
Enter fullscreen mode Exit fullscreen mode

Expected output:

PASS naive result depends on completion order; generation guard preserves newest request
Enter fullscreen mode Exit fullscreen mode

The test deliberately completes request B before A and proves that late A is marked superseded while model B remains active.

Generations are only one part of the protocol

Define all of these:

Concern Contract
Duplicate request Same request ID returns the same operation, without a second restart
Competing request New generation supersedes old intent, or admission rejects while busy
Late success Recorded for audit; cannot overwrite a newer generation
Restart failure Active model remains the last successfully applied generation
Process crash Reconciler compares requested, applied, and runtime-observed state
Session loading Bound to the specific operation and generation
Credential binding Runtime token and cached config correspond to the applied model

Serialization is a valid alternative: acquire a per-task lock or queue switches. It simplifies overlap but requires lease expiry, crash recovery, fairness, and a user-visible “switch in progress” state. A generation guard remains useful as defense against stale workers.

Validate with controlled interleavings

Unit tests should pause operations at credential creation, switch-record creation, restart dispatch, restart response, and persistence. Release A and B in every relevant order. Assert:

  1. newest accepted intent is the only applied generation;
  2. every attempt remains auditable;
  3. duplicate request IDs do not trigger duplicate restarts;
  4. failure cannot erase a prior valid model;
  5. reconciliation converges after a crash between runtime restart and database update.

The invariant is concise:

active model == successful result for the greatest non-superseded generation
Enter fullscreen mode Exit fullscreen mode

Once model switching is treated as a protocol, UI labels, audit records, retries, tokens, and persistence can all agree on what “current” means.

Disclosure: I contribute to the MonkeyCode project. Current fields and flow are based on the linked source at commit c58bcd4. The concurrency risk is a bounded source-review question, not a confirmed vulnerability; the generation/CAS simulator was tested locally.

Top comments (0)