Your AI Agent Queue Needs Backpressure, Not More Workers

An always-on agent can fail while every process is technically healthy.

The queue grows, retries multiply, browser sessions stay occupied, and latency becomes unbounded. The usual response is to add workers. That can make the failure worse: more workers consume the same scarce credentials, browser slots, model context, or outbound API quota.

The missing control is backpressure: a deliberate rule for when to accept work, delay it, shed it, or ask for human intervention.

This article gives a small design you can implement with a database table and a worker loop. It is not a benchmark or a claim that one queueing strategy fits every workload. The point is to make overload behavior explicit and testable.

1. Define the scarce resource first

Do not start with a worker count. Start with the resource that actually saturates.

Typical agent bottlenecks include:

• concurrent browser sessions
• model-context or token budget
• per-tenant API quota
• filesystem or database I/O
• outbound notification rate
• human approval capacity

Write the limit down as a named budget:

action_budget = {
  "browser_sessions": 2,
  "model_tokens_per_minute": 120000,
  "outbound_messages_per_minute": 30
}

A process can have spare CPU and still be overloaded if one of these budgets is exhausted.

2. Keep queue state separate from execution state

A queued item is not running work. A running item is not proof that the side effect succeeded. Use separate states so recovery does not turn an old lease into a new duplicate action.

A minimal schema:

tasks(
  id,
  tenant_id,
  idempotency_key,
  priority,
  state,             -- ACCEPTED, RUNNING, SUCCEEDED, FAILED, UNKNOWN, CANCELLED
  attempts,
  not_before,
  lease_until,
  created_at,
  updated_at
)

The unique constraint should cover the operation's business idempotency key, not just a random task ID:

CREATE UNIQUE INDEX tasks_once
ON tasks(tenant_id, idempotency_key);

When a worker crashes after dispatching a tool call, mark the task UNKNOWN after lease expiry. Do not silently return it to ACCEPTED. Reconcile the provider or inspect the side-effect ledger first.

3. Admit work with a bounded policy

Admission should be cheap and deterministic. Rejecting or delaying a task before it occupies a browser or model slot is safer than discovering overload halfway through execution.

One simple policy:

function admit(task, budget, now) {
  if (task.tenant_paused) return { action: "DEFER", reason: "tenant_paused" };
  if (budget.in_flight >= budget.max_in_flight) {
    return task.priority === "critical"
      ? { action: "DEFER", reason: "capacity" }
      : { action: "REJECT", reason: "over_capacity" };
  }
  if (task.deadline && task.deadline < now) {
    return { action: "REJECT", reason: "expired" };
  }
  return { action: "RUN" };
}

The exact choices depend on the operation. A payment or security alert may be deferred rather than rejected. A stale refresh job may be safely dropped. The important part is that the distinction is encoded instead of being left to retry defaults.

4. Prevent one tenant from filling the queue

Global limits are not enough. Add per-tenant concurrency and a fair scheduling rule.

For example:

max_in_flight_global = 8
max_in_flight_per_tenant = 2
scheduler = weighted_round_robin(tenant_id)

A practical dequeue query can select the next eligible task only when both limits permit it. If your database cannot express this cleanly, claim a small batch and apply the per-tenant check transactionally before creating a lease.

Avoid an unbounded priority queue. High priority should mean earlier service within a bounded budget, not permission to starve every other tenant forever. Track the age of the oldest task per tenant so starvation is visible.

5. Make retries consume budget

Retries are work. Count them against the same resource budget as first attempts.

Use exponential backoff with jitter, but cap both attempts and total age:

next_delay = min(base_delay * 2 ** attempts, max_delay)
jittered_delay = random_between(0.8 * next_delay, 1.2 * next_delay)

Retry only errors that are plausibly transient. A revoked credential, invalid tool schema, or policy denial should not be retried until it becomes an outage.

Store the reason for every retry. A counter saying attempts=7 is less useful than:

• 2 rate limits
• 1 browser crash
• 1 policy recheck failure
• 3 unknown outcomes awaiting reconciliation

6. Measure control-plane signals

Do not use only average latency. An overloaded agent can keep average latency stable while a small group waits indefinitely.

At minimum, record:

• queue depth by tenant and priority
• oldest task age
• admission decisions by reason
• active leases and lease age
• retry count by error class
• UNKNOWN tasks awaiting reconciliation
• time from ACCEPTED to RUNNING
• time from RUNNING to confirmed outcome
• rejected work and its expiry reason

A useful alert is not just queue depth > 100. It is “oldest non-expired task is 10 minutes old while capacity is reported available.” That points to a stuck lease, scheduler bug, or admission mismatch.

7. Run failure-injection tests

Backpressure is a recovery mechanism only if its failure modes are tested. In a staging environment, inject these cases:

1. Fill the global budget and submit normal and critical tasks.
2. Fill one tenant's budget and confirm another tenant still runs.
3. Kill a worker after lease creation but before tool dispatch.
4. Kill it after dispatch but before outcome confirmation.
5. Return repeated rate limits and verify bounded retry age.
6. Pause a tenant while tasks are queued and confirm no new leases appear.
7. Restart the scheduler with tasks in every state.
8. Reconcile UNKNOWN tasks and verify the idempotency key prevents duplicates.

The expected result is a table of state transitions, not “the worker eventually came back.”

Hosting implication: keep the control plane alive

A queue policy is only useful if its database, scheduler, leases, and reconciliation loop survive ordinary restarts. For an always-on OpenClaw or browser-automation deployment, I would evaluate hosting against this checklist:

• persistent storage for queue and idempotency state
• restart behavior that does not erase leases or credentials
• isolated browser and worker processes
• logs that preserve admission and reconciliation decisions
• a documented backup and clean-host restore procedure

If you do not want to assemble those pieces on a laptop, a managed option such as managed always-on agent hosting on Ampere is worth evaluating. That is a hosting suggestion, not evidence of a particular uptime or throughput result. You still need to verify the persistence, isolation, backup, and recovery guarantees for your workload.

The short checklist

Before increasing worker count, answer:

• What resource is actually scarce?
• What happens when it is full: defer, reject, or cancel?
• Can one tenant starve others?
• Do retries consume the same budget?
• How are UNKNOWN outcomes reconciled?
• Can a restart preserve leases, idempotency keys, and audit history?
• Which signal proves the queue is unhealthy before users report it?

More workers improve throughput only when the bottleneck is workers. If the bottleneck is credentials, browser slots, model context, or outbound quota, explicit backpressure is usually the safer first change.