When the user is busy, don't drop the UI action: a small replay queue in Node.js

Disclosure: I'm the developer of the tool described here, and this post was drafted and published by an AI agent from the project's source code. Relaymote is unofficial; not affiliated with or endorsed by Anthropic.

Relaymote (MIT, Node.js) is a phone remote for Claude Desktop's Code-tab sessions. A few of its jobs are plain UI automation on the desktop: moving a session into a sidebar group, renaming a session. Before any of that runs, an idle gate checks that nobody is using the keyboard or mouse, because automation typing into a window you are using is worse than no automation.

That gate creates a second problem. If you are at the PC for an hour, the action is refused for an hour. Return an error and the request is simply lost; the caller has usually moved on. So the refused action goes into a small queue and runs at the next idle window. This post is that queue, about 110 lines, and the rules that keep it from doing something stale or surprising.

Rule 1: only replay what is safe to replay late

Not every UI action means the same thing an hour later. "Approve this prompt" an hour late could approve something you never saw. "Put session X in group Y" means the same at 10:00 and at 11:00. So the queue holds an allow-list and refuses everything else:

const ALLOWED = new Set(['setGroup', 'renameSession']);

function enqueue(action, args, meta = {}) {
  if (!ALLOWED.has(action)) return null;   // not replayable: caller gets the plain refusal
  // ...
}

Two idempotent, organisational actions. Anything that answers a question, sends text or approves something is not on the list, and the caller gets the refusal straight back.

Rule 2: the same request twice is one item

A caller that gets "user active" will often just ask again. Without care the queue fills with ten copies of the same rename. Each item gets a key made from the action and its arguments. A repeat refreshes the existing item's timestamp instead of adding a new one:

const key = action + ':' + JSON.stringify(args);
const existing = q.items.find(i => i.key === key && !i.doneAt);
if (existing) { existing.requestedAt = new Date().toISOString(); save(q); return existing; }

Refreshing requestedAt matters for the next rule: a request that is still being asked for is not stale.

Rule 3: an old request expires instead of firing

Some intents go out of date even when the action stays safe. If you renamed the session by hand in the meantime, a six-hour-old rename would undo your change. Every item carries its request time, and the drain drops anything older than the replay window (six hours by default):

if (Date.now() - Date.parse(item.requestedAt) > MAX_AGE_MS) {
  item.doneAt = new Date().toISOString(); item.lastError = 'expired';
  dropped.push({ id: item.id, action: item.action, reason: 'older than the replay window' });
  continue;
}

Expired items are reported as dropped, not silently deleted, so whoever asked can see why it never happened.

Rule 4: check idle before each item, not once per drain

The drain runs on a 45-second timer. It checks the gate once at the start (no point loading the desktop module when you are typing) and then again before every item:

if (!opts.force && !idle.isIdle(min)) {
  return { skipped: 'user-active', pending: pending.length, idleMs: idle.idleMs() };
}
for (const item of pending) {
  // ...expiry check...
  if (!opts.force && !idle.isIdle(min)) break;   // the user came back mid-drain
  // ...run it...
}

A drain of three items takes a few seconds of window work. If you touch the mouse between item one and item two, the rest wait for the next window. The queue file is saved after each item, so a break or a crash never loses or repeats work already done.

Since the queue already passed the gate, it calls the action with { idle: false }. Otherwise the action would run its own gate, wait, and possibly re-queue itself.

Rule 5: bounded retries, then give up loudly

A UI action can fail for reasons that will not fix themselves, for example the session it targets was archived. Each item gets three attempts. After the third failure it is marked done with its last error and written to an audit log:

if (item.attempts >= MAX_ATTEMPTS) {
  item.doneAt = new Date().toISOString();
  dropped.push({ id: item.id, action: item.action, reason: item.lastError, attempts: item.attempts });
  audit(`UIQUEUE gave up on ${item.action}: ${item.lastError} after ${item.attempts} attempts`);
}

Finished items stay in the file for an hour, so status() can show what just ran, and are then pruned. The file never grows without bound.

The caller's side

The idle-gated wrapper waits a while for an idle window. If none comes, it enqueues the action and says so in its result rather than pretending it worked:

return {
  ok: false, error: 'USER_ACTIVE', action: name,
  queued: queued ? { id: queued.id, willRunAtNextIdleWindow: true } : undefined,
  message: 'The user has been active the whole time, so this was NOT performed rather than interrupting them. ...',
};

ok: false stays honest: nothing happened yet. The queued field tells the caller it doesn't need to retry, and cancel(id) lets it withdraw the request if the intent changes.

What carries over to other projects

• Before you queue a refused action, ask whether it means the same thing later. Replay only those that do, by allow-list.
• Make queue items idempotent by key, and refresh the timestamp on a repeat instead of adding a duplicate.
• Give every deferred intent an expiry, and report expiries as results, not silence.
• Re-check the precondition before each step of a batch, and persist progress after each step.
• Cap retries and log the give-up; a queue that retries forever is just a slow infinite loop.

---

Relaymote is MIT-licensed: github.com/ashwarsadh/relaymote. The queue is lib/uiqueue.js; the gate it relies on is lib/idle.js. Unofficial; not affiliated with or endorsed by Anthropic.