Building a Real-Time Step Progress UI with Flask + JavaScript

A Bot creation flow that takes 16 steps — here's how we made the wait feel transparent and fast.

---

Background

When you hit "Create Bot" in our home lab dashboard, the backend runs through this sequence:

1. SSH connection check
2. Bot name collision check
3. Home directory creation
4. Template file copy
5. Register in agents.list
6. Telegram Bot binding
7. auth-profiles.json symlink creation
8. OpenClaw config file write
9. Agent config file (openclaw.json) creation
10. Gateway restart
11. Startup verification (log monitoring)
12. Dev directory creation
13. TOOLS.md generation
14. Symlink final verification
15. Gateway restart wait (5 seconds)
16. Completion notification

Even when everything succeeds, this takes 30–40 seconds. Just showing a spinner makes it feel frozen. Unexplained waiting time feels roughly 3× longer than it actually is.

---

Design Options

There are several standard patterns for real-time progress:

1. Polling

Client repeatedly hits GET /api/job-status/{id}. Simple, but requires job ID management, concurrency control, and cleanup.

2. Server-Sent Events (SSE)

HTTP streaming response. Lightweight for one-way (server → client) updates. In Flask: Response(generator, mimetype='text/event-stream').

3. WebSocket

For bidirectional communication. Overkill here.

4. Synchronous response — return all steps at once

This is what we chose.

@app.route('/api/create-bot-steps', methods=['POST'])
def create_bot_steps():
    steps = []

    # Step 1: SSH check
    ok, out = ssh_cmd_with_retry(node, 'echo ok', retries=3)
    steps.append({'step': 1, 'label': 'SSH connection check', 'ok': ok, 'detail': out})
    if not ok:
        return jsonify({'success': False, 'steps': steps})

    # Step 2: Name collision check
    ok, out = ssh_cmd(node, f'test -d {agent_dir} && echo exists || echo ok')
    if 'exists' in out:
        steps.append({'step': 2, 'label': 'Name collision check', 'ok': False, 'detail': 'Already exists'})
        return jsonify({'success': False, 'steps': steps})
    steps.append({'step': 2, 'label': 'Name collision check', 'ok': True, 'detail': ''})

    # ... continues for all 16 steps

    return jsonify({'success': True, 'steps': steps})

It blocks synchronously, but since Flask runs multi-threaded, other requests aren't affected. Steps accumulate in an array, all returned at once at the end.

---

Client Side: Fake Progress Animation

The problem: "button press → 30 seconds of nothing → all steps appear at once." No sense of progress.

Solution: animate the steps sequentially after the response arrives.

async function createBot(formData) {
  showProgressModal();

  const res = await fetch('/api/create-bot-steps', {
    method: 'POST',
    body: JSON.stringify(formData),
    headers: {'Content-Type': 'application/json'}
  });
  const data = await res.json();

  // Display steps one at a time with delay
  for (const step of data.steps) {
    await appendStep(step);
    await sleep(120);  // 120ms between each step
  }

  showResult(data.success);
}

async function appendStep(step) {
  const icon = step.ok ? '✅' : '❌';
  const el = document.createElement('div');
  el.className = `step-item ${step.ok ? 'ok' : 'fail'}`;
  el.innerHTML = `${icon} Step ${step.step}: ${step.label}`;
  if (step.detail) {
    el.innerHTML += `<span class="detail">${step.detail}</span>`;
  }
  progressLog.appendChild(el);
  el.scrollIntoView({behavior: 'smooth'});
}

At 120ms per step, 16 steps takes about 2 seconds to animate. Even though the actual processing took 30 seconds, the "sense of progress" is satisfied in those first 2 seconds.

---

Why We Didn't Use SSE

"Sync response + client-side fake animation" is sufficient for most cases.

SSE makes sense when:

• Processing can be interrupted or cancelled (e.g., user wants to abort mid-run)
• Processing is very long (5+ minutes)
• Multiple clients need to watch the same progress in real time

Bot creation: 30–40 seconds, single user, no cancel needed. SSE would have been overkill.

Separating "actual real-time" from "perceived real-time" makes the design decision obvious.

---

Fail Fast: Abort on First Failure

When a step fails, the API immediately returns — skipping all remaining steps. The client animates whatever steps it received, then appends a red error at the bottom.

if (!data.success) {
  const failedStep = data.steps.find(s => !s.ok);
  appendError(`Failed at Step ${failedStep.step}: ${failedStep.detail}`);
}

Users can see exactly where things broke. Debugging time dropped noticeably.

---

Summary

• Under 30 seconds, single user → sync response + client animation is enough
• Return steps as an array → frontend display logic stays simple
• Perceived speed often matters more than actual speed
• SSE and WebSocket are powerful, but they carry implementation cost — don't pull them in for simple cases

For home lab tooling especially: a working, maintainable implementation beats a technically perfect stack every time.