DEV Community

Cover image for I learned the hard way that Slack is the worst place to find out your website agent skipped the important part
Lars Winstand
Lars Winstand

Posted on • Originally published at standardcompute.com

I learned the hard way that Slack is the worst place to find out your website agent skipped the important part

Slack is great for approvals and alerts.

It is a terrible source of truth for supervising long-running agents.

That sounds dramatic until you run a real website update workflow through it.

I’m talking about the common setup now:

  • an agent updates site copy
  • touches a CMS
  • maybe runs a script
  • maybe calls an internal API
  • posts progress into Slack so a human can approve the final step

On paper, this looks clean.

In practice, Slack turns rich execution traces into vibes.

And vibes are not enough when an agent is editing production content.

The failure mode is extremely normal

I ran into this while looking at agent supervision patterns for website update automation.

A thread on r/openclaw captured the exact problem:
https://reddit.com/r/openclaw/comments/1v1nmnk/how_to_get_agent_commentary_and_tool_calls_to/

The user had already enabled Slack streaming with commentary, narration, and tool progress:

{
  "channels": {
    "slack": {
      "streaming": {
        "mode": "progress",
        "progress": {
          "commentary": true,
          "render": "rich",
          "narration": true,
          "toolProgress": true
        }
      }
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

That is not a lazy setup.

That is someone trying to do agent supervision correctly.

And after a bunch of testing, they still ended up with Slack showing header-like fragments instead of the details they actually needed.

The key line was basically:

I can see the commentary and tool calls in the OpenClaw dashboard, but I want them in Slack too.

That’s the whole issue.

The dashboard has the truth.
Slack has the summary.

If your human supervisor only watches Slack, they are watching a compressed version of reality.

Why this happens

Because Slack is a chat app.

Agents are not chat-shaped anymore.

Modern agent workflows emit structured events:

  • planning steps
  • tool calls
  • partial tool output
  • retries
  • approval checkpoints
  • state transitions
  • final actions

OpenAI Responses API streams event-like output around tool use and response items.
Anthropic Messages API streaming exposes granular blocks like tool_use and content deltas.

That is how agents behave in the real world.

A trace looks more like this:

  1. inspect page state
  2. compare requested copy changes
  3. call CMS tool
  4. get validation error
  5. retry with corrected field
  6. generate diff summary
  7. request approval
  8. publish

A dashboard can preserve that structure.

A Slack thread flattens it into text.

Once you flatten it, you lose the exact context a human needs to decide whether the agent is being careful or just sounding confident.

The real problem: lying by omission

This is the part that matters.

If Slack says:

  • Commentary
  • bash
  • tool running

that is not transparency.
That is a label.

It does not answer the useful questions:

  • What command ran?
  • What file changed?
  • What API payload was sent?
  • Did the first attempt fail?
  • Did the agent retry?
  • Did it touch staging or production?

Those are very different stories.

But Slack can make them all look identical.

For website automation, that gets dangerous fast.

There’s a huge difference between:

  • resizing an image
  • updating a blog title
  • replacing pricing copy
  • changing SEO metadata
  • triggering publish in Webflow, WordPress, or Contentful

If all Slack shows is tool running, your supervision layer is missing the important part.

Slack’s limits are a bad fit for agent traces

This is not just a UX complaint.

Slack’s API constraints are fine for chatbots and notifications.
They are not fine for high-fidelity agent streaming.

Slack recommends keeping message text under 4,000 characters and says messages over 40,000 characters may be truncated.

Slack also rate-limits message posting to roughly 1 message per second per channel, plus broader workspace limits.

For a normal bot, no problem.

For an agent emitting:

  • commentary
  • tool progress
  • command output
  • retries
  • approval checkpoints

that becomes a bottleneck.

What breaks first

Usually not the agent.

The supervision layer breaks first.

When you push too much execution detail into Slack, one of these happens:

  • updates get batched into vague summaries
  • updates arrive late or out of order
  • updates truncate
  • updates disappear under load

And then the human says, “the agent was weird.”

Maybe.

But a lot of the time, the trace was fine and Slack turned it into mush.

Bad pattern: treating Slack like an observability system

I keep seeing teams do this:

Agent -> tool call -> tool output -> progress update -> retry -> approval request -> publish -> final summary
Enter fullscreen mode Exit fullscreen mode

All streamed directly into one Slack thread.

This feels convenient because everyone already lives in Slack.

It is also the fastest way to create false confidence.

A thread full of updates looks like visibility.
It is not the same thing as execution visibility.

Better pattern: Slack for attention, trace UI for truth

This is the pattern I’d use every time for website update automation.

Option What happens in real life
Slack thread only Fast for human attention, bad for dense traces, raw tool output, and debugging; truncation and rate limits show up quickly
Dashboard / trace UI only Best for full fidelity, spans, retries, tool input/output, and replay; worse for quick approvals because humans are not staring at it all day
Hybrid: Slack + dashboard Best tradeoff; Slack handles summaries and approvals, dashboard holds the canonical trace

That hybrid setup is the one that survives contact with production.

What I’d actually build

1. Keep Slack short and decision-oriented

Slack messages should answer:

  • What is happening?
  • Does a human need to act?
  • Where is the full trace?

Good Slack milestones:

  • planned change
  • running tool step
  • awaiting approval
  • completed
  • failed and needs review

Bad Slack content:

  • full command output
  • raw diffs
  • long reasoning streams
  • every retry event
  • every intermediate tool payload

Example Slack message:

{
  "text": "Website agent updated homepage hero copy in staging. Awaiting approval before publish. Full trace: https://your-trace-ui/runs/abc123"
}
Enter fullscreen mode Exit fullscreen mode

That is enough.

2. Put the real execution record in a trace UI

Every Slack checkpoint should link to the trace.

That trace should include:

  • raw tool input/output
  • command text
  • file diffs
  • timestamps
  • retries
  • approval events
  • model used for each step

If you are using OpenClaw, LangSmith, or your own tracing layer, this is where the real debugging happens.

One reply in that OpenClaw thread mentioned trying this too:

"commandText": "raw"
Enter fullscreen mode Exit fullscreen mode

That may improve visibility in some setups.

Still, I would not make Slack the canonical log even if raw command text shows up.

3. Approve in Slack, investigate in the trace

This is the clean split.

Slack is good for:

  • “Approve this publish?”
  • “This step failed.”
  • “The agent is waiting on you.”

The trace UI is good for:

  • “Why did it touch this field?”
  • “What command actually ran?”
  • “Did the first attempt fail?”
  • “Which model decided to call the tool?”

Different interface, different job.

Concrete implementation sketch

Here’s a simple pattern for an agent that updates a site and posts into Slack.

Agent flow

plan -> inspect -> edit draft -> validate -> summarize diff -> request approval -> publish
Enter fullscreen mode Exit fullscreen mode

What gets stored in the trace

{
  "run_id": "site-update-4821",
  "model": "gpt-5.4",
  "steps": [
    {
      "type": "inspect",
      "target": "homepage hero",
      "result": "Current headline fetched"
    },
    {
      "type": "tool_call",
      "tool": "contentful.updateEntry",
      "input": {
        "entryId": "hero_01",
        "field": "headline"
      },
      "output": {
        "status": "draft updated"
      }
    },
    {
      "type": "approval_required",
      "summary": "Headline changed from A to B"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

What gets sent to Slack

Website agent prepared a homepage update.

Change: hero headline updated
Status: awaiting approval
Trace: https://trace.example.com/runs/site-update-4821
Enter fullscreen mode Exit fullscreen mode

That separation keeps Slack readable and the trace useful.

Why this matters even more as agents get better

Here’s the weird part: richer supervision usually increases traffic.

Better agent operations do not always mean fewer messages.
They often mean:

  • more checkpoints
  • more retries surfaced
  • more tool events
  • more review loops
  • more experiments with prompts and routing

That creates pressure on both:

  • the human-facing channel
  • the underlying model/tool budget

This is where pricing starts affecting architecture.

If every extra trace, retry, and review loop feels expensive, teams suppress visibility.
They log less.
They supervise less.
They avoid useful checkpoints because each one costs money.

That is a bad incentive if you are running agents in n8n, Make, Zapier, OpenClaw, or custom workflows.

For agent-heavy automations, flat monthly compute is a lot easier to reason about than per-token billing.

If your team wants more supervision, more traces, and more iteration without constantly watching usage, that’s exactly why products like Standard Compute exist:
https://standardcompute.com

It gives you an OpenAI-compatible API with flat monthly pricing, so you can run agent workflows without every extra checkpoint feeling like a billing event.

That matters more than people think.

Because the better your oversight gets, the more model activity you usually generate.

My rule now

If a human might need to ask:

what exactly did the agent do?

Slack cannot be the only place that answer lives.

Use Slack as the front desk.
Use a trace UI as the security camera footage.

That’s the clean lesson here.

Slack is great for attention, approvals, and escalation.

It is not where I want the only copy of a production agent’s execution history.

Once you see that clearly, the architecture gets simpler:

  • Slack for summaries
  • dashboard for evidence
  • humans approve in chat
  • humans debug in traces

That setup is less flashy than streaming everything into a thread.

It is also the one I trust.

Practical takeaway

If you’re building website update automation this week, do this:

1. Stream milestones to Slack
2. Store full traces elsewhere
3. Link every Slack update to the trace
4. Keep approvals in Slack
5. Keep debugging out of Slack
Enter fullscreen mode Exit fullscreen mode

If you skip step 2, you are not supervising an agent.

You are reading its status messages and hoping they tell the full story.

Top comments (0)