Building a Slack Deploy Queue Bot: Lessons from NestJS, BullMQ, and Redis in Production

Over the past few months I built a side project that taught me more about production system design than any course. A Slack bot for deploy queue management. This isn't about the business side of it, it's a technical breakdown of the architecture decisions, the real problems I hit building a Slack app with NestJS, and what I learned solving each one.

The problem

Every engineering team has lived this. Two people deploy at the same time, one overwrites the other, and it turns into "who's touching prod right now?" shouted into a Slack channel. Sounds simple. Solving it properly across multiple teams, multiple environments, with timeouts, without ever locking anyone out, is not.

Stack and why

NestJS + TypeScript on the backend, PostgreSQL via Prisma, Redis + BullMQ for background jobs, @slack/bolt for the Slack integration.

Choosing NestJS wasn't just preference. Its modular architecture (modules, providers, guards, interceptors) mapped really well to the domain. Every entity (Workspace, Project, Environment, Queue) became its own module, with the repository pattern keeping Prisma out of the business logic layer.

The hardest problem: dynamic modals in Slack Block Kit

Slack's Block Kit doesn't natively support a select input that reloads its options based on another select's value, within the same form. If you want "pick a project, then load that project's environments," there's no built-in prop for that.

The solution combines two Bolt event types. A block_actions listener on the project select fetches the environments, then re-renders the whole modal via views.update:

app.action('project_select', async ({ ack, body, client }) => {
  await ack();
  const selectedProjectId = body.actions[0].selected_option.value;
  const environments = await environmentService.findByProject(selectedProjectId);

  await client.views.update({
    view_id: body.view.id,
    view: buildModalWithEnvironments(environments),
  });
});

The detail that tripped me up the most: the block_id of dependent fields needs to change dynamically (like environment_${projectId}) so Slack bypasses its internal state cache and actually renders the new initial_option you're trying to set.

A queue with timeouts that survive a restart

Every queue entry needs a timer. "If this person doesn't confirm within X minutes, cancel and advance the queue." The obvious move is setTimeout, but that dies the moment the process restarts, whether that's a deploy or a crash.

Fixed it with BullMQ using a deterministic jobId based on the entry's own ID:

await queue.add(
  'confirmation-timeout',
  { entryId: entry.id },
  {
    jobId: `confirmation-timeout:${entry.id}`,
    delay: confirmationTimeoutMs,
  }
);

The fixed jobId prevents duplication. If the code somehow tries to schedule the same job twice, BullMQ just ignores the second attempt. Since the job lives in Redis, not in the Node process's memory, it survives any restart with zero extra plumbing.

Multi-tenancy and the field that almost corrupted everything

I modeled queue position as a computed field, never stored:

SELECT COUNT(*) FROM "QueueEntry"
WHERE "environmentId" = $1
AND status IN ('WAITING', 'PENDING_CONFIRMATION', 'IN_PROGRESS')
AND "enteredAt" < $2

First version stored a fixed position column. Realized pretty quick that corrupts the moment someone leaves the queue mid-way. You'd need to rewrite N rows on every exit, and under concurrency that's a guaranteed race condition. Computing it at runtime just removes the whole problem.

Reconciliation: not every webhook arrives

Expensive lesson working with any third-party webhook integration: webhooks fail silently. Server down for a few seconds, a network timeout, a transient bug, any of these can make you lose an event and your database quietly drifts from reality.

Fixed it with a daily reconciliation job comparing the local database against the external provider's actual state in both directions.

Provider to database: lists recent state changes on the provider's side, checks whether the database reflects them. Catches the case where the event fired but the webhook handler never received it.

Database to provider: checks local records that should've been updated by now, confirms with the provider whether that's still true. Catches the case where the follow-up event never arrived at all.

The interesting part was making the provider-to-database pass never auto-correct. It only logs and alerts, because silently applying state changes without a second confirmation felt too risky. The database-to-provider pass does auto-correct though, since leaving a stale, more permissive local state unhandled is the worse failure mode.

What I'm taking away from this

Modular architecture pays for itself even on a solo project. BullMQ + Redis solves "code that must run in the future, even if the server dies" way more reliably than any homemade solution. Third-party integrations are never just "webhook arrives, update the database," you always need a second line of defense. And designing for multi-tenancy from day one is much cheaper than retrofitting it later.

This is the system that became Antline, a deploy queue bot for Slack (antline.dev). Not the point of this post, just where all of the above ended up living.

Curious if anyone's solved deploy coordination differently. Drop a comment.