Stephen Goldberg for Harper

Posted on Feb 26

Build Your Own AI Ops Assistant — Part 4: Slack Integration

#vibecoding #ai #aiops #rag

This is Part 4 of a 6-part series. Part 3 covers the AI orchestrator.

Slash Commands, Events & Rich Responses

Your AI brain is built. Now let's put it where your team actually works: Slack. By the end of this part, your engineers will be able to type /incident replication lag on us-east or @harper-eye how does sharding work? and get a response that looks like this:

That's a real response to a real question from one of our engineers. Every @-mention, every source citation, every step — generated by the orchestrator from Part 3, formatted with Block Kit, and delivered right where the conversation is already happening.

Why Slack-First Matters

You could build the world's best web dashboard and your team wouldn't use it. Not because the dashboard is bad, but because context switching kills adoption. Your engineers live in Slack. When something breaks at 2 am, they're already in a channel. Making them open a separate tool, log in, type a query, wait for a response, then copy-paste the answer back into Slack, you've already lost them.

The Slack integration isn't a nice-to-have. It is the product for most of your team.

Step 1: Create a Slack App

Go to api.slack.com/apps and create a new app. Choose "From an app manifest" and use this JSON:

{
  "display_information": {
    "name": "Harper Eye",
    "description": "AI-powered ops assistant",
    "background_color": "#1a1a2e"
  },
  "features": {
    "bot_user": {
      "display_name": "Harper Eye",
      "always_online": true
    },
    "slash_commands": [
      {
        "command": "/incident",
        "url": "https://your-cluster.your-org.harperfabric.com/SlackEvents",
        "description": "Analyze an incident with AI",
        "usage_hint": "describe the issue",
        "should_escape": false
      },
      {
        "command": "/harper-ask",
        "url": "https://your-cluster.your-org.harperfabric.com/SlackEvents",
        "description": "Ask Harper a question",
        "usage_hint": "your question about the platform",
        "should_escape": false
      }
    ]
  },
  "oauth_config": {
    "scopes": {
      "bot": [
        "app_mentions:read",
        "chat:write",
        "commands",
        "channels:history",
        "groups:history",
        "users:read"
      ]
    }
  },
  "settings": {
    "event_subscriptions": {
      "request_url": "https://your-cluster.your-org.harperfabric.com/SlackEvents",
      "bot_events": ["app_mention"]
    },
    "interactivity": {
      "is_enabled": true,
      "request_url": "https://your-cluster.your-org.harperfabric.com/SlackInteractivity"
    }
  }
}

Replace your-cluster.your-org.harperfabric.com with your actual Fabric URL. Install the app to your workspace and grab the tokens from the "OAuth & Permissions" and "Basic Information" pages. Add them to your CONFIG.env:

SLACK_BOT_TOKEN=xoxb-...
SLACK_VERIFICATION_TOKEN=...
SLACK_SIGNING_SECRET=...

Step 2: Build the Slack Event Handler

This is the main Slack endpoint. It handles URL verification, slash commands, and @-mention events — all in a single Resource Class.

Create resources/SlackEvents.js:

import { Resource, tables } from 'harperdb';
import crypto from 'crypto';
import { config } from '../lib/config.js';
import { orchestrate } from '../lib/orchestrator.js';
import { generateEmbedding } from '../lib/embeddings.js';
import { searchKnowledgeBase } from '../lib/knowledge-base.js';
import {
  formatIncidentResponse,
  formatPlainText,
  formatAskResponse,
  formatAskPlainText,
} from '../lib/slack-formatter.js';
import { WebClient } from '@slack/web-api';

let slackClient;
function getSlackClient() {
  if (!slackClient) slackClient = new WebClient(config.slack.botToken());
  return slackClient;
}

export class SlackEvents extends Resource {
  static loadAsInstance = false;

  async post(target, data) {
    const body = data;

    // Slack URL verification — MUST return before any auth check
    if (body?.type === 'url_verification') {
      return { challenge: body.challenge };
    }

    // Verify request came from Slack
    if (!verifySlackRequest(body)) {
      const err = new Error('Invalid request');
      err.statusCode = 401;
      throw err;
    }

    // Slash commands: /incident and /harper-ask
    if (body?.command === '/incident') return handleSlashCommand(body, 'incident');
    if (body?.command === '/harper-ask') return handleSlashCommand(body, 'ask');

    // Event callbacks (@mentions)
    if (body?.type === 'event_callback') return handleEventCallback(body);

    return { ok: true };
  }
}

There are two critical things to notice here:

1. URL verification returns BEFORE auth. When you first configure your Slack app's event URL, Slack sends a challenge request. If you verify the signature first and the verification fails (which it can during initial setup), Slack never gets its challenge response and rejects your URL. Always handle url_verification first.

2. The post() method signature. Remember from Part 2: data is the parsed request body (second argument), not target (first argument). This is the most common mistake when building Harper Resource Classes.

Step 3: Handle Slash Commands

Slash commands have a critical constraint: Slack requires a response within 3 seconds. Your orchestrator takes 5-15 seconds. So the pattern is: acknowledge immediately, process asynchronously, then post the results via Slack's response_url.

function handleSlashCommand(body, mode = 'incident') {
  const { text, user_id, channel_id, response_url } = body;
  const isAsk = mode === 'ask';

  if (!text?.trim()) {
    return {
      response_type: 'ephemeral',
      text: isAsk
        ? 'Usage: `/harper-ask <your question>`'
        : 'Usage: `/incident <describe the issue>`',
    };
  }

  // Acknowledge immediately (returns within 3s)
  // Process asynchronously — fire and forget
  processQueryAsync({
    query: text.trim(),
    userId: user_id,
    channelId: channel_id,
    responseUrl: response_url,
    mode,
  });

  return {
    response_type: 'in_channel',
    text: ':mag: Searching across knowledge base...',
  };
}

The processQueryAsync function runs the orchestrator, formats the response, and posts it back:

async function processQueryAsync({
  query, userId, channelId, responseUrl, threadTs, updateTs, mode = 'incident',
}) {
  try {
    // Fast path: check KB before hitting Claude
    let result;
    let kbHit = false;

    try {
      const embedding = await generateEmbedding(query);
      if (embedding) {
        const kbResult = await searchKnowledgeBase(query, embedding);
        if (kbResult.match === 'exact' && kbResult.entry?.answer) {
          const cached = JSON.parse(kbResult.entry.answer);
          result = {
            summary: cached.summary ?? cached.answer ?? null,
            sources: cached.sources ?? [],
            steps: cached.steps ?? [],
            customerImpact: cached.customerImpact ?? null,
            escalation: cached.escalation ?? null,
            fromKnowledgeBase: true,
            knowledgeBaseEntryId: kbResult.entry.id,
          };
          kbHit = true;
        }
      }
    } catch (kbErr) {
      console.error('KB fast path failed:', kbErr.message);
    }

    // Full path: orchestrate if no KB hit
    if (!kbHit) {
      result = await orchestrate(query, { channelId, mode });
    }

    // Save to audit trail
    const queryId = crypto.randomUUID();
    await tables.IncidentQuery.put({
      id: queryId,
      slackUserId: userId,
      slackChannelId: channelId,
      query,
      response: JSON.stringify(result),
      sourcesUsed: result.sources?.map((s) => s.url ?? s.title) ?? [],
      createdAt: new Date().toISOString(),
    });
    result.queryId = queryId;

    // Format for Slack
    const isAsk = mode === 'ask';
    const blocks = isAsk ? formatAskResponse(result) : formatIncidentResponse(result);
    const fallbackText = isAsk ? formatAskPlainText(result) : formatPlainText(result);

    if (responseUrl) {
      // Slash command → post via response_url
      await fetch(responseUrl, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({
          response_type: 'in_channel',
          replace_original: true,
          blocks,
          text: fallbackText,
        }),
      });
    } else if (updateTs) {
      // @mention → update the ack message
      const slack = getSlackClient();
      await slack.chat.update({
        channel: channelId,
        ts: updateTs,
        blocks,
        text: fallbackText,
      });
    }
  } catch (err) {
    console.error('Error processing query:', err);
    const errorText = `:warning: Something went wrong: ${err.message}`;
    if (responseUrl) {
      await fetch(responseUrl, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ response_type: 'ephemeral', replace_original: true, text: errorText }),
      });
    }
  }
}

Notice the fast path: before ever calling the orchestrator (and spending API tokens), we check if the knowledge base already has a verified answer. If someone asked this question last week and another engineer confirmed the answer was good, we return it instantly. This is the knowledge loop at work; we'll build it fully in Part 5.

Step 4: Handle @-Mentions with Threaded Conversations

@-mentions are more powerful than slash commands because they support threaded follow-up conversations. Your engineer can ask a question, read the response, then reply in the thread with a follow-up, and Harper Eye has the full conversation context.

async function handleEventCallback(payload) {
  const event = payload.event;

  // Ignore bot messages to prevent loops
  if (event.bot_id || event.subtype === 'bot_message') {
    return { ok: true };
  }

  if (event.type === 'app_mention') {
    // Strip the @mention from the query text
    const query = event.text.replace(/<@[A-Z0-9]+>/g, '').trim();

    if (!query) {
      const slack = getSlackClient();
      await slack.chat.postMessage({
        channel: event.channel,
        thread_ts: event.ts,
        text: 'Ask me anything! Example: `@harper-eye how does replication work?`',
      });
      return { ok: true };
    }

    // Check for thread context (follow-up questions)
    const threadTs = event.thread_ts ?? event.ts;
    let conversationHistory = [];

    if (event.thread_ts) {
      // This is a reply in an existing thread — fetch conversation context
      conversationHistory = await getThreadConversationContext(
        event.channel,
        event.thread_ts
      );
    }

    // Post acknowledgment in the thread
    const slack = getSlackClient();
    const ack = await slack.chat.postMessage({
      channel: event.channel,
      thread_ts: threadTs,
      text: ':mag: Searching across knowledge base...',
    });

    // Process async, update the ack message when done
    processQueryAsync({
      query,
      userId: event.user,
      channelId: event.channel,
      threadTs,
      updateTs: ack.ts,
      conversationHistory,
    });
  }

  return { ok: true };
}

The thread context fetcher pairs user messages with bot responses to build conversation history:

async function getThreadConversationContext(channelId, threadTs) {
  try {
    const slack = getSlackClient();
    const result = await slack.conversations.replies({
      channel: channelId,
      ts: threadTs,
      limit: 20,
    });

    const history = [];
    let lastUserQuery = null;

    for (const msg of result.messages ?? []) {
      if (!msg.bot_id && !msg.subtype && msg.text) {
        lastUserQuery = msg.text.replace(/<@[A-Z0-9]+>/g, '').trim();
      } else if (msg.bot_id && lastUserQuery) {
        const summary = extractSummaryFromBlocks(msg.blocks) || msg.text;
        if (summary?.length > 20) {
          history.push({ query: lastUserQuery, summary });
        }
        lastUserQuery = null;
      }
    }
    return history;
  } catch (err) {
    console.error('Failed to fetch thread context:', err.message);
    return [];
  }
}

Step 5: Format Responses with Block Kit

Slack's Block Kit gives you rich, structured messages with headers, dividers, formatted text, and interactive buttons. Here's the formatter that turns orchestrator results into Slack blocks.

Create lib/slack-formatter.js:

/**
 * Format an incident analysis result into Slack Block Kit blocks.
 */
export function formatIncidentResponse(result) {
  const blocks = [];

  blocks.push({
    type: 'header',
    text: { type: 'plain_text', text: 'Incident Analysis', emoji: true },
  });

  // Knowledge base indicator
  if (result.fromKnowledgeBase) {
    blocks.push({
      type: 'context',
      elements: [{
        type: 'mrkdwn',
        text: ':brain: _Retrieved from knowledge base (verified past answer)._',
      }],
    });
  }

  // Summary
  if (result.summary) {
    blocks.push(
      { type: 'divider' },
      {
        type: 'section',
        text: { type: 'mrkdwn', text: `*Situation Summary*\n${result.summary}` },
      }
    );
  }

  // Customer Impact
  if (result.customerImpact) {
    blocks.push(
      { type: 'divider' },
      {
        type: 'section',
        text: { type: 'mrkdwn', text: `*Customer Impact*\n${result.customerImpact}` },
      }
    );
  }

  // Steps
  if (result.steps?.length) {
    const stepsText = result.steps.map((s, i) => `${i + 1}. ${s}`).join('\n');
    blocks.push(
      { type: 'divider' },
      {
        type: 'section',
        text: { type: 'mrkdwn', text: `*Recommended Steps*\n${stepsText}` },
      }
    );
  }

  // Sources with clickable links
  if (result.sources?.length) {
    const sourcesText = result.sources.map((s) => {
      if (typeof s === 'string') return `- ${s}`;
      if (s.url && s.title) return `- <${s.url}|${s.title}>`;
      return `- ${s.title ?? s.url ?? JSON.stringify(s)}`;
    }).join('\n');
    blocks.push(
      { type: 'divider' },
      {
        type: 'section',
        text: { type: 'mrkdwn', text: `*Sources*\n${sourcesText}` },
      }
    );
  }

  // Escalation
  if (result.escalation) {
    blocks.push(
      { type: 'divider' },
      {
        type: 'section',
        text: { type: 'mrkdwn', text: `:rotating_light: *Escalation*\n${result.escalation}` },
      }
    );
  }

  // Feedback buttons
  const buttonValue = JSON.stringify({
    queryId: result.queryId ?? null,
    knowledgeEntryId: result.knowledgeBaseEntryId ?? null,
  });

  blocks.push(
    { type: 'divider' },
    {
      type: 'actions',
      block_id: 'feedback_actions',
      elements: [
        {
          type: 'button',
          text: { type: 'plain_text', text: ':thumbsup: Helpful', emoji: true },
          action_id: 'feedback_positive',
          value: buttonValue,
          style: 'primary',
        },
        {
          type: 'button',
          text: { type: 'plain_text', text: ':thumbsdown: Not Helpful', emoji: true },
          action_id: 'feedback_negative',
          value: buttonValue,
        },
      ],
    }
  );

  return blocks;
}

Step 6: Handle Button Interactions

When someone clicks "Helpful" or "Not Helpful," Slack sends a payload to your interactivity endpoint. Create resources/SlackInteractivity.js:

import { Resource, tables } from 'harperdb';
import { config } from '../lib/config.js';
import { storeKnowledgeEntry, storeNegativeFeedback } from '../lib/knowledge-base.js';
import { WebClient } from '@slack/web-api';

let slackClient;
function getSlackClient() {
  if (!slackClient) slackClient = new WebClient(config.slack.botToken());
  return slackClient;
}

export class SlackInteractivity extends Resource {
  static loadAsInstance = false;

  async post(target, data) {
    // Slack sends interactivity payloads as form-encoded with a "payload" field
    const payloadStr = data?.payload ?? data;
    const payload = typeof payloadStr === 'string' ? JSON.parse(payloadStr) : payloadStr;

    if (payload.type === 'block_actions') {
      for (const action of payload.actions ?? []) {
        const meta = JSON.parse(action.value ?? '{}');

        if (action.action_id === 'feedback_positive') {
          await handlePositiveFeedback(meta, payload);
        }

        if (action.action_id === 'feedback_negative') {
          await handleNegativeFeedback(meta, payload);
        }
      }
    }

    return { ok: true };
  }
}

async function handlePositiveFeedback({ queryId }, payload) {
  if (!queryId) return;

  try {
    const entry = await tables.IncidentQuery.get(queryId);
    if (!entry) return;

    // Store as verified knowledge for future queries
    await storeKnowledgeEntry({
      query: entry.query,
      answer: entry.response,
      sources: entry.sourcesUsed ?? [],
      originalIncidentId: queryId,
      approvedByUserId: payload.user?.id,
      channelId: entry.slackChannelId,
    });

    // Acknowledge in Slack
    const slack = getSlackClient();
    await slack.chat.postMessage({
      channel: payload.channel?.id,
      thread_ts: payload.message?.ts,
      text: ':white_check_mark: Thanks! This answer has been saved to the knowledge base.',
    });
  } catch (err) {
    console.error('Failed to store positive feedback:', err.message);
  }
}

async function handleNegativeFeedback({ queryId, knowledgeEntryId }, payload) {
  // Open a modal asking what was wrong
  const slack = getSlackClient();
  await slack.views.open({
    trigger_id: payload.trigger_id,
    view: {
      type: 'modal',
      callback_id: 'negative_feedback_modal',
      private_metadata: JSON.stringify({ queryId, knowledgeEntryId }),
      title: { type: 'plain_text', text: 'What went wrong?' },
      blocks: [
        {
          type: 'input',
          block_id: 'category',
          element: {
            type: 'static_select',
            action_id: 'category_select',
            options: [
              { text: { type: 'plain_text', text: 'Inaccurate' }, value: 'inaccurate' },
              { text: { type: 'plain_text', text: 'Outdated' }, value: 'outdated' },
              { text: { type: 'plain_text', text: 'Too Generic' }, value: 'too_generic' },
              { text: { type: 'plain_text', text: 'Missing Info' }, value: 'missing_info' },
            ],
          },
          label: { type: 'plain_text', text: 'Category' },
        },
        {
          type: 'input',
          block_id: 'details',
          optional: true,
          element: {
            type: 'plain_text_input',
            action_id: 'details_input',
            multiline: true,
            placeholder: { type: 'plain_text', text: 'Any specific feedback?' },
          },
          label: { type: 'plain_text', text: 'Details' },
        },
      ],
      submit: { type: 'plain_text', text: 'Submit' },
    },
  });
}

When the user submits the modal, it comes back as a view_submission event. You'd handle that in the same post() method to call storeNegativeFeedback() — which we'll build fully in Part 5.

Deploy & Test

Deploy your updated code:

npm run deploy

Once deployed, verify your Slack app's event URL is pointing at your Fabric URL. Then try it:

/harper-ask how does replication work?

You should see the "🔍 Searching..." acknowledgment, followed by a rich Block Kit response with your answer, sources, and feedback buttons.

Try the @mention in a channel:

@harper-eye what happens when a node goes down?

Then reply in the thread:

@harper-eye what about if it's the primary node?

Harper Eye will include the previous Q&A as context for the follow-up, with no repetition and full continuity.

What You Have Now

Your team can now interact with Harper Eye entirely from Slack:

/incident <description> — incident analysis with remediation steps
/harper-ask <question> — general questions about your platform
@harper-eye <question> — @mention anywhere, supports threaded follow-ups
Feedback buttons — thumbs up saves to knowledge base, thumbs down collects structured feedback
Rich formatting — headers, sections, clickable source links, escalation callouts

All of it running on a single Harper instance. No Lambda functions, no SQS queues, no separate API gateway. One deploy target.

What's Next

In Part 5, we build the knowledge loop, the system that makes Harper Eye smarter every time your team uses it. Vector-indexed knowledge base, semantic similarity search, feedback-driven degradation, and the self-healing answer pipeline.

DEV Community