DEV Community: Ram Ji Tripathi

Redis + BullMQ for AI Workloads: Retries Are Not Enough — You Also Need Idempotency and Ordering

Ram Ji Tripathi — Sun, 04 Oct 2026 12:43:57 +0000

Moving AI analysis out of an HTTP request improves the request path. Queueing that work does not make it correct.

In my AI Support Assistant, a new message triggers BullMQ analysis of the ticket conversation. The request waits for database writes and queue publication, then returns without waiting for the AI results.

But a retryable job can still run more than once, reload different context, race with another job, repeat provider work or overwrite newer results. The message can also be saved without its analysis job reaching Redis.

Examining the current implementation makes those boundaries concrete. The scenarios below are correctness risks exposed by the code, not production incidents; the proposed fixes are not implemented features.

The System This Came From

The first article described the broader React, Node.js, PostgreSQL, Redis and OpenAI architecture.

This article focuses on one workflow: a message is added to a support ticket, and background processing generates a summary, sentiment and priority. Those values are written back to the ticket. The worker also records an AIInteraction containing the conversation, summary response and summary token usage.

The queue is named ai-processing. Its job type is generate-ticket-summary, although the processor does more than summarize. It analyzes the whole ticket conversation, rather than just the message that triggered publication.

Why the AI Work Went to BullMQ

Saving a message and enriching a ticket have different completion requirements. The message is the user-authored record. AI metadata can arrive later.

The current boundary looks like this:

React client → Node.js API → PostgreSQL: Message + Ticket timestamp
                         → Redis/BullMQ: publish analysis job
                         → HTTP 201 after publication

Redis/BullMQ → separate worker → load ticket messages
                              → summary → sentiment → priority
                              → update Ticket → insert AIInteraction

Publication makes the job available to the worker. The worker may start before HTTP 201 is sent; the diagram shows two execution paths, not a guarantee that background processing begins after the response.

The important boundary is what the request awaits: persistence and publication, followed by a return to the caller. Summary, sentiment and priority belong to the worker's lifecycle.

The Actual Producer Flow

From backend/src/modules/messages/message.service.ts, lines 26–58. Authorization precedes this excerpt. The excerpt is otherwise complete; blank lines are condensed.

const message = await createMessageRepository({
  content: data.content,
  ticketId: data.ticketId,
  senderId: data.senderId,
});

await updateTicketRepository(data.ticketId, {
  updatedAt: new Date(),
});

await aiQueue.add(
  GENERATE_TICKET_SUMMARY_JOB,
  {
    ticketId: data.ticketId,
    messageId: message.id,
  },
  {
    attempts: 3,
    backoff: {
      type: "exponential",
      delay: 5000,
    },
    removeOnComplete: true,
    removeOnFail: false,
  },
);

return message;

The repository helpers execute prisma.message.create and prisma.ticket.update. These are separate writes, followed by publication to Redis. There is no encompassing transaction or outbox in this flow.

The controller awaits this service before returning HTTP 201. Consequently, a message can already exist even if the service does not reach its successful return. Queue publication is part of request completion; AI processing is not.

The payload includes messageId, but the worker's processing logic only consumes ticketId. The message ID currently provides neither a processing boundary nor an idempotency key.

What the Worker Actually Does

The worker dispatches generate-ticket-summary to processTicketSummaryJob using only the ticket ID. Its configuration in backend/src/queues/ai.worker.ts, lines 71–75, is:

{
  connection: redis as any,
  concurrency: 5,
},

This is the complete options object from the worker constructor, with blank lines condensed.

Five jobs can run concurrently within this worker instance, including jobs for the same ticket. This is a local worker limit, not a global or per-ticket limit. See BullMQ’s worker concurrency guide.

The processor loads messages with where: { ticketId }, includes sender role and name, and sorts by createdAt: "asc". It returns early when there are no messages. Otherwise it formats the conversation and executes this sequence from lines 113–127:

const aiResult = await generateTicketSummary(
  {
    conversation,
  },
  aiContext,
);

const sentimentResult = await generateSentiment(
  {
    conversation,
  },
  aiContext,
);

const priorityResult = await detectTicketPriority(conversation, aiContext);

These operations are sequential within one job. Different jobs can still overlap. The service routes generation through an OpenAI-primary orchestrator with optional Gemini fallback for eligible failures, so three logical operations do not necessarily mean exactly three external HTTP requests.

After sentiment and priority validation, lines 133–155 perform two separate writes:

await prisma.ticket.update({
  where: {
    id: ticketId,
  },
  data: {
    aiSummary: aiResult.summary,
    sentiment,
    priority,
  },
});

await createAIInteraction({
  ticketId,
  prompt: conversation,
  response: aiResult.summary || "",
  tokensUsed: aiResult.tokensUsed,
});

Blank lines are condensed. createAIInteraction is a plain Prisma insert. There is no job identity or conversation version attached to that record, and no conditional version check on the ticket update.

The worker runs through a separate entry point in backend/src/worker.ts. That separates its execution lifecycle from HTTP handling; it does not add coordination between jobs for one ticket.

Retries Solve Failure Recovery, Not Correctness

The producer configures three total attempts, including the initial execution, with exponential backoff and a five-second base. For ordinary thrown-error retries, that gives nominal delays of five and ten seconds before the remaining attempts; scheduling can add waiting time. See BullMQ's retry documentation.

Completed jobs are removed. Failed jobs are retained. Those choices affect recovery and diagnosis, but they do not make processing idempotent.

Retry asks, “Should I try this work again after failure?” Idempotency asks, “If I perform it again, will the resulting state remain correct?” Deduplication asks, “Should equivalent work be admitted or executed again at all?”

Here, retries restart a processor that calls providers and performs database writes. The retry configuration supplies no rule for deciding whether those effects have already happened.

Failure Mode 1: The Database Commits but the Queue Does Not

Consider the producer's actual order:

The message insert succeeds.
The ticket timestamp update succeeds.
Publication rejects, or the process exits before publication completes.

The database now contains the conversation change without assured corresponding queue work. Retrying a worker cannot recover a job that was never published.

A publication error can also leave an ambiguous outcome: Redis might have accepted the job before the caller lost its acknowledgement. Recovery must handle both missing work and repeated publication.

The earlier boundary also matters: if the ticket update fails after message creation, the message remains persisted and publication is never reached. Wrapping those two database writes in a transaction would address that database-only partial state. It would not atomically commit a Redis job with PostgreSQL data.

A proposed transactional outbox changes where publication intent becomes durable:

BEFORE — CURRENT
Message insert → Ticket update → Redis publication
     database writes complete       can fail separately

IMPROVED — PROPOSED
PostgreSQL transaction:
  Message insert + Ticket update + Outbox event
                         ↓ committed publication intent
Outbox dispatcher → Redis/BullMQ → worker
       ↓
mark event published after successful queue publication

The dispatcher retries unpublished events. If it publishes and crashes before marking the event, it may publish again. An outbox therefore needs stable event identity and duplicate-safe consumption; it does not confer exactly-once execution.

This adds a table, dispatcher lifecycle, backlog monitoring and retention policy. For optional enrichment in an early application, explicit reconciliation may be an interim choice. If every accepted message must eventually be analyzed, I would prioritize an outbox regardless of traffic volume. Reliability requirements, rather than scale alone, determine when it is needed.

Failure Mode 2: Retrying AI Work Is Not Automatically Idempotent

Suppose summary generation succeeds and sentiment generation fails with an error that escapes the provider layer. BullMQ can retry the processor. It starts again by loading messages and generating the summary. There is no checkpoint that reuses the previous summary.

Now move the failure later. Summary, sentiment and priority succeed; the ticket update commits; the interaction insert fails. The ticket already contains AI metadata, but the job has failed. A retry repeats the AI sequence and attempts both database writes again.

The resulting values need not match the previous output. More messages may also have arrived before the retry reloads the conversation. The same job payload therefore does not necessarily identify the same input snapshot.

A duplicate execution that reaches the insert can append another AIInteraction: its schema uses a generated UUID and has no unique analysis key. Interruption after application writes but before queue completion is recorded creates another window for repeated effects. The relevant effects here are provider requests, ticket metadata writes and interaction records.

Deterministic job identity helps at publication

A proposed identity such as ticket-analysis-<messageId> could suppress repeated publication for the same saved message while that queue job exists. It would not suppress jobs triggered by two different messages, nor deduplicate message creation if a repeated HTTP request creates a new message ID.

BullMQ's job ID documentation explains that an existing custom ID prevents another job with that ID from being added. Once the job is removed, the ID no longer protects against duplicates. That is directly relevant to this application's removeOnComplete: true setting.

Durable idempotency belongs at the application effect

For this workflow, I would define a logical analysis key from ticket identity, conversation revision and analysis/prompt version. A durable record with a unique constraint could track whether that analysis has already been applied.

The ticket update, interaction insert and successful analysis marker should commit together, with concurrent duplicate attempts resolved by database constraints and conditional writes. A separate initial “already done?” read is insufficient because two workers can both pass it.

That would protect database effects. Provider requests made before commit could still repeat. Reusing stored generation results would require checkpointing and invalidation rules; I would start with atomic result persistence and durable identity.

Failure Mode 3: Ordering Is a Business Constraint

Two messages arrive quickly on ticket T:

Message A is saved and job A starts. It reads the conversation containing A.
Message B is saved and job B starts in another available worker slot. It reads A and B.
Job B finishes first and writes metadata based on A and B.
Job A finishes later and overwrites that metadata using its older context.

The current ticket update matches only id: ticketId. Nothing prevents step four. This is a possible stale, last-writer-wins result visible from the architecture.

Both jobs could also start after B exists, read overlapping or identical context, and perform redundant analysis. Because messageId is unused, neither job is restricted to the conversation as it stood when its triggering message was saved.

Sorting messages inside each job does not order jobs or their writes. The timestamp sort also has no explicit tie-breaker for equal timestamps.

Global throughput and per-ticket correctness are separate constraints. Lowering one worker's concurrency to one would reduce overlap there, but also serialize unrelated tickets and would not establish a cross-process ordering policy.

Idempotency, Deduplication and Ordering Are Different

Concept	Question it answers	Example in this system
Retry	Should failed work be attempted again?	Retry a ticket analysis after a transient failure.
Deduplication	Should equivalent work be admitted again?	Proposed: suppress publication while the same job ID exists.
Idempotency	Will repeating the work preserve correct state?	Proposed: commit one analysis record for a ticket revision.
Ordering	In what sequence may work take effect?	Proposed: apply ticket analyses in conversation-revision order.
Concurrency	How much work may run at the same time?	Allow five active jobs within this worker instance.

Only retries and worker concurrency are configured in the current path. The other examples describe proposed behavior. A version check can reject stale results without executing every job in strict order; serialization still needs a policy for retries and earlier failures.

A Better Per-Ticket Processing Model

The following is a proposed model, not current code. For summary, sentiment and priority, the business goal is usually a current conversation view. It may not require publishing every intermediate analysis.

Messages A, B, C → atomically advance ticket conversation revision
                                   ↓
                       record latest requested revision
                                   ↓
                 coalesce pending work for this ticket
                                   ↓
                 one active analysis per ticket, if needed
                                   ↓
                    read consistent snapshot at revision R
                                   ↓
                            generate AI results
                                   ↓
                  atomic apply only if revision is still R
                          ↙                      ↘
                stale: discard              current: commit
                    ↓                    results + analysis record
         ensure latest revision remains scheduled

Version checks are the first correctness protection I would add. A dedicated conversation revision should advance atomically with message persistence. The worker must read the revision and messages as a consistent snapshot, then conditionally apply results in a short database transaction. A separate check followed by an unconditional update leaves a race.

I would avoid using Ticket.updatedAt as the conversation revision. Prisma marks it @updatedAt, so AI writes and unrelated ticket edits also change it. A dedicated revision makes the input contract explicit. The tradeoff is schema and transaction work; discarded stale analyses still consume processing. This protection is useful now because the current concurrency already permits stale writes.

Coalescing can reduce unnecessary work. Several pending triggers can represent one request to analyze the latest conversation. A short debounce period trades freshness for fewer intermediate analyses. BullMQ offers deduplication modes, but their lifetime semantics must match the application.

In particular, simply suppressing every new job while one analysis is active can miss updates arriving after that analysis loaded its messages. The design must durably track the latest requested revision and arrange another pass when needed. Checking once at the end without an atomic handoff can still lose a concurrent update. Coalescing is a pragmatic next step when message bursts cause redundant work; it is not a substitute for version checks.

Per-ticket serialization can limit overlap. A coordinated lease or partitioned processing scheme can allow different tickets to progress concurrently while restricting one ticket's active analysis. An in-memory mutex covers only one process. Distributed leases introduce expiry, renewal and crash-recovery concerns; conditional version checks remain valuable when a lease expires while work is still running.

If a future job must act on every message in business order, it needs explicit sequence tracking and a policy for earlier failures. Coalescing intermediate analyses would no longer satisfy that requirement.

What I Would Change First

These are recommendations for this application, not features already implemented.

Define analysis identity and protect freshness together. Introduce a conversation revision, conditional result application and an atomic transaction for ticket metadata, interaction and an analysis marker. Then use deterministic publication identity for the chosen unit of work. This addresses correctness exposed by today's concurrency; it costs a schema change and careful transaction design.
Coalesce pending ticket analysis where the product permits it. Treat the job as “refresh this ticket” rather than “produce every intermediate result,” while preserving a durable follow-up for changes during active processing. This costs scheduling complexity and may delay freshness slightly. It becomes more useful when bursts create redundant work. Add coordinated per-ticket serialization if overlap remains significant; version protection comes first.
Close the database-to-queue gap when eventual analysis is required. Add an outbox and replay-safe dispatcher. Also review producer connection policy: the shared Redis module sets maxRetriesPerRequest: null, and the message service has no explicit publication deadline. BullMQ’s connection guidance distinguishes worker recovery from bounded waiting in HTTP producers. The operational cost is real, but the need depends on the reliability contract, not a hypothetical high-traffic milestone.
Build recovery on the visibility already present. The worker logs receipt, completion, failures, ticket/job IDs and duration. Before using the attempt fields for alerts, I would verify their meaning at receipt and failure events. Add exhausted-attempt alerts, queue age/backlog visibility, stale-result counters and revision correlation. These are useful now; a larger telemetry platform can wait.
Add a deliberate replay policy. Retained failed jobs are a starting point, not an implemented dead-letter workflow. An operator should distinguish transient failures from invalid input, replay after repair, and know whether replay means the original snapshot or latest conversation. Controlled replay tooling and retention add maintenance work. A separate dead-letter queue is optional until volume or operational ownership warrants it.

I would evaluate these changes through duplicate work, result freshness and publication failures. Infrastructure changes should follow a demonstrated constraint.

What I Would Not Add Yet

I would not introduce Kafka, Kubernetes or decompose the application into microservices solely because it has a queue.

The concrete concerns are durable publication intent, safe repeated writes and valid application of conversation-derived results. They can be addressed within PostgreSQL, Redis, BullMQ and the existing API/worker boundary.

More infrastructure may become appropriate under specific throughput, ownership or deployment constraints. It does not remove the need to define what makes an analysis current or safe to repeat.

Why AI Workloads Make This More Visible

In this worker, three sequential generation operations separate the initial conversation read from the final ticket update. While those external requests are in flight, another message can arrive and another job can complete. The queue can function normally while the result becomes obsolete.

Provider calls can be relatively slow, and successful generation consumes usage billed under the provider’s terms; OpenAI documents token-based API pricing. Repeating a completed summary because sentiment failed can therefore repeat meaningful work, even when no database row has yet changed. This is a reason to control repetition, not a measured cost or latency claim about this application.

Generated text can also differ between attempts. Overwriting aiSummary with a second output is not automatically idempotent just because both writes target the same row. Meanwhile, this processor reloads messages on every attempt, so a retry can change both its input and its output.

For this workload, the useful contract is: which conversation revision does this analysis describe, and when may it take effect? The queue schedules execution. The application must answer that question.

Final Takeaway

BullMQ gives this application a clear boundary between message creation and background AI enrichment. Correctness still needs an application-level design: durable publication intent, duplicate-safe effects and protection against stale conversation results.

Retries make another attempt possible. Idempotency and ordering rules determine whether that attempt is safe to apply.

Originally part of the engineering work behind my AI Support Assistant architecture.

Originally published on my engineering blog:
https://blog.ramjwork.in/ai-engineering/redis-bullmq-ai-background-jobs

How I Architected an AI Customer Support System with React, Node.js, PostgreSQL, Redis & OpenAI

Ram Ji Tripathi — Sun, 04 Oct 2026 05:11:40 +0000

Adding AI wasn’t the difficult part. Deciding where AI belongs in the application architecture was.

Adding an LLM API to an application is relatively straightforward.

The harder questions begin after the first API call works.

Should AI run inside the user’s HTTP request? What happens when the provider is slow? Which AI operations should be asynchronous? What belongs in React state versus server state? Should an AI-generated support reply be sent automatically? What happens when a background job retries after partially completing?

I ran into these questions while building an AI-powered customer support application with React, TypeScript, Node.js, PostgreSQL, Redis, BullMQ and OpenAI.

The interesting part of the project wasn’t the prompt.

It was deciding where AI should—and should not—sit inside the system.

This article walks through that architecture, shows selected code from the actual implementation, and discusses several things I would change before scaling the design further.

What the Application Does

The application revolves around support tickets and conversations.

It has three roles:

Customers create tickets, read their own tickets and add messages.
Agents work with assigned tickets, update their status and request AI-generated reply suggestions.
Admins have broader ticket access and assignment capabilities.

AI assists the workflow in two different ways.

First, when a new message is added, background processing analyzes the ticket conversation to generate:

a ticket summary,
sentiment,
priority.

Second, an agent or admin can explicitly request an AI-generated reply suggestion.

Those two AI workloads look similar at first—they both call an LLM—but they have very different execution requirements.

That distinction shaped much of the architecture.

High-Level Architecture

At a high level, the system looks like this:

                    ┌─────────────────────────┐
                    │       React Client      │
                    │                         │
                    │ Auth / Tickets /        │
                    │ Messages / AI Replies   │
                    └────────────┬────────────┘
                                 │
                              REST API
                                 │
                    ┌────────────▼────────────┐
                    │     Node.js / Express   │
                    │                         │
                    │ Auth → Controller →     │
                    │ Service → Repository    │
                    └───────┬─────────┬───────┘
                            │         │
                            │         │ enqueue
                            │         ▼
                    ┌───────▼───┐  ┌──────────────┐
                    │PostgreSQL │  │ Redis/BullMQ │
                    └───────────┘  └──────┬───────┘
                                         │
                                         ▼
                                  ┌─────────────┐
                                  │  AI Worker  │
                                  │             │
                                  │ Summary     │
                                  │ Sentiment   │
                                  │ Priority    │
                                  └──────┬──────┘
                                         │
                                    LLM provider

The application is deliberately not split into many network services.

The API is a modular monolith, while AI background execution runs in a separate worker process.

That gives me an important boundary without introducing a distributed-service architecture prematurely:

HTTP request handling and background AI processing have independent execution lifecycles.

Frontend State: Don’t Put Everything in One Store

One frontend decision was to separate state based on who owns it.

I use three broad categories:

authentication/session state → React Context,
remote server state → TanStack Query,
transient UI state → local component state.

For example, tickets are server-owned resources, so their fetching and cache lifecycle belong to TanStack Query:

export function useTickets(params?: TicketListParams) {
  return useQuery({
    queryKey: ticketKeys.list(params),
    queryFn: () => getTickets(params),
  });
}

export function useTicket(ticketId: string) {
  return useQuery({
    queryKey: ticketKeys.detail(ticketId),
    queryFn: () => getTicketById(ticketId),
    enabled: Boolean(ticketId),
  });
}

export function useCreateTicket() {
  const queryClient = useQueryClient();

  return useMutation({
    mutationFn: (payload: CreateTicketRequest) => createTicket(payload),

    onSuccess: () => {
      queryClient.invalidateQueries({ queryKey: ticketKeys.lists() });
    },
  });
}

The important part here isn’t the library.

It’s the ownership model.

A ticket fetched from the API is not fundamentally client state. The server remains authoritative.

TanStack Query handles fetching, caching and invalidation, while component-local state handles things such as editable drafts and temporary UI interactions.

This prevents a common React architecture problem: turning one global store into a mirror of the backend.

The Backend Is a Modular Monolith

The backend follows a layered flow that is roughly:

Route
  ↓
Authentication / Authorization / Validation
  ↓
Controller
  ↓
Service
  ↓
Repository
  ↓
Prisma
  ↓
PostgreSQL

This gives features such as tickets and messages clear boundaries without requiring each feature to become an independent service.

The service layer owns application workflows.

Repositories isolate most persistence operations.

Controllers remain focused primarily on translating HTTP requests and responses.

AI is a partial exception because some worker-side operations interact more directly with Prisma, but the overall HTTP architecture follows this separation.

For the current scope, I prefer this to creating multiple microservices.

A service boundary should solve an actual deployment, ownership or scaling problem—not exist simply because the system has multiple features.

Why I Moved AI Analysis Out of the Request Path

Consider what happens when a customer adds a message.

The important user-facing operation is:

Persist my message successfully.

Summary, sentiment and priority enrichment are useful, but the user should not have to wait for several LLM operations to complete before the message request can finish.

The service therefore persists the message and publishes a BullMQ job:

export const createMessageService = async (data: CreateMessageInput) => {
  await getAuthorizedTicketService(data.ticketId, data.accessContext);

  const message = await createMessageRepository({
    content: data.content,
    ticketId: data.ticketId,
    senderId: data.senderId,
  });

  await updateTicketRepository(data.ticketId, {
    updatedAt: new Date(),
  });

  await aiQueue.add(
    GENERATE_TICKET_SUMMARY_JOB,
    {
      ticketId: data.ticketId,
      messageId: message.id,
    },
    {
      attempts: 3,
      backoff: {
        type: "exponential",
        delay: 5000,
      },
      removeOnComplete: true,
      removeOnFail: false,
    },
  );

  return message;
};

There is an important architectural detail here.

The HTTP request does wait for queue publication, but it does not wait for the AI analysis itself.

Conceptually:

POST message
     │
     ▼
Authorize access
     │
     ▼
Persist message
     │
     ▼
Update ticket timestamp
     │
     ▼
Publish BullMQ job
     │
     ├──────────────► HTTP response
     │
     ▼
Background worker
     │
     ▼
AI processing

That removes LLM execution time from the main request path while still ensuring the API knows whether it successfully handed work to the queue.

It also allows the worker lifecycle to be operated separately from the HTTP server.

What the Background Worker Actually Does

BullMQ uses Redis as the queue infrastructure.

A separate worker consumes AI-processing jobs:

export const aiWorker = new Worker(
  AI_QUEUE,

  async (job) => {
    const startedAt = Date.now();
    jobStartTimes.set(String(job.id), startedAt);

    const ticketId =
      typeof job.data?.ticketId === "string" ? job.data.ticketId : undefined;

    workerLogger.info(
      {
        event: "ai_job_received",
        jobId: job.id,
        jobName: job.name,
        ticketId,
        attempt: job.attemptsMade + 1,
      },
      "Job received",
    );

    switch (job.name) {
      case GENERATE_TICKET_SUMMARY_JOB:
        await processTicketSummaryJob(ticketId ?? job.data.ticketId);
        break;

      default:
        workerLogger.warn(
          {
            event: "ai_job_unknown",
            jobId: job.id,
            jobName: job.name,
            ticketId,
          },
          "Unknown job",
        );
    }
  },

  {
    connection: redis as any,
    concurrency: 5,
  },
);

Inside the ticket-processing workflow, messages are loaded in conversation order and transformed into the conversation passed to the AI layer.

The relevant part of the processing pipeline is:

const conversation = formatConversation(messages);

const aiResult = await generateTicketSummary(
  {
    conversation,
  },
  aiContext,
);

const sentimentResult = await generateSentiment(
  {
    conversation,
  },
  aiContext,
);

const priorityResult = await detectTicketPriority(
  conversation,
  aiContext,
);

const sentiment = validateSentiment(
  sentimentResult.sentiment || "",
) as any;

const priority = validatePriority(
  priorityResult.priority || "",
) as any;

Summary, sentiment and priority are currently generated sequentially inside a job.

The normalized results are then used to update ticket-level AI fields, and an AI interaction record is created for the summary workflow.

This design gives the application retryable asynchronous execution without coupling LLM latency directly to message creation.

But queues introduce their own problems.

I’ll return to those shortly.

Not Every AI Operation Belongs in a Queue

Once background processing existed, it would have been easy to conclude:

AI is slow, therefore every AI operation should be queued.

I don’t think that is the right abstraction.

The correct execution model depends on what the user is doing.

Ticket analysis is enrichment. It can happen asynchronously.

An agent clicking Generate Reply, however, is explicitly waiting for an answer.

For that workflow, the backend generates the suggestion synchronously:

const conversation = formatConversation(messages);

const aiResult = await generateReplySuggestion(conversation, {
  ticketId,
  requestId: typeof req.id === "string" ? req.id : undefined,
});

res
  .status(200)
  .json(new ApiResponse("Reply suggestion generated", aiResult));

The important distinction is:

Background analysis
Message → Queue → Worker → AI → Ticket metadata

Interactive assistance
Agent → Request suggestion → AI → Suggested text → Agent

The second flow doesn’t automatically create or send a support message.

That is intentional.

Human-in-the-Loop AI

A generated support reply can be wrong, incomplete or inappropriate for the context.

So the system treats AI output as a suggestion, not an autonomous action.

On the frontend:

<ReplySuggestion
  ticketId={data.id}
  onUseReply={(content) => {
    setComposerDraft(content);
    setComposerKey((current) => current + 1);
  }}
/>

<MessageComposer
  key={composerKey}
  ticketId={data.id}
  initialContent={composerDraft}
  onMessageSent={() =>
    setMessageRefreshToken((current) => current + 1)
  }
/>

When the user selects Use Reply, the suggestion is moved into the normal message composer.

The agent can then:

Generate
   ↓
Review
   ↓
Edit
   ↓
Send

The AI does not own the final action.

That boundary matters more to me than simply adding another confirmation dialog.

The application architecture itself separates generation from execution.

This is a pattern I would reuse in many AI-assisted workflows where the output affects another person or an important business process.

AI Provider Logic Belongs Behind an Application Boundary

The application currently uses an OpenAI-backed adapter for operations such as:

summarization,
sentiment analysis,
priority detection,
reply suggestions.

There is also provider orchestration that can use Gemini as a fallback for selected classes of provider, quota, rate-limit, network or timeout failures.

I don’t want ticket or message business logic to know the details of every provider.

Conceptually:

Business workflow
       │
       ▼
AI operation
       │
       ▼
Provider orchestration
       │
       ├── OpenAI
       │
       └── Gemini fallback

This boundary becomes useful even if the application never changes providers.

It gives one place to reason about:

prompts,
provider failures,
normalization,
fallback rules,
model configuration.

Provider abstraction does not mean every model is interchangeable.

Different models behave differently.

The goal is simply to stop provider-specific concerns from leaking through the rest of the application.

Authentication Is Not Authorization

The application uses JWT authentication, but authentication alone does not answer:

Is this user allowed to access this particular ticket?

A valid token identifies the user and role.

Resource authorization still has to be enforced.

For example:

customers should only access their own tickets,
agents operate on tickets available to their role and assignment rules,
admins have broader access.

The frontend also hides or exposes functionality according to role, but that is a UX decision—not a security boundary.

The backend remains authoritative.

This distinction is especially important in React applications because hiding a button is easy to mistake for enforcing permission.

It isn’t.

Redis Has More Than One Responsibility

Redis is not used only because BullMQ needs it.

In this project it also supports:

short-lived ticket caching,
distributed rate-limit counters.

That makes Redis shared infrastructure serving different concerns:

Redis
├── BullMQ queue infrastructure
├── ticket cache
└── rate-limit counters

Those responsibilities should still remain conceptually separate even if they use the same underlying technology.

A queue is not a cache.

A cache is not a source of truth.

The database remains the authoritative store for ticket and conversation data.

What the Architecture Gets Right

There are several boundaries in this design that I would keep.

1. User-facing writes are separated from AI enrichment

Message creation does not wait for summary, sentiment and priority generation.

2. Interactive AI and background AI use different execution models

The system chooses between synchronous and asynchronous execution based on the user workflow rather than treating all AI calls identically.

3. Server state remains server state

TanStack Query manages API-backed ticket state rather than copying everything into a global client store.

4. AI suggestions do not automatically become actions

Generated replies remain editable suggestions.

5. HTTP and worker lifecycles are separate

The API server and AI worker can fail, restart and evolve independently at the process level.

But none of those decisions makes the architecture finished.

Several weaknesses become more important as concurrency and operational requirements increase.

What I Would Change Before Scaling

This is the part of the architecture I find most useful to examine.

Queues solve one category of problem while creating others.

1. Fix the Database-to-Queue Consistency Gap

Look again at message creation:

Write message to PostgreSQL
        ↓
Update ticket
        ↓
Publish BullMQ job

These are separate operations.

Imagine:

PostgreSQL successfully saves the message.
Redis becomes unavailable before aiQueue.add() succeeds.

The application now has committed business data without the corresponding background event.

Retries do not solve this consistency problem.

A stronger design would introduce a transactional outbox.

Conceptually:

             PostgreSQL transaction
        ┌────────────────────────────┐
        │ Save message               │
        │ Save outbox event          │
        └──────────────┬─────────────┘
                       │ commit
                       ▼
                Outbox publisher
                       │
                       ▼
                   BullMQ
                       │
                       ▼
                    Worker

The message and the intent to perform AI processing would be committed atomically in PostgreSQL.

A separate publisher could then reliably move pending outbox events into BullMQ.

That doesn’t make the entire system exactly-once.

It closes a specific and important consistency gap.

2. Retries Need Idempotency

The queue currently configures multiple attempts with exponential backoff.

That improves recoverability from transient failures.

But:

retryable does not automatically mean safe to retry.

Suppose a job:

calls the AI provider,
updates the ticket,
writes an AI interaction,
fails somewhere around those operations,
retries.

Without explicit idempotency, repeated attempts can repeat side effects.

A stronger implementation would give each logical processing operation a stable identity and make persisted effects safe to replay.

Possible approaches include:

deterministic job IDs,
processing-version records,
unique database constraints,
upsert semantics,
explicit processed-event records.

The exact mechanism depends on the workflow.

The architectural requirement is simpler:

A retry should not accidentally turn one logical operation into multiple business effects.

3. Protect Against Same-Ticket Concurrency

The worker has concurrency greater than one.

That’s useful for processing unrelated tickets.

But it also means two jobs for the same ticket can overlap.

Consider:

Ticket conversation V1
        │
        └── Job A starts

New message arrives
        │
Ticket conversation V2
        │
        └── Job B starts

Job B finishes first → stores V2 analysis
Job A finishes later → stores older V1 analysis

The final ticket can now contain stale AI metadata.

This is not solved by generic worker concurrency configuration alone.

Before scaling the workflow, I would introduce some combination of:

per-ticket serialization,
conversation version numbers,
compare-before-write logic,
coalescing pending analysis jobs,
stale-result rejection.

The important principle is that global concurrency and entity-level ordering are different problems.

4. Give the UI an Explicit AI-Result Delivery Mechanism

Background processing creates another question:

How does the browser learn that AI analysis has finished?

The current architecture does not implement a dedicated real-time result-delivery channel.

At larger scale or with a richer UI, I would make that explicit.

Depending on the product requirements:

Simplest
Polling

        ↓

More event-driven
Server-Sent Events

        ↓

Bidirectional realtime needs
WebSockets

I would not automatically choose WebSockets.

If the browser only needs server-to-client status updates, SSE may be enough.

Architecture should follow the interaction requirement, not the popularity of the technology.

5. Tighten Cache Freshness Rules

Caching ticket data can reduce repeated reads, but mutable support conversations create invalidation challenges.

Every path that changes data relevant to a cached ticket needs a clear freshness strategy.

Before relying more heavily on caching, I would explicitly document:

what is cached,
the cache key,
TTL,
which writes invalidate it,
whether stale reads are acceptable,
what happens when invalidation fails.

A short TTL is useful, but TTL alone isn’t a complete consistency model.

6. Improve the AI Context and Evaluation Layer

The current AI pipeline is intentionally straightforward.

There are several improvements I would investigate before treating AI behavior as a mature subsystem:

explicit structured outputs,
conversation-size/token management,
better context construction,
prompt/model version tracking,
evaluation datasets,
regression testing for AI behavior,
provider health telemetry,
clearer usage accounting,
privacy and retention policies for model inputs and outputs.

I would add these based on actual product requirements rather than building an elaborate AI platform prematurely.

The important shift is from:

Does the model return something?

to:

Can I measure whether this AI behavior remains useful and reliable as the application changes?

What I Would Not Add Yet

Scaling discussions can easily turn into architecture shopping lists.

I would not automatically add:

Kafka,
Kubernetes,
multiple microservices,
vector databases,
WebSockets,
event sourcing,
multiple caches.

None of those technologies is inherently an upgrade.

For this system, I would first strengthen the guarantees around the architecture that already exists:

DB/queue consistency
        ↓
Idempotent processing
        ↓
Per-ticket ordering/versioning
        ↓
Observable job state
        ↓
AI evaluation
        ↓
Scale infrastructure when evidence requires it

Complexity should be purchased with a requirement.

The Most Important Lesson: AI Is a Workload, Not the Architecture

It is tempting to describe this application as an OpenAI integration.

That misses most of the engineering.

The LLM is one dependency inside a larger system.

The architecture still has to answer ordinary software-engineering questions:

Who owns state?
Which operations block the user?
What should happen asynchronously?
Where is authorization enforced?
What happens after partial failure?
Are retries safe?
Can concurrent work overwrite newer state?
How does the UI learn that background work finished?
Where does human approval belong?

Those questions existed before LLMs.

AI simply makes several of them more visible because model calls are external, comparatively slow, probabilistic and failure-prone.

Final Architecture

The current design can be summarized as:

                           React + TypeScript
                                  │
                 ┌────────────────┼────────────────┐
                 │                │                │
            Auth Context     TanStack Query    Local UI State
                 │                │                │
                 └────────────────┼────────────────┘
                                  │
                                  ▼
                           Express REST API
                                  │
                    Auth / AuthZ / Validation
                                  │
                                  ▼
                       Application Services
                         │               │
                         │               │
                         ▼               ▼
                    PostgreSQL      Redis / BullMQ
                                         │
                                         ▼
                                     AI Worker
                                         │
                             ┌───────────┼───────────┐
                             │           │           │
                          Summary    Sentiment    Priority
                             │           │           │
                             └───────────┼───────────┘
                                         │
                                    AI Providers

Interactive reply path:

Agent → API → AI reply suggestion → editable composer → human sends

It is intentionally simpler than a large distributed system.

And it has known limitations.

I consider both of those facts healthy.

Good architecture is not about pretending the current system can handle every future requirement.

It is about knowing which guarantees the system currently provides, which ones it doesn’t, and where the next architectural pressure points will appear.

Closing

Building this project changed how I think about adding AI to existing applications.

I wouldn’t start by asking:

Where can I call the LLM?

I would start with:

What role should AI have in this workflow?

Then decide:

synchronous or asynchronous,
advisory or autonomous,
user-facing or background,
retryable or idempotent,
cached or authoritative,
human-reviewed or automatically executed.

Once those boundaries are clear, choosing the API is usually the easier part.

In the next article in this architecture series, I’ll go deeper into BullMQ background jobs for AI workloads—especially retries, idempotency and ordering, because adding a queue is only the beginning of making background processing reliable.

About the author

I’m Ram Ji Tripathi, a senior frontend-focused full-stack engineer working across React, TypeScript, Node.js and AI-enabled applications.

I write about frontend architecture, backend systems, AI engineering, system design and the engineering decisions behind building real software.

More engineering notes and projects are available on my portfolio.

Originally published on my engineering blog:
https://blog.ramjwork.in/ai-engineering/ai-customer-support-architecture