DEV Community: Mehedi Hassan Shifat

Can your Odoo automations trigger each other forever?

Mehedi Hassan Shifat — Wed, 11 Nov 2026 10:12:30 +0000

With enough automated actions, one of them will eventually write a field that another one watches. Odoo's automation module defines exactly when that happens: create triggers, write triggers, and special triggers (stage, state, priority, user, tags, archive) that watch a specific field.

Disclosure: I make a paid Odoo module that draws this map (link at the bottom).

That means the chain is computable. For each automation, look at what its actions write (object-write actions give a field path, Python actions can be read for write({...}), record.field = ... and env[...].create), and link it to every active automation that listens to that model and field. Strongly connected groups are loops, and two automations writing the same field on the same event is a conflict.

I built a module that draws that map, grouped by model, and flags loops, conflicts, rules with no action, code that no longer compiles, and schedules that keep failing. It's called Automation Map, 39 EUR on the Apps Store: https://apps.odoo.com/apps/modules/19.0/automation_map/

It reads Python statically, so code that builds model or field names dynamically can't be followed.

Test-render every Odoo report after an upgrade, without printing anything

Mehedi Hassan Shifat — Wed, 11 Nov 2026 10:11:18 +0000

After an upgrade, the reports nobody opens are the ones that break. You can test them all without a PDF engine: render each report's QWeb as HTML on a couple of real records, inside a savepoint that you always roll back.

Disclosure: I make a paid Odoo module that automates this (link at the bottom).

Two details matter. First, an error from QWeb wraps the real cause; the exception's __cause__ has it, and the message names the template and element. Second, some standard reports refuse on purpose: Odoo's "Hash integrity result" raises a UserError ("contact your accountant") when its conditions aren't met. That's not breakage, so classify UserError as skipped.

On a demo database with 50 reports, 32 rendered fine, 16 were skipped because there was no record to test with, and only the two I had broken on purpose failed. Nothing in Odoo's own reports was flagged.

I turned this into a module called Report Breakage Checker, 29 EUR on the Apps Store: https://apps.odoo.com/apps/modules/19.0/report_breakage_checker/

Odoo's filestore: duplicates cost less than you think, orphans cost more

Mehedi Hassan Shifat — Wed, 11 Nov 2026 10:10:07 +0000

Odoo's filestore stores a file by its checksum, so attaching the same PDF to ten records stores it once. A "duplicates" report that adds up sizes will overstate what you could save. The number that matters is the stored size, counting each file once.

Disclosure: I make a paid Odoo module that reports on this (link at the bottom).

Orphans are the other surprise. If an attachment's record was deleted (by SQL, or a failed cascade), Odoo hides the attachment from every user except the superuser, so the Attachments list never shows it. They take space and nobody can see them.

A third case is a database row whose file is gone from the filestore. Pages that show attachments fail on those.

I wrote a module that reports total and stored size, the biggest files, duplicate groups, orphans (with a confirmed delete that re-checks first) and missing files. It's called Attachment Analyser, 39 EUR on the Apps Store: https://apps.odoo.com/apps/modules/19.0/attachment_analyser/

Finding the slowest call in an Odoo database in five minutes

Mehedi Hassan Shifat — Wed, 11 Nov 2026 10:08:50 +0000

Odoo has a profiler, but you have to know which page to profile. The web client sends almost every action to one URL, /web/dataset/call_kw/<model>/<method>, so a request log by URL tells you nothing.

Disclosure: I make a paid Odoo module that does this measuring (link at the bottom).

What helps is naming the call by model and method, removing ids so all orders count as one page, and measuring duration and query count per call. Buffer the measurements in memory and write them in one batch every 15 seconds, so it adds no database write per request.

On a demo database I found something I hadn't looked for: the Contacts app calls res.company.iap_enrich_auto on every load and it took about 2 seconds each time, because it tries to reach Odoo's IAP service, which my sandbox can't. On a server with internet access it would likely be faster, but it is the sort of thing you'd never spot by feel.

I turned this into a module that lists pages slowest first by total time, with the average, the time 95% of calls stay within, queries per call, and the latest slow requests with who ran them. It's called Slow Page Finder, 19 EUR on the Apps Store: https://apps.odoo.com/apps/modules/19.0/slow_page_finder/

It measures server time only, not browser rendering.

Why a new Odoo record still shows English in a French database

Mehedi Hassan Shifat — Wed, 11 Nov 2026 10:07:09 +0000

Odoo stores translatable fields as JSON, one key per language: {"en_US": "Partner", "fr_FR": "Partenaire"}. If the French key is missing, Odoo shows the English text. That's the first kind of gap.

Disclosure: I make a paid Odoo module that scans for these (link at the bottom).

The second kind surprised me. Create a record from a French session and Odoo stores the text as both English and French values, so the French value exists and is just the English word. The record has a "translation" and nobody ever translated it. Looking for missing keys can't find it, so you also have to compare the two values.

A third detail is about .po files: entries marked fuzzy are loaded and shown to users, so they aren't gaps, only worth a review.

I made a module that scans every translatable field per installed language (counting missing and same-as-English), lists the module terms that a .po leaves empty, and opens the exact records in that language so you can fix them. It's called Translation Gap Finder, 19 EUR on the Apps Store: https://apps.odoo.com/apps/modules/19.0/translation_gap_finder/

Before you upgrade to Odoo 20: PostgreSQL 16, Python 3.12, and 616 removed fields

Mehedi Hassan Shifat — Wed, 11 Nov 2026 10:05:56 +0000

Two hard requirements changed between Odoo 19 and 20, straight from odoo/release.py: the minimum PostgreSQL goes from 13 to 16, and the minimum Python from 3.10 to 3.12. If you're still on PostgreSQL 14 or 15, that is the first job, and it has nothing to do with Odoo itself.

Disclosure: I make a paid Odoo module for the rest of this (link at the bottom).

Then there's your data. I installed a broad set of Community apps on both versions and compared every model and field. Odoo 20 has 39 fewer models and 616 fewer fields in what I compared. Examples: res.partner.company_type, the whole hr.contract.type model, stock.scrap.

That matters because customisations don't follow the data. A view that shows company_type is valid on 19, and on 20 the field it points at is gone. The same goes for saved filters, record rules, and fields you added from the interface that point at a removed model.

I made a module that reads your database, checks it against that comparison, and lists the custom views, filters, rules and fields that will break, plus stuck modules, non-standard modules and dangling external IDs. It's called Upgrade Checker, 59 EUR on the Apps Store: https://apps.odoo.com/apps/modules/19.0/upgrade_checker/

Limits: the field comparison covers Community modules, so Enterprise fields aren't in it.

Odoo can send an email with no recipient and tell you nothing

Mehedi Hassan Shifat — Wed, 11 Nov 2026 10:04:18 +0000

Odoo checks a mail template when you save it. After that, nothing checks it again. A field gets removed by an upgrade, a module is uninstalled, or an expression meets a record with an empty value, and the template breaks.

Disclosure: I make a paid Odoo module that tests templates (link at the bottom). The technique below works without it.

I tried it on Odoo 19: _generate_template with a broken placeholder in email_to returns an empty recipient and no error. The email just has nobody to go to. Calling _render_field on the same field raises the real error. (Subject and body failures do raise.)

So to test templates properly, render each field with _render_field, on more than one record, because a template like {{ object.parent_id.name.upper() }} works on a contact with a parent and fails on one without. Do it inside a savepoint you always roll back, so nothing is created or sent.

I built that into a module that checks every template and reports errors, placeholders left as text, and whole records printed as res.partner(36, 19). On the 33 standard templates of a demo database it raised no false alarm (31 OK, 2 skipped). It's called Email Template Checker, 19 EUR on the Apps Store: https://apps.odoo.com/apps/modules/19.0/email_template_checker/

It does not render attachments or PDF reports.

What does uninstalling an Odoo module really delete? I measured it for CRM

Mehedi Hassan Shifat — Wed, 11 Nov 2026 10:03:06 +0000

The uninstall button warns you about dependent apps and nothing else. So I took a demo database, wrote down what I expected to be removed, uninstalled CRM, and compared the database before and after.

Disclosure: I make a paid Odoo module that previews this (link at the bottom). The numbers below don't depend on it.

CRM takes 5 other modules with it (6 in total). The database loses 32 tables and 31 columns from other tables, and 286 records go with them, leads included.

The rule Odoo follows: everything the modules own through an external ID is deleted, unless a module that stays also owns it. Models lose their tables, fields their columns, and selection values that records still use are a problem on their own.

I turned that into a module. It computes the list and shows it before you click, and I checked its numbers against a real uninstall: modules, tables, columns and record counts all matched. It's called Safe Uninstall Preview, 39 EUR on the Apps Store: https://apps.odoo.com/apps/modules/19.0/safe_uninstall_preview/

The numbers come from a demo database. On yours they will differ, which is the point of looking before you click.

Odoo 20 runs on Owl 3. What stops working in your custom module?

Mehedi Hassan Shifat — Wed, 11 Nov 2026 10:00:31 +0000

If you maintain custom Odoo modules, the Owl change is probably the biggest thing in the 19 to 20 jump. I compared the two releases instead of trusting blog posts, so here is what I found.

Disclosure: I make a paid Odoo module that checks for this (link at the bottom). The findings below stand without it.

Owl itself. Odoo 19 ships Owl 2.8.4. Odoo 20 ships Owl 3.0.0-alpha.50. These names are no longer exported by Owl: loadFile, onRendered, onWillRender, reactive, status, useChildSubEnv, useComponent, useEnv, useExternalListener, useRef, useState, useSubEnv, validate.

The compatibility layer. Odoo 20 includes web/static/src/owl2/owl3_compatibility_layer.js, which describes itself as temporary. It puts back Component, App, mount, status, onWillRender, useComponent, useEnv and useSubEnv, but not useState, useRef, reactive, useExternalListener, onRendered, useChildSubEnv, validate or loadFile. It also asks you to rename t-portal to t-custom-portal, t-model to t-custom-model, and useEffect to useLayoutEffect.

To see how much Odoo itself moved: useState is imported in 486 JavaScript files of Odoo 19 and in 2 of Odoo 20. useRef goes from 270 files to none.

Python is changing too. In the 20.0 image I checked, odoo.osv.expression, odoo.service.db and odoo.tools.pycompat no longer exist, ir.rule and ir.model.access were replaced by ir.access, and _check_recursion is no longer a method of every model.

Checking all of that by hand across several modules is slow, so I wrote a scanner that reads a module's Python, XML and JavaScript and lists each line that breaks, with the reason. It's called Odoo 20 Code Scanner, 39 EUR on the Apps Store: https://apps.odoo.com/apps/modules/19.0/odoo20_code_scanner/

The comparison covers Community modules only, and the scan is static: code that builds names at run time can't be followed.

XML-RPC is going away in Odoo 22. Here's what the JSON-2 version looks like

Mehedi Hassan Shifat — Wed, 11 Nov 2026 09:54:12 +0000

If you have scripts or integrations that talk to Odoo over /xmlrpc/2 or /jsonrpc, Odoo 19 already logs a warning every time they run. The warning says those endpoints are scheduled for removal in Odoo 22. They still work in 20, so nothing is on fire. But the replacement works differently enough that it's worth knowing before you have fifteen integrations to move.

Disclosure: I make a paid Odoo module that helps with this (more at the bottom). The rest of this post doesn't need it.

Before

import xmlrpc.client
common = xmlrpc.client.ServerProxy(f"{url}/xmlrpc/2/common")
uid = common.authenticate(db, login, password, {})
models = xmlrpc.client.ServerProxy(f"{url}/xmlrpc/2/object")
rows = models.execute_kw(db, uid, password, "res.partner", "search_read",
                         [[["is_company", "=", True]]], {"fields": ["name"], "limit": 5})

After

import requests
rows = requests.post(
    f"{url}/json/2/res.partner/search_read",
    headers={"Authorization": f"bearer {api_key}"},
    json={"domain": [["is_company", "=", True]], "fields": ["name"], "limit": 5},
).json()

A few things that tripped me up while checking this:

There's no login call. You make an API key for the user and send it as a bearer token.
Arguments are named. In XML-RPC you pass them by position, so you have to look at the Python signature of the method to see what each one is called. search_read takes domain, fields, offset, limit, order.
Methods that work on records want an ids key, e.g. {"ids": [1, 2], "vals": {...}} for write.
Only public methods work. Anything starting with an underscore is refused.
The database isn't in the request. The key decides it.

The annoying part of a migration isn't writing the new call. It's finding every client that still uses the old one. The log tells you something is calling, not who. You can get a rough answer from your reverse proxy's access log by filtering on the three paths.

I wanted more than that (which client, which model, which method, how many times), so I wrote an Odoo module that counts legacy calls and shows the JSON-2 equivalent next to each one. It never stores arguments or passwords. It's called API Migration Checker and it's on the Apps Store: https://apps.odoo.com/apps/modules/19.0/api_migration_checker/

One date caveat: several blog posts say removal happens in Odoo 20. Odoo's own deprecation notice says 22, and I confirmed the endpoints still answer on a 20.0 server.

Building a Production-Ready Shopify App with Background Jobs: A Complete Architecture Guide

Mehedi Hassan Shifat — Tue, 09 Dec 2025 17:05:35 +0000

Building a Production-Ready Shopify App with Background Jobs: A Complete Architecture Guide

Building a Shopify app that handles heavy data processing requires careful architectural decisions. In this comprehensive guide, I'll walk you through how we built a scalable Shopify pricing app using Remix, BullMQ, Redis, and a dedicated worker process to handle background jobs efficiently.

Architecture Overview
Project Structure
Setting Up the Foundation
Building the Queue System
Creating Workers
Handling Job Recovery
Deployment Strategy
Best Practices & Lessons Learned

Architecture Overview

Our Shopify app processes bulk price updates, which can involve thousands of products. Here's the high-level architecture:

┌─────────────────┐
│  Shopify Admin  │
└────────┬────────┘
         │
         ▼
┌─────────────────────────┐
│   Remix Web Server      │
│  (Handles UI & API)     │
└────────┬────────────────┘
         │
         ▼
┌─────────────────────────┐
│   Redis + BullMQ        │
│  (Job Queues)           │
└────────┬────────────────┘
         │
         ▼
┌─────────────────────────┐
│   Worker Process        │
│  (Background Jobs)      │
└─────────────────────────┘

Why This Architecture?

Separation of Concerns: Web server handles HTTP requests; worker handles heavy lifting
Scalability: Workers can scale independently from the web server
Reliability: Jobs persist in Redis; if a worker crashes, jobs can be recovered
Performance: Non-blocking operations keep the web server responsive

Project Structure

Here's how we organized our codebase:

bevy-price-rules/
├── app/
│   ├── routes/                    # Remix routes (API & UI)
│   │   ├── api.sales.ts          # Enqueue price update jobs
│   │   ├── api.upload.ts         # Handle CSV uploads
│   │   └── api.revert.ts         # Revert operations
│   ├── services/
│   │   └── queue/                # Queue infrastructure
│   │       ├── config.server.ts          # Redis & queue config
│   │       ├── request-queue.server.ts   # Request queue
│   │       ├── request-worker.server.ts  # Request worker
│   │       ├── processing-queue.server.ts    # Processing queue
│   │       ├── processing-worker.server.ts   # Processing worker
│   │       ├── waiting-queue.server.ts       # Waiting queue
│   │       └── waiting-worker.server.ts      # Waiting worker
│   ├── types/
│   │   └── queue.type.ts         # TypeScript interfaces
│   └── db.server.ts              # Prisma client
├── worker.ts                     # Main worker entry point
├── tsconfig.worker.json          # Worker-specific TypeScript config
├── package.json                  # Scripts for building workers
├── Dockerfile                    # Docker configuration
└── heroku.yml                    # Heroku deployment config

Setting Up the Foundation

1. Install Dependencies

{
  "dependencies": {
    "@remix-run/node": "^2.16.1",
    "@shopify/shopify-app-remix": "^3.7.0",
    "@prisma/client": "^6.2.1",
    "bullmq": "^5.56.0",
    "ioredis": "^5.6.1",
    "dotenv": "^17.0.1"
  },
  "devDependencies": {
    "typescript": "^5.0.0",
    "ts-node": "^10.9.1"
  }
}

2. Configure Package Scripts

{
  "scripts": {
    "build": "remix vite:build",
    "start": "remix-serve ./build/server/index.js",
    "worker:build": "tsc -p ./tsconfig.worker.json && tsc-esm-fix --target=dist",
    "worker:start": "node ./dist/worker.js",
    "worker:dev": "node --loader ts-node/esm --no-warnings --experimental-specifier-resolution=node ./worker.ts"
  }
}

3. Worker TypeScript Configuration

Create tsconfig.worker.json to exclude UI-related files:

{
  "compilerOptions": {
    "module": "ESNext",
    "esModuleInterop": true,
    "target": "ES2022",
    "moduleResolution": "bundler",
    "outDir": "dist",
    "baseUrl": ".",
    "strict": true,
    "skipLibCheck": true
  },
  "include": [
    "./worker.ts",
    "./app/**/*.ts"
  ],
  "exclude": [
    "node_modules",
    "./app/**/*.tsx",
    "./app/components/**/*",
    "./app/routes/**/*",
    "./app/pages/**/*",
    "./app/providers/**/*"
  ]
}

4. Optimize Vite Configuration

Exclude worker-related dependencies from the web bundle:

// vite.config.ts
export default defineConfig({
  optimizeDeps: {
    exclude: [
      "bullmq", 
      "./app/services/*",
      "@sentry/node"
    ],
  },
  resolve: {
    alias: {
      fs: path.resolve("./app/shims/fs.ts"),
    },
  },
});

Building the Queue System

1. Redis Configuration

Create a robust Redis connection with retry logic:

// app/services/queue/config.server.ts
import Redis from 'ioredis';
import type { RedisOptions } from 'ioredis';

export const redisConfig: RedisOptions = {
  tls: process.env.REDIS_TLS === 'true' 
    ? { rejectUnauthorized: false } 
    : undefined,
  db: 0,
  family: 4,
  enableAutoPipelining: true,
  maxRetriesPerRequest: null,
  enableReadyCheck: true,
  connectTimeout: 60000,

  retryStrategy: (times) => {
    const maxDelay = 30000;
    const maxAttempts = 20;

    if (times >= maxAttempts) {
      console.error(`Redis unreachable after ${maxAttempts} attempts`);
      return null;
    }

    const delay = Math.min(100 * Math.pow(2, times), maxDelay);
    console.warn(`Redis retry #${times} in ${delay}ms...`);
    return delay;
  },

  reconnectOnError: (err) => {
    const recoverableErrors = ['READONLY', 'ECONNRESET', 'ETIMEDOUT'];
    return recoverableErrors.some(error => err?.message?.includes(error));
  },
};

export const redisClient = new Redis(process.env.REDIS_URI!, redisConfig);

export const defaultQueueOptions = {
  attempts: 3,
  backoff: {
    type: 'exponential',
    delay: 10_000,
  },
  removeOnComplete: true,
  removeOnFail: true,
};

export const defaultStreams = {
  events: {
    maxLen: 5, // Limit stream size
  },
};

2. Define Queue Names & Job Data Types

// app/types/queue.type.ts
export enum QueueJobNames {
  requestQ = 'bulk-price-update-request',
  processingQ = 'bulk-price-processing',
  waitingQ = 'bulk-price-waiting',
  computeEngineQ = 'compute-engine',
  productWebhookQ = 'product-webhook',
}

export interface IRequestJobData {
  shop: string;
  shopId: number;
  identifier: string; // UUID for tracking
  entityId: number;   // Price rule or upload ID
  type: SyncLogType;
  source: BulkOperationSourceType;
  action: ActionType;
  bulkOperationJobId: number;
  priority?: number;
}

3. Create the Request Queue

// app/services/queue/request-queue.server.ts
import { Queue, QueueEvents } from 'bullmq';
import { redisClient, QueueJobNames, defaultQueueOptions } from './config.server';

const RequestQueue = new Queue(QueueJobNames.requestQ, {
  connection: redisClient,
  defaultJobOptions: defaultQueueOptions,
});

const queueEvents = new QueueEvents(QueueJobNames.requestQ, {
  connection: redisClient,
});

queueEvents.on('failed', ({ jobId, failedReason }) => {
  console.error(`Job ${jobId} failed: ${failedReason}`);
});

export async function queueBulkOperationRequest(
  data: IRequestJobData
): Promise<{ 
  success: boolean; 
  message: string; 
  duplicateJob: boolean; 
  jobId: string | null 
}> {
  try {
    // Validate required fields
    if (!data?.shop || !data?.shopId || !data?.bulkOperationJobId) {
      return { 
        success: false, 
        message: 'Missing required fields', 
        duplicateJob: false, 
        jobId: null 
      };
    }

    // Create deterministic job ID to prevent duplicates
    const jobId = `${data.shop}__${data.source}__${data.entityId}`;
    const existingJob = await RequestQueue.getJob(jobId);

    if (existingJob) {
      const state = await existingJob.getState();

      // Block duplicate if job is active
      if (state === 'waiting' || state === 'active' || state === 'delayed') {
        console.warn(`Duplicate job blocked: ${jobId} is ${state}`);

        await prisma.bulkOperationJob.update({
          where: { id: data.bulkOperationJobId },
          data: { 
            status: BulkOperationJobStatus.FAILED,
            errorMessage: 'Operation already queued or running',
          },
        });

        return { 
          success: false, 
          message: 'Operation already queued', 
          duplicateJob: true,
          jobId, 
        };
      }

      // Remove completed/failed jobs
      if (state === 'completed' || state === 'failed') {
        await existingJob.remove();
      }
    }

    // Add job to queue
    await RequestQueue.add(
      QueueJobNames.requestQ, 
      data,
      {
        jobId,
        priority: data?.priority || 1,
        attempts: 3,
        removeOnComplete: true,
        removeOnFail: { count: 1, age: 1000 },
        delay: 50,
        backoff: {
          type: 'exponential', 
          delay: 10_000         
        },
      }
    );

    // Update database
    await prisma.bulkOperationJob.update({
      where: { id: data.bulkOperationJobId },
      data: { 
        requestQueueJobId: jobId, 
        requestQueueJobData: JSON.stringify(data),
        status: BulkOperationJobStatus.QUEUED,
      },
    });

    console.log(`Queued job ${jobId} for shop: ${data.shop}`);

    return { success: true, message: 'ok', duplicateJob: false, jobId };
  } catch (error: any) {
    console.error(`Failed to queue job for ${data.shop}:`, error);
    throw error;
  }
}

export default RequestQueue;

4. Using the Queue from API Routes

// app/routes/api.sales.ts
import { json } from "@remix-run/node";
import { queueBulkOperationRequest } from "~/services/queue/request-queue.server";

export async function action({ request }: ActionFunctionArgs) {
  const { admin, session } = await authenticate.admin(request);

  // Create bulk operation job record
  const bulkOperationJob = await prisma.bulkOperationJob.create({
    data: {
      shopId: session.shopId,
      identifier: uuidv4(),
      type: ActionType.CREATE,
      sourceType: BulkOperationSourceType.PRICE_RULE,
      status: BulkOperationJobStatus.PENDING,
    },
  });

  // Queue the job
  const result = await queueBulkOperationRequest({
    shop: session.shop,
    shopId: session.shopId,
    identifier: bulkOperationJob.identifier,
    entityId: priceRuleId,
    type: SyncLogType.CREATE_PRICE_RULES,
    source: BulkOperationSourceType.PRICE_RULE,
    action: ActionType.CREATE,
    bulkOperationJobId: bulkOperationJob.id,
  });

  if (!result.success) {
    return json({ error: result.message }, { status: 400 });
  }

  return json({ success: true, jobId: result.jobId });
}

Creating Workers

1. Request Worker

The request worker validates and routes jobs to the appropriate processing queue:

// app/services/queue/request-worker.server.ts
import { Worker, type Job } from 'bullmq';
import { redisClient, QueueJobNames } from './config.server';
import { queueProcessBulkOperationRequest } from './processing-queue.server';

const MAX_STALLS = 3;
const REQ_STALL_MS = 30 * 1000;
const REQ_LOCK_TTL_MS = 60 * 1000;

export const RequestWorker = () => {
  console.log('Starting Request Worker with concurrency: 1');

  const worker = new Worker(
    QueueJobNames.requestQ,
    async (job: Job) => {
      const { shop, shopId, bulkOperationJobId } = job.data;

      console.log(`Processing request for shop: ${shop}, job ${job.id}`);

      try {
        job.updateProgress(0);

        if (!shop || !shopId || !bulkOperationJobId) {
          throw new Error('Missing required fields');
        }

        // Check stall count
        const stalledCount = (job as any).stalledCounter ?? 0;
        if (stalledCount >= MAX_STALLS) {
          throw new Error('Job stalled too many times');
        }

        // Forward to processing queue
        const result = await queueProcessBulkOperationRequest(job.data);

        if (!result?.success) {
          throw new Error(result?.message);
        }

        return { success: true, message: 'ok', shop };
      } catch (error: any) {
        console.error(`Failed to process job ${job.id}:`, error);
        throw error; 
      }
    },
    { 
      connection: redisClient,
      concurrency: 1,
      stalledInterval: REQ_STALL_MS,
      lockDuration: REQ_LOCK_TTL_MS, 
      drainDelay: 5, 
      removeOnComplete: {
        count: 1,
        age: 1000
      },
      removeOnFail: { 
        count: 1,
        age: 1000
      }
    }
  );

  // Event handlers
  worker.on('completed', async (job) => {
    console.log(`Job ${job.id} completed`);
  });

  worker.on('failed', async (job, err) => {
    const attemptsMade = job?.attemptsMade || 1;
    const jobAttemptsLimit = job?.opts?.attempts || 1;
    const isFinal = attemptsMade >= jobAttemptsLimit;

    if (isFinal) {
      console.error(`Job ${job.id} failed permanently:`, err);

      await prisma.bulkOperationJob.update({
        where: { id: job.data.bulkOperationJobId },
        data: { 
          status: BulkOperationJobStatus.FAILED,
          errorMessage: err?.message,
        },
      });
    }
  });

  worker.on('stalled', async (jobId) => {
    console.warn(`Job ${jobId} has stalled`);
  });

  worker.on('error', (error) => {
    console.error('Worker error:', error);
  });

  return worker;
};

2. Processing Worker

The processing worker handles the actual business logic:

// app/services/queue/processing-worker.server.ts
import { Worker, type Job } from 'bullmq';
import { redisClient, QueueJobNames } from './config.server';

export const ProcessingWorker = () => {
  console.log('Starting Processing Worker with concurrency: 5');

  const worker = new Worker(
    QueueJobNames.processingQ,
    async (job: Job) => {
      const { shop, shopId, entityId, source, action } = job.data;

      console.log(`Processing ${action} for ${source} ${entityId}`);

      try {
        job.updateProgress(10);

        // Mark as active in DB
        await prisma.bulkOperationJob.update({
          where: { id: job.data.bulkOperationJobId },
          data: { 
            status: BulkOperationJobStatus.PROCESSING,
            activeInWorker: true,
          },
        });

        // Perform the actual work (Shopify API calls, etc.)
        switch (action) {
          case ActionType.CREATE:
            await handleCreateAction(job.data);
            break;
          case ActionType.UPDATE:
            await handleUpdateAction(job.data);
            break;
          case ActionType.DELETE:
            await handleDeleteAction(job.data);
            break;
        }

        job.updateProgress(100);

        // Mark as complete
        await prisma.bulkOperationJob.update({
          where: { id: job.data.bulkOperationJobId },
          data: { 
            status: BulkOperationJobStatus.COMPLETED,
            activeInWorker: false,
            completedAt: new Date(),
          },
        });

        return { success: true };
      } catch (error: any) {
        console.error(`Processing failed for job ${job.id}:`, error);

        await prisma.bulkOperationJob.update({
          where: { id: job.data.bulkOperationJobId },
          data: { 
            status: BulkOperationJobStatus.FAILED,
            activeInWorker: false,
            errorMessage: error.message,
          },
        });

        throw error;
      }
    },
    { 
      connection: redisClient,
      concurrency: 5, // Process multiple jobs in parallel
      stalledInterval: 60 * 1000,
      lockDuration: 120 * 1000,
    }
  );

  return worker;
};

3. Main Worker Entry Point

// worker.ts
import * as dotenv from 'dotenv';
import { RequestWorker } from './app/services/queue/request-worker.server';
import { ProcessingWorker } from './app/services/queue/processing-worker.server';
import { WaitingRequestWorker } from './app/services/queue/waiting-worker.server';
import prisma, { disconnectPrisma } from "./app/db.server";

dotenv.config();

(async () => {
  try {
    // Start all workers
    const workers = [
      RequestWorker(),
      ProcessingWorker(),
      WaitingRequestWorker(),
    ];

    console.log('Running Workers:', workers.map(w => w?.name));

    // Resume incomplete operations (see next section)
    await resumeIncompleteBulkOperations();

    // Graceful shutdown
    async function shutdown(signal: string) {
      console.log(`Received ${signal}, shutting down gracefully...`);

      try {
        // Mark all jobs as inactive
        await prisma.bulkOperationJob.updateMany({
          where: { activeInWorker: true },
          data: { activeInWorker: false },
        });

        // Close workers
        await Promise.all(workers.map(w => w.close()));

        // Disconnect from database
        await disconnectPrisma();

        console.log('Shutdown complete');
        process.exit(0);
      } catch (error) {
        console.error('Error during shutdown:', error);
        process.exit(1);
      }
    }

    process.on('SIGTERM', () => shutdown('SIGTERM'));
    process.on('SIGINT', () => shutdown('SIGINT'));

  } catch (error) {
    console.error('Failed to start workers:', error);
    process.exit(1);
  }
})();

Handling Job Recovery

One of the most critical features is recovering from crashes. Here's how we handle it:

// Inside worker.ts
async function resumeIncompleteBulkOperations() {
  try {
    await sleep(1000); // Wait for Redis to stabilize
    console.log('[Recovery] Scanning for incomplete operations...');

    // Reset activeInWorker flag
    await prisma.bulkOperationJob.updateMany({
      where: { activeInWorker: true },
      data: { activeInWorker: false },
    });

    const twentyFourHoursAgo = new Date(Date.now() - 24 * 60 * 60 * 1000);
    const pageSize = 10;
    let page = 0;

    while (true) {
      // Find incomplete jobs
      const incompleteJobs = await prisma.bulkOperationJob.findMany({
        where: {
          status: {
            in: [
              BulkOperationJobStatus.QUEUED,
              BulkOperationJobStatus.PROCESSING,
              BulkOperationJobStatus.ACTIVE,
              BulkOperationJobStatus.PENDING,
            ]
          },
          activeInWorker: false,
          createdAt: { gte: twentyFourHoursAgo },
        },
        include: {
          shop: { select: { id: true, domain: true, status: true } },
        },
        orderBy: { createdAt: 'asc' },
        skip: page * pageSize,
        take: pageSize,
      });

      if (incompleteJobs.length === 0) break;

      for (const job of incompleteJobs) {
        try {
          if (job.status === BulkOperationJobStatus.PROCESSING && 
              job.processingQueueJobData) {
            // Re-queue to processing
            const jobData = JSON.parse(job.processingQueueJobData);
            await queueProcessBulkOperationRequest(jobData);
            console.log(`[Recovery] Re-queued processing job ${job.id}`);
          } 
          else if (job.status === BulkOperationJobStatus.QUEUED && 
                   job.requestQueueJobData) {
            // Re-queue to waiting
            const jobData = JSON.parse(job.requestQueueJobData);
            await queueBulkOperationToWaitingQueue(jobData);
            console.log(`[Recovery] Re-queued request job ${job.id}`);
          }
        } catch (err) {
          console.error(`[Recovery] Error resuming job ${job.id}:`, err);
        }
      }

      page++;
    }

    console.log('[Recovery] Job recovery complete');
  } catch (err) {
    console.error('[Recovery] Failed to scan for incomplete jobs:', err);
  }
}

Key Recovery Features:

Runs on worker startup
Only recovers jobs from last 24 hours
Preserves job state and data
Handles pagination for large datasets
Logs all recovery actions

Deployment Strategy

1. Dockerfile

FROM node:22

WORKDIR /app

COPY package.json package-lock.json* ./
COPY prisma ./prisma/

# Clean install
RUN rm -f package-lock.json && \
    npm install --legacy-peer-deps

RUN npx prisma generate

# Copy application code
COPY . .

# Build web server
RUN npm run build

# Build worker
RUN npm run worker:build

2. Heroku Configuration

# heroku.yml
build:
  docker:
    web: Dockerfile
    worker: Dockerfile

run:
  web: npm run docker-start
  worker:
    command:
      - npm run worker:start
    image: web

release:
  image: web
  command:
    - echo 'Starting server after deployment!'

3. Environment Variables

# Redis
REDIS_URI=redis://user:password@host:port
REDIS_TLS=true

# Database
DATABASE_URL=postgresql://user:password@host:port/dbname

# Shopify
SHOPIFY_API_KEY=your_api_key
SHOPIFY_API_SECRET=your_api_secret

# Worker Configuration
REQ_QUEUE_MAX_STALLS=3
REQ_STALL_MS=30000
REQ_LOCK_TTL_MS=60000

4. Scaling on Heroku

# Scale web dynos
heroku ps:scale web=2:standard-2x

# Scale worker dynos
heroku ps:scale worker=2:standard-2x

# Monitor
heroku logs --tail --dyno=worker

Best Practices & Lessons Learned

1. Use Deterministic Job IDs

Prevent duplicate jobs by creating IDs from shop + source + entity:

const jobId = `${shop}__${source}__${entityId}`;

2. Implement Retry Logic with Backoff

{
  attempts: 3,
  backoff: {
    type: 'exponential',
    delay: 10_000 // 10s, then 20s, then 40s
  }
}

3. Track Job State in Database

Never rely solely on Redis for state. Always maintain a source of truth in your database:

await prisma.bulkOperationJob.update({
  where: { id: jobId },
  data: { 
    status: BulkOperationJobStatus.PROCESSING,
    requestQueueJobData: JSON.stringify(data),
    processingQueueJobData: JSON.stringify(data),
  },
});

4. Handle Stalled Jobs

Jobs can stall due to crashes or network issues. Detect and recover:

const stalledCount = (job as any).stalledCounter ?? 0;
if (stalledCount >= MAX_STALLS) {
  throw new Error('Job stalled too many times');
}

5. Implement Graceful Shutdown

Always clean up on shutdown:

process.on('SIGTERM', async () => {
  await prisma.bulkOperationJob.updateMany({
    where: { activeInWorker: true },
    data: { activeInWorker: false },
  });
  await Promise.all(workers.map(w => w.close()));
  await disconnectPrisma();
  process.exit(0);
});

6. Use Multiple Queues for Different Priorities

Request Queue: Fast validation, concurrency 1
Processing Queue: Heavy lifting, concurrency 5+
Waiting Queue: Rate-limited operations

7. Clean Up Completed Jobs

Prevent Redis memory bloat:

{
  removeOnComplete: true,
  removeOnFail: { count: 1, age: 1000 }
}

8. Monitor Redis Connection Health

Implement robust retry strategies:

retryStrategy: (times) => {
  if (times >= maxAttempts) return null;
  return Math.min(100 * Math.pow(2, times), maxDelay);
}

9. Separate Worker TypeScript Configuration

Exclude UI dependencies to reduce worker bundle size:

{
  "exclude": [
    "./app/components/**/*",
    "./app/routes/**/*",
    "./app/pages/**/*"
  ]
}

10. Use Job Progress for User Feedback

Update progress to show users real-time status:

await job.updateProgress(50);

Performance Considerations

Redis Optimization

Enable autoPipelining for batched commands
Use maxLen on streams to prevent unbounded growth
Set appropriate TTL values for locks

Worker Optimization

Adjust concurrency based on workload (I/O-bound vs CPU-bound)
Set stalledInterval to detect hung jobs quickly
Use lockDuration longer than expected job duration

Database Optimization

Index frequently queried fields (shopId, status, identifier)
Use connection pooling
Implement pagination for bulk queries

Monitoring & Debugging

Log Critical Events

worker.on('completed', (job) => {
  console.log(`[Worker] Job ${job.id} completed`);
});

worker.on('failed', (job, err) => {
  console.error(`[Worker] Job ${job.id} failed:`, err);
});

worker.on('stalled', (jobId) => {
  console.warn(`[Worker] Job ${jobId} stalled`);
});

Health Checks

Implement health check endpoints:

// app/routes/health.ts
export async function loader() {
  try {
    await redisClient.ping();
    await prisma.$queryRaw`SELECT 1`;
    return json({ status: 'healthy' });
  } catch (error) {
    return json({ status: 'unhealthy' }, { status: 503 });
  }
}

BullMQ Dashboard

Use Bull Board for visual monitoring:

npm install @bull-board/api @bull-board/express

Conclusion

Building a production-ready Shopify app with background jobs requires careful planning, but the payoff is huge:

✅ Scalable: Handle thousands of concurrent operations

✅ Reliable: Jobs survive crashes and get recovered

✅ Performant: Non-blocking, keeps your app responsive

✅ Maintainable: Clear separation of concerns

The architecture we've built processes millions of price updates reliably, scales horizontally, and recovers gracefully from failures.

Key Takeaways

Use BullMQ for reliable job processing
Implement multiple queues for different workloads
Always store job state in your database
Build in job recovery from day one
Deploy web and worker as separate processes

Resources

Questions or suggestions? Feel free to reach out

Happy coding! 🚀

DEV Community: Mehedi Hassan Shifat

Can your Odoo automations trigger each other forever?

Test-render every Odoo report after an upgrade, without printing anything

Odoo's filestore: duplicates cost less than you think, orphans cost more

Finding the slowest call in an Odoo database in five minutes

Why a new Odoo record still shows English in a French database

Before you upgrade to Odoo 20: PostgreSQL 16, Python 3.12, and 616 removed fields

Odoo can send an email with no recipient and tell you nothing

What does uninstalling an Odoo module really delete? I measured it for CRM

Odoo 20 runs on Owl 3. What stops working in your custom module?

XML-RPC is going away in Odoo 22. Here's what the JSON-2 version looks like

Building a Production-Ready Shopify App with Background Jobs: A Complete Architecture Guide

Building a Production-Ready Shopify App with Background Jobs: A Complete Architecture Guide

Table of Contents

Architecture Overview

Why This Architecture?

Project Structure

Setting Up the Foundation

1. Install Dependencies

2. Configure Package Scripts

3. Worker TypeScript Configuration

4. Optimize Vite Configuration

Building the Queue System

1. Redis Configuration

2. Define Queue Names & Job Data Types

3. Create the Request Queue

4. Using the Queue from API Routes

Creating Workers

1. Request Worker

2. Processing Worker

3. Main Worker Entry Point

Handling Job Recovery

Deployment Strategy

1. Dockerfile

2. Heroku Configuration

3. Environment Variables

4. Scaling on Heroku

Best Practices & Lessons Learned

1. Use Deterministic Job IDs

2. Implement Retry Logic with Backoff

3. Track Job State in Database

4. Handle Stalled Jobs

5. Implement Graceful Shutdown

6. Use Multiple Queues for Different Priorities

7. Clean Up Completed Jobs

8. Monitor Redis Connection Health

9. Separate Worker TypeScript Configuration

10. Use Job Progress for User Feedback

Performance Considerations

Redis Optimization

Worker Optimization

Database Optimization

Monitoring & Debugging

Log Critical Events

Health Checks

BullMQ Dashboard

Conclusion

Key Takeaways

Resources