DEV Community: Egeo Minotti

I built a saga workflow engine for Bun (no Temporal, no Redis, no Kafka)

Egeo Minotti — Sat, 11 Apr 2026 11:46:02 +0000

TL;DR

I built a workflow engine inside bunqueue with first-class saga compensation, retries with exponential backoff, branching, parallel steps, loops, sub-workflows, and human-in-the-loop signals. It runs on Bun + SQLite. Zero external infrastructure. The DSL is plain TypeScript and reads like pseudocode. Source: bunqueue.dev/guide/workflow.

The problem nobody wants to deal with

Distributed transactions are the most underestimated problem in modern backend.

An order comes in. You need to validate it, reserve inventory, charge the card, notify the warehouse, send a confirmation email, update analytics. Six services, six failure points.

What happens if the charge succeeds but the warehouse is offline? You just took money from a customer for a product that will never ship.

The academic answer is two-phase commit. The pragmatic answer, since 1987, is the Saga Pattern: split the transaction into steps, each with its own compensation, and if anything goes wrong, run the rollbacks in reverse order.

It's elegant. It's also a nightmare to implement by hand.

Why I built another workflow engine

I evaluated the existing players:

Temporal: powerful but operationally heavy. You're running a cluster.
Inngest: cloud-only by default. Vendor lock-in.
Trigger.dev: nice DX, but again, infrastructure to run.
AWS Step Functions: locked to AWS, JSON-based DSL, no local dev story.

I wanted sagas without any of that. Just TypeScript, a SQLite file, and a fluent DSL that reads like pseudocode. So I built one inside bunqueue, the job queue I already maintain for Bun.

The whole thing is ~1500 lines and lives at src/client/workflow/. Each step is a regular bunqueue job on an internal __wf:steps queue. Persistence is the same SQLite store. State survives restarts, crashes, deploys.

The smallest possible example

import { Workflow, Engine } from 'bunqueue/workflow';

const flow = new Workflow('greet')
  .step('hello', async (ctx) => {
    return { message: `Hi, ${(ctx.input as { name: string }).name}` };
  });

const engine = new Engine({ embedded: true });
engine.register(flow);

const run = await engine.start('greet', { name: 'Ada' });

That's the entire surface area for a one-step workflow. embedded: true means no separate server — bunqueue spins up an in-process SQLite store and runs steps as background jobs.

Compensation: the part that actually matters

Here's where the saga pattern earns its keep. You attach a compensate handler to any step, and if a later step fails, the engine walks back through everything that completed and runs the compensations in reverse.

const checkout = new Workflow('checkout')
  .step('reserve-inventory',
    async (ctx) => {
      const items = (ctx.input as { items: Item[] }).items;
      return { reservationId: await stock.reserve(items) };
    },
    {
      compensate: async (ctx) => {
        const id = (ctx.steps['reserve-inventory'] as { reservationId: string }).reservationId;
        await stock.release(id);
      },
    },
  )
  .step('charge-card',
    async (ctx) => {
      const amount = (ctx.input as { amount: number }).amount;
      return { txId: await payments.charge(amount) };
    },
    {
      retry: 3,
      compensate: async (ctx) => {
        const txId = (ctx.steps['charge-card'] as { txId: string }).txId;
        await payments.refund(txId);
      },
    },
  )
  .step('dispatch-warehouse', async (ctx) => {
    return await warehouse.dispatch(ctx.steps['reserve-inventory']);
  })
  .step('send-receipt', async (ctx) => {
    return await email.send((ctx.input as { email: string }).email);
  });

If dispatch-warehouse throws after charge-card succeeded, the engine automatically:

Refunds the card (compensate for charge-card)
Releases the inventory (compensate for reserve-inventory)

In that order. You write zero rollback orchestration code. No try/catch pyramids, no hand-rolled state machines, no boolean flags scattered across the database to remember what you've already done.

The execution state during compensation is 'compensating', so you can hook into it via the event emitter and surface it to your observability stack.

Retries with exponential backoff

Every step accepts a retry count. The engine retries with min(500ms * 2^attempt + jitter, 30s). The jitter prevents thundering herds when you have 10k workflows hitting the same flaky downstream.

.step('flaky-api-call', handler, { retry: 5, timeout: 10_000 })

Failed attempts emit a step:retry event with the attempt number, so you can wire this into Datadog or whatever you use without a polling loop.

Branching

Routes execution based on runtime context. Only the matching path runs; the others are skipped entirely. Steps after the branch always execute.

new Workflow('kyc')
  .step('assess-risk', async (ctx) => ({ tier: classifyRisk(ctx.input) }))
  .branch((ctx) => (ctx.steps['assess-risk'] as { tier: string }).tier)
    .path('low',    (w) => w.step('auto-approve', autoApprove))
    .path('medium', (w) => w.step('request-docs', requestDocs))
    .path('high',   (w) => w.step('manual-review', manualReview))
  .step('finalize', finalize);

Parallel steps

Promise.allSettled under the hood. If any step in the group fails, the whole group fails and compensation kicks in.

.parallel((w) => w
  .step('notify-warehouse', notifyWarehouse)
  .step('send-email', sendEmail)
  .step('update-analytics', updateAnalytics)
)

Loops and iteration

Three flavors, all with a maxIterations safety limit (default 100):

// doUntil — run, then check
.doUntil(
  (ctx) => (ctx.steps['poll'] as { ready: boolean }).ready,
  (w) => w.step('poll', pollStatus),
  { maxIterations: 20 },
)

// doWhile — check, then run
.doWhile(
  (ctx, i) => i < 5,
  (w) => w.step('process-batch', processBatch),
)

// forEach — iterate over a dynamic list
.forEach(
  (ctx) => (ctx.input as { items: Item[] }).items,
  'process-item',
  async (ctx) => {
    const item = ctx.steps.__item as Item;
    const idx = ctx.steps.__index as number;
    return await process(item, idx);
  },
)

forEach results are stored under indexed names (process-item:0, process-item:1, ...) so you can fan back in afterward.

Sub-workflows

Workflows can call other workflows. Parent pauses, child runs, results nest under ctx.steps['sub:<child-name>']. Failures propagate upward and trigger parent compensation.

.subWorkflow('payment', (ctx) => ({ amount: ctx.input.total }))
.step('after-payment', async (ctx) => {
  const result = ctx.steps['sub:payment'];
  // ...
});

This is how you keep individual workflows small and composable instead of building one monstrous flow.

Human-in-the-loop

This is my favorite feature.

Call .waitFor('event-name') and execution pauses. State becomes 'waiting'. The workflow sits there for hours or days until an external system calls engine.signal(executionId, 'event-name', payload). The signal payload becomes available in ctx.signals['event-name'].

const deploy = new Workflow('deploy')
  .step('build', buildArtifacts)
  .step('deploy-staging', deployStaging)
  .waitFor('approval', { timeout: 48 * 60 * 60 * 1000 }) // 48h
  .step('deploy-prod', async (ctx) => {
    const approval = ctx.signals['approval'] as { approver: string };
    return await deployProd({ approver: approval.approver });
  });

// Later, from your approval UI:
await engine.signal(runId, 'approval', { approver: 'alice@corp.io' });

If nobody approves within 48 hours, the signal times out, the workflow fails, and compensation runs. Perfect for approval gates, manual reviews, KYC compliance, production deploys requiring human sign-off.

You don't need a separate scheduler. You don't need to poll. You don't need to manage timer state in Postgres. The workflow is genuinely asleep until either the signal arrives or the timeout fires.

Schema validation (bring your own)

Steps accept inputSchema and outputSchema. Anything with a .parse() method works — Zod, ArkType, Valibot, your own custom validator. Duck typing means I don't ship a hard dep on any particular library.

import { z } from 'zod';

const InputSchema = z.object({ orderId: z.string(), amount: z.number().positive() });

.step('validate', handler, { inputSchema: InputSchema })

If validation throws, the step fails like any other step — retries, compensation, and event emission all kick in.

Observability

The engine emits typed events for everything that matters:

workflow:started, workflow:completed, workflow:failed, workflow:waiting, workflow:compensating
step:started, step:completed, step:failed, step:retry
signal:received, signal:timeout

You can subscribe globally via EngineOptions.onEvent or per-execution:

const unsub = engine.subscribe(run.id, (event) => {
  if (event.type === 'step:retry') {
    metrics.increment('workflow.step.retry', { step: event.stepName });
  }
});

For history management there's engine.cleanup({ maxAge }) and engine.archive({ maxAge }) — same idea, one deletes, the other moves to cold storage.

A complete real-world example

Here's an e-commerce checkout end-to-end. Validation, parallel notifications, conditional fulfillment based on customer tier, and full saga compensation on failure:

import { Workflow, Engine } from 'bunqueue/workflow';

const checkout = new Workflow('checkout')
  .step('validate-cart', async (ctx) => {
    const cart = (ctx.input as { cart: Cart }).cart;
    if (cart.items.length === 0) throw new Error('Empty cart');
    return { total: cart.items.reduce((s, i) => s + i.price, 0) };
  })
  .step('reserve-inventory',
    async (ctx) => ({ resId: await stock.reserve(ctx.input) }),
    { compensate: async (ctx) => stock.release((ctx.steps['reserve-inventory'] as any).resId) },
  )
  .step('charge-card',
    async (ctx) => ({ txId: await payments.charge((ctx.steps['validate-cart'] as any).total) }),
    {
      retry: 3,
      compensate: async (ctx) => payments.refund((ctx.steps['charge-card'] as any).txId),
    },
  )
  .branch((ctx) => (ctx.input as { tier: string }).tier)
    .path('vip', (w) => w.step('priority-ship', priorityShip))
    .path('standard', (w) => w.step('standard-ship', standardShip))
  .parallel((w) => w
    .step('notify-warehouse', notifyWarehouse)
    .step('send-receipt', sendReceipt)
    .step('update-analytics', updateAnalytics)
  )
  .step('finalize', async () => ({ status: 'complete' }));

const engine = new Engine({
  embedded: true,
  dataPath: './data/checkout.db',
  onEvent: (e) => logger.info(e),
});

engine.register(checkout);
const run = await engine.start('checkout', { cart, tier: 'vip' });

If anything in priority-ship, the parallel block, or finalize fails, the engine refunds the card and releases the inventory automatically. You wrote one compensate per side-effecting step — that's it.

How it compares

Feature	bunqueue	Temporal	Inngest	Trigger.dev	Step Functions
Saga compensation	First-class	Manual	Limited	Limited	Manual
Infrastructure	None (SQLite)	Cluster	Cloud	Cloud/self-host	AWS
Local dev	Same as prod	Heavy	Limited	OK	None
Language	TypeScript	Multi (heavy SDKs)	TypeScript	TypeScript	JSON DSL
Vendor lock-in	None	None	Yes (cloud)	Partial	Total
Human-in-the-loop	Built-in	Built-in	Built-in	Built-in	Built-in

bunqueue isn't trying to replace Temporal at planet-scale. If you're running a million workflows per minute across regions, use Temporal. If you're building a SaaS, an internal tool, or anything that fits comfortably on one or two boxes, bunqueue gets you 90% of the value with 1% of the operational cost.

Getting started

bun add bunqueue

import { Workflow, Engine } from 'bunqueue/workflow';

const flow = new Workflow('hello')
  .step('greet', async (ctx) => ({ msg: `Hi, ${ctx.input.name}` }));

const engine = new Engine({ embedded: true });
engine.register(flow);
const run = await engine.start('hello', { name: 'world' });

Full guide: bunqueue.dev/guide/workflow

GitHub, npm, and the rest are linked from the docs.

Closing thoughts

The Saga Pattern has been around since 1987. The reason it doesn't get used more is that the implementation tax is brutal — you end up rebuilding half a workflow engine from scratch every time. Putting it inside a tool you already use, with a TypeScript DSL and zero infrastructure, removes that tax.

If you've ever implemented sagas by hand, you know how much plumbing it takes. If you've ever evaluated Temporal and bounced off the operational overhead, this is for you.

Feedback and contributions welcome. Especially curious to hear from people running checkout flows, KYC pipelines, or CI/CD orchestration — those are the use cases I designed it around.

No Redis Required: Zero-Config Job Queue for Bun

Egeo Minotti — Mon, 02 Feb 2026 01:24:31 +0000

I stopped configuring Redis. Here's what I use instead.

The Setup

import { Queue, Worker } from 'bunqueue/client';

const queue = new Queue('tasks', { embedded: true });

await queue.add('process', { userId: 123 });

new Worker('tasks', async (job) => {
  return { done: true };
}, { embedded: true });

That's the entire infrastructure. No Redis. No Docker. No connection strings. No server process.

What `embedded: true` Actually Does

When you set embedded: true, bunqueue creates a SQLite database in your project and manages everything in-process:

Your App Process
├── Your Code
├── Queue (in-memory priority queues)
├── Worker (processes jobs)
└── SQLite (./data/bunq.db)

One process. One file. Done.

Real Example: Background Email Processing

import { Queue, Worker } from 'bunqueue/client';

interface EmailJob {
  to: string;
  template: string;
  data: Record<string, unknown>;
}

const emails = new Queue<EmailJob>('emails', { embedded: true });

// Add jobs from your API routes
await emails.add('welcome', {
  to: 'user@example.com',  template: 'welcome',
  data: { name: 'John' }
}, {
  attempts: 3,
  backoff: 5000,
  priority: 10
});

// Process in the same app
const worker = new Worker<EmailJob>('emails', async (job) => {
  await job.updateProgress(10, 'Loading template');

  const html = await renderTemplate(job.data.template, job.data.data);

  await job.updateProgress(50, 'Sending');
  await sendEmail(job.data.to, html);

  return { sent: true };
}, {
  embedded: true,
  concurrency: 5
});

worker.on('failed', (job, err) => {
  console.error(`Email to ${job.data.to} failed: ${err.message}`);
});

Install

bun add bunqueue

I Built a Job Queue That's 32x Faster Than BullMQ (No Redis Required)

Egeo Minotti — Fri, 30 Jan 2026 08:10:30 +0000

I got tired of spinning up Redis just to run background jobs. So I built bunqueue - a job queue that uses SQLite instead.

The Problem

Every time I start a new project:

# Here we go again...
docker run -d redis
npm install bullmq ioredis

For small to medium projects, Redis is overkill. You need:

Another container to manage
Another service to monitor
Another thing that can fail at 3 AM

The Solution

bun add bunqueue

That's it. No Redis. No Docker. Just SQLite.

import { Queue, Worker } from 'bunqueue/client';

// Create a queue
const queue = new Queue('emails');

// Add a job
await queue.add('welcome', {
  to: 'user@example.com'
});

// Process jobs
new Worker('emails', async (job) => {
  await sendEmail(job.data);
  return { sent: true };
});

Why SQLite?

Bun has native SQLite support (bun:sqlite) that's incredibly fast. With WAL mode enabled:

500,000+ ops/sec for queue operations
32x faster than BullMQ for common patterns
Zero network latency - it's just a file

Features You Actually Need

bunqueue isn't a toy. It has everything for production:

Feature
Retries with backoff	✅
Job priorities	✅
Delayed jobs	✅
Cron scheduling	✅
Dead Letter Queue	✅
Stall detection	✅
Progress tracking	✅
S3 backups	✅
Sandboxed workers	✅

Real Example: Email Queue

import { Queue, Worker } from 'bunqueue/client';

interface EmailJob {
  to: string;
  subject: string;
  template: string;
}

const emailQueue = new Queue<EmailJob>('emails');

// Add with retry options
await emailQueue.add('welcome', {
  to: 'user@example.com',
  subject: 'Welcome!',
  template: 'welcome'
}, {
  attempts: 3,        // Retry 3 times
  backoff: 5000,      // Wait 5s between retries
  priority: 10,       // Higher = processed first
});

// Worker with concurrency
const worker = new Worker<EmailJob>('emails', async (job) => {
  await job.updateProgress(10, 'Loading template');

  const html = await renderTemplate(job.data.template);

  await job.updateProgress(50, 'Sending');

  await sendEmail({
    to: job.data.to,
    subject: job.data.subject,
    html
  });

  await job.updateProgress(100, 'Done');
  return { sent: true };
}, {
  concurrency: 5  // Process 5 emails in parallel
});

worker.on('completed', (job, result) => {
  console.log(`✅ Email sent to ${job.data.to}`);
});

worker.on('failed', (job, error) => {
  console.error(`❌ Failed: ${job.data.to}`, error.message);
});

CPU-Intensive? Use Sandboxed Workers

For heavy tasks that might crash or leak memory:

import { SandboxedWorker } from 'bunqueue/client';

const worker = new SandboxedWorker('video-processing', {
  processor: './video-processor.ts',
  concurrency: 4,
  timeout: 300000,    // 5 min timeout
  maxMemory: 512,     // MB per worker
  maxRestarts: 10,    // Auto-restart on crash
});

worker.start();

Each job runs in an isolated Bun subprocess. If it crashes, only that worker restarts.

BullMQ Migration

Already using BullMQ? The API is compatible:

- import { Queue, Worker } from 'bullmq';
+ import { Queue, Worker } from 'bunqueue/client';

// Your code stays the same
const queue = new Queue('my-queue');

When to Use bunqueue

✅ Use bunqueue when:

Single server or small cluster
You want simplicity
You don't want to manage Redis
You need fast local queues

❌ Stick with BullMQ/Redis when:

Multi-region distributed systems
You already have Redis infrastructure
You need pub/sub beyond job queues

Get Started

bun add bunqueue

import { Queue, Worker } from 'bunqueue/client';

const queue = new Queue('tasks');

await queue.add('hello', { message: 'world' });

new Worker('tasks', async (job) => {
  console.log(job.data.message);
});

Links:

📦 npm
📖 Docs
⭐ GitHub

Built with Bun. Because sometimes the simple solution is the right one.

Building Multi-Stage LLM Pipelines with Job Dependencies

Egeo Minotti — Thu, 22 Jan 2026 23:59:23 +0000

Modern AI apps don't call a single LLM. They chain multiple steps: embed a query, search a vector DB, generate a response. Each step depends on the previous one.

The Pattern

┌─────────┐     ┌─────────┐     ┌──────────┐
│  Embed  │ ──▶ │ Search  │ ──▶ │ Generate │
└─────────┘     └─────────┘     └──────────┘

The Code

import { Queue, Worker } from 'flashq';
import OpenAI from 'openai';

const openai = new OpenAI();
const queue = new Queue('ai-pipeline');

// Define the pipeline
async function ask(question: string) {
  const embed = await queue.add('embed', { text: question });
  const search = await queue.add('search', { topK: 5 }, { dependsOn: [embed.id] });
  const generate = await queue.add('generate', {}, { dependsOn: [search.id] });

  return queue.finished(generate.id);
}

// Worker handles all stages
new Worker('ai-pipeline', async (job) => {
  switch (job.name) {
    case 'embed':
      const embed = await openai.embeddings.create({
        input: job.data.text,
        model: 'text-embedding-3-small'
      });
      return { embedding: embed.data[0].embedding, text: job.data.text };

    case 'search':
      const results = await vectorDB.search({
        vector: job.parentResult.embedding,
        topK: job.data.topK
      });
      return { documents: results, query: job.parentResult.text };

    case 'generate':
      const context = job.parentResult.documents.map(d => d.content).join('\n\n');
      const response = await openai.chat.completions.create({
        model: 'gpt-4',
        messages: [
          { role: 'system', content: 'Answer based on context.' },
          { role: 'user', content: `Context:\n${context}\n\nQuestion: ${job.parentResult.query}` }
        ]
      });
      return response.choices[0].message.content;
  }
});

// Usage
const answer = await ask('What is the capital of France?');

https://github.com/egeominotti/flashq
https://flashq.dev/

Key Points

dependsOn: [jobId] chains jobs in sequence
job.parentResult accesses the previous job's output
queue.finished(id) waits for the entire pipeline
Failed jobs stop the pipeline — configure attempts for retries

Works great for RAG, document processing, content generation, or any multi-step AI workflow.

Binja: Finally, a Fast Jinja2/Django Template Engine for Bun

Egeo Minotti — Thu, 22 Jan 2026 18:18:46 +0000

If you've ever tried to use Django-style templates in the JavaScript ecosystem, you know the pain. Nunjucks is the go-to option, but it's slow and missing features. I needed something better for a Bun project, so I built binja.

The Problem

I was migrating a Django application to Bun/Elysia and hit a wall: my templates relied heavily on Django Template Language features that Nunjucks either didn't support or handled poorly. Template inheritance worked, but many filters were missing, and performance wasn't great.

What binja Does Differently

Runtime rendering is 2-4x faster than Nunjucks across all benchmarks:

Benchmark	binja	Nunjucks
Simple Template	371K ops/s	96K ops/s
HTML Escaping	985K ops/s	242K ops/s
Nested Loops	76K ops/s	26K ops/s

But the real win is AOT compilation. Pre-compile your templates at server startup and rendering becomes synchronous and incredibly fast:

import { compile } from 'binja'

// Compile once at startup
const templates = {
  home: compile(await Bun.file('./views/home.html').text()),
  user: compile(await Bun.file('./views/user.html').text()),
}

// Sync rendering - no await needed
app.get('/', () => templates.home({ title: 'Welcome' }))

AOT-compiled templates run at 14+ million ops/sec for simple templates. That's ~160x faster than Nunjucks.

Full Django/Jinja2 Compatibility

This was my main goal. binja includes:

84 built-in filters covering both Jinja2 and Django
28 built-in tests (is even, is defined, etc.)
All Django loop variables (forloop.counter, forloop.first, forloop.parentloop)
Django-specific tags: {% csrf_token %}, {% cycle %}, {% regroup %}, {% lorem %}
Template inheritance with {% extends %} and {% block %}
Autoescape enabled by default

If you're porting Django templates, most should work without changes.

Multi-Engine Support

A feature I didn't expect to need but ended up loving: binja can parse Handlebars, Liquid, and Twig syntax too.

import * as handlebars from 'binja/engines/handlebars'
import * as liquid from 'binja/engines/liquid'

await handlebars.render('{{#each items}}{{this}}{{/each}}', { items: ['a', 'b'] })
await liquid.render('{% for item in items %}{{ item }}{% endfor %}', { items: ['a', 'b'] })

All engines share the same optimized runtime, so you get consistent performance regardless of syntax.

Framework Integration

First-class adapters for Hono and Elysia:

import { Elysia } from 'elysia'
import { binja } from 'binja/elysia'

const app = new Elysia()
  .use(binja({ root: './views', cache: true }))
  .get('/', ({ render }) => render('index', { title: 'Home' }))
  .listen(3000)

Getting Started

bun add binja

Basic usage:

import { render } from 'binja'

const html = await render('Hello, {{ name|title }}!', { name: 'world' })
// Output: Hello, World!

With an environment for template inheritance:

import { Environment } from 'binja'

const env = new Environment({
  templates: './templates',
  autoescape: true,
})

https://flashq.dev/blog/binja-template-engine.html

const html = await env.render('pages/home.html', { user })

When to Use binja

Migrating Django/Flask apps to Bun
Need high-performance server-side rendering
Want a single template engine that handles multiple syntaxes
Building with Hono or Elysia and need proper template support

Check it out:

GitHub: github.com/egeominotti/binja
npm: npmjs.com/package/binja

PRs welcome.

What template engine are you using with Bun? I'd be curious to hear about your setup.

Migrate from BullMQ to flashQ in 5 Minutes

Egeo Minotti — Thu, 22 Jan 2026 00:58:01 +0000

flashQ is BullMQ-compatible. Migration is mostly find-and-replace.

Why?

No Redis — single binary
1.9M jobs/sec vs ~50K
<1ms latency vs 5-10ms

1. Start flashQ

docker run -d -p 6789:6789 flashq/flashq

2. Swap packages

npm uninstall bullmq ioredis
npm install flashq

3. Update imports

// Before
import { Queue, Worker } from 'bullmq';

// After
import { Queue, Worker } from 'flashq';

4. Change port

// 6379 → 6789
connection: { host: 'localhost', port: 6789 }

Done. The rest works unchanged: add(), addBulk(), updateProgress(), events.

API Differences

BullMQ	flashQ
`QueueEvents` class	Events on `Queue` directly
`FlowProducer` with children	`depends_on: [jobId]` option
`repeat: { cron }`	`queue.addCron()`
Worker `limiter`	`queue.setRateLimit()`

flashQ + Elysia & Hono.js: Background Jobs for Modern Bun Apps

Egeo Minotti — Thu, 22 Jan 2026 00:55:25 +0000

Elysia and Hono.js are two of the fastest TypeScript frameworks out there. Combine them with flashQ and you get a stack capable of handling millions of background jobs with sub-millisecond API response times.

Why This Stack?

	Elysia	Hono.js	flashQ
Performance	~2.5M req/sec	~1.5M req/sec	~1.9M jobs/sec
Runtime	Bun-native	Multi-runtime	Rust-powered
Type Safety	End-to-end	Full TypeScript	Typed SDK

Your API responds instantly. Heavy work happens in the background.

Elysia Integration

Setup

bun create elysia flashq-elysia
cd flashq-elysia
bun add flashq

Queue Configuration

// src/queue.ts
import { FlashQ } from 'flashq';

let client: FlashQ | null = null;

export async function getClient(): Promise<FlashQ> {
  if (!client) {
    client = new FlashQ({
      host: process.env.FLASHQ_HOST || 'localhost',
      port: parseInt(process.env.FLASHQ_PORT || '6789'),
      token: process.env.FLASHQ_TOKEN,
    });
    await client.connect();
  }
  return client;
}

export const QUEUES = {
  EMAIL: 'email',
  AI_PROCESSING: 'ai-processing',
} as const;

Elysia Plugin

// src/plugins/flashq.ts
import { Elysia } from 'elysia';
import { getClient, QUEUES } from '../queue';

export const flashqPlugin = new Elysia({ name: 'flashq' })
  .decorate('queue', {
    async push<T>(queue: string, data: T, options?: any) {
      const client = await getClient();
      return client.push(queue, data, options);
    },
    async getJob(jobId: string) {
      const client = await getClient();
      return client.getJob(jobId);
    },
    async waitForResult(jobId: string, timeout = 30000) {
      const client = await getClient();
      return client.finished(jobId, timeout);
    },
    QUEUES,
  });

API Routes

// src/index.ts
import { Elysia, t } from 'elysia';
import { flashqPlugin } from './plugins/flashq';

const app = new Elysia()
  .use(flashqPlugin)

  .post('/api/email', async ({ body, queue }) => {
    const job = await queue.push(queue.QUEUES.EMAIL, body, {
      attempts: 5,
      backoff: 5000,
    });
    return { success: true, jobId: job.id };
  }, {
    body: t.Object({
      to: t.String({ format: 'email' }),
      subject: t.String({ minLength: 1 }),
      template: t.String(),
      data: t.Record(t.String(), t.Any()),
    }),
  })

  .post('/api/generate', async ({ body, query, queue }) => {
    const job = await queue.push(queue.QUEUES.AI_PROCESSING, body, {
      priority: body.priority || 5,
      timeout: 120000,
    });

    // Sync mode: wait for result
    if (query.sync === 'true') {
      const result = await queue.waitForResult(job.id, 60000);
      return { success: true, result };
    }

    return { success: true, jobId: job.id };
  }, {
    body: t.Object({
      prompt: t.String({ minLength: 1 }),
      model: t.Optional(t.Union([
        t.Literal('gpt-4'),
        t.Literal('claude-3'),
      ])),
      userId: t.String(),
    }),
  })

  .get('/api/jobs/:id', async ({ params, queue }) => {
    const job = await queue.getJob(params.id);
    return job || { error: 'Job not found' };
  })

  .listen(3000);

Hono.js Integration

Setup

bun create hono@latest flashq-hono
cd flashq-hono
bun add flashq zod @hono/zod-validator

Queue Middleware

// src/middleware/queue.ts
import { createMiddleware } from 'hono/factory';
import { FlashQ } from 'flashq';

let client: FlashQ | null = null;

async function getClient(): Promise<FlashQ> {
  if (!client) {
    client = new FlashQ({
      host: process.env.FLASHQ_HOST || 'localhost',
      port: parseInt(process.env.FLASHQ_PORT || '6789'),
    });
    await client.connect();
  }
  return client;
}

export const QUEUES = { EMAIL: 'email', AI: 'ai-processing' } as const;

export const queueMiddleware = createMiddleware(async (c, next) => {
  const flashq = await getClient();
  c.set('queue', {
    push: (name, data, options) => flashq.push(name, data, options),
    getJob: (id) => flashq.getJob(id),
    finished: (id, timeout) => flashq.finished(id, timeout),
  });
  await next();
});

Routes

// src/index.ts
import { Hono } from 'hono';
import { zValidator } from '@hono/zod-validator';
import { z } from 'zod';
import { queueMiddleware, QUEUES } from './middleware/queue';

const app = new Hono();
app.use('/api/*', queueMiddleware);

const emailSchema = z.object({
  to: z.string().email(),
  subject: z.string().min(1),
  template: z.string(),
  data: z.record(z.any()),
});

app.post('/api/email', zValidator('json', emailSchema), async (c) => {
  const body = c.req.valid('json');
  const queue = c.get('queue');

  const job = await queue.push(QUEUES.EMAIL, body, {
    attempts: 5,
    backoff: 5000,
  });

  return c.json({ success: true, jobId: job.id });
});

app.get('/api/jobs/:id', async (c) => {
  const queue = c.get('queue');
  const job = await queue.getJob(c.req.param('id'));
  return job ? c.json(job) : c.json({ error: 'Not found' }, 404);
});

export default { port: 3000, fetch: app.fetch };

Worker (Shared)

Works with both Elysia and Hono:

// worker/index.ts
import { Worker } from 'flashq';
import { Resend } from 'resend';

const resend = new Resend(process.env.RESEND_API_KEY);

const emailWorker = new Worker('email', async (job) => {
  const { to, subject, template, data } = job.data;
  const html = renderTemplate(template, data);

  const result = await resend.emails.send({
    from: 'noreply@yourdomain.com',
    to,
    subject,
    html,
  });

  return { emailId: result.id };
}, {
  connection: {
    host: process.env.FLASHQ_HOST || 'localhost',
    port: parseInt(process.env.FLASHQ_PORT || '6789'),
  },
  concurrency: 10,
});

emailWorker.on('completed', (job) => {
  console.log(`✓ Job ${job.id} completed`);
});

emailWorker.on('failed', (job, error) => {
  console.error(`✗ Job ${job.id} failed: ${error.message}`);
});

Advanced: Job Workflows

Chain jobs with dependencies:

app.post('/api/pipeline', async (c) => {
  const client = await getClient();

  // Step 1: Extract
  const extractJob = await client.push('extract', { documentUrl: body.url });

  // Step 2: Summarize (waits for extract)
  const summarizeJob = await client.push('summarize', {
    sourceJobId: extractJob.id,
  }, {
    depends_on: [extractJob.id],
  });

  // Step 3: Embed (waits for extract)
  const embedJob = await client.push('embed', {
    sourceJobId: extractJob.id,
  }, {
    depends_on: [extractJob.id],
  });

  return c.json({
    jobs: {
      extract: extractJob.id,
      summarize: summarizeJob.id,
      embed: embedJob.id,
    },
  });
});

Docker Compose

version: '3.8'

services:
  flashq:
    image: ghcr.io/egeominotti/flashq:latest
    ports:
      - "6789:6789"
    environment:
      - DATABASE_URL=postgres://flashq:flashq@postgres:5432/flashq
    depends_on:
      - postgres

  postgres:
    image: postgres:16-alpine
    environment:
      - POSTGRES_USER=flashq
      - POSTGRES_PASSWORD=flashq
      - POSTGRES_DB=flashq

  api:
    build: .
    ports:
      - "3000:3000"
    environment:
      - FLASHQ_HOST=flashq
      - FLASHQ_PORT=6789
    depends_on:
      - flashq

  worker:
    build:
      context: .
      dockerfile: Dockerfile.worker
    environment:
      - FLASHQ_HOST=flashq
    depends_on:
      - flashq
    deploy:
      replicas: 3

TL;DR

Elysia: Bun-native, end-to-end type safety with t schemas
Hono.js: Multi-runtime (Bun, Node, Cloudflare Workers), Zod validation
flashQ: 1.9M jobs/sec, BullMQ-compatible API, Rust-powered

Both frameworks integrate seamlessly. Pick Elysia for pure Bun projects, Hono for portability.

Links:

io_uring How flashQ Achieves Kernel-Level Async IO Performance

Egeo Minotti — Wed, 21 Jan 2026 16:28:17 +0000

When building flashQ, I faced a fundamental challenge: how do you build a job queue that can handle hundreds of thousands of operations per second while maintaining low latency?

The answer lies in one of Linux's most significant kernel innovations in the past decade: io_uring.

The Problem with Traditional Async I/O

Before io_uring, Linux applications had two primary options for handling I/O.

Blocking I/O with Thread Pools

The traditional approach: spawn threads, let them block on I/O operations. Simple, but inefficient:

Context switching overhead: Each thread switch costs 1-10 microseconds
Memory overhead: Each thread requires its own stack (typically 8MB)
Scalability limits: Thousands of concurrent connections = thousands of threads

Event-driven I/O (epoll/kqueue)

Modern async runtimes like Tokio use epoll (Linux) or kqueue (macOS/BSD):

int epfd = epoll_create1(0);
epoll_ctl(epfd, EPOLL_CTL_ADD, sockfd, &event);

while (1) {
    int n = epoll_wait(epfd, events, MAX_EVENTS, -1);  // syscall
    for (int i = 0; i < n; i++) {
        read(events[i].data.fd, buf, len);  // syscall
        process(buf);
        write(events[i].data.fd, response, len);  // syscall
    }
}

Better than threads, but still problematic:

Syscall overhead: Every read(), write(), accept() requires a kernel transition
Data copying: Data must be copied between kernel and user space
Notification only: epoll tells you a socket is ready, but you still need syscalls to do actual I/O

For high-throughput applications, these syscalls become the bottleneck. A server handling 100K requests/second makes 300K+ syscalls per second just for basic I/O.

Enter io_uring: A Paradigm Shift

Introduced in Linux 5.1 (2019) by Jens Axboe, io_uring fundamentally changes how applications interact with the kernel for I/O operations.

The Core Innovation: Shared Ring Buffers

io_uring creates two ring buffers shared between user space and kernel space:

+-------------------------------------------------------------+
|                      User Space                              |
|  +---------------------+    +---------------------+         |
|  |  Submission Queue   |    |  Completion Queue   |         |
|  |  (SQ) - Requests    |    |  (CQ) - Results     |         |
|  |                     |    |                     |         |
|  |  [read fd=5, ...]   |    |  [done, 1024 bytes] |         |
|  |  [write fd=7, ...]  |    |  [done, 512 bytes]  |         |
|  |  [accept fd=3, ...] |    |  [error, EAGAIN]    |         |
|  +----------+----------+    +----------^----------+         |
|             | shared memory            |                     |
+-------------|--------------------------+---------------------+
|             |      Kernel Space        |                     |
|             v                          |                     |
|  +-------------------------------------+---+                |
|  |           io_uring Subsystem            |                |
|  |                                         |                |
|  |   - Processes SQ entries                |                |
|  |   - Performs actual I/O                 |                |
|  |   - Posts results to CQ                 |                |
|  +-----------------------------------------+                |
+-------------------------------------------------------------+

Key Benefits at a Glance

Feature	Traditional (epoll)	io_uring
Syscalls per I/O	1-3 per operation	0 (batched submission)
Data copies	User - Kernel	Zero-copy possible
Batching	Not native	Submit hundreds at once
Kernel polling	No	Yes (SQPOLL mode)
Fixed buffers	No	Yes (registered buffers)

How flashQ Uses io_uring

flashQ is written in Rust, leveraging the tokio-uring crate for io_uring support. Here's how we integrate it.

Runtime Detection

flashQ automatically detects the optimal I/O backend at startup:

pub fn select_io_backend() -> IoBackend {
    #[cfg(target_os = "linux")]
    {
        if io_uring_available() && kernel_version() >= (5, 1) {
            return IoBackend::IoUring;
        }
        return IoBackend::Epoll;
    }

    #[cfg(target_os = "macos")]
    return IoBackend::Kqueue;

    #[cfg(target_os = "windows")]
    return IoBackend::Iocp;
}

You'll see the active backend in the startup logs:

# Linux with io_uring
INFO flashq_server::runtime: IO backend: io_uring (kernel-level async)

# macOS
INFO flashq_server::runtime: IO backend: kqueue (poll-based async)

Batched Operations

One of io_uring's biggest advantages is batching. Instead of making individual syscalls, flashQ batches multiple operations:

async fn handle_connections(ring: &IoUring) {
    let mut submissions = Vec::with_capacity(32);

    // Collect pending operations
    for conn in pending_connections.drain(..) {
        submissions.push(ReadOp::new(conn.fd, conn.buffer));
    }

    // Submit all at once - single syscall for 32 operations
    ring.submit_batch(&submissions).await;

    // Process completions
    for completion in ring.completions() {
        handle_completion(completion);
    }
}

This reduces syscall overhead by 95%+ under high load.

Zero-Copy Networking

With registered buffers, flashQ can perform true zero-copy I/O:

// Register buffers once at startup
let buffers = IoUring::register_buffers(
    (0..BUFFER_COUNT)
        .map(|_| vec![0u8; BUFFER_SIZE])
        .collect()
);

// Use registered buffers for I/O - no copying!
async fn read_message(fd: RawFd, buf_idx: u16) -> io::Result<usize> {
    ring.read_fixed(fd, buf_idx, 0).await
}

Data flows directly from network card to application memory without intermediate kernel buffer copies.

SQPOLL Mode for Ultra-Low Latency

In SQPOLL mode, the kernel continuously polls the submission queue without requiring any syscalls:

let ring = IoUring::builder()
    .setup_sqpoll(2000)  // Poll for 2ms before sleeping
    .build()?;

// Submissions are picked up automatically by kernel thread
// No io_uring_enter() syscall needed!

This is ideal for latency-sensitive workloads where every microsecond counts.

Performance Impact

I benchmarked flashQ with and without io_uring on identical hardware (AMD EPYC 7763, 64 cores, 128GB RAM).

Throughput Comparison

Metric	epoll	io_uring	Improvement
Jobs pushed/sec	245,000	312,000	+27%
Jobs processed/sec	180,000	228,000	+26%
P99 latency (push)	1.8ms	0.9ms	-50%
P99 latency (fetch)	2.1ms	1.1ms	-48%
CPU usage at 100K/s	45%	31%	-31%
Syscalls/sec at 100K ops	~320,000	~12,000	-96%

Latency Distribution

Push Latency Distribution (100K jobs/sec sustained)

epoll:
  P50:  0.4ms  ################
  P90:  1.2ms  ########################################
  P99:  1.8ms  ############################################################
  P999: 4.2ms  ######################################################################

io_uring:
  P50:  0.2ms  ########
  P90:  0.6ms  ####################
  P99:  0.9ms  ##############################
  P999: 1.8ms  ############################################################

Getting Started

Docker (Recommended)

# Our official image has io_uring enabled
docker run -d -p 6789:6789 ghcr.io/egeominotti/flashq:latest

# Verify io_uring is active
docker logs flashq | grep "IO backend"

Building from Source

git clone https://github.com/egeominotti/flashq.git
cd flashq

# Build with io_uring feature
cargo build --release --features io-uring

./target/release/flashq-server

Requirements:

Linux kernel 5.1+ (5.10+ recommended)
liburing installed
For SQPOLL mode: CAP_SYS_ADMIN or root

Platform Compatibility

flashQ runs everywhere with automatic backend selection:

Platform	I/O Backend	Notes
Linux (kernel 5.1+)	io_uring	Fastest, kernel-level async
Linux (older kernels)	epoll	Fast, poll-based
macOS	kqueue	Native, optimal for macOS
Windows	IOCP	Native, optimal for Windows

When io_uring Makes the Biggest Difference

High Connection Count - With thousands of concurrent connections, the syscall reduction is dramatic.

High Throughput Workloads - AI workloads pushing hundreds of thousands of jobs benefit enormously from the 27% throughput improvement.

Latency-Sensitive Applications - The 50% P99 latency reduction matters for real-time applications.

CPU-Constrained Environments - The 31% CPU reduction means you can handle more load on the same hardware.

The Future of io_uring

io_uring continues to evolve:

Linux 5.19+: Multishot operations
Linux 6.0+: Zero-copy send support
Linux 6.1+: Improved buffer ring management
Upcoming: User-space networking (io_uring + XDP)

flashQ will continue adopting new features as they stabilize.

Conclusion

io_uring represents a fundamental shift in how high-performance applications interact with the Linux kernel. For flashQ users, this translates to:

27% higher throughput on Linux systems
50% lower tail latency for time-sensitive workloads
31% less CPU usage for the same workload
Better scalability under high connection counts

The best part? It's automatic. Deploy flashQ on a modern Linux system, and you get io_uring performance out of the box.

Have questions about io_uring or flashQ? Drop a comment below or check out the GitHub repo.

egeominotti / flashq

Blazingly fast job queue server. Process millions of jobs/sec with sub-millisecond latency. Built with Rust.

flashQ

High-performance job queue server built with Rust. No Redis required.

Website • Docs • Blog

Features

1.9M jobs/sec push throughput, 280K jobs/sec processing
BullMQ-compatible API - easy migration from Redis-based queues
SQLite persistence - embedded WAL mode, zero external dependencies
S3 backup - AWS S3, Cloudflare R2, MinIO, Backblaze B2 compatible
Real-time dashboard - WebSocket-powered monitoring UI
Advanced scheduling - priorities, delays, cron jobs, rate limiting
Dead letter queues - automatic retry with exponential backoff
Job dependencies - workflow orchestration with parent/child jobs

Architecture

┌─────────────┐     ┌─────────────────┐     ┌──────────────┐
│   Client    │────▶│  flashQ Server  │────▶│    SQLite    │
│  (SDK/API)  │     │   (Rust/Axum)   │     │  (embedded)  │
└─────────────┘     └─────────────────┘     └──────────────┘
      TCP/HTTP              │                      │
      Binary/JSON           ▼                      ▼
                    ┌───────────────┐      ┌──────────────┐
                    │   Dashboard   │      │  S3 Backup   │
                    │  (WebSocket)  │      │  (optional)  │
                    └───────────────┘      └──────────────┘

Quick Start

# Start server
docker run -d -p 6789:6789 -p 6790:6790 -e HTTP=1 ghcr.io/egeominotti/flashq:latest

SDKs

TypeScript / JavaScript

…

View on GitHub

I Built a 2M ops/sec Job Queue in Rust to Replace Redis

Egeo Minotti — Wed, 21 Jan 2026 03:57:57 +0000

Last week I got frustrated with Redis.

Not because Redis is bad — it’s great for caching. But using it as a job queue backend? That’s where things get painful.

Every job = network round trip. Every poll = commands to Redis. BullMQ doing 10-15 Redis commands just to check if there’s work. My Upstash bill climbing because of polling, not actual jobs.

So I built flashQ — a job queue server in Rust that doesn’t need Redis.

The Problem with Redis-based Queues

Here’s what happens when you push 10,000 jobs with BullMQ:

10,000 jobs × 1 network round-trip = 10,000 TCP calls
Average RTT: 0.3ms
Total time: ~3 seconds

And that’s just pushing. Pulling, acknowledging, retrying — each operation hits Redis over the network.

Redis is single-threaded. One CPU core handles ALL your queue operations. When you need more throughput, you start clustering, sharding, managing multiple Redis instances.

The flashQ Approach

What if the queue lived in-process? What if batch operations were truly atomic?

┌─────────────────────────────────────────┐
│            flashQ Server                 │
│  ┌────────────────────────────────────┐ │
│  │      32 Parallel Shards            │ │
│  │   (one per CPU core)               │ │
│  └────────────────────────────────────┘ │
│         In-Process: ~100 nanoseconds    │
└─────────────────────────────────────────┘
            ↑
      Single TCP connection
            ↑
┌─────────────────────────────────────────┐
│        Your App (batch push)            │
└─────────────────────────────────────────┘

Same 10,000 jobs:

10,000 jobs × 1 batch command = 1 TCP call
Processing time: ~5ms

That’s 600x faster for batch operations.

Real Benchmarks

Tested on Apple M2. No synthetic tests, no asterisks.

Operation	flashQ	BullMQ	Improvement
Batch throughput	2,127,660 ops/sec	36,232 ops/sec	58x
Pull + Ack	519,388 ops/sec	~10,000 ops/sec	52x
P99 Latency	127-196 μs	606-647 μs	3-5x
Memory per 1M jobs	~200 MB	~2 GB	10x less

BullMQ-Compatible API

I didn’t want to reinvent the wheel. If you know BullMQ, you know flashQ:

// Before (BullMQ)
import { Queue, Worker } from 'bullmq';

// After (flashQ)
import { Queue, Worker } from 'flashq';

// Same code works
const queue = new Queue('emails');

await queue.add('send', { 
  to: 'user@example.com',
  subject: 'Welcome!'
});

const worker = new Worker('emails', async (job) => {
  await sendEmail(job.data);
  return { sent: true };
});

Change one import. That’s it.

Features

Everything you’d expect from a production queue:

Priority queues — critical jobs first
Delayed jobs — schedule for later
Job dependencies — DAG-style workflows
Rate limiting — protect your APIs
Dead letter queue — failed jobs isolated
Retries with backoff — exponential, configurable
Cron scheduling — repeatable jobs
Real-time dashboard — monitor everything

Built with Rust

Why Rust?

No garbage collector — predictable latency
Memory safe — no segfaults in production
Fearless concurrency — 32 shards, zero data races
Small footprint — 30MB base memory

Libraries used:

tokio for async runtime
tonic for gRPC
parking_lot for fast locks
fxhash for fast hashing
mimalloc for memory allocation
simd-json for fast parsing

When to Use Redis (Still)

Redis is still great for:

✅ Caching
✅ Pub/Sub
✅ Session storage
✅ Low volume queues (<1K jobs/sec)

When to Use flashQ

✅ High throughput (>10K jobs/sec)
✅ Low latency (<1ms P99)
✅ Batch operations at scale
✅ No Redis infrastructure to manage
✅ Cost efficiency (less RAM, fewer servers)

Try It

# Docker
docker run -p 6789:6789 -p 6790:6790 flashq/flashq

# Dashboard at http://localhost:6790

# npm
bun add flashq

GitHub: github.com/egeominotti/flashq

Docs: flashq.dev/docs

It’s MIT licensed, open source, and I’m actively developing it.

Currently at v0.2 — battle-testing welcome. If you try it, let me know what breaks.

Questions? Drop a comment or open an issue.

DEV Community: Egeo Minotti

I built a saga workflow engine for Bun (no Temporal, no Redis, no Kafka)

TL;DR

The problem nobody wants to deal with

Why I built another workflow engine

The smallest possible example

Compensation: the part that actually matters

Retries with exponential backoff

Branching

Parallel steps

Loops and iteration

Sub-workflows

Human-in-the-loop

Schema validation (bring your own)

Observability

A complete real-world example

How it compares

Getting started

Closing thoughts

No Redis Required: Zero-Config Job Queue for Bun

The Setup

What embedded: true Actually Does

Real Example: Background Email Processing

Install

I Built a Job Queue That's 32x Faster Than BullMQ (No Redis Required)

The Problem

The Solution

Why SQLite?

Features You Actually Need

Real Example: Email Queue

CPU-Intensive? Use Sandboxed Workers

BullMQ Migration

When to Use bunqueue

Get Started

Building Multi-Stage LLM Pipelines with Job Dependencies

The Pattern

The Code

Key Points

Binja: Finally, a Fast Jinja2/Django Template Engine for Bun

The Problem

What binja Does Differently

Full Django/Jinja2 Compatibility

Multi-Engine Support

Framework Integration

Getting Started

When to Use binja

Migrate from BullMQ to flashQ in 5 Minutes

Why?

1. Start flashQ

2. Swap packages

3. Update imports

4. Change port

API Differences

Links

flashQ + Elysia & Hono.js: Background Jobs for Modern Bun Apps

Why This Stack?

Elysia Integration

Setup

Queue Configuration

Elysia Plugin

API Routes

Hono.js Integration

Setup

Queue Middleware

Routes

Worker (Shared)

Advanced: Job Workflows

Docker Compose

TL;DR

io_uring How flashQ Achieves Kernel-Level Async IO Performance

The Problem with Traditional Async I/O

Blocking I/O with Thread Pools

Event-driven I/O (epoll/kqueue)

Enter io_uring: A Paradigm Shift

The Core Innovation: Shared Ring Buffers

Key Benefits at a Glance

How flashQ Uses io_uring

Runtime Detection

Batched Operations

Zero-Copy Networking

What `embedded: true` Actually Does