DEV Community

Cover image for Supabase Just Got More Powerful: Queue, Cron, and Background Tasks in Edge Functions
Taishi
Taishi

Posted on β€’ Edited on

15 7 7 7 7

Supabase Just Got More Powerful: Queue, Cron, and Background Tasks in Edge Functions

This was my first reaction when Supabase announced their new Queue, Cron and Background Tasks in an Edge Function features.
Let me show you why this is such a big deal.

The Old World: Juggling Multiple Services

Picture this: You're building an (AI) app that needs to handle long-running processes, like AI-powered audio transcription. Until now, you had to piece together a puzzle of different services:

  • Queue management with Bull or Trigger.dev
  • A separate hosting platform for your background jobs (ex: Render, Railway)
  • Redis for job state management
  • And of course, Supabase for your main database

I experienced this while building AutoRepurpose AI - a tool that transforms YouTube videos into engaging social media content. The process involves:

  1. Extracting video subtitles
  2. Creating and storing embeddings in Supabase (PostgreSQL)
  3. Generating tailored social posts using these embeddings

For these tasks, I had to set up Bull on a Node.js instance hosted on Railway, plus Redis for managing the background processes.

Previous Architecture with Multiple Services

The New World: Supabase Queue Changes Everything

Now, everything lives under one roof. No more context switching between services. No more managing multiple platforms. Just pure, streamlined efficiency.

New Simplified Architecture with Supabase

Here's all you need to implement background processing with Supabase:

  1. Add items to your queue
  2. Create an edge function to process queued items
  3. Set up a cron job to trigger the processing

Let's see how this looks in code ✨:

Add a queue item: React component (Next.js)

const supabase = createClient<Database>(
  process.env.NEXT_PUBLIC_SUPABASE_URL!,
  process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY!,
  { db: { schema: "pgmq_public" } }
);

const sendToQueue = async () => {
  await supabase.rpc("send", {
    queue_name: "ai-background-job",
    message: { userId, videoId },
  });
};

return <button onClick={sendToQueue}>Create a new project</button>
Enter fullscreen mode Exit fullscreen mode

ai-process edge function (Supabase Edge Functions)

import { createClient } from "https://esm.sh/@supabase/supabase-js@2.47.7";
import process from "node:process";
import type { Database } from "../supabase.ts";

const supabase = createClient<Database>(
  process.env.SUPABASE_URL!,
  process.env.SUPABASE_ANON_KEY!,
);

const supabasePgmqSchema = createClient<Database>(
  process.env.SUPABASE_URL!,
  process.env.SUPABASE_ANON_KEY!,
  { db: { schema: "pgmq_public" } },
);

Deno.serve(async (req) => {
  const queueToProcess = await supabasePgmqSchema.rpc("pop", {
    queue_name: "ai-background-job",
  });

  if (
    queueToProcess.error || !queueToProcess.data ||
    !Array.isArray(queueToProcess.data)
  ) {
    return new Response(
      JSON.stringify({ success: false }),
      { headers: { "Content-Type": "application/json" }, status: 500 },
    );
  }

  const { userId, videoId } = queueToProcess.data[0].message as {
    userId: string;
    videoId: string;
  };

  // Fetch the transcription of the video, generate social posts,
  // and save them in the database in the background
  EdgeRuntime.waitUntil(processAIStuff({ userId, videoId }));

  return new Response(
    JSON.stringify({ success: true }),
    { headers: { "Content-Type": "application/json" } },
  );
});
Enter fullscreen mode Exit fullscreen mode

Set up a cron job

The new cron interface is quite straightforward. Just a few clicks in the dashboard, and you're done:

Image description

Image description

The Complete Backend Package

Supabase now offers everything you need in one platform:

  • Queuing system
  • Simple, powerful cron jobs
  • Edge Functions with serverless computing and background task capabilities
  • authentication
  • PostgreSQL database with superpowers

They didn't just add a missing piece - they completed the backend puzzle. As developers, we can now focus on building features instead of managing infrastructure.

Supabase truly has become the backend that eats backends! ⚑️

Do your career a big favor. Join DEV. (The website you're on right now)

It takes one minute, it's free, and is worth it for your career.

Get started

Community matters

Top comments (4)

Collapse
 
loong profile image
Long Hoang β€’

I've used pg_cron before to trigger jobs that run as Edge Functions. Now, Queue was exactly the missing piece I needed!

Collapse
 
taishi profile image
Taishi β€’

Yes! We were able to use corn with pg_cron but now they have a UI for it + Queue!!

Collapse
 
jumski profile image
Wojtek Majewski β€’ β€’ Edited

Registered just to chime in - I have built something that makes working with queues and background tasks on Supabase easier and more reliable: a task queue worker running on top of Edge Functions (yes, really!).

I made it super easy to set up. Consuming queues with my worker is a breeze.
It's open source and part of a bigger project I have been working on since November (a deeply supabase-integrated Postgres-first workflow engine).
I'm collaborating with Supabase devs to integrate it more deeply.

I would love to hear your thoughts or connect to exchange ideas!
You can check the docs here: pgflow.dev or hit me up on X (I'm "pgflow_dev")

cc @taishi @loong @piyush_priyadarshi_f77534

Collapse
 
piyush_priyadarshi_f77534 profile image
Piyush Priyadarshi β€’

Nice article ,
But How can we use multiple consumers to process messages concurrently ?
Would love to know your thoughts on this @loong @taishi

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

πŸ‘‹ Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay