Building a Priority-Based Task Scheduler with FastAPI, Redis, and C++

OJASVI SWAMI — Sun, 25 Oct 2026 17:58:10 +0000

Introduction :-
Not every backend operation should be executed as part of an HTTP request.
Some operations require additional computation, may take unpredictable amounts of time, or need to run independently of the API layer. Executing all of them synchronously can tie request handling to processing time and make it harder to control how work is scheduled.
A common solution is to separate task submission from task execution.
In this article, I will walk through a task-processing system I built using Python, FastAPI, Redis, PostgreSQL, and a C++ processing service powered by OpenSSL. The system accepts operations through a REST API, organizes tasks into priority queues, and processes them asynchronously.

The goal is not to introduce another queue implementation. Instead, I want to explore the engineering decisions behind the system: why the components are separated, how tasks move through the architecture, and what trade-offs arise when building an asynchronous processing pipeline.
The complete source code is available on GitHub.

1. Problem Statement:-
Consider an API that accepts computational tasks such as encryption, decryption, hashing, and compression.

If the API performs each operation before responding, the request remains coupled to the execution time of that operation. This becomes inconvenient when processing needs to happen independently or when different tasks have different priorities.

I wanted to build a system having following capablities:-
Asynchronous processing:

Accept a task without waiting for its computation to finish.
Priority-based scheduling: Route tasks into high, normal, and low-priority queues.
Resource control: Apply a per-user request rate limit.
Separation of responsibilities: Keep API handling separate from the C++ processing service.
Status tracking: Allow clients to check whether a task is pending, running, completed, or failed.
Persistent results: Store task-related results in PostgreSQL.

These are the requirements for building architecture of the project.

2. Architectural overview :-
The system has four main services: FastAPI, Redis, a C++ processing service, and PostgreSQL.

The responsibilities are deliberately separated:

FastAPI handles incoming requests, validates task parameters, and exposes endpoints for task submission and status checks.
Redis provides priority queues for pending tasks and supports request rate limiting.
The C++ service executes the supported processing operations using OpenSSL and the project's RLE implementation.
PostgreSQL stores task-related state and results that need to remain available for subsequent status requests.

3. Following a task through the system :-
Here is a brief walk through the lifecycle of task.

Step 1: Task submission

A client submits a request to POST /process.

For example:

FastAPI validates the request and applies the rate-limiting middleware. The system allows up to 10 requests per minute for a given x-user-id header, according to the configured rate limit.

Once the task is accepted, the API returns a request ID and a PENDING status rather than waiting for the processing operation to finish.

Step 2: Priority-based scheduling
The scheduler routes tasks into one of three Redis queues:

This makes priority an explicit part of the scheduling model.

A worker can check higher-priority queues before lower-priority queues, allowing urgent tasks to be selected first. However, this also raises an important design question: what happens to low-priority tasks if high-priority work arrives continuously?

Strict priority can cause starvation. A more advanced scheduler might introduce weighted scheduling or aging, where a task's effective priority increases as it waits.

The current architecture establishes separate priority queues; starvation prevention would be a further scheduling enhancement.

Step 3: Asynchronous task execution

A background Python worker retrieves queued tasks using Redis blocking queue operations and dispatches them to the C++ processing service through HTTP.

The processing service supports five operations:

ENCRYPT and DECRYPT using AES-256-CBC.

HASH using SHA-256.

TRANSFORM and DECOMPRESS using run-length encoding (RLE).

The Python API and worker do not need to implement these operations themselves. They coordinate task execution while the C++ service owns the processing logic.

This boundary also creates an independent place to evolve and test the processing implementation.

Step 4: Persisting results and checking status

As processing progresses, the task moves through the following states:

PENDING → IN_PROGRESS → DONE / FAILED

The result and relevant task information are stored in PostgreSQL. Clients can subsequently call GET /status/{request_id} to retrieve the current state, output data, and processing time when available.

This polling model is straightforward to implement and integrate with ordinary HTTP clients.Its trade-off is that clients make repeated requests even when a task's state has not changed. _Webhooks or server-sent events _could be explored if clients later need push-based updates.

*4. Why Use Redis for Priority Queues? *

Redis provides a lightweight way to maintain pending tasks without forcing the API to execute them immediately.

In this design, the task payload is placed directly into the Redis queue. The worker receives the information it needs to dispatch the task without making an additional database query just to reconstruct the queued request.

This is a deliberate trade-off: it reduces the need for an extra database read during dispatch, but duplicates some task information between Redis and PostgreSQL.

The design also raises reliability questions.

The worker uses Redis BLPOP, which removes a task from the queue as soon as it retrieves it. If the worker crashes before completing the task, Redis does not automatically put it back into the queue. The task may therefore remain stuck in PENDING or IN_PROGRESS in PostgreSQL.

A possible improvement would be to introduce an in-flight queue, acknowledgements, and a recovery mechanism for unfinished tasks. The current implementation does not yet provide this recovery behavior.

The right choice depends on the delivery guarantees the application actually needs.

5. Why Separate the C++ Processing Service?
The project uses Python for API orchestration and C++ for the processing operations.This offers several benefits including-

The API does not have to execute the processing logic directly inside its request handler.
The processing implementation can be tested independently.
The C++ service can be scaled separately if workload measurements justify doing so.
Each component has a narrower responsibility.

There is also a cost. Calling the C++ service over HTTP introduces serialization, network communication, and additional failure points.
It is therefore important not to assume that a separate C++ service is automatically faster than implementing the same operation in Python. The benefit depends on the workload, data size, execution cost, and communication overhead.

6. Rate Limiting at the API Layer

The application limits each x-user-id to 10 requests per minute using Redis-backed keys with expiration.

This protects the API from an individual identifier submitting requests too quickly and provides a simple mechanism for controlling admission.

When the limit is exceeded, the API returns HTTP 429 Too Many Requests, along with a retry-after value.

One important distinction is that rate limiting controls the rate at which tasks are submitted. It does not, by itself, control how many tasks a worker can execute concurrently or how large the queue can grow.

For a more complete overload-control strategy, I would also evaluate queue-depth limits, bounded worker concurrency, and backpressure when processing capacity falls behind incoming demand.

The current implementation uses separate Redis operations to read, increment, and expire the counter. Concurrent requests can race, and the initial increment and expiration are not atomic. A Redis Lua script could combine these operations to provide more consistent rate-limit enforcement under concurrency.

7. Testing the Components

The project includes Python tests using pytest and C++ tests using GoogleTest.

The C++ tests cover operations such as encryption/decryption round trips, SHA-256, and RLE compression/decompression. Python tests cover API behavior, rate limiting, and processor integration.

These tests provide a foundation for validating the individual components. Additional tests would be valuable for system-level failure scenarios:

A worker crashes after retrieving a task.
The C++ service becomes unavailable.
PostgreSQL temporarily rejects writes.
The same task is submitted or processed more than once.
A high-priority workload continuously occupies the workers.
Incoming tasks exceed the processing capacity.

These cases matter because successful execution tests alone do not establish reliable delivery under failure.

8. What I Would Improve Next

The current implementation is a useful starting point, but several areas deserve further investigation.

Reliable task delivery: Introduce an explicit recovery strategy for tasks removed from Redis but not completed.

Idempotent execution: Give tasks stable identities and define how duplicate execution is handled, particularly when a worker fails after performing the operation but before recording completion.

Bounded concurrency and backpressure: Limit in-flight work and define what happens when the queue grows faster than workers can drain it.

Observability: Record queue wait time, processing duration, task failure rate, queue depth, and end-to-end latency. These measurements would help distinguish slow computation from scheduling or persistence overhead.

Security: AES-CBC provides confidentiality but does not authenticate ciphertext. For applications that need authenticated encryption, I would evaluate AES-GCM or another authenticated encryption construction, with appropriate nonce management and key handling.

Performance evaluation: Benchmark realistic payload sizes and concurrent workloads before claiming that one architecture or implementation is faster than another.

These improvements would move the system toward stronger delivery guarantees and more measurable operational behavior.

Conclusion

Building the task scheduler was an exercise in separating concerns across an asynchronous backend pipeline.

FastAPI handles the API, Redis organizes pending work, a Python worker coordinates execution, C++ performs the supported operations, and PostgreSQL retains task information and results.

The interesting engineering questions go beyond choosing a technology. They involve deciding what happens when a worker fails, how priority affects fairness, how to control overload, and how to measure the real cost of communication between services.

Those trade-offs are what make a task scheduler a useful system-design case study—and what I would investigate next as the system evolves.

Source Code

OJASVISWAMI22 / task-scheduler

Async task processing system built with FastAPI, Redis rate limiting, and a C++ processing service backed by PostgreSQL

Task Scheduler

An asynchronous task-processing system built with FastAPI, C++ (OpenSSL), Redis and PostgreSQL, fully containerized with Docker Compose.

Clients send tasks (encrypt, hash, compress etc.) through REST API. Queues are prioritized in Redis and handled by a C++ service. Results go to PostgreSQL.

Architecture

flowchart TD
    Client([Client])

    subgraph FastAPI["FastAPI Service"]
        RL["Rate Limiter\n(Redis TTL · 10 req/min)"]
        PP["POST /process"]
        GS["GET /status/{request_id}"]
        WK["Worker\n(asyncio · blpop)"]
    end

    R[(Redis\nqueue:high / normal / low)]
    CPP["C++ Service\n(AES-256-CBC · SHA-256 · RLE)"]
    PG[(PostgreSQL\nrequests · results)]

    Client -->|POST /process| RL
    Client -->|GET /status| GS
    RL --> PP
    PP -->|enqueue| R
    WK -->|blpop| R
    WK -->|HTTP POST /process| CPP
    WK -->|save result| PG
    GS -->|read| PG

Features

Priority queues — tasks routed to queue:high, queue:normal, or queue:low based on request priority (3 / 2 / 1)
Rate limiting — 10 requests/minute per x-user-id header, enforced in middleware via Redis TTL keys
Async processing — background…

View on GitHub

DEV Community: OJASVI SWAMI

Building a Priority-Based Task Scheduler with FastAPI, Redis, and C++

OJASVISWAMI22 / task-scheduler

Async task processing system built with FastAPI, Redis rate limiting, and a C++ processing service backed by PostgreSQL

Architecture

Features