Real-Time vs Non-Real-Time Systems: Breaking Down the Key Concepts

Zeporath — Sun, 27 Apr 2025 13:39:42 +0000

As software developers, we often encounter the term "real-time." But what does it truly mean from a software perspective, particularly when working with C++ in a Linux environment? What one domain considers "real-time" might not be fast enough for another.

We can categorize the real-time behavior of software into a few key areas:

Hard Real-Time (e.g., Airbag Deployment Systems): In these systems, missing a deadline is catastrophic. Failure to meet timing constraints can lead to severe consequences.
Soft Real-Time (e.g., Sensor Data Readings for Speed): Here, missing a deadline is tolerable, although it may degrade performance or user experience.
Non-Real-Time (e.g., Video Streaming Over a Dashboard): The primary focus is on minimizing latency and maximizing throughput, rather than strict adherence to deadlines.

Ultimately, the classification of a system as hard, soft, or non-real-time depends on its criticality and the design choices made during development. We'll delve into those design choices in future blog posts. For now, let's recap and solidify our understanding of the fundamental concepts surrounding the "what" and "how" of real-time tasks, especially as they relate to C++ development on Linux platforms.

What is REAL TIME:

Real-time does not mean fast. It means predictable and guaranteed response times.
They are everywhere, from embedded devices (robot arms) to specialized servers (stock trading engines).
So if we can characterize the real-time and non-real time tasks as:

Aspect	Real-time Threads	Non-real-time Threads
Scheduling	Priority-based, often fixed-priority (e.g., FIFO, RR)	Fairness-based (e.g., CFS in Linux)
Timing Guarantees	Must meet deadlines	Best-effort, no timing guarantee
Preemption	High priority to preempt lower	May not preempt immediately
Examples	Motor control, medical devices	Web servers, user applications

Who enables/gives REAL TIME:

It's your code where you specify for a thread to be a real-time or not. And the OS while executing will honor your demands, provided its capable of doing so.

Roles of CPU Execution vs Kernel Scheduling in real-time executions:

CPU simply executes instructions.
It’s the OS (kernel scheduler) that determines who runs when.
Real-time OS or Linux with real-time extensions ensures deadlines by:
- Priority-based preemptive scheduling (SCHED_FIFO, SCHED_RR, SCHED_DEADLINE).
- Avoiding blocking I/O.
- Locking memory into RAM (mlockall) etc.

🔥 Can a process on Linux have both real-time and non-real-time threads?

✅ YES.
In Linux, each thread has its own scheduling policy (SCHED_OTHER, SCHED_FIFO, SCHED_RR, etc.).
So, one process can absolutely have:

Some threads running real-time (e.g., SCHED_FIFO policy).
Some threads running normal (e.g., SCHED_OTHER policy).

👉 In fact, it's common to design this way:
Real-time thread: handles sensor readings, motor control, etc.
Non-real-time thread: logs data to disk, handles user input, sends reports over network

Enough with concepts, let's look at a demo code:

#include <iostream>
#include <thread>
#include <vector>
#include <queue>
#include <chrono>
#include <cmath>
#include <mutex>
#include <condition_variable>
#include <complex>
#include <cstring>
#include <pthread.h>
#include <sched.h>
#include <unistd.h>

using namespace std::chrono_literals;

// Shared data queue
std::queue<std::vector<double>> dataQueue;
std::mutex queueMutex;
std::condition_variable dataCond;
bool finished = false;

constexpr int DATA_SIZE = 1024;
constexpr int DEADLINE_US = 220; // Real-time deadline

// Dummy FFT function (simulate work)
void dummyFFT(const std::vector<double>& input, std::vector<std::complex<double>>& output) {
    for (size_t i = 0; i < input.size(); ++i) {
        output[i] = std::complex<double>(std::sin(input[i]), std::cos(input[i]));
    }
}

// Set CPU affinity 
void set_cpu_affinity(int cpu_id) {
    cpu_set_t cpuset;
    CPU_ZERO(&cpuset);
    CPU_SET(cpu_id, &cpuset); // Set CPU ID (for example, CPU 0)

    int s = pthread_setaffinity_np(pthread_self(), sizeof(cpu_set_t), &cpuset);
    if (s != 0) {
        std::cerr << "Error setting CPU affinity: " << strerror(s) << std::endl;
    } else {
        std::cout << "[Info] Thread bound to CPU " << cpu_id << std::endl;
    }
}

// Set thread to real-time scheduling policy
void set_realtime_priority() {
    struct sched_param sched;
    sched.sched_priority = 80; // Priority from 1-99 (higher = more important)
    if (pthread_setschedparam(pthread_self(), SCHED_FIFO, &sched) != 0) {
        std::cerr << "[Warning] Failed to set real-time priority: " << std::strerror(errno) << "\n";
    } else {
        std::cout << "[Info] Real-time priority set.\n";
    }
}

// Real-time consumer thread
void realTimeThread() {
    set_realtime_priority();
    set_cpu_affinity(0);      // Set CPU affinity to core 0 
    std::vector<std::complex<double>> fftOutput(DATA_SIZE); // Preallocated buffer

    while (true) {
        std::vector<double> data;
        {
            std::unique_lock lock(queueMutex);
            dataCond.wait(lock, [] { return !dataQueue.empty() || finished; });
            if (finished && dataQueue.empty()) break;
            data = std::move(dataQueue.front());
            dataQueue.pop();
        }

        auto start = std::chrono::steady_clock::now();

        dummyFFT(data, fftOutput);

        auto end = std::chrono::steady_clock::now();
        auto duration = std::chrono::duration_cast<std::chrono::microseconds>(end - start).count();

        std::cout << "[RealTimeThread] Processing Time: " << duration << " us"
                  << (duration > DEADLINE_US ? " -- Deadline Missed!" : " -- OK")
                  << "\n";
    }
}

// Non-real-time consumer thread
void nonRealTimeThread() {
    while (true) {
        std::vector<double> data;
        {
            std::unique_lock lock(queueMutex);
            dataCond.wait(lock, [] { return !dataQueue.empty() || finished; });
            if (finished && dataQueue.empty()) break;
            data = std::move(dataQueue.front());
            dataQueue.pop();
        }
        auto start = std::chrono::steady_clock::now();

        std::vector<std::complex<double>> fftOutput(DATA_SIZE); // Dynamic allocation each time
        dummyFFT(data, fftOutput);
        auto end = std::chrono::steady_clock::now();
        auto duration = std::chrono::duration_cast<std::chrono::microseconds>(end - start).count();

        std::cout << "[NonRealTimeThread] Processed a frame. Processing Time: " << duration << " us\n";
    }
}

// Data producer thread
void producerThread(int frames) {
    for (int frame = 0; frame < frames; ++frame) {
        std::vector<double> data(DATA_SIZE);
        for (int i = 0; i < DATA_SIZE; ++i) {
            data[i] = std::sin(2 * M_PI * i / DATA_SIZE);
        }

        {
            std::lock_guard lock(queueMutex);
            dataQueue.push(std::move(data));
        }
        dataCond.notify_all();

        std::this_thread::sleep_for(10us); // Simulate streaming every 10us
    }

    // Signal consumers to exit
    {
        std::lock_guard lock(queueMutex);
        finished = true;
    }
    dataCond.notify_all();
}

int main() {
    std::thread producer(producerThread, 100); // produce 100 frames
    std::thread rtThread(realTimeThread);
    std::thread nrtThread(nonRealTimeThread);

    producer.join();
    rtThread.join();
    nrtThread.join();

    std::cout << "All done!\n";
    return 0;
}

We need to carefully plan our deadline, which in my case I kept 200us.
Planning your real-time task's attributes is completely an investigation based design choice, the right and wrongs are in your hand, OS only promises time-bound response.
In the code, we are setting two main real-time "attributes" for the thread:

Real-time scheduling policy:
We set the thread to SCHED_FIFO with a priority of 80 inside set_realtime_priority().
CPU affinity:
We pin the real-time thread to CPU 0 using set_cpu_affinity(0).

What We Did Right	What Problems Still Exist (Causing Deadline Miss)
Set SCHED_FIFO (real-time) scheduling.	- Linux is not a "hard" real-time OS (it's soft real-time at best without PREEMPT-RT patch).
Set a high priority (80/99).	- Interrupts (IRQ) are still served by the same CPU and can preempt your real-time thread.
Pinned to a single CPU core (CPU 0).	- Higher priority threads/processes (e.g., kernel daemons, IRQ handlers) can preempt your thread.
	- Cache misses and memory contention can cause random latency spikes.
	- Power-saving features (like CPU frequency scaling) can add jitter.
	- Lock contention (if other threads are using mutexes, condition variables) can delay wakeup.

So if we pay attention to output:

sshekhar@sshekhar:~/workspace/practice$ g++ nrt_rt_demo.cpp -lpthread -o realtime_demo
sshekhar@sshekhar:~/workspace/practice$ sudo ./realtime_demo 
[Info] Real-time priority set.
[Info] Thread bound to CPU 0
[RealTimeThread] Processing Time: 264 us -- Deadline Missed!
[NonRealTimeThread] Processed a frame. Processing Time: 421 us
[RealTimeThread] Processing Time: 258 us -- Deadline Missed!
[RealTimeThread] Processing Time: 215 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 489 us
[RealTimeThread] Processing Time: 229 us -- Deadline Missed!
[NonRealTimeThread] Processed a frame. Processing Time: 250 us
[RealTimeThread] Processing Time: 246 us -- Deadline Missed!
[NonRealTimeThread] Processed a frame. Processing Time: 290 us
[RealTimeThread] Processing Time: 220 us -- OK
[RealTimeThread] Processing Time: 226 us -- Deadline Missed!
[NonRealTimeThread] Processed a frame. Processing Time: 336 us
[RealTimeThread] Processing Time: 202 us -- OK
[RealTimeThread] Processing Time: 201 us -- OK
[RealTimeThread] Processing Time: 201 us -- OK
[RealTimeThread] Processing Time: 200 us -- OK
[RealTimeThread] Processing Time: 216 us -- OK
[RealTimeThread] Processing Time: 201 us -- OK
[RealTimeThread] Processing Time: 205 us -- OK
[RealTimeThread] Processing Time: 200 us -- OK
[RealTimeThread] Processing Time: 203 us -- OK
[RealTimeThread] Processing Time: 203 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 330 us
[RealTimeThread] Processing Time: 205 us -- OK
[RealTimeThread] Processing Time: 207 us -- OK
[RealTimeThread] Processing Time: 226 us -- Deadline Missed!
[RealTimeThread] Processing Time: 205 us -- OK
[RealTimeThread] Processing Time: 202 us -- OK
[RealTimeThread] Processing Time: 204 us -- OK
[RealTimeThread] Processing Time: 216 us -- OK
[RealTimeThread] Processing Time: 205 us -- OK
[RealTimeThread] Processing Time: 200 us -- OK
[RealTimeThread] Processing Time: 206 us -- OK
[RealTimeThread] Processing Time: 226 us -- Deadline Missed!
[NonRealTimeThread] Processed a frame. Processing Time: 183 us
[RealTimeThread] Processing Time: 204 us -- OK
[RealTimeThread] Processing Time: 201 us -- OK
[RealTimeThread] Processing Time: 200 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 278 us
[RealTimeThread] Processing Time: 201 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 225 us
[RealTimeThread] Processing Time: 203 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 223 us
[RealTimeThread] Processing Time: 224 us -- Deadline Missed!
[RealTimeThread] Processing Time: 216 us -- OK
[RealTimeThread] Processing Time: 210 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 351 us
[RealTimeThread] Processing Time: 224 us -- Deadline Missed!
[RealTimeThread] Processing Time: 200 us -- OK
[RealTimeThread] Processing Time: 201 us -- OK
[RealTimeThread] Processing Time: 201 us -- OK
[RealTimeThread] Processing Time: 221 us -- Deadline Missed!
[NonRealTimeThread] Processed a frame. Processing Time: 313 us
[RealTimeThread] Processing Time: 225 us -- Deadline Missed!
[NonRealTimeThread] Processed a frame. Processing Time: 271 us
[RealTimeThread] Processing Time: 246 us -- Deadline Missed!
[RealTimeThread] Processing Time: 206 us -- OK
[RealTimeThread] Processing Time: 201 us -- OK
[RealTimeThread] Processing Time: 202 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 296 us
[RealTimeThread] Processing Time: 218 us -- OK
[RealTimeThread] Processing Time: 228 us -- Deadline Missed!
[RealTimeThread] Processing Time: 259 us -- Deadline Missed!
[RealTimeThread] Processing Time: 232 us -- Deadline Missed!
[RealTimeThread] Processing Time: 229 us -- Deadline Missed!
[RealTimeThread] Processing Time: 242 us -- Deadline Missed!
[RealTimeThread] Processing Time: 252 us -- Deadline Missed!
[RealTimeThread] Processing Time: 266 us -- Deadline Missed!
[NonRealTimeThread] Processed a frame. Processing Time: 300 us
[RealTimeThread] Processing Time: 216 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 311 us
[RealTimeThread] Processing Time: 234 us -- Deadline Missed!
[RealTimeThread] Processing Time: 212 us -- OK
[RealTimeThread] Processing Time: 206 us -- OK
[RealTimeThread] Processing Time: 208 us -- OK
[RealTimeThread] Processing Time: 231 us -- Deadline Missed!
[RealTimeThread] Processing Time: 213 us -- OK
[RealTimeThread] Processing Time: 210 us -- OK
[RealTimeThread] Processing Time: 206 us -- OK
[RealTimeThread] Processing Time: 224 us -- Deadline Missed!
[RealTimeThread] Processing Time: 250 us -- Deadline Missed!
[RealTimeThread] Processing Time: 241 us -- Deadline Missed!
[RealTimeThread] Processing Time: 232 us -- Deadline Missed!
[RealTimeThread] Processing Time: 207 us -- OK
[RealTimeThread] Processing Time: 256 us -- Deadline Missed!
[RealTimeThread] Processing Time: 246 us -- Deadline Missed!
[RealTimeThread] Processing Time: 231 us -- Deadline Missed!
[NonRealTimeThread] Processed a frame. Processing Time: 320 us
[RealTimeThread] Processing Time: 215 us -- OK
[RealTimeThread] Processing Time: 211 us -- OK
[RealTimeThread] Processing Time: 237 us -- Deadline Missed!
[RealTimeThread] Processing Time: 241 us -- Deadline Missed!
[RealTimeThread] Processing Time: 222 us -- Deadline Missed!
[RealTimeThread] Processing Time: 252 us -- Deadline Missed!
[RealTimeThread] Processing Time: 249 us -- Deadline Missed!
[NonRealTimeThread] Processed a frame. Processing Time: 327 us
[RealTimeThread] Processing Time: 236 us -- Deadline Missed!
[RealTimeThread] Processing Time: 224 us -- Deadline Missed!
[RealTimeThread] Processing Time: 219 us -- OK
[RealTimeThread] Processing Time: 211 us -- OK
All done!
sshekhar@sshekhar:~/workspace/practice$

⌚ Rooms for improvement

✅ Setting SCHED_FIFO means your thread can preempt normal (non-real-time) threads immediately.

✅ Pinning to CPU 0 reduces migration cost (cache misses) and context-switch overheads — but CPU 0 is often very busy (it often handles kernel housekeeping tasks and interrupts). Maybe pinning to CPU 1, 2, 3 would be slightly better.

And by just a small change in cpu pinning to CPU 3:

sshekhar@sshekhar:~/workspace/practice$ g++ nrt_rt_demo.cpp -lpthread -o realtime_demo
sshekhar@sshekhar:~/workspace/practice$ sudo ./realtime_demo 
[sudo] password for sshekhar: 
[Info] Real-time priority set.
[Info] Thread bound to CPU 3
[NonRealTimeThread] Processed a frame. Processing Time: 141 us
[RealTimeThread] Processing Time: 51 us -- OK
[RealTimeThread] Processing Time: 50 us -- OK
[RealTimeThread] Processing Time: 50 us -- OK
[RealTimeThread] Processing Time: 50 us -- OK
[RealTimeThread] Processing Time: 47 us -- OK
[RealTimeThread] Processing Time: 47 us -- OK
[RealTimeThread] Processing Time: 47 us -- OK
[RealTimeThread] Processing Time: 47 us -- OK
[RealTimeThread] Processing Time: 47 us -- OK
[RealTimeThread] Processing Time: 47 us -- OK
[RealTimeThread] Processing Time: 96 us -- OK
[RealTimeThread] Processing Time: 93 us -- OK
[RealTimeThread] Processing Time: 95 us -- OK
[RealTimeThread] Processing Time: 83 us -- OK
[RealTimeThread] Processing Time: 88 us -- OK
[RealTimeThread] Processing Time: 87 us -- OK
[RealTimeThread] Processing Time: 89 us -- OK
[RealTimeThread] Processing Time: 90 us -- OK
[RealTimeThread] Processing Time: 65 us -- OK
[RealTimeThread] Processing Time: 91 us -- OK
[RealTimeThread] Processing Time: 91 us -- OK
[RealTimeThread] Processing Time: 88 us -- OK
[RealTimeThread] Processing Time: 88 us -- OK
[RealTimeThread] Processing Time: 97 us -- OK
[RealTimeThread] Processing Time: 95 us -- OK
[RealTimeThread] Processing Time: 87 us -- OK
[RealTimeThread] Processing Time: 97 us -- OK
[RealTimeThread] Processing Time: 96 us -- OK
[RealTimeThread] Processing Time: 88 us -- OK
[RealTimeThread] Processing Time: 101 us -- OK
[RealTimeThread] Processing Time: 88 us -- OK
[RealTimeThread] Processing Time: 89 us -- OK
[RealTimeThread] Processing Time: 91 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 121 us
[RealTimeThread] Processing Time: 96 us -- OK
[RealTimeThread] Processing Time: 90 us -- OK
[RealTimeThread] Processing Time: 80 us -- OK
[RealTimeThread] Processing Time: 83 us -- OK
[RealTimeThread] Processing Time: 80 us -- OK
[RealTimeThread] Processing Time: 87 us -- OK
[RealTimeThread] Processing Time: 91 us -- OK
[RealTimeThread] Processing Time: 90 us -- OK
[RealTimeThread] Processing Time: 90 us -- OK
[RealTimeThread] Processing Time: 105 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 121 us
[RealTimeThread] Processing Time: 83 us -- OK
[RealTimeThread] Processing Time: 92 us -- OK
[RealTimeThread] Processing Time: 79 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 120 us
[NonRealTimeThread] Processed a frame. Processing Time: 111 us
[NonRealTimeThread] Processed a frame. Processing Time: 117 us
[RealTimeThread] Processing Time: 80 us -- OK
[RealTimeThread] Processing Time: 80 us -- OK
[RealTimeThread] Processing Time: 79 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 124 us
[RealTimeThread] Processing Time: 79 us -- OK
[RealTimeThread] Processing Time: 88 us -- OK
[RealTimeThread] Processing Time: 79 us -- OK
[RealTimeThread] Processing Time: 80 us -- OK
[RealTimeThread] Processing Time: 82 us -- OK
[RealTimeThread] Processing Time: 82 us -- OK
[RealTimeThread] Processing Time: 79 us -- OK
[RealTimeThread] Processing Time: 82 us -- OK
[RealTimeThread] Processing Time: 80 us -- OK
[RealTimeThread] Processing Time: 80 us -- OK
[RealTimeThread] Processing Time: 81 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 119 us
[RealTimeThread] Processing Time: 84 us -- OK
[RealTimeThread] Processing Time: 81 us -- OK
[RealTimeThread] Processing Time: 90 us -- OK
[RealTimeThread] Processing Time: 80 us -- OK
[RealTimeThread] Processing Time: 83 us -- OK
[RealTimeThread] Processing Time: 81 us -- OK
[RealTimeThread] Processing Time: 78 us -- OK
[RealTimeThread] Processing Time: 82 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 133 us
[RealTimeThread] Processing Time: 83 us -- OK
[RealTimeThread] Processing Time: 79 us -- OK
[RealTimeThread] Processing Time: 88 us -- OK
[RealTimeThread] Processing Time: 78 us -- OK
[RealTimeThread] Processing Time: 92 us -- OK
[RealTimeThread] Processing Time: 82 us -- OK
[RealTimeThread] Processing Time: 97 us -- OK
[RealTimeThread] Processing Time: 83 us -- OK
[RealTimeThread] Processing Time: 83 us -- OK
[RealTimeThread] Processing Time: 83 us -- OK
[RealTimeThread] Processing Time: 82 us -- OK
[RealTimeThread] Processing Time: 83 us -- OK
[RealTimeThread] Processing Time: 83 us -- OK
[RealTimeThread] Processing Time: 90 us -- OK
[RealTimeThread] Processing Time: 83 us -- OK
[RealTimeThread] Processing Time: 92 us -- OK
[NonRealTimeThread] Processed a frame. Processing Time: 122 us
[RealTimeThread] Processing Time: 83 us -- OK
[RealTimeThread] Processing Time: 84 us -- OK
[RealTimeThread] Processing Time: 95 us -- OK
[RealTimeThread] Processing Time: 95 us -- OK
[RealTimeThread] Processing Time: 99 us -- OK
[RealTimeThread] Processing Time: 87 us -- OK
All done!

There are now 0 deadline misses and execution time is less than half of threshold.

And believe me we can refine it further! But its too lengthy for this blog.

To Summarize:

🧠 What in real-time execution actually happens:

✅ The CPU:

Is always dumb — it just fetches and executes instructions, cycle-by-cycle.
It doesn’t "know" about real-time or non-real-time.

✅ The Kernel (OS Scheduler):

Controls which thread gets to run next on the CPU.
When you ask for real-time (SCHED_FIFO, SCHED_RR, etc.), you're telling the kernel:
- "Hey, always prioritize this thread over others!"

✅ The Real-Time Priority:

High priority real-time threads preempt low-priority threads immediately.
Even inside the kernel itself (with PREEMPT_RT patches).
So if you're running at SCHED_FIFO priority 90, and a logger at 10, the logger won't disturb you.

✅ Memory locking (`mlockall`):

Prevents page faults.
- In normal Linux, if your code touches new memory, the kernel might say, "Wait, let me fetch this page from disk..."
- That "disk fetch" = hundreds of microseconds → instant death for real-time.
- mlockall(MCL_CURRENT | MCL_FUTURE); keeps all your current and future allocations in RAM.

✅ Lockless programming:

In RT, you want to never block on locks, semaphores, etc.
Use lock-free data structures, atomics, or message queues designed for RT.

In following blogs, me try to contribute about what kind of design choices should we make while developing an application related to real-time demands.

DEV Community: Zeporath