DEV Community: GhostSocket

Blocking Unlabelled Pods with Kyverno: From ClusterPolicy to CEL on a Local kind Cluster

GhostSocket — Fri, 06 Nov 2026 11:35:59 +0000

Note: This is an independent walkthrough by a learner and is not officially endorsed by the CNCF or Kyverno maintainers.

Summary
In this walkthrough, I enforced mandatory team labeling on Kubernetes Pods using both Kyverno's legacy ClusterPolicy and the modern CEL-based ValidatingPolicy. I learned how Kyverno extends the Kubernetes API with CRDs and how CEL-based policies run in Kyverno's admission webhook using a standard expression language instead of Kyverno's custom pattern-matching engine. This was built locally on a kind cluster to compare the two architectures side-by-side.
Environment Setup
Everything was tested locally using a standard kind cluster. I Note: This is an independent walkthrough by a learner and is not officially endorsed by the CNCF or Kyverno maintainers.

1. Summary

In this walkthrough, I enforced mandatory team labeling on Kubernetes Pods using both Kyverno's legacy ClusterPolicy and the modern CEL-based ValidatingPolicy. I learned how Kyverno extends the Kubernetes API with CRDs and how CEL-based policies run in Kyverno's admission webhook using a standard expression language instead of Kyverno's custom pattern-matching engine. This was built locally on a kind cluster to compare the two architectures side-by-side.

2. Environment Setup

Everything was tested locally using a standard kind cluster. I installed Kyverno via Helm, using chart version kyverno-3.9.2 (App version v1.19.2) that supports CEL policies.

Out of the box, Kubernetes has no idea what a Kyverno policy is. To verify the installation, I checked if Kyverno successfully registered its Custom Resource Definitions (CRDs), which extend the Kubernetes API so it can understand our custom policy objects:

kubectl get crd | findstr "policies.kyverno.io"

Output:

cleanuppolicies.kyverno.io                              Namespaced   v2(storage),v2beta1            2026-10-24T15:40:36Z
clustercleanuppolicies.kyverno.io                       Cluster      v2(storage),v2beta1            2026-10-24T15:40:36Z
clusterpolicies.kyverno.io                              Cluster      v1(storage),v2beta1            2026-10-24T15:40:37Z
deletingpolicies.policies.kyverno.io                    Cluster      v1,v1alpha1,v1beta1(storage)   2026-10-24T15:40:36Z
generatingpolicies.policies.kyverno.io                  Cluster      v1,v1alpha1,v1beta1(storage)   2026-10-24T15:40:36Z
imagevalidatingpolicies.policies.kyverno.io             Cluster      v1,v1alpha1,v1beta1(storage)   2026-10-24T15:40:36Z
mutatingpolicies.policies.kyverno.io                    Cluster      v1,v1alpha1,v1beta1(storage)   2026-10-24T15:40:36Z
namespaceddeletingpolicies.policies.kyverno.io          Namespaced   v1,v1beta1(storage)            2026-10-24T15:40:36Z
namespacedgeneratingpolicies.policies.kyverno.io        Namespaced   v1,v1beta1(storage)            2026-10-24T15:40:36Z
namespacedimagevalidatingpolicies.policies.kyverno.io   Namespaced   v1,v1beta1(storage)            2026-10-24T15:40:36Z
namespacedmutatingpolicies.policies.kyverno.io          Namespaced   v1,v1beta1(storage)            2026-10-24T15:40:36Z
namespacedvalidatingpolicies.policies.kyverno.io        Namespaced   v1,v1beta1(storage)            2026-10-24T15:40:36Z
policies.kyverno.io                                     Namespaced   v1(storage),v2beta1            2026-10-24T15:40:37Z
policyexceptions.policies.kyverno.io                    Namespaced   v1,v1alpha1,v1beta1(storage)   2026-10-24T15:40:36Z
validatingpolicies.policies.kyverno.io                  Cluster      v1,v1alpha1,v1beta1(storage)   2026-10-24T15:40:36Z

Seeing validatingpolicies.policies.kyverno.io in the output confirmed the CRDs were installed.

3. Policy v1: The Legacy ClusterPolicy

Before trying CEL, I wanted to see how this is traditionally done. In older versions of Kyverno, you create a ClusterPolicy that relies on Kyverno's custom pattern-matching engine.

Based on the official Kyverno Policy Library, a basic policy to enforce a label looks like this:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-team-label
spec:
  validationFailureAction: Enforce
  rules:
  - name: check-team-label
    match:
      any:
      - resources:
          kinds:
          - Pod
    validate:
      message: "The label `team` is required."
      pattern:
        metadata:
          labels:
            team: "?*"

Notice the syntax: it tells the Kyverno controller to watch for Pod resources, and it uses a custom wildcard "?*" to say "this label must exist and have a value."

When I applied this policy and tried to create an unlabelled pod, the Kyverno controller caught it and blocked the request, sending back this rejection message:

Error from server: admission webhook "validate.kyverno.svc-fail" denied the request:
resource Pod/default/c1 was blocked due to the following policies
require-team-label:
  check-team-label: 'validation error: The label `team` is required. rule check-team-label failed at path /metadata/labels/team/'

4. Policy v2: The CEL ValidatingPolicy

Next, I deleted the legacy ClusterPolicy and replaced it with a modern ValidatingPolicy written in Common Expression Language (CEL).

Here is the exact ValidatingPolicy manifest I applied to the cluster:

apiVersion: policies.kyverno.io/v1
kind: ValidatingPolicy
metadata:
  name: require-team-label
spec:
  validationActions:
    - Deny
  matchConstraints:
    resourceRules:
      - apiGroups: [""]
        apiVersions: ["v1"]
        operations: ["CREATE", "UPDATE"]
        resources: ["pods"]
  validations:
    - expression: "has(object.metadata.labels) && 'team' in object.metadata.labels"
      message: "The label `team` is required."

Breaking Down the CEL Logic

The core of this policy is the CEL expression:
has(object.metadata.labels) && 'team' in object.metadata.labels

has(object.metadata.labels) safely checks if the incoming Pod object has any labels attached at all (preventing a null reference error).
'team' in object.metadata.labels evaluates whether the specific key team exists within that metadata map.

Testing the CEL Policy

Negative Test (Missing Label):

First, I attempted to launch an unlabelled Pod:

kubectl run c1 --image=nginx

As expected, the request was immediately denied by the admission webhook:

Error from server: admission webhook "vpol.validate.kyverno.svc-fail" denied the request: Policy require-team-label failed: The label `team` is required.

Positive Test (Valid Label Attached):

Next, I ran the test command providing the required label:

kubectl run c2 --image=nginx --labels=team=payments

Output:

pod/c2 created

5. Comparison: What Changed?

Having deployed and tested both, the shift from traditional Kyverno policies to CEL-based validation introduces some major architectural changes. Here is a quick breakdown of the core differences:

Feature	Legacy `ClusterPolicy` / `Policy`	Modern CEL `ValidatingPolicy`
Kinds	One kind holds validate, mutate, and generate rules	A separate kind per purpose: Validating, Mutating, Generating, Deleting, ImageValidating
Shape	`spec.rules[]`, each with `match` and a `validate` or `mutate` block	`matchConstraints` once, then `validations[]` with a CEL expression and a message (modeled on Kubernetes’ `ValidatingAdmissionPolicy`)
Matching	Kinds like `Pod`, plus names, selectors, and more	API groups, versions, operations, and plural resources like `pods`
Scope	`ClusterPolicy` cluster-wide, `Policy` per namespace	`ValidatingPolicy` cluster-wide, `NamespacedValidatingPolicy` per namespace
Enforcement	Failure action (Enforce/Audit)	Validation actions (Deny/Audit/Warn)

6. Takeaways & Next Steps

Transitioning from traditional Kyverno ClusterPolicy manifests to modern CEL-based ValidatingPolicy represents a significant architectural step forward:

Standardization: CEL rules use the same expression language and a similar structure as Kubernetes' own ValidatingAdmissionPolicy rather than relying on custom pattern-matching dialects.
Efficiency: CEL provides deterministic typing and a standard expression language, replacing Kyverno's custom pattern-matching dialect.

What I Haven't Tested

Policy reports: Using Kyverno's PolicyReport and ClusterPolicyReport CRDs to aggregate and query validation results over time.
Native ValidatingAdmissionPolicy generation: Generating upstream Kubernetes ValidatingAdmissionPolicy resources from Kyverno CEL policies (via kyverno generate).installed Kyverno via Helm, using chart version kyverno-3.9.2 (App version v1.19.2) that supports CEL policies.

kubectl get crd | findstr "policies.kyverno.io"

Output:

cleanuppolicies.kyverno.io Namespaced v2(storage),v2beta1 2026-10-24T15:40:36Z
clustercleanuppolicies.kyverno.io Cluster v2(storage),v2beta1 2026-10-24T15:40:36Z
clusterpolicies.kyverno.io Cluster v1(storage),v2beta1 2026-10-24T15:40:37Z
deletingpolicies.policies.kyverno.io Cluster v1,v1alpha1,v1beta1(storage) 2026-10-24T15:40:36Z
generatingpolicies.policies.kyverno.io Cluster v1,v1alpha1,v1beta1(storage) 2026-10-24T15:40:36Z
imagevalidatingpolicies.policies.kyverno.io Cluster v1,v1alpha1,v1beta1(storage) 2026-10-24T15:40:36Z
mutatingpolicies.policies.kyverno.io Cluster v1,v1alpha1,v1beta1(storage) 2026-10-24T15:40:36Z
namespaceddeletingpolicies.policies.kyverno.io Namespaced v1,v1beta1(storage) 2026-10-24T15:40:36Z
namespacedgeneratingpolicies.policies.kyverno.io Namespaced v1,v1beta1(storage) 2026-10-24T15:40:36Z
namespacedimagevalidatingpolicies.policies.kyverno.io Namespaced v1,v1beta1(storage) 2026-10-24T15:40:36Z
namespacedmutatingpolicies.policies.kyverno.io Namespaced v1,v1beta1(storage) 2026-10-24T15:40:36Z
namespacedvalidatingpolicies.policies.kyverno.io Namespaced v1,v1beta1(storage) 2026-10-24T15:40:36Z
policies.kyverno.io Namespaced v1(storage),v2beta1 2026-10-24T15:40:37Z
policyexceptions.policies.kyverno.io Namespaced v1,v1alpha1,v1beta1(storage) 2026-10-24T15:40:36Z
validatingpolicies.policies.kyverno.io Cluster v1,v1alpha1,v1beta1(storage) 2026-10-24T15:40:36Z

Seeing validatingpolicies.policies.kyverno.io in the output confirmed the CRDs were installed.

Policy v1: The Legacy ClusterPolicy Before trying CEL, I wanted to see how this is traditionally done. In older versions of Kyverno, you create a ClusterPolicy that relies on Kyverno's custom pattern-matching engine.

Based on the official Kyverno Policy Library, a basic policy to enforce a label looks like this:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
name: require-team-label
spec:
validationFailureAction: Enforce
rules:

name: check-team-label match: any:
- resources: kinds:
  - Pod validate: message: "The label team is required." pattern: metadata: labels: team: "?*"

Notice the syntax: it tells the Kyverno controller to watch for Pod resources, and it uses a custom wildcard "?*" to say "this label must exist and have a value."

When I applied this policy and tried to create an unlabelled pod, the Kyverno controller caught it and blocked the request, sending back this rejection message:

Error from server: admission webhook "validate.kyverno.svc-fail" denied the request:
resource Pod/default/c1 was blocked due to the following policies
require-team-label:
check-team-label: 'validation error: The label team is required. rule check-team-label failed at path /metadata/labels/team/'

Policy v2: The CEL ValidatingPolicy Next, I deleted the legacy ClusterPolicy and replaced it with a modern ValidatingPolicy written in Common Expression Language (CEL).

Here is the exact ValidatingPolicy manifest I applied to the cluster:

apiVersion: policies.kyverno.io/v1
kind: ValidatingPolicy
metadata:
name: require-team-label
spec:
validationActions:
- Deny
matchConstraints:
resourceRules:
- apiGroups: [""]
apiVersions: ["v1"]
operations: ["CREATE", "UPDATE"]
resources: ["pods"]
validations:
- expression: "has(object.metadata.labels) && 'team' in object.metadata.labels"
message: "The label team is required."

Breaking Down the CEL Logic
The core of this policy is the CEL expression: has(object.metadata.labels) && 'team' in object.metadata.labels

has(object.metadata.labels) safely checks if the incoming Pod object has any labels attached at all (preventing a null reference error).
'team' in object.metadata.labels evaluates whether the specific key team exists within that metadata map.
Testing the CEL Policy
Negative Test (Missing Label):
First, I attempted to launch an unlabelled Pod:

kubectl run c1 --image=nginx

As expected, the request was immediately denied by the admission webhook:

Error from server: admission webhook "vpol.validate.kyverno.svc-fail" denied the request: Policy require-team-label failed: The label team is required.

Positive Test (Valid Label Attached):
Next, I ran the test command providing the required label:

kubectl run c2 --image=nginx --labels=team=payments

Output:

pod/c2 created

Comparison: What Changed? Having deployed and tested both, the shift from traditional Kyverno policies to CEL-based validation introduces some major architectural changes. Here is a quick breakdown of the core differences:

Feature Legacy ClusterPolicy / Policy Modern CEL ValidatingPolicy
Kinds One kind holds validate, mutate, and generate rules A separate kind per purpose: Validating, Mutating, Generating, Deleting, ImageValidating
Shape spec.rules[], each with match and a validate or mutate block matchConstraints once, then validations[] with a CEL expression and a message (modeled on Kubernetes’ ValidatingAdmissionPolicy)
Matching Kinds like Pod, plus names, selectors, and more API groups, versions, operations, and plural resources like pods
Scope ClusterPolicy cluster-wide, Policy per namespace ValidatingPolicy cluster-wide, NamespacedValidatingPolicy per namespace
Enforcement Failure action (Enforce/Audit) Validation actions (Deny/Audit/Warn)

Takeaways & Next Steps Transitioning from traditional Kyverno ClusterPolicy manifests to modern CEL-based ValidatingPolicy represents a significant architectural step forward:

Standardization: CEL rules use the same expression language and a similar structure as Kubernetes' own ValidatingAdmissionPolicy rather than relying on custom pattern-matching dialects.
Efficiency: CEL provides deterministic typing and a standard expression language, replacing Kyverno's custom pattern-matching dialect.
What I Haven't Tested
Policy reports: Using Kyverno's PolicyReport and ClusterPolicyReport CRDs to aggregate and query validation results over time.
Native ValidatingAdmissionPolicy generation: Generating upstream Kubernetes ValidatingAdmissionPolicy resources from Kyverno CEL policies (via kyverno generate).

Handling AI Traffic Spikes: Decoupling Express APIs with Redis and BullMQ

GhostSocket — Fri, 07 Aug 2026 18:06:46 +0000

This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry.

Executive Summary & Roadmap

Imagine 100 students hitting "Submit" on a final exam at exactly 11:00 AM.

In a traditional CRUD app, this is a standard traffic spike. But when every submission requires evaluating multiple short-answer questions using an AI model, that standard spike turns into a bottleneck that can easily take down your entire backend.

At MOD-U-GO, an online examination platform, we wanted to automate the grading of text-based answers using Small Language Models (SLMs). We chose the Groq LLM API for its incredibly fast inference speeds. However, we quickly realized that bolting an LLM call directly into our synchronous HTTP request lifecycle was a recipe for disaster.

Here is how we solved this by decoupling our AI grading process from the submission lifecycle—designing an Asynchronous Architecture using Node.js, Bull, and Redis to grade exams at scale without ruining the user experience.

Topics to look forward to:

The Pitfalls of Synchronous LLM Calls: Why outbound socket exhaustion and provider-side queueing trigger 504 Gateway Timeouts under load.
The "Fire-and-Forget" Pattern: Decoupling user submissions from heavy background processing using Node.js, Bull, and Redis.
Architectural Trade-offs & Zero Data Loss: Handling API timeouts safely with a pending_review state so no student exam is ever lost or scored zero.
Scaling Strategies: Queue-level global rate limiting and horizontal worker scaling to handle large-scale exam traffic.

The Problem: Why Synchronous LLM Grading Fails
Architecture & Data Flow: The "Fire-and-Forget" Pattern
Queue Configuration & System Resilience
Edge Cases & Architectural Trade-offs
Future Enhancements & Scalability

1. The Problem: Why Synchronous LLM Grading Fails

When building AI features, the most common anti-pattern is treating an LLM call like a database query. If we bolt an LLM directly into a synchronous HTTP request lifecycle, the math works against us.

The Mathematics of HTTP Timeouts

Let’s break down the exact numbers. A single call to an SLM to evaluate a short-answer response takes ~1.5 seconds on average. Assume we have an examination of 10 short-answer questions.

10 questions × 1.5 seconds = 15 seconds just to grade a single exam.

On paper, 15 seconds is under the standard 30-second Cloudflare/AWS ALB gateway timeout. But what happens when 30 students hit "Submit" at the exact same moment when the exam timer ends?

In a synchronous setup, 30 concurrent submissions create a burst of 300 outbound HTTP requests to the LLM provider. This triggers two immediate bottlenecks:

Outbound Socket Queuing: Node.js limits outbound concurrent sockets to prevent host memory exhaustion. As 300 requests fight for socket availability, later requests would wait in our server's local connection pool before ever reaching the network.
Provider Latency Degradation: Flooding an LLM gateway with 300 burst requests forces the provider to queue our jobs, degrading per-call latency from 1.5s to 3.5s+.

Because the questions are processed sequentially, an increase of just 2 seconds per inference call inflates total execution time to 10 * 3.5 = 35 seconds.

Hence the connection drops, returning a 504 Gateway Timeout.

The API Rate Limit Wall

Beyond timeouts, it immediately hit Limit. Those 30 concurrent exams generate 300 simultaneous API calls (30 students × 10 questions).

LLM providers strictly enforce RPM (Requests Per Minute) and TPM (Tokens Per Minute) quotas. Flooding an API with 300 rapid-fire inference requests will instantly trigger a wave of 429 Too Many Requests errors. Because the backend is processing this synchronously, those failures bubble straight up to the user, completely breaking the exam submission flow.

To prevent this cascading failure, we must decouple the submission from the evaluation.

2. Architecture & Data Flow: The "Fire-and-Forget" Pattern

To survive massive traffic spikes, we shifted to an Event-Driven Architecture backed by a job queue using Bull and Redis. The golden rule of this architecture is the Fire-and-Forget pattern: secure the data, acknowledge the request, and keep the heavy lifting for future.

Text-based questions are marked for Evaluation. The submission is saved to the database with a grading status, and the API immediately fires a 201 Created response back to the client. As far as the student is concerned, the exam is safely submitted with near-zero latency.

The End-to-End Submission Journey

Submission: The student submits their exam via a POST request.
Synchronous Split: The backend grades all MCQs instantly.
Immediate Client Response: The API returns a success response to the student without waiting for the LLM.(Near - Zero Latency)
Fire-and-Forget Enqueueing: The API pushes a lightweight job containing only the submissionId into the Bull queue, keeping Redis memory footprint small.
Worker Processing: A background worker process pops the job from Redis, fetches the full exam context from the database, and iterates through the text answers.
SLM Inference: The worker makes independent HTTP requests to the Groq API for each answer, asking the SLM to evaluate semantic similarity against a baseline model answer.
Persistence & Notification: The returned SLM scores (0.0 to 1.0) are normalized into actual marks. The database is updated, and the student receives a push notification that their final grade is ready.

System Architecture Diagram and Bull working

To understand more about Redis and it's working you can read - https://dev.to/socket_a64c6489500ee7/demystifying-redis-how-i-built-a-custom-in-memory-store-in-c-5fg4

3. Queue Configuration & System Resilience

Networks are uncertain, and third-party APIs go down. A grading system cannot afford to lose student data or assign a score of zero due to a dropped connection. We built resilience into the pipeline at two distinct levels: the queue configuration and the application layer.

Exponential Backoff & Retry Strategies

We configured our Bull queue with an Exponential Backoff strategy. If the Groq API drops a request, the worker doesn't immediately hammer the API again. It retries up to 3 times, waiting 2 seconds, then 4 seconds, and finally 8 seconds before giving up. We also keep a tight limit on job history (removeOnComplete: 100, removeOnFail: 50) to avoid Redis memory bloat.

// Bull queue robust default options
const queueOptions = {
  defaultJobOptions: {
    attempts: 3,
    backoff: {
      type: "exponential",
      delay: 2000, // Starts at 2s, exponentially increases
    },
    removeOnComplete: 100,
    removeOnFail: 50,
  },
};

In-Memory Graceful Fallback (Operating Without Redis)

In a rigid architecture, the grading pipeline would break entirely in case redis conenction failure We built an In-Memory Graceful Fallback directly into the queue producer. If the redisAvailable flag is false during enqueueing, the system catches the failure and immediately falls back to processing the grading job in the background memory of the Node.js process.

// Gracefully fallback if Redis is down
async function enqueueGrading(submissionId) {
  if (redisAvailable && gradingQueue) {
    try {
      await gradingQueue.add("grade-submission", { submissionId });
      return { async: true };
    } catch (error) {
      console.warn("Queue add failed (falling back to background memory)");
    }
  }

  // Background fallback: process without Redis
  processGradingJob(submissionId).catch((err) =>
    console.error("Background grading error:", err)
  );
  return { async: true };
}

While processing heavy tasks on the main Node event loop isn't ideal for massive scale, this fallback ensures that MOD-U-GO stays alive and student exams continue to be graded even during a total Redis outage.

4. Edge Cases & Architectural Trade-offs

Moving to an asynchronous, queue-driven pipeline solved our scale and timeout problems, but it introduced new challenges in user experience and edge-case handling.

The UX Dilemma: Immediate Feedback vs. Eventual Consistency(Trade-off)

In an ideal world, a student clicks "Submit" and sees their final grade instantly. By decoupling the grading pipeline, we had to embrace Eventual Consistency. We explicitly traded instant feedback for guaranteed system stability.

Zero Data Loss: The `pending_review` Fallback Strategy(Edge Case)

What happens when the worst-case scenario occurs? Suppose the Groq API goes down completely, and our Bull worker exhausts all 3 of its exponential backoff retries. In a naive implementation, the student might recieve score of zero . We engineer Zero Data Loss strategy. If the respone of SLM is a timeout or error , we gracefully handle it

} catch (error) {
  const isTimeout = error.code === "ECONNABORTED";

  // Gracefully fail this specific question, not the whole exam
  return {
    isCorrect: false,
    slmScore: null,
    marksAwarded: 0,
    gradingStatus: "pending_review", // Hand over to human teacher
    gradingMethod: "slm_semantic",
    error: isTimeout ? "SLM inference timeout (>30s)" : `SLM error: ${error.message}`,
  };
}

The status will be updated to pending review and alert the teacher that manual intervention is needed.

4. Future Enhancements & Scalability

Our asynchronous architecture solved our immediate bottleneck, but engineering is an iterative process. As MOD-U-GO scales to support larger institutions, it would require updates.

Queue-Level Rate Limiting

Currently, our Bull workers process jobs as fast as possible. If we scale out horizontally by adding more Node.js instances, our concurrent API calls will multiply, risking RPM limit. The next step is implementing Queue-Level Rate Limiting. By configuring Bull’s internal rate limiter (e.g., limiter: { max: 50, duration: 60000 }), which will act as a shared atomic clock and counter then limiting request overall the workers.

Horizontal Worker Auto-Scaling

Because our worker logic is completely decoupled from the main API servers, we can scale it independently. By monitoring Redis for Queue Depth (the number of pending jobs), we can configure Kubernetes (or our cloud provider) to dynamically spin up additional worker instances during peak exam periods (like midterms and finals), and scale back down to zero when the queue is empty to save compute costs.

Batch Prompting to Minimize Network Overhead

Right now, evaluating a 5-question exam requires 5 separate HTTP calls to the Groq API. This introduces significant network latency overhead. To optimize this, we can implement Batch Prompting. By aggregating all 5 text answers into a single, structured JSON prompt, we can evaluate the entire submission in a single API round-trip. This dramatically reduces network overhead and minimizes our total API request volume.

Conclusion

Building software with Third-Party AI integrations requires a fundamental mindset shift: never trust external API latencies inside your synchronous request paths.

By decoupling MOD-U-GO's exam submission endpoint from the SLM evaluation engine using Bull and Redis, we guaranteed near-zero submission latency for students, shielded our infrastructure from API rate-limit walls, and implemented graceful fallback states to ensure zero data loss—even when external services fail.

Architecting for scale isn't about preventing failures; it's about making sure your system degrades gracefully when they inevitably happen.

Thanks for reading! If you’ve built similar event-driven pipelines or have thoughts on queue optimization strategies with Node.js and Redis, I’d love to hear your feedback in the comments below!

Demystifying Redis: How I Built a Custom In-Memory Store in C++

GhostSocket — Tue, 04 Aug 2026 15:14:16 +0000

Executive Summary & Roadmap

Rebuilding a system you rely on every day is the ultimate way to demystify its internal architecture.

In this article, we break down Redis-Lite—a multi-threaded, RESP-compliant in-memory key-value store built entirely from scratch in modern C++. We trace the exact execution path of a database command: from raw TCP byte stream ingestion over POSIX sockets, through protocol parsing, to concurrent in-memory execution and file persistence.

Topics to look forward to:

Low-Level Socket Programming: How POSIX TCP sockets handle concurrent connections using a Thread-per-Connection model.
Protocol Parsing: Safely deserializing the Redis Serialization Protocol (RESP) without buffer overflows.
Concurrency & Memory Safety: Protecting std::unordered_map with C++ RAII mutex guards and comparing coarse-grained locking against single-threaded event loops (epoll).
Real-World Application: Building a BullMQ-style Distributed Priority Task Queue on top of our custom database commands (HMSET, LPUSH, RPUSH, LPOP).

Architectural Overview & System Design
The TCP Server Loop and Threading Model
Parsing the RESP Protocol
Memory Storage & Concurrency
Advanced Features: Expiration (TTL) & RDB State Persistence
Building a Distributed Task Queue
Performance Takeaways & Conclusion

1. Architectural Overview & System Design

Building an in-memory database requires carefully orchestrating network I/O, string parsing, and thread-safe data storage. Redis-Lite is designed around a straightforward, blocking architectural flow:

TCP Server Loop: The main server loop listens for incoming client connections.
Thread-per-Connection Execution: Upon accepting a connection, the server immediately hands off the socket to a newly spawned std::thread, preventing the main loop from blocking.
RESP Parser: The worker thread reads the raw network byte stream and parses it according to the Redis Serialization Protocol (RESP) to extract executable commands.
In-Memory Store: Parsed commands are routed to the database, which uses coarse-grained mutex locking to safely execute reads and writes against std::unordered_map data structures.

High-Level Architecture Flow

2. The TCP Server Loop and Threading Model

At its core, the server relies on low-level POSIX socket programming to communicate over the network. We initialize a TCP socket, bind it to a port, and begin listening. Because we aren't using high-level web frameworks, we manually manage the raw byte streams traveling across these socket file descriptors.

Handling multiple concurrent clients is the biggest challenge in a custom database. Unlike native Redis (which is single-threaded and uses an event loop), Redis-Lite uses a Thread-per-Connection concurrency model.

The server sits in an infinite loop blocking on the accept() system call. When a client connects, it spins up a detached POSIX thread via C++ std::thread to handle the entire lifecycle of that specific client socket.

Here is the exact implementation of it:

while(running) {
    int client_socket = accept(server_socket, nullptr, nullptr);
    if (client_socket < 0) {
        if (running) cerr << "Couldn't accept client connection\n";
        break;
    }

    // Spin up a new thread for this specific client connection
    threads.emplace_back([client_socket, &cmdHandler]() {
        char buffer[1024];
        while (true) {
            memset(buffer, 0, sizeof(buffer));
            // Prevent buffer overflow by restricting read size
            int bytes = recv(client_socket, buffer, sizeof(buffer) - 1, 0); 
            if (bytes <= 0) break;

            string request(buffer, bytes);
            string response = cmdHandler.processCommand(request);
            send(client_socket, response.c_str(), response.size(), 0);
        }
        close(client_socket);
    });
}

3. Parsing the RESP Protocol

Once the thread reads the raw socket bytes, it must translate the Redis Serialization Protocol (RESP). The protocol prefixes Arrays with * and Bulk Strings with $, followed by their respective lengths.

Instead of guessing where strings end, the parser uses these prefixes to perfectly extract data. Here is the core parsing logic:

size_t pos = 0;

// 1. Read the '*' to get the Array size
if(input[pos] != '*') return tokens; 
pos++;

size_t crlf = input.find("\r\n", pos);
int numElements = stoi(input.substr(pos, crlf - pos));
pos = crlf + 2; // Move past \r\n

// 2. Iterate and extract each Bulk String
for(int i = 0; i < numElements; i++) {
    if(pos >= input.size() || input[pos] != '$') break;
    pos++;

    crlf = input.find("\r\n", pos);
    int len = stoi(input.substr(pos, crlf - pos));
    pos = crlf + 2; 

    // Extract exact string safely using length 'len'
    string token = input.substr(pos, len);
    tokens.push_back(token);

    pos += len + 2; 
}

4. Memory Storage & Concurrency

Our primary storage mechanism leverages C++'s std::unordered_map to provide average O(1) time complexity for lookups and insertions. However, because our server uses a Thread-per-Connection model, multiple client threads might attempt to read or modify the hash map simultaneously.

To prevent memory corruption (race conditions), we use a global std::mutex paired with std::lock_guard.

// Storage definition: std::unordered_map<std::string, std::string> kv_Store;
// Global lock: std::mutex db_mutex;

void RedisDatabase::set(const std::string& key, const std::string& value) {
    // RAII wrapper automatically acquires lock here
    std::lock_guard<std::mutex> lock(db_mutex); 
    kv_Store[key] = value;
} // Lock is automatically released upon exiting function scope

bool RedisDatabase::get(const std::string& key, std::string& value) {
    std::lock_guard<std::mutex> lock(db_mutex);
    purgeExpired(); // Lazy expiration cleanup

    auto it = kv_Store.find(key);
    if(it != kv_Store.end()) {
        value = it->second;
        return true;
    }
    return false;
}

Preventing Race Conditions with RAII

By wrapping the mutex in std::lock_guard, we utilize the C++ RAII (Resource Acquisition Is Initialization) pattern. The lock is safely acquired when the function begins and is strictly guaranteed to be released when the scope ends—even if the function returns early or throws an exception. This ensures that only one thread can ever access the database at a time.

The Bottleneck of a Global Lock vs. Native Redis

While a single global db_mutex guarantees thread safety, it acts as a coarse-grained lock. If Thread A is reading a key, Thread B is entirely blocked from writing to a completely unrelated key. Under heavy traffic, client threads spend most of their time waiting for the lock to be released, severely bottlenecking throughput.

How production Redis solves this: Native Redis bypasses this locking problem entirely by being fundamentally single-threaded. Instead of spinning up threads for every client, it uses an asynchronous event loop (like epoll) to multiplex thousands of non-blocking sockets onto a single execution core. Because only one thread processes commands, race conditions simply do not exist, eliminating the need for slow mutexes altogether.

5. Advanced Features: Expiration (TTL) & RDB State Persistence

A production-grade database must do more than just store strings in memory. It must manage the lifecycle of volatile data (Time-To-Live) and ensure that critical data survives a server crash or reboot.

Core Data Stores Explained

Our database orchestrates four separate std::unordered_map structures to handle different workloads efficiently:

kv_Store: Maps string to string. The fastest way to handle 1-to-1 caching.
list_store: Maps string to std::vector<string>. Vectors provide incredibly cache-friendly memory locality and $O(1)$ appends (RPUSH).
hash_Store: Maps string to an inner unordered_map. This nested map allows $O(1)$ lookup for specific fields inside large objects.
expiry_map: Maps string to a std::chrono::time_point. This records the exact future timestamp when a key should be evicted.

Lazy Expiration (TTL)

To manage key expiration, we use a Lazy / On-Demand Expiration model. When a client sets an EXPIRE time, we calculate the absolute future time_point and store it in expiry_map. Instead of running a continuous background loop that scans for expired keys, we evaluate expiration synchronously during data access. Whenever a command like GET or SET is called, it first triggers purgeExpired():

//check all the expiry_amp timings with current time
void RedisDatabase::purgeExpired() {
    auto now = std::chrono::steady_clock::now();
    for (auto it = expiry_map.begin(); it != expiry_map.end(); ) {
        if (now > it->second) {
            // Remove from all stores
            kv_Store.erase(it->first);
            list_store.erase(it->first);
            hash_Store.erase(it->first);
            it = expiry_map.erase(it);
        } else {
            ++it;
        }
    }
}

Why this is efficient: We only pay the CPU cost for deletion when we actively interact with the dataset, completely eliminating the need for complex, lock-heavy background threads.

RDB-Style File Persistence

Because RAM is volatile, shutting down the server wipes all data. To solve this, we implemented a custom dump.my_rdb serialization protocol. When the server undergoes a graceful shutdown, it writes the in-memory state to a binary file on the disk.

bool RedisDatabase::dump(const std::string& filename) {
    std::lock_guard<std::mutex> lock(db_mutex);
    std::ofstream ofs(filename, std::ios::binary);

    // Serialize Key-Value Store
    for(const auto& kv : kv_Store) {
        ofs << "K " << kv.first << " " << kv.second << "\n";
    }

    // Serialize Lists
    for(const auto& kv : list_store) {
        ofs << "L " << kv.first;
        for(const auto& item : kv.second) ofs << " " << item;
        ofs << "\n";
    }

    // (Hashes are serialized similarly)
    return true; 
}

When the server boots up, the load() function reads this file line-by-line to repopulate the hash maps. This hybrid approach guarantees data survival across restarts while maintaining the blazing-fast speeds of an in-memory execution engine.

6. Building a Distributed Task Queue

To demonstrate the real-world utility of Redis-Lite, I built a Distributed Task Queue (similar to BullMQ) entirely on top of the custom Redis commands.

This system separates work into two independent processes: Producers that generate tasks, and Workers that asynchronously consume and process them.

Enqueuing and Dequeuing Logic

The task queue heavily utilizes Redis Hashes and Lists:

Producer: When a producer creates a task, it first uses HMSET to store the task's metadata (type, priority, timestamp) in a Redis Hash. Then, it pushes the taskId into a Redis List acting as a queue. For urgent tasks, it uses LPUSH (putting it at the front of the line), and for standard tasks, it uses RPUSH (putting it at the back).
Worker: The worker runs an infinite loop, constantly querying the server using LPOP. Because LPOP removes the item from the list and returns it in one atomic operation, it is impossible for two workers to accidentally grab the same task. Once popped, the worker updates the hash status to "processing", simulates the work, and finally marks it "completed".

Task Flow Architecture

Priority Queues & Worker Distribution

Priority Handling: We enforce priority natively by creating four separate list queues: critical, high, normal, and low. Inside the worker's getNextTask() method, the worker explicitly attempts to LPOP from the critical queue first. Only if critical returns null does it move on to high, and so on. This guarantees that background tasks never block urgent workloads.

7. Performance Takeaways & Conclusion

Building Redis-Lite revealed several critical performance realities about concurrent systems programming:

The Mutex Bottleneck: Despite having a multi-threaded architecture, throughput plateaus under heavy load. Because all operations share a single global db_mutex, threads constantly queue up waiting for access.A production-ready fix would be transitioning to fine-grained locking or sharding.
Context Switching Overhead: The Thread-per-Connection model is incredibly intuitive to program, but it scales poorly. If 10,000 clients connect, the OS attempts to schedule 10,000 threads. The CPU wastes massive amounts of time just context-switching between them, and many threads sit idle blocking on recv(). This perfectly illustrates why native Redis uses a single-threaded asynchronous event loop (like epoll) to multiplex connections instead.
Algorithmic Complexity Matters: I chose std::vector for lists because appending (RPUSH) is O(1) and cache-friendly. However, inserting at the front (LPUSH) forces the vector to shift all elements, degrading to O(n). Understanding these underlying C++ container mechanics is vital when building a database meant for microsecond latencies.

Final Thoughts

Rebuilding a tool you use every day is the ultimate way to demystify it. Redis-Lite successfully implements the RESP protocol, multiplexes connections, and safely stores data—but more importantly, it taught me the exact engineering hurdles that led to the architectural decisions behind the real Redis.

DEV Community: GhostSocket

Blocking Unlabelled Pods with Kyverno: From ClusterPolicy to CEL on a Local kind Cluster

1. Summary

2. Environment Setup

3. Policy v1: The Legacy ClusterPolicy

4. Policy v2: The CEL ValidatingPolicy

Breaking Down the CEL Logic

Testing the CEL Policy

5. Comparison: What Changed?

6. Takeaways & Next Steps

What I Haven't Tested

Handling AI Traffic Spikes: Decoupling Express APIs with Redis and BullMQ

Executive Summary & Roadmap

Topics to look forward to:

Table of Contents

1. The Problem: Why Synchronous LLM Grading Fails

The Mathematics of HTTP Timeouts

The API Rate Limit Wall

2. Architecture & Data Flow: The "Fire-and-Forget" Pattern

The End-to-End Submission Journey

System Architecture Diagram and Bull working

3. Queue Configuration & System Resilience

Exponential Backoff & Retry Strategies

In-Memory Graceful Fallback (Operating Without Redis)

4. Edge Cases & Architectural Trade-offs

The UX Dilemma: Immediate Feedback vs. Eventual Consistency(Trade-off)

Zero Data Loss: The pending_review Fallback Strategy(Edge Case)

4. Future Enhancements & Scalability

Queue-Level Rate Limiting

Horizontal Worker Auto-Scaling

Batch Prompting to Minimize Network Overhead

Conclusion

Demystifying Redis: How I Built a Custom In-Memory Store in C++

Executive Summary & Roadmap

Topics to look forward to:

Table of Contents

1. Architectural Overview & System Design

High-Level Architecture Flow

2. The TCP Server Loop and Threading Model

3. Parsing the RESP Protocol

4. Memory Storage & Concurrency

Preventing Race Conditions with RAII

The Bottleneck of a Global Lock vs. Native Redis

5. Advanced Features: Expiration (TTL) & RDB State Persistence

Core Data Stores Explained

Lazy Expiration (TTL)

RDB-Style File Persistence

6. Building a Distributed Task Queue

Enqueuing and Dequeuing Logic

Task Flow Architecture

Priority Queues & Worker Distribution

7. Performance Takeaways & Conclusion

Final Thoughts

Zero Data Loss: The `pending_review` Fallback Strategy(Edge Case)