HexCore: Zero-Trust Paged KV Cache Allocator

Rasuljanov Muhammadali — Thu, 30 Jul 2026 16:43:07 +0000

Efficient memory management is one of the biggest bottlenecks when scaling Large Language Model (LLM) inference runtimes. Memory fragmentation and unpredictable allocation latencies can severely degrade token generation performance.

To tackle this problem, I built HexCore v2.2.0 — a zero-trust, ultra-low latency Paged KV Cache Allocator written in C++20 with CUDA kernels.

🔑 Key Features

Paged KV Cache Allocator: Minimizes latency and memory fragmentation during autoregressive decoding.
CUDA Accelerated Kernels: Custom GPU implementations for Paged Attention and quantized memory layouts.
Zero-Trust Memory Safety: Fully guarded boundary checking verified with AddressSanitizer (ASan) and UndefinedBehaviorSanitizer (UBSan).
Python/PyTorch Bindings: High-performance C++20 bindings for seamless integration into modern AI runtimes.

🛠 Tech Stack

Core: C++20, CUDA
Integration: Python, PyTorch (pybind11)
Build System & CI: CMake, GitHub Actions CI/CD

🚀 Open Source & CI/CD Pipeline

The repository includes complete GitHub Actions CI/CD workflows testing host bindings, sanitizer safety, and build integrity across multiple platforms.

Check out the full source code and documentation on GitHub:

👉 GitHub Repository: password162156/hexcore-llm

I'd love to hear feedback, thoughts, or contributions from the C++ and AI systems engineering community! ⭐

Beyond open(): Designing a Crash-Safe, Atomic Hot-Patching Engine in Python

Rasuljanov Muhammadali — Sun, 21 Jun 2026 12:56:58 +0000

Why standard file operations are a liability in production
In high-availability systems, file I/O is the "hidden killer." If a server process updates a configuration file and crashes midway, you're left with a corrupted state—or worse, a silent failure. I analyzed this bottleneck and built Shadow Kernel, an atomic hot-patching engine designed to handle critical data integrity with zero third-party dependencies.

The Engineering Challenge
Traditional open('w') operations are non-atomic. A sudden power loss or process kill leaves the file in an indeterminate state. My goal was to create a mechanism that guarantees:

Atomicity: The file is either fully updated or left completely untouched.

Exclusivity: Preventing concurrent race conditions during the patch cycle.

Syntax Validation: Ensuring no malformed code ever hits the disk.

The Architecture
Shadow Kernel doesn't just write to a file; it executes a four-phase safety protocol:

Phase 1: Pre-flight AST Validation. Before any I/O occurs, I leverage Python's ast.parse() to validate the syntax. If the code is broken, the engine aborts immediately, protecting the production state.

Phase 2: Dual-Layer Locking. I implemented a two-tier locking system: threading.Lock for in-process safety, combined with fcntl.flock (exclusive, non-blocking) to prevent race conditions across different system processes.

Phase 3: The Atomic Commit. Data is written to a temporary file (.tmp). Using fdatasync, I flush the kernel buffers directly to physical media. Finally, a POSIX os.replace operation—which is an atomic system-level swap—replaces the target file.

Phase 4: Integrity Verification. A pre- and post-patch SHA-256 hash comparison ensures that the bytes written match the intended source. If there is a mismatch, the engine triggers an automatic rollback.

Why this matters
This isn't just about updating source code; it's about building resilient systems that can self-heal. By relying solely on the Python standard library, I've created a portable, high-integrity solution that can be dropped into any environment without adding dependency bloat.

Closing Thoughts
Engineering is the art of making software predictable in the face of inevitable hardware and system failures. Shadow Kernel is my attempt to bring the robustness of kernel-level file handling to Python application logic.