DEV Community

Austin
Austin

Posted on

28K TPS Single-Node Resource Scheduling Engine [Architecture Showcase]

#dotnet #architecture #actor #ai

⚠️ Disclaimer:

This project was formerly a high-frequency routing and resource scheduling backbone carrying complex business logic. To strip sensitive business attributes and protect data privacy, the complete business source code has been physically destroyed.
This repository serves strictly as an Architecture Showcase, preserving core design philosophies, benchmarks, and de-identified "hardcore" source code snippets (e.g., Lock-free Actor Dispatcher, Augmented Interval Trees, etc.).
Benchmarks were conducted in a "noisy" development environment: Mac Studio M4 (36GB), 5+ VS Code instances, 10+ browser tabs, video playback, Docker (approx. 10 containers running), pgAdmin4, and other daily productivity tools.

💡 Genesis & Experiment: The "One-Man Army" Leverage in the AI Era

In an era dominated by distributed systems, the default reflex for high concurrency is to split microservices and introduce Redis clusters with distributed locks. While a reasonable compromise for rapid iteration, the cost is staggering: massive hardware overhead, network I/O latency, and the nightmare of debugging distributed deadlocks.

I wanted to conduct an extreme reverse exploration: What happens if we return to a monolithic architecture and squeeze local RAM and CPU to their absolute physical limits?

Furthermore, this was a stress test for AI-Native Engineering.
From initial requirement decomposition and architectural derivation to core algorithm design, database optimization, Docker deployment, and UI construction—everything was completed by a solo developer collaborating with Large Language Models (LLMs) within 3 months.

My role was to provide architectural intuition, define data structure boundaries, and handle high-performance trade-offs; the AI handled the heavy lifting of code weaving and foundational implementation. This human-machine synergy allowed a highly complex low-level engine to materialize at an incredible velocity.

🏗️ Architecture Overview

Design Principle: Discovery first, Route second, Asynchronous Persistence.

💥 Benchmark: 28,000 TPS (End-to-End)

To simulate a realistic production environment, I performed stress tests on a heavily loaded dev machine:

  • Hardware Baseline: Mac M4 (36GB RAM)
  • Interference: Host machine running full dev suites, browser clusters, and PostgreSQL within a local Docker container.
  • Results: Sustained 28,000+ TPS with P99 latency maintained at sub-millisecond levels.
  • Core: Asynchronous state self-healing and eventual consistency monitoring logic based on the Awaitable-Signal pattern.
  • Brutalist Engineering: Throughput simulation squeezing single-node multi-core capacity under 200 concurrent workers.
  • Observability: Utilizing IEventBus for low-overhead asynchronous tracking of distributed Actor state changes.

This is not a theoretical "Hello World" concurrent test. Every request penetrates the full business lifecycle:
Auth/Security Check -> Interval Tree Multi-dimensional Addressing -> Atomic Memory Quota Deduction -> FSM State Transition -> Async Micro-batch Persistence -> Result Feedback.

🧠 Design Philosophy & Trade-offs

To shave every millisecond off the hot path, the system makes aggressive, targeted compromises:

1. Eliminating Locks (Zero-Lock) & Strong-Typed Actor Model

Under high-frequency burst traffic, any Mutex leads to a context-switching catastrophe.
The system abandons shared-state concurrency in favor of a strong-typed Actor System built on .NET Channels. Entity states are encapsulated within independent Actors, and instructions enter a Mailbox for strictly serial consumption.

  • Trade-off: Sacrifices the intuitiveness of synchronous code and raises the debugging bar.
  • Benefit: Completely eliminates Data Races, allowing the CPU to focus 100% on computation.

2. O(log N) Augmented Interval Tree

Multi-dimensional matching of massive resource pools based on dynamic weights would kill any DB via table scans.
The system maintains a customized Augmented Interval Tree in memory. By utilizing a multi-dimensional weight dynamic pruning algorithm, it aggressively discards non-optimal branches early in the traversal, keeping addressing time in the sub-millisecond range.

3. Micro-Batching IO

The Iron Rule: Core logic never waits for Disk I/O.
The PersistenceCoordinator acts as a background "janitor," intercepting tens of thousands of memory mutations per second and aggregating them every few milliseconds into macro-transactions using PostgreSQL's UNNEST for bulk SQL execution. Even during transient DB jitters, the memory engine continues to operate smoothly.

4. Memory Discipline & GC Combat

Running at full throttle in C# means the GC is your primary adversary.
From message packets to queue nodes, all high-frequency lifecycle objects are pinned within an ObjectPool. Combined with Span<T> memory slicing, this minimizes Gen0 collection frequency to the absolute limit.

🛡️ Industrial-Grade Reliability: Beyond Speed (Chaos Engineering)

To ensure absolute data consistency at 28,000 TPS, I built a comprehensive Black-box/White-box integration test suite. The system has been verified against the following extremes:

  • Actor Passivation & Reactivation: Verified that resource entities can be released from memory (Passivation) during inactivity and 100% restored from snapshots upon wake-up.
  • Crash Recovery (Power-off Simulation): Simulates a system crash mid-process leaving "half-finished" transactions in the DB. Upon reboot, Actors use Deterministic IDs to identify stale tasks and self-heal.
  • Atomicity & Idempotency: Concurrent stress testing prevents "Double Allocation" and "State Oscillation," ensuring eventual consistency between memory and the persistence layer under a 200-thread onslaught.

* Geo-Aware Matching: Verified the Augmented Interval Tree in multi-dimensional spatial addressing, ensuring the system prioritizes "same-city/same-province" routing before falling back to global optima.

🚀 Business Applications & Roadmap

This lock-free in-memory foundation is naturally suited for "Heavy & Fast" battlegrounds:

  • Real-time intelligent dispatching for massive fleets/orders (Ride-hailing, Food delivery).
  • Ultra-high concurrency flash sales and global quota allocation centers.
  • Foundational abstractions for high-frequency financial trading.

Issue #1: Why abandon traditional Mutex/ReaderWriterLock for Channel-based Actors?

Labels: Architecture Performance

Description:
During early R&D, we attempted to use ReaderWriterLockSlim and ConcurrentDictionary for resource management. At 2,000+ TPS, we observed massive Context Switching and Kernel Preemption.

Technical Details:

  • Pain Point: Lock contention caused CPU Pipeline Stalls. High-frequency R/W led to False Sharing on the L3 cache, making cache-line synchronization overhead astronomical.
  • Solution: Refactored to a strong-typed Actor dispatcher based on .NET System.Threading.Channels (see ActorDispatcher.cs).
  • Logic:
    • Encapsulate each Aggregate Root in an independent single-threaded consumption loop.
    • Use Channels to achieve Ownership Transfer rather than shared memory.
  • Outcome: Throughput doubled from 8K to a stable 28K+ TPS, achieving near-linear scalability on M4 cores.

Issue #2: Dynamic Pruning Logic for Augmented Interval Trees at Scale

Labels: Algorithm Optimization

Description:
The core matching logic relies on an AugmentedIntervalTree. Standard interval tree complexity is $O(\log N + K)$, but under extreme load, simple matching isn't enough to find the "weighted optimal solution."

Deep Dive:
We added auxiliary metadata MaxSubtreeWeight to tree nodes:

  1. Logic: During depth-first traversal, if the current best score found is already greater than the MaxSubtreeWeight of a branch, the entire subtree is pruned.
  2. Snippet (De-identified): 
// Context-aware pruning in QueryRecursive
if (topNQueue.Count == topN) {
    int maxPossibleScoreInSubtree = node.MaxSubtreeScore + LocationBonus;
    if (maxPossibleScoreInSubtree <= currentWorstInQueue) return; // Pruning triggered
}
Enter fullscreen mode Exit fullscreen mode
  1. Result: This optimization reduced node visits by over 60% in a test set of 10M random weights, which is critical for maintaining sub-millisecond latency.

Issue #3: Solving PostgreSQL Transaction Bloat & Write-Amplification at 28K Mutations/sec

Labels: Persistence PostgreSQL

Description:
Allowing business Actors to write directly to the DB would kill any instance via WAL bottlenecks and Autovacuum backlogs, regardless of sharding.

Trade-off:
We engineered the PersistenceCoordinator with a Write-Amplification Suppression strategy:

  • Buffer Layer: Actor state mutations first enter a high-speed async backend buffer.
  • Macro-Transaction Aggregation: 50ms latency window to aggregate writes using PostgreSQL’s UNNEST function.

  • Core SQL Pattern:

    INSERT INTO ResourcePool (...) 
SELECT * FROM unnest(@BatchData) 
ON CONFLICT (Id) DO UPDATE SET ...

  • Effect: Tens of thousands of single-row updates are collapsed into a few bulk writes. IOPS seen by the DB dropped by two orders of magnitude, completely eliminating VACUUM backlog risks.

Issue #4: Zero Gen2 GCs: Memory Discipline in High-Frequency Scenarios

Labels: Memory Management GC-Tuning

Description:
At 28K TPS, even a small new object() will fill Gen0 instantly, triggering expensive collections.

Optimization Path:

  1. Universal Pooling: From IActorCommand packets to SAGA state machine contexts, everything is leased via ObjectPool<T>.
  2. Zero-Copy Slicing: Extensive use of Span<T> and Memory<T> when parsing binary data streams to avoid intermediate string or array allocations.
  3. Monitoring: Built-in ActorLoadMeter to monitor memory allocation slopes per batch in real-time.

Result: During a 1-hour stress test, Gen2 GC remained at 0, with Gen0 collection frequency kept in the single digits per minute.

Issue #5: AI-Native Development: How Architects Drive AI to Deliver Hardcore Middleware

Labels: AI-Engineering Productivity

Description:
This project is not just a technical experiment but a validation of Solo-Developer/AI Synergy.

Paradigm:

  • Architect's Role: I defined Actor isolation boundaries, State Transition Matrices, and fallback strategies for concurrency conflicts.
  • AI's Role: Generated tedious Dapper mappings, Postgres stored procedure conversions, and high-coverage concurrent unit tests.
  • Insights:
    • The most successful collaboration was AI assisting in creating edge-case deadlock tests for the StripedSemaphore.

    * One Man, Three Months, 100x Efficiency: This mode allowed me to escape the "boilerplate swamp" and focus on "architectural beauty" and "extreme tuning."

    🧠 Core Insights: Non-Standard Thinking on Performance

#1 Regarding Memory Barriers

In the MatchEngine, I didn't get bogged down in the "lock-free algorithm" trap. I eliminated competition at the source through the Actor Isolation Model. The philosophy is: Do not communicate by sharing memory; instead, share memory by communicating. Protecting the CPU pipeline from lock oscillation is more effective than writing 100 Interlocked calls.

#2 Regarding Postgres VACUUM & High-Freq Writes

28K QPS hitting a DB directly will kill it via WAL limits. Through the PersistenceCoordinator, I implemented Write-Amplification Suppression. Postgres sees "ordered bulk writes" rather than a "scatter gun" of single updates. In this architecture, VACUUM is just a walk in the park.

🔬 Roadmap: The AI-Native Co-Pilot Kernel

The 28K TPS foundation solves "execution efficiency." The next phase—The Private AI Intelligent Brain—aims to solve the "creativity bottleneck."

The goal isn't just automation; it's deep AI intervention in the software lifecycle to free the architect from grunt work, achieving a "One Man as a Hundred" efficiency lever.

1. The AI-Native Stack

  • Context Layer: Utilizing the MCP (Model Context Protocol) to break barriers between IDEs, codebases, DBs, and LLMs, giving AI a "God's eye view" of system state.
  • Knowledge Layer: Establishing Graph-RAG. More than just document retrieval—it's about deep parsing of architectural topology to ensure AI understands the global logic.
  • Action Layer: Encapsulating Atomic Skills. Tools designed for low-level optimization (e.g., auto-memory allocation, lock-free logic rewriting) so AI can output "production-grade hardcore code."
  • Orchestration Layer: Multi-Agent Cross-Domain Synergy.
    • Architect Agent: Assists in boundary definition and trade-off derivation.
    • Coder Agent: High-precision code weaving.
    • Guard Agent: Automated unit testing, stress test generation, and 24/7 quality audits.

2. Evolution: From "Developer" to "Commander"

  • From Complexity to Precision: All boilerplate and grunt work (琐碎的实现, config, bug fixing) is offloaded to the AI core. I focus 90% of my energy on architectural evolution and design beauty.
  • Autonomous Decision Making: The Intelligent Brain won't just be an assistant; it will be a digital twin making decisions based on my "design philosophy," choosing optimal algorithms and providing quantitative trade-offs for my final approval.

🤝 Let's Connect

I am a senior developer passionate about "architectural extremes" and "low-level tuning." I am sharing this architectural concept to step beyond the confines of daily business and connect with the broader tech ecosystem.

If you are a:

  • CTO / Tech Executive: Facing severe performance bottlenecks and needing a "Lead Surgeon" or Architect who understands the low-level and AI empowerment.
  • Senior Tech Recruiter: Looking for infrastructure experts with a high-level vision for Tier-1 firms or star unicorns.
  • Geek: A fellow traveler with an obsession for the Actor model and squeezing out every cycle of performance.

I look forward to an online deep dive or a coffee to discuss the art of architecture and potential collaborations.

Talk is cheap. Show me the benchmark. ⚡️

Top comments (0)