Introduction
In the previous article, we explored Brighter’s messaging pump architecture, including the Reactor and Proactor patterns. This article focuses on how to configure the number of messaging pump instances and explains Brighter’s design decision to use dedicated threads for message processing, even in asynchronous scenarios.
Messaging Pump Recap
Brighter’s messaging pump abstracts message consumption from the underlying transport (e.g., RabbitMQ, Kafka). Two key implementations exist:
- Reactor Pattern: Synchronous message processing using a single thread per pump.
- Proactor Pattern: Asynchronous message processing via async/await, leveraging the SynchronizationContext to preserve thread affinity.
Configuring the Number of Messaging Pumps
By default, Brighter configures one messaging pump instance per subscription. To scale horizontally, adjust the noOfPerformers
parameter:
new <Provider>Subscription<SomeRequest>(
new SubscriptionName("some-name"),
new ChannelName("some-queue"),
new RoutingKey("some-topic"),
noOfPerformers: 16 // Set to match your concurrency needs
)
Messaging Pump Execution Model
Brighter uses a preemptive multitasking approach:
- Each messaging pump runs on its own dedicated thread.
- Threads are explicitly managed to avoid contention and ensure predictable performance.
Why Not Use Task.Run or Task.Factory.StartNew?
Brighter avoids Task.Run
for long-running operations because:
-
ThreadPool Overhead: Long-running Tasks can exhaust the
ThreadPool
, leading to thread pool starvation. - Deterministic Control: Threads provide finer-grained control over CPU affinity and priority compared to Tasks.
- Consistency: Matches the Reactor pattern’s thread-per-pump model, ensuring alignment with synchronous and asynchronous workflows.
As noted in the ADR for async pipeline support, Brighter's design prioritizes explicit thread management to avoid issues with async/await in pooled environments.
Async Context and Thread Affinity
For the Proactor
pattern, Brighter uses a custom SynchronizationContext
to ensure:
- Thread Affinity: Maintains the same thread before and after await to avoid context switches.
- Predictable Execution: Prevents race conditions by isolating async pipelines to their own threads.
This approach resolves the tension between the Proactor
pattern’s non-blocking I/O and the need for thread-local state management in distributed systems.
Key Design Rationale
- Thread Safety: Dedicated threads eliminate shared state conflicts during message processing.
-
Scalability:
noOfPerformers
allows horizontal scaling without altering the application’s threading model. -
Async Reliability: The custom
SynchronizationContext
ensures async pipelines behave deterministically, avoiding issues like thread pool starvation.
Conclusion
Configuring the number of messaging pumps in Brighter balances scalability and resource management. By dedicating threads to each pump:
- Reactor ensures predictable, ordered processing.
- Proactor enables high-throughput async workflows while preserving thread affinity.
This design, rooted in the principles of preemptive multitasking and explicit thread management, ensures reliability in both synchronous and asynchronous pipelines. For advanced async scenarios, refer to the ADR on async pipeline support.
Top comments (0)