We are told that to scale Ruby background workloads across multiple CPU cores, we have two choices: process forking (e.g., Pitchfork) which incurs massive Copy-on-Write memory overhead and risk of OOM kills under load, or database-backed queues (e.g., Solid Queue) which introduce heavy synchronous disk I/O and row-locking bottlenecks [1.8, 1.10].
To break this cycle, I built SolidJobs (v0.5.0): an open-source, Ractor-local Redis task runtime engineered from scratch around a strict shared-nothing architecture [1.8].
- The Architecture: Warm IPC Pool over Unix Sockets
Instead of forcing workers to share mutable application states, SolidJobs completely isolates concurrency multiplexing at the OS kernel level [1.8, 1.10].
• Local Unix Sockets (0o600): Communication transport overhead is compressed to ~40 µs [1.8]. The Linux kernel handles the backlog natively [1.8].
• solid-resp-ractor: A zero-dependency, RESP3-compliant stream encoder/decoder designed to be deeply frozen (Ractor.make_shareable). It processes binary payloads inside isolated Ractors without ever triggering a Ruby IsolationError [1.8].
- Certified Benchmarks (Sustained Workload)
Under heavy execution testing (mixed CPU + I/O load), this model completely bypasses GVL contention [1.8]:
• Throughput: Reaches up to 3,073 jobs/s (a 5.4× raw performance improvement over thread-bound Sidekiq clusters) [1.8].
• Latency: Atomic Redis reservations via custom Lua scripts execute with a p50 of 9 µs and a p99 of 39 µs.
• Memory Efficiency: Object allocations drop by 36%, dramatically decreasing Garbage Collector frequency and eliminating memory bloat [1.8].
• Backward Compatibility: It runs out-of-the-box on Ruby 3.2+, allowing teams to exploit true multi-core parallel execution without upgrading their entire production stack [1.10].
- The Multi-Core Telemetry Blind Spot & SolidTrace
Once your workers are locked inside strict, isolated Ractors, standard APM instrumentation (Datadog, New Relic, standard Rails loggers) shatters [1.10]. Forcing workers to push telemetry to a shared mutable heap either triggers isolation errors or introduces massive locks that kill parallel performance [1.10].
To bridge this telemetry void, I developed SolidTrace [1.8]. It utilizes an out-of-band, low-overhead structured event journaling system running locally inside each Ractor worker [1.8, 1.10]. It maps per-core object allocations and proactively tracks queue saturation before the kernel fires an OOM kill [1.8, 1.10].
The core runtime engine is 100% open-source (LGPL-3.0), and because multi-core clusters shouldn't run blind, I am opening a 1-month free trial sandbox for the SolidTrace commercial dashboard [1.8, 1.10].
I’d love to get your feedback on the IPC architecture, the RESP3 parser implementation, and how you approach multi-core telemetry in shared-nothing runtimes [1.8].
• GitHub: https://github.com/nicolasva/solid-jobs
• RubyGems: https://rubygems.org/gems/solid-jobs
Top comments (0)