Quick Disclosure: I want to disclose that I work for the company that built this box. I'm going to do my best to give an unbiased opinion here: the good, the bad, and the ugly.
It Begins
My first hands-on experience with the QuietBox 2 was pretty cool. It arrived later in the day, so I didn't have to wait until I got off work to start playing with it.
The first thing I noticed was the heft of the box itself. This felt a lot more like server equipment than a simple PC tower. Upon opening it, I was greeted by an obviously custom case with two gigantic fans and some pretty cool cosmetic touches. The box itself isn't obtrusive or gaudy. It just looks like it would fit neatly on someone's desk, and it would be just as comfortable pretending to be a gaming rig as it would be the sleeper AI Ferrari that it actually is. The liquid cooling channels on the front were a nice touch. Even before I booted it up, it looked pretty amazing.
Diving a little deeper, the box came with a speaker and some decals, which I set aside for later. I can totally see the intention behind this computer. It's clearly meant to be a household-appliance-style AI server.
More Than Just an Accelerator
The box itself is an incredibly powerful unit, and I don't just mean as an inference server. As a usable desktop workstation, or even a medium to heavy load server, it's quite capable. 256 GB of RAM, a high-end consumer CPU, and enough storage for pretty much any task make it clear that this box isn't the equivalent of just buying a GPU accelerator. It's more than that. It's trying to be a full solution.
As someone who has been building out a home lab for years, this is intriguing, because it means I can treat the box as a truly self-contained appliance. Something like a home assistant could theoretically be built out exclusively on this box, with all the different parts and complex pieces running locally instead of spread out across boxes on a network.
The liquid cooling and the big radiator fans make this an almost illegitimately quiet box for a config with zero hardware trade-offs. If I were building a system like this myself, I'd have to spend quite a lot of money to get anything remotely close to this noise profile at this power level. That said, if you really wanted to, I'm sure you could pull it off.
The Accelerator Cards
Let's talk about the accelerator cards for a second, because these are unique to this system. When you first boot up and look at what's attached, the cards show up as four, even though the box itself only has two physical cards installed. This comes down to the dual-chip design of each P300C card. The end result is something close to four P150A cards' worth of compute in a single box, while only taking up two PCIe slots.
There are a few reasons for this. Anyone who has tried to design a four-card system with full-size, dual-width, high-power-draw cards can tell you it presents a unique challenge. You're pretty much off consumer hardware from the get-go and onto server-grade EPYC or Threadripper CPUs before you even start, and the problem only gets harder from there. That's why these two dual-chip cards make sense in this build. They let the price stay well below what a true four-card system would cost, while still providing the power and bandwidth to handle what's actually running.
The two cards are linked together in what's called a 1x4 fabric configuration. That means each chip functions like an individual card on its own, but they're networked together into a mesh so the resources get pooled when needed. The end result is 128 GB of usable memory (GDDR6) across the accelerator setup, which lets you load models much bigger than you'd manage on a single GPU.
Built Like a Tank
Let me change gears for a moment and talk about the case. This thing feels tough, and I don't mean tough like rigid plastic. It feels like it was built out of armor. It weighs a little over 50 lb, and moving it by the carry handles feels more like carrying an upright server than a desktop tower. You won't find any flimsy aluminum paneling here, just raw, no-nonsense, compact form factor power, contained in a chassis that feels tough enough to shield you from flying debris in a natural disaster.
The case is pretty cool beyond that too. It uses a modified clamshell design where both side panels open to reveal what's essentially a bifurcated physical layout. Each of the P300C cards sits on either side of the motherboard, upright. Both are monsters, coming in at about 7 lb each and taking up three slots, even with the more compact liquid cooling. When the box really gets going, it gives off a ton of heat, but again, nothing a high-end gaming rig wouldn't already do.
Another cool feature is the liquid reservoir built into the front of the case. You can actually see the fluid moving through it while the computer is running. This is definitely one of the more interesting liquid cooling solutions I've seen, and it gives the whole thing a unique vibe.
A Week In
In the week since I started using it, I've been pleasantly surprised. It's quickly become a workhorse in my home lab. I've thrown a lot at it and haven't come close to taxing it fully. I've got a number of projects that I'll hopefully be able to blog about soon, but until then, you'll have to make do with this teaser.
So What Is This Thing, Really?
I'd like to point out what this box isn't. It very clearly isn't what you should buy if you want a scalable system. There's no path to link these together like there would be with Tenstorrent's Galaxy system, and if you only need one or two cards, buying them separately and building your own box will be cheaper.
This box doesn't really feel like it's trying to compete with either of those scenarios though. It's something unique, more like an appliance that can handle everything, and it wouldn't surprise me if this type of system eventually becomes as common in a house as a refrigerator, washer, or dryer.
Just the brain of the house. It's what you get if you don't want to pay a subscription to a major LLM provider, and beyond that, it can run all your home automation and anything else you need. Various privacy movements, like the de-Googleify movement, would make great use of hardware like this.
Now for the Bad (and the Ugly)
The software stack's maturity definitely isn't where GPUs have been for a while now. That means while the box can run very large models, it also has some limitations around bottlenecks, and the kernels just aren't as refined as what you'd get on a lot of GPUs. Throughput isn't always quite where you'd want it to be.
That said, concurrency is this box's strong suit, and it can outpace any other setup I've had the privilege of testing, many times over. Running a single model with 30 open agents, all batching their work against the accelerator cards, is the kind of power you need to run a bunch of different systems on a single box. That, more than anything, is where this box shines.
Set your model up and go to town. This is a powerhouse and a workhorse. It's not trying to be anything else, and it doesn't need to be.
Final Verdict
So where does the QuietBox 2 actually land? If you need something that scales out into a real cluster, this isn't your box, and it was never trying to be. What it is, is one of the more interesting all-in-one AI appliances I've gotten my hands on. It's quiet, it's built like a tank, and it has more than enough headroom to run as a genuine home brain instead of just another box tucked in a rack.
The software stack still shows its sharp edges in places, and that's worth going in with eyes open. But the concurrency numbers make up for a lot, and this ecosystem is moving fast enough that today's rough edges probably won't stick around for long.
Between this and the P150A cluster I've been building out, I've now got two very different answers to the same question: how do you put real AI compute in your house without handing your data, or your wallet, over to someone else every month. One approach is fully DIY. The other shows up ready to go out of the box. I'll keep living with both and report back on how they stack up.
That's a wrap on my first week with the QuietBox 2. Thanks for reading, and stay tuned.
Top comments (0)