Have you ever gotten tired of vibe coding only serious stuff? Yeah, me too. So based on my previous work with perf vibes, I decided to build an entire Doom environment just to look at Linux telemetry.
I also had Claude cut me a release trailer, which is viewable here
Read on to join me for this wild ride of vibing and discovery!
Check back later for a video of me demo-ing the actual playground with use-practice and investigating a random perf issue.
Quick digression on the new world
I can't believe that this project actually happened, and it wouldn't have just been merely very difficult before the age of AI...it would have been completely and without a doubt impossible.
I mean, mindbogglingly out of reach, not even in the realm of possibility; utterly unfathomable even.
To think that just via prompting I could get a typescript and DOOM wasm project to completely regenerate a custom map with new elements and sprites that represent system telemetry from a live Linux system. It's something that there's 100% zero chance I could have done on my own.
Not only would it have been completely out of my reach technically (8% of the project is just raw C, a language I do not use, nor do I have any reason to learn), it wouldn't have even crossed my mind to attempt such a thing.
But now, it's possible to just prompt it into existence.
I will caveat that this was never intended to be anything close to a project that runs in a production system, and so it was much more vibe-friendly than it otherwise might have been
A bit of background
You may have already read my other article (linked above) on the two tools I created to help teach myself systems performance a little better. While those were fun ways to re-imagine the experience of investigating performance issues (in particular, via Brendan Gregg's USE methodology), it was entirely via the command line.
Now don't get me wrong, I think a healthy appreciation and interest in the command line will serve you well in any event, but I wanted something that could present the concepts in a slightly different way...something more...visual.
My main intention was that instead of needing to parse the words and numbers in a terminal into a conceptual framework of what's happening inside a cpu, it might be easier to see it.
Since even if you DO run a command like vm stat 1 5 in the terminal and see something like:
$ vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
3 0 2096956 1131988 185976 5249136 0 0 9 51 1 0 6 2 92 0 0
0 0 2096956 1140632 186008 5249876 0 0 0 496 5984 8913 8 3 89 0 0
0 0 2096956 1134680 186092 5249572 0 0 0 744 7536 12975 13 4 82 1 0
3 0 2096956 1134964 186112 5249684 0 0 0 308 6881 10712 10 3 87 0 0
0 0 2096956 1136128 186136 5249892 0 0 0 484 4982 5922 7 3 90 0 0
...and furthermore, even if you know that 3 under the r column means three threads on the run queue...is that good or bad?
Similarly, if you ran uptime and saw load average: 2.00, 2.22, 3.57...is that good or bad?
You would have to know how many logical CPUs are on your system, and then compare that to the number you got to identify not only whether you're seeing any current or historical saturation, but even get a sense of how much.
Compare that to an image representing the uptime command from Doom-Perf, and it's much more obvious at a glance:
A browser game is born
So naturally, I wanted to see these visualizations in Doom...but I had a problem.
How do I run Doom in the browser? A bit of google, and I found the wonderful doom-typescript project (for which I opened the one and only PR, ha), and immediately set to work chatting to my friend Claude about it.
Turns out, we can rip out all the stuff we don't want, and put in all the stuff we do want.
...well, not ALL the stuff.
Because of the particular way that Doom is rendered (a limitation of the WAD format and how the engine renders), we're a bit constrained on the shapes and textures and yadda yadda (I've already got another idea in the works to revamp this, but in a much more modern game engine like godot or something).
But the point is that I can get a game running in the browser (awesome, webtech is hashtag delicious!), AND have that browser served from a domain that also has a Server-Sent-Events endpoint with access to system telemetry (in Golang, naturally).
It took about 4 months of sporadic evening prompting, but we got there in the end.
Stuff I learned about Linux
ZOMG!!1 Linux queues are everywhere. One of the reasons it took me so long to finish is that I was trying to faithfully represent the 3 queues in the networking stack between the application and the actual wire (3 for each of TX/RX, so actually 6, and they're not exactly the same each way).
But yes, it was great to build this alongside reading Brendan Gregg's excellent Systems Performance book.
There are also the disk device IO queues as well as IO scheduler queues (kind of like the CPU run queue), the inactive page list for memory (which is kinda like a queue), identifying pages for eviction from memory...oh, and caches are also basically at every level of a lot of these sub-systems (though those didn't feature as heavily in the representations, since I was mostly looking at utilization and saturation, which tend to involve queues more).
I also had the complete wrong idea about page faults, which are literally always happening (the minor ones anyway), and even the major ones are very common and not a super big deal (though do indicate a bit of memory saturation). I was expecting those to not ever be happening except in a dire emergency, but I guess they happened like 10,000 times while I was typing this sentence.
The "So What"
While interesting in its own right, the real fun part is when we mix this game with the use-practice tool I made to create real system load in just such a scenario as this.
Try it out at https://labs.iximiuz.com/playgrounds/doom-perf-c0bd32e1, and follow the instructions to generate a random load and see it in Doom!
Source: https://github.com/lpmi-13/doom-perf
PS - There's an Easter Egg homage to Brendan Gregg hidden in one of the wings...
Top comments (0)