The local models behind my AI reports had been running with the wrong sampling settings for weeks, and I also closed out last week's graphics-card incident.
This is the English version of a post originally written in Korean for my algorithmic trading system devlog(new tab).
"Wait, did we ever actually discuss that setting?"
Today started with revisiting the sampling settings (the parameters that control how random a model's output is) for the local models behind my AI reporting pipeline.
The trigger was a simple question I asked myself: had this setting ever actually been discussed and deliberately chosen?
I asked an AI to trace through the code and check, and the answer came back close to "not really."
A few weeks earlier I'd run a comparison experiment that changed this setting. It turned out the client code relaying that value was silently dropping it partway through.
So instead of comparing "changed" against "unchanged," I'd effectively compared the same setting against itself. The experiment's result was invalid.
Digging further turned up something more fundamental. The main production model is deliberately run with its "deep thinking" mode turned off — but the sampling values actually in effect were the defaults meant for when that mode is on.
The mode was off, but the settings behind it assumed it was on. Nobody had changed this on purpose; whenever the value wasn't explicitly set, it quietly fell back to whatever default was bundled with the model file.
Finding the right values and fixing it
I asked an AI to look up the recommended settings for the mode I'm actually running.
It turned out the model vendor publishes separate official recommendations for "thinking mode off" versus "on," and my setup had been running with the wrong one all along.
I applied the correct values to the main production model. At the same time, I applied the same fix to a separate observation-only model that runs during market hours purely to watch performance — it has no say in actual trades.
I verified the fix both with unit tests and by checking the actual runtime logs, then restarted every resident process to apply it.
While I was at it, I also revisited whether keeping "thinking mode" off was still the right call. The conclusion held: it stays off.
I also flagged the earlier invalidated experiment in my records, so its result doesn't get cited as evidence for anything else going forward.
Also wrapped up last week's graphics-card incident
I closed out the remaining follow-up work from last week's graphics-card recognition failures today too.
First, I added a diagnostic step that flags permission-related causes directly in the failure message, so this kind of issue is easier to spot at a glance next time.
I also moved the time the OS installs automatic updates to a window outside trading hours. The auto-update process wasn't the actual cause of last week's incident, but shifting it removes even the small chance of it colliding with anything in the future.
Lastly, I found and cleaned up one of my own incident notes from a previous write-up that turned out to be a broken link — the file it pointed to no longer existed.
What's next
The new sampling settings are still in an early observation phase. I'll keep an eye on how they affect report quality tomorrow morning and over the weekend.
Top comments (0)