<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: MilkyWay008</title>
    <description>The latest articles on DEV Community by MilkyWay008 (@milkyway008).</description>
    <link>https://dev.to/milkyway008</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4075791%2F36d12367-22b5-4e63-b6b6-ee85957624b5.jpg</url>
      <title>DEV Community: MilkyWay008</title>
      <link>https://dev.to/milkyway008</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/milkyway008"/>
    <language>en</language>
    <item>
      <title>PowerShell autocompletes on every keystroke: it's PSReadLine predictive IntelliSense</title>
      <dc:creator>MilkyWay008</dc:creator>
      <pubDate>Tue, 13 Oct 2026 03:23:42 +0000</pubDate>
      <link>https://dev.to/milkyway008/powershell-autocompletes-on-every-keystroke-its-psreadline-predictive-intellisense-2nab</link>
      <guid>https://dev.to/milkyway008/powershell-autocompletes-on-every-keystroke-its-psreadline-predictive-intellisense-2nab</guid>
      <description>&lt;p&gt;Somebody on r/powershell described it like this: they built a Win32 Intune package, and right after that their PowerShell window started completing things on every keystroke. &lt;code&gt;clear&lt;/code&gt; bought them a few seconds of calm, then it came straight back. Only closing the window and opening a fresh one actually held.&lt;/p&gt;

&lt;p&gt;I've run into this before. If I remember correctly, the package is almost never the culprit. It's PSReadLine......&lt;/p&gt;

&lt;h2&gt;
  
  
  What PSReadLine actually is
&lt;/h2&gt;

&lt;p&gt;PSReadLine is the module that owns your interactive line editor. It isn't the shell and it isn't your terminal app. Every time you type at a &lt;code&gt;PS&amp;gt;&lt;/code&gt; prompt, PSReadLine is what draws the line and decides what happens when you press Tab. It ships with Windows PowerShell 5.1 and PowerShell 7+, so it's always there whether you installed it or not.&lt;/p&gt;

&lt;p&gt;Since PSReadLine 2.2.6, one of its features is on by default: predictive IntelliSense. That's the thing drawing ghost text ahead of your cursor and guessing your next command from history or from a plugin.&lt;/p&gt;

&lt;p&gt;Version 2.1.0 introduced the feature in 2020, but with &lt;code&gt;PredictionSource&lt;/code&gt; set to &lt;code&gt;None&lt;/code&gt;. 2.2.6 flipped the default on for any terminal that supports virtual terminal sequences, which is roughly every terminal you'd be using today.&lt;/p&gt;

&lt;p&gt;With &lt;code&gt;PredictionViewStyle&lt;/code&gt; left at its default of &lt;code&gt;InlineView&lt;/code&gt;, the guess gets drawn into the line you're typing, ahead of the cursor. Nothing is accepted, no Tab is pressed, but the line keeps redrawing every time the guess changes. That's the "it's typing for me" feeling. Switch to &lt;code&gt;ListView&lt;/code&gt; and you also get a drop-down grid under the line, which is what makes some setups unusable.&lt;/p&gt;

&lt;h2&gt;
  
  
  Look before you change anything
&lt;/h2&gt;

&lt;p&gt;Run these two and read three things: the version, &lt;code&gt;PredictionSource&lt;/code&gt;, and &lt;code&gt;PredictionViewStyle&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Get-PSReadLineOption&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Select-Object&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;PredictionSource&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;PredictionViewStyle&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="n"&gt;Get-Module&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;PSReadLine&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Select-Object&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Version&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Path&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If &lt;code&gt;PredictionSource&lt;/code&gt; says &lt;code&gt;History&lt;/code&gt; or &lt;code&gt;Plugin&lt;/code&gt; and you never asked for predictions, you found it. Check the version too. Windows PowerShell 5.1 historically shipped PSReadLine 2.0.0, and a profile that imports a different copy can give you behaviour the docs don't describe. Worth listing what's installed side by side:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Get-Module&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;PSReadLine&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-ListAvailable&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Select-Object&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Version&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Path&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Turning it off
&lt;/h2&gt;

&lt;p&gt;To stop it outright:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Set-PSReadLineOption&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-PredictionSource&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;None&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If you like history hints and just don't want the ghost text moving around on you, keep predictions and force the inline view:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Set-PSReadLineOption&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-PredictionSource&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;History&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="n"&gt;Set-PSReadLineOption&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-PredictionViewStyle&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;InlineView&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Either way that only applies to the session you ran it in. Microsoft is explicit about it: to persist any of these options, they have to go in a profile script.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;notepad&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="bp"&gt;$PROFILE&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="c"&gt;# add the line you want, e.g.&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="c"&gt;# Set-PSReadLineOption -PredictionSource None&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If &lt;code&gt;$PROFILE&lt;/code&gt; doesn't exist yet, create it first with &lt;code&gt;New-Item -Path $PROFILE -ItemType File -Force&lt;/code&gt;. Otherwise you'll edit an empty nothing and wonder why nothing changed. Been there.&lt;/p&gt;

&lt;h2&gt;
  
  
  The second cause, which looks identical
&lt;/h2&gt;

&lt;p&gt;There's an uglier variant with the same symptoms. PSReadLine caches where your line starts in the console buffer, along with the starting column. When the terminal gets resized, or the buffer is narrowed and then widened again, that anchor isn't always recalculated. From then on, every keystroke re-renders in the wrong column. That's the chaotic version where the line jumps around and you can't type at all.&lt;/p&gt;

&lt;p&gt;Upstream PR #5190 spells it out: the edit anchor is never restored for the rest of that &lt;code&gt;ReadLine&lt;/code&gt; call.&lt;/p&gt;

&lt;p&gt;This is also why &lt;code&gt;clear&lt;/code&gt; feels like a fix. &lt;code&gt;Clear-Host&lt;/code&gt; wipes the visible display and leaves your session state alone, so the badly aligned drawing disappears. The bad anchor doesn't. Next keystroke, back to the mess.&lt;/p&gt;

&lt;p&gt;For this class, clear the display, or resize the window once and press Enter for a fresh prompt, then update the module:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Update-Module&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;PSReadLine&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Force&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You'll want a new shell after that.&lt;/p&gt;

&lt;h2&gt;
  
  
  Two isolation checks
&lt;/h2&gt;

&lt;p&gt;If none of the above changes anything, PSReadLine probably isn't the problem, and something else is setting it. Start a shell without your profile:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;pwsh&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-NoProfile&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If typing is normal there, a profile script or another module is doing it. From there it's also worth trying the same session in Windows Terminal and in the old conhost window, since host-specific rendering bugs exist and they don't behave the same in both.&lt;/p&gt;

&lt;h2&gt;
  
  
  The honest caveats
&lt;/h2&gt;

&lt;p&gt;The thread I pulled this from had zero replies and no accepted answer, so I can't tell you which of the two mechanisms hit that person. The Intune theory in the post doesn't hold up either. &lt;code&gt;IntuneWinAppUtil.exe&lt;/code&gt; packs a folder into an &lt;code&gt;.intunewin&lt;/code&gt; container. It doesn't read or write PSReadLine state, &lt;code&gt;$PROFILE&lt;/code&gt;, or &lt;code&gt;$PSModulePath&lt;/code&gt;. It could only matter indirectly, if a packaging script itself ran &lt;code&gt;Set-PSReadLineOption&lt;/code&gt; or wrote a profile, and there's no evidence of that in the thread.&lt;/p&gt;

&lt;p&gt;Treat the Intune angle as unproven. Treat "PSReadLine is drawing ghost text, or drawing in the wrong column" as the thing to check first. Both are documented upstream and both are fixable in about a minute.&lt;/p&gt;

&lt;p&gt;Worth a try. Hope it helps.&lt;/p&gt;

</description>
      <category>tutorial</category>
      <category>productivity</category>
      <category>devops</category>
    </item>
    <item>
      <title>0xC0000005 on Windows: how to find which module crashed</title>
      <dc:creator>MilkyWay008</dc:creator>
      <pubDate>Thu, 01 Oct 2026 17:16:52 +0000</pubDate>
      <link>https://dev.to/milkyway008/0xc0000005-on-windows-how-to-find-which-module-crashed-3gl8</link>
      <guid>https://dev.to/milkyway008/0xc0000005-on-windows-how-to-find-which-module-crashed-3gl8</guid>
      <description>&lt;p&gt;Some app on your Windows box crashes. You search the crash code and get a wall of confident answers: update your graphics driver, run sfc /scannow, turn off XMP, reinstall Windows.&lt;/p&gt;

&lt;p&gt;Most of that is a guess wearing a lab coat.&lt;/p&gt;

&lt;p&gt;0xC0000005 is STATUS_ACCESS_VIOLATION. Windows defines it as the thread trying to read from or write to a virtual address for which it does not have the appropriate access. If your tool reports decimals instead of hex, the same code shows up as 3221225477. It's the most common crash code on Windows and the least useful one on its own, because almost anything can produce it.&lt;/p&gt;

&lt;p&gt;Windows already wrote down what faulted. You just have to read the evidence in the right order.&lt;/p&gt;

&lt;h2&gt;
  
  
  Event 1000 names the module
&lt;/h2&gt;

&lt;p&gt;Event Viewer, Windows Logs, Application, source &lt;code&gt;Application Error&lt;/code&gt;, event ID 1000. I'd start there every time. If you'd rather not click through the UI:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Get-WinEvent&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-FilterHashtable&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;@{&lt;/span&gt;&lt;span class="nx"&gt;LogName&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s1"&gt;'Application'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;&lt;span class="nx"&gt;ProviderName&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s1"&gt;'Application Error'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;&lt;span class="nx"&gt;Id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-MaxEvents&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;20&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="n"&gt;Select-Object&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;TimeCreated&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;@{&lt;/span&gt;&lt;span class="nx"&gt;N&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s1"&gt;'Exception'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;&lt;span class="nx"&gt;E&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="bp"&gt;$_&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Properties&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;Value&lt;/span&gt;&lt;span class="p"&gt;}},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;@{&lt;/span&gt;&lt;span class="nx"&gt;N&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s1"&gt;'Module'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;&lt;span class="nx"&gt;E&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="bp"&gt;$_&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Properties&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;Value&lt;/span&gt;&lt;span class="p"&gt;}},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;@{&lt;/span&gt;&lt;span class="nx"&gt;N&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s1"&gt;'PID'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;&lt;span class="nx"&gt;E&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="bp"&gt;$_&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Properties&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;Value&lt;/span&gt;&lt;span class="p"&gt;}},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;@{&lt;/span&gt;&lt;span class="nx"&gt;N&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s1"&gt;'ModulePath'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;&lt;span class="nx"&gt;E&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="bp"&gt;$_&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Properties&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;11&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;Value&lt;/span&gt;&lt;span class="p"&gt;}},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;@{&lt;/span&gt;&lt;span class="nx"&gt;N&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s1"&gt;'ReportId'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;&lt;span class="nx"&gt;E&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="bp"&gt;$_&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Properties&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;Value&lt;/span&gt;&lt;span class="p"&gt;}}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The Module column is the whole point. &lt;code&gt;0xc0000005&lt;/code&gt; in the Exception column confirms you're looking at an access violation and not something else wearing the same exit code.&lt;/p&gt;

&lt;p&gt;One gotcha: when the faulting module is ntdll.dll or kernelbase.dll, you're usually looking at the messenger, not the culprit. Those are the most common places for a crash to surface because they do the memory work for everything else. The module you want is the last non-Microsoft one in the dump's stack.&lt;/p&gt;

&lt;p&gt;Property order has been stable for years, but if your columns look shifted, open a single event's Details tab and read the field names instead of trusting the index numbers.&lt;/p&gt;

&lt;h2&gt;
  
  
  If there is no event 1000, turn dumps on
&lt;/h2&gt;

&lt;p&gt;No Application Error entry means Windows Error Reporting never got a chance. Some apps die below the crash handler, usually very early in startup, and that absence is a clue in itself: it points at the GPU process or a native module load rather than application logic.&lt;/p&gt;

&lt;p&gt;You can force dumps to be kept. From an admin shell:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$k&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;'HKLM:\SOFTWARE\Microsoft\Windows\Windows Error Reporting\LocalDumps'&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="n"&gt;New-Item&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Path&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;$k&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Force&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="n"&gt;New-ItemProperty&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Path&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;$k&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Name&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;DumpFolder&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-PropertyType&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;ExpandString&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Value&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;'%LOCALAPPDATA%\CrashDumps'&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Force&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="n"&gt;New-ItemProperty&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Path&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;$k&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Name&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;DumpType&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-PropertyType&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;DWord&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Value&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;2&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Force&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="n"&gt;New-ItemProperty&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Path&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;$k&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Name&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;DumpCount&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-PropertyType&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;DWord&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Value&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;10&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Force&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;DumpType 2 is a full dump, which is what you want while the cause is still unknown. Dumps land in &lt;code&gt;%LOCALAPPDATA%\CrashDumps&lt;/code&gt;; WER's own archives live under &lt;code&gt;C:\ProgramData\Microsoft\Windows\WER\ReportArchive&lt;/code&gt; and &lt;code&gt;ReportQueue&lt;/code&gt;. Then reproduce the crash.&lt;/p&gt;

&lt;h2&gt;
  
  
  Read the dump without installing Visual Studio
&lt;/h2&gt;

&lt;p&gt;You don't need the IDE. Install just the "Debugging Tools for Windows" component from the Windows SDK, then point cdb at the dump:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;cdb.exe &lt;span class="nt"&gt;-z&lt;/span&gt; &lt;span class="s2"&gt;"%LOCALAPPDATA%&lt;/span&gt;&lt;span class="se"&gt;\C&lt;/span&gt;&lt;span class="s2"&gt;rashDumps&lt;/span&gt;&lt;span class="se"&gt;\a&lt;/span&gt;&lt;span class="s2"&gt;pp.exe.1234.dmp"&lt;/span&gt; &lt;span class="nt"&gt;-c&lt;/span&gt; &lt;span class="s2"&gt;"!analyze -v; q"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Three things in that output matter. FAULTING_IP is the instruction that faulted. The EXCEPTION_RECORD gives you the ExceptionAddress and the code. STACK_TEXT is the call stack, and it names the module whose code was actually running. DEFAULT_BUCKET_ID is a loose classification you can search on to see if other people hit the same shape of crash.&lt;/p&gt;

&lt;p&gt;The exception parameters also carry direction. &lt;code&gt;ExceptionInformation[0]&lt;/code&gt; is 0 for a read, 1 for a write, 8 for an execute. A read violation means the thread dereferenced a pointer it couldn't read: garbage, or memory that was already freed. A write violation means something wrote where it had no business writing. Both are useful, but neither names a culprit without the stack.&lt;/p&gt;

&lt;h2&gt;
  
  
  Rule out hardware and drivers first
&lt;/h2&gt;

&lt;p&gt;The System log tells you whether you're chasing software at all:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;WHEA-Logger events 17 and 18 are corrected hardware errors; 19 is uncorrected. If those show up next to your crashes, stop and test the RAM with mdsched.exe (MemTest86 if you want a real answer). Memory and PCIe are the usual sources.&lt;/li&gt;
&lt;li&gt;"Display driver amdwddmg stopped responding and has recovered" is a TDR, event 4101. That's the GPU and driver path, and it explains a lot of access violations in GPU-accelerated apps.&lt;/li&gt;
&lt;li&gt;Kernel-Power 41 shows up after any unclean shutdown. Everyone reads it as a diagnosis. It isn't one. It's a marker that the machine went down hard, and on its own it names no cause.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Be suspicious of the confident answers
&lt;/h2&gt;

&lt;p&gt;Two claims I'd flag, partly because I've repeated them myself:&lt;/p&gt;

&lt;p&gt;The first: "the fault offset is always the same but the bad address changes, so it's bad RAM." That pattern does look more like corruption than a logic bug, and testing memory is a fair response to it. But it isn't documented as proof of anything. It can be bad RAM, a freed object, or a native module built against the wrong SDK, and only the dump separates those.&lt;/p&gt;

&lt;p&gt;The second: "update the GPU driver, or switch the inference engine, and it goes away." Sometimes it does. While reading threads for this piece I found one where the suggested engine switch was tried by a second person and crashed for them too. Gone was the fix; the experiment stayed. Treat driver updates and engine switches as cheap things to try, not as advice you can hand someone else as an answer.&lt;/p&gt;

&lt;p&gt;Same story with the launch-flag soup you'll find in older threads. &lt;code&gt;--disable-gpu&lt;/code&gt; is a legitimate diagnostic because it moves rendering out of the GPU process. &lt;code&gt;--no-sandbox&lt;/code&gt; is not a fix. It disables the sandbox. Chromium's own documentation says it's for testing, so don't leave it sitting in your daily configuration.&lt;/p&gt;

&lt;p&gt;If the crash disappears after a clean boot, that's your answer: antivirus, overlays like RTSS or MSI Afterburner, and shell extensions all inject code into live processes, and all of them cause access violations.&lt;/p&gt;

&lt;h2&gt;
  
  
  The short version
&lt;/h2&gt;

&lt;p&gt;0xC0000005 means "bad memory access". It does not mean bad RAM, and it doesn't mean the module named in the log is buggy. Event 1000 gets you the module, the dump gets you the function, and WHEA plus TDR tell you whether you're in hardware territory at all. Ten minutes of reading, most of it spent waiting for a dump to load.&lt;/p&gt;

&lt;p&gt;I could be wrong about your specific crash. But if you're staring at this code with nowhere to start, the Event 1000 module name is the best first click you can make.&lt;/p&gt;

</description>
      <category>devops</category>
      <category>tutorial</category>
      <category>ai</category>
    </item>
    <item>
      <title>GPU offload says Max, but only 54 of 65 layers loaded: how to check where your model actually runs</title>
      <dc:creator>MilkyWay008</dc:creator>
      <pubDate>Wed, 30 Sep 2026 17:10:23 +0000</pubDate>
      <link>https://dev.to/milkyway008/gpu-offload-says-max-but-only-54-of-65-layers-loaded-how-to-check-where-your-model-actually-runs-7om</link>
      <guid>https://dev.to/milkyway008/gpu-offload-says-max-but-only-54-of-65-layers-loaded-how-to-check-where-your-model-actually-runs-7om</guid>
      <description>&lt;p&gt;A 27B model at Q4 was generating 12-18 tokens per second on an RTX 3090. Same model, same settings a week earlier, it was doing 60-70. Nothing in the app looked broken. The model config still said GPU Offload: Max.&lt;/p&gt;

&lt;p&gt;That's the trap. Max doesn't mean all layers on the GPU. It means as many as the app's size estimate thinks will fit. When that estimate comes out pessimistic, the loader quietly leaves layers on the CPU, and you pay for it on every token instead of noticing it once.&lt;/p&gt;

&lt;p&gt;Here's how to check what actually happened, and what to try.&lt;/p&gt;

&lt;h2&gt;
  
  
  The load log is the source of truth, not the dropdown
&lt;/h2&gt;

&lt;p&gt;LM Studio writes a log per model load. They live under your user profile in &lt;code&gt;.lmstudio&lt;/code&gt;. The paths I've seen quoted are &lt;code&gt;%USERPROFILE%\.lmstudio\logs\&lt;/code&gt; for the app log and &lt;code&gt;%USERPROFILE%\.lmstudio\server-logs\&lt;/code&gt; for the per-load server log, so have a poke around if yours sit somewhere else.&lt;/p&gt;

&lt;p&gt;Load a model, open the newest log, and look for this pair:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Limit weight offload to dedicated GPU Memory: ON
Model load size estimate with adjusted num offload layers '54'
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;"Adjusted" is the word doing the damage. The setting asked for Max. The estimator adjusted it down to 54 because its model of your VRAM said the rest wouldn't fit.&lt;/p&gt;

&lt;p&gt;Then llama.cpp prints the final answer on its own line:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;offloaded 54/65 layers to GPU
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If both numbers match, everything landed. If they don't, you're running a hybrid and the CPU is handling part of every token. On a 27B that's the difference between roughly 45 tokens per second and 15.&lt;/p&gt;

&lt;h2&gt;
  
  
  The check that doesn't need logs
&lt;/h2&gt;

&lt;p&gt;Watch GPU and CPU utilization while the model generates.&lt;/p&gt;

&lt;p&gt;Full offload looks like GPU utilization pinned high (the reporter measured around 97%) with the CPU mostly idle. Partial offload looks like low, spiky GPU utilization, a busy CPU, and throughput down by 2-3x. If it looks like the second case while the settings say Max, the estimator is your suspect.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why the estimate trims layers at all
&lt;/h2&gt;

&lt;p&gt;The machinery shows up clearly in another report in the same tracker. With the VRAM cap armed, the estimator runs and adjusts. With the cap off, it skips the adjustment step entirely:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[LM Studio] Model load size estimate with raw num offload layers 'max' and context length '8192'
[LM Studio] Strict GPU VRAM cap is OFF: GPU offload layers will not be checked for adjustment
[LM Studio] Resolved GPU config options:   Num Offload Layers: max
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;So the setting behind that first log line, something like "Limit weight offload to dedicated GPU memory" in the load settings, is what arms the trimming. It isn't a silly feature. It stops you from requesting more layers than the card can hold and getting worse performance for it. It just isn't always right.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to try, in order
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Switch the runtime backend. This was the change that reliably worked for the reporter. The engine was set to the CUDA 12 runtime; switching to the plain CUDA runtime, or to Vulkan, restored full offload. Throughput went from 12-18 tok/s to 41.9-45.4 tok/s at roughly 97% GPU utilization, across four different 27B models and a range of CUDA 12 runtime versions. A dropdown change for about a 3x speedup.&lt;/li&gt;
&lt;li&gt;Set an explicit layer count instead of Max, so nothing is left to the estimator. Unload and reload, then read the log line again. If you set 65 and the log still says 54, the trimming is happening anyway and you're back to step 1.&lt;/li&gt;
&lt;li&gt;Reduce the context length. The KV cache for a long context eats into the budget the estimator is working with, so a shorter context can leave more room for weights. Worth trying, though the reporter found it didn't reliably stop the behaviour.&lt;/li&gt;
&lt;li&gt;From the CLI: &lt;code&gt;lms load &amp;lt;model&amp;gt; --gpu max&lt;/code&gt; (or &lt;code&gt;off&lt;/code&gt;, or a fraction like &lt;code&gt;0.5&lt;/code&gt;), &lt;code&gt;--context-length&lt;/code&gt; to cap context, and &lt;code&gt;--estimate-only&lt;/code&gt; to see what the loader plans before you commit to the load. Those are in LM Studio's CLI docs.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  The honest part
&lt;/h2&gt;

&lt;p&gt;The CUDA 12 trimming is an open issue with no official fix as of late September 2026, so everything above is diagnosis plus a workaround, not a patch. Nobody has confirmed why the CUDA 12 path over-trims while the regular CUDA and Vulkan runtimes don't. The running theory in the thread is that the CUDA 12 build reserves more memory for graph and KV buffers, which makes the estimate come out larger, but that's a commenter's theory rather than a maintainer's finding. Treat it as a hypothesis.&lt;/p&gt;

&lt;p&gt;Worth saying out loud too: partial offload is sometimes the correct outcome. If your card genuinely can't hold the model, turning the cap off doesn't create VRAM. It just lets you overcommit, and overcommitting usually performs worse than a clean split.&lt;/p&gt;

&lt;p&gt;One related pattern, because it's the same class of problem on AMD hardware: an RX 6900 XT (RDNA2) on Windows can fail outright with "RDNA2 not supported with the ROCm version you are using" when the ROCm engine gets selected, while Vulkan works fine on the same card. Different message, same lesson. The backend picker matters more than the offload slider, so bisect the runtime before changing anything else.&lt;/p&gt;

&lt;p&gt;If you remember one thing from this: after loading a model, grep the load log for the offload count. Max is a request, not a receipt.&lt;/p&gt;

&lt;p&gt;The source report, with the full logs, is here: &lt;a href="https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/2312" rel="noopener noreferrer"&gt;https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/2312&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>tutorial</category>
      <category>devops</category>
    </item>
    <item>
      <title>[ASN1] nested asn1 error on Windows: one bad cert in the store breaks every Python tool</title>
      <dc:creator>MilkyWay008</dc:creator>
      <pubDate>Tue, 29 Sep 2026 17:17:35 +0000</pubDate>
      <link>https://dev.to/milkyway008/asn1-nested-asn1-error-on-windows-one-bad-cert-in-the-store-breaks-every-python-tool-3mo6</link>
      <guid>https://dev.to/milkyway008/asn1-nested-asn1-error-on-windows-one-bad-cert-in-the-store-breaks-every-python-tool-3mo6</guid>
      <description>&lt;p&gt;I was scrolling through bug reports a while back and one of them made me stop: a Python tool that dies on import with &lt;code&gt;ssl.SSLError: [ASN1] nested asn1 error&lt;/code&gt;. Machine was fine, network was fine, code was fine...... the user had just upgraded the app. Rolling back fixed it, which is the sort of detail that sends you hunting in the wrong place for an hour.&lt;/p&gt;

&lt;p&gt;I dug into it, and as far as I can tell the app was never the problem. One certificate sitting in the Windows certificate store was.&lt;/p&gt;

&lt;h2&gt;
  
  
  What is actually blowing up
&lt;/h2&gt;

&lt;p&gt;On Windows, CPython's &lt;code&gt;ssl&lt;/code&gt; module does not just read a bundled CA file. When something asks for a default SSL context, it walks the Windows certificate stores and collects the certs into one big buffer. Here is the code from &lt;code&gt;Lib/ssl.py&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;_load_windows_store_certs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;storename&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;purpose&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;certs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;bytearray&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;cert&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;encoding&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;trust&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;enum_certificates&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;storename&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
            &lt;span class="c1"&gt;# CA certs are never PKCS#7 encoded
&lt;/span&gt;            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;encoding&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;x509_asn&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;trust&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="n"&gt;purpose&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;oid&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;trust&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
                    &lt;span class="n"&gt;certs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;extend&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cert&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="n"&gt;PermissionError&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;warnings&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;warn&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;unable to enumerate Windows certificate store&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;certs&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;load_verify_locations&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cadata&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;certs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;certs&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Read that last part again. Every cert goes into &lt;code&gt;certs&lt;/code&gt;, and then the whole pile is handed to &lt;code&gt;load_verify_locations&lt;/code&gt; in a single call. The stores it walks are &lt;code&gt;('CA', 'ROOT')&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;So if one cert in that pile is malformed, the DER parse fails for the entire buffer:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;ssl.SSLError: [ASN1] nested asn1 error
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The bad one might be the 45th out of 45. The rest are perfectly good. Doesn't matter. The call throws, the context never gets built, and whichever import or TLS call comes first is the one that dies. Your code never gets a say in it.&lt;/p&gt;

&lt;p&gt;Depending on how the bytes are mangled you may get a different message for the same underlying problem: &lt;code&gt;SSLError(0, 'not enough data: cadata does not contain a certificate')&lt;/code&gt;. I hit both while testing.&lt;/p&gt;

&lt;h2&gt;
  
  
  I reproduced it in about 20 lines
&lt;/h2&gt;

&lt;p&gt;I didn't want to take this one on faith, so I ran it. Enumerating the ROOT store on my Windows 11 box gave 45 certs, 48,107 bytes of DER. Loaded whole, everything is happy. Then I flipped a single byte inside one certificate and loaded the exact same buffer:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;python&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;3.11&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;15&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="n"&gt;openssl&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;OpenSSL&lt;/span&gt; &lt;span class="mf"&gt;3.5&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt;
&lt;span class="n"&gt;ROOT&lt;/span&gt; &lt;span class="n"&gt;store&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;45&lt;/span&gt; &lt;span class="n"&gt;certs&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;48107&lt;/span&gt; &lt;span class="nb"&gt;bytes&lt;/span&gt; &lt;span class="n"&gt;DER&lt;/span&gt;
&lt;span class="n"&gt;baseline&lt;/span&gt; &lt;span class="nf"&gt;load_verify_locations&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;all&lt;/span&gt; &lt;span class="n"&gt;store&lt;/span&gt; &lt;span class="n"&gt;certs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;OK&lt;/span&gt;
&lt;span class="n"&gt;REPRO&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="n"&gt;corrupted&lt;/span&gt; &lt;span class="n"&gt;cert&lt;/span&gt; &lt;span class="n"&gt;among&lt;/span&gt; &lt;span class="mi"&gt;45&lt;/span&gt; &lt;span class="n"&gt;valid&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;SSLError&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;ASN1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="n"&gt;nested&lt;/span&gt; &lt;span class="n"&gt;asn1&lt;/span&gt; &lt;span class="nf"&gt;error &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;_ssl&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;4057&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One byte. That is the whole bug. If you want to poke at it on your own machine:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;ssl&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;_ssl&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;enum_certificates&lt;/span&gt;

&lt;span class="n"&gt;certs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;enc&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;trust&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;enum_certificates&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ROOT&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;enc&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;x509_asn&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="n"&gt;ctx&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;ssl&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;SSLContext&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ssl&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;PROTOCOL_TLS_CLIENT&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;load_verify_locations&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cadata&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sa"&gt;b&lt;/span&gt;&lt;span class="sh"&gt;""&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;certs&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;   &lt;span class="c1"&gt;# works ... unless one cert is bad
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Why SSL_CERT_FILE does not save you here
&lt;/h2&gt;

&lt;p&gt;The instinct is to point Python at a known-good bundle and move on. On Windows that does not stop the scan, because of the order these things run in. &lt;code&gt;SSLContext.load_default_certs&lt;/code&gt; walks the stores first and only afterwards calls &lt;code&gt;set_default_verify_paths()&lt;/code&gt;, which is where &lt;code&gt;SSL_CERT_FILE&lt;/code&gt; and &lt;code&gt;SSL_CERT_DIR&lt;/code&gt; are actually consulted:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;load_default_certs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;purpose&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;Purpose&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;SERVER_AUTH&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="bp"&gt;...&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;sys&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;platform&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;win32&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;storename&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_windows_cert_stores&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;_load_windows_store_certs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;storename&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;purpose&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set_default_verify_paths&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The bad cert blows up the buffer before your environment variable is ever read. Same story for any tool that builds its context with a bare &lt;code&gt;ssl.create_default_context()&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;I'm reading this off CPython 3.11 and 3.12. If you're on something newer, &lt;code&gt;.../Lib/ssl.py&lt;/code&gt; is right there on disk. The ordering has been tweaked between releases, so check yours rather than trusting my summary of it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Find the bad cert
&lt;/h2&gt;

&lt;p&gt;This part is read-only and quick. PowerShell 5.1, no admin needed:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="c"&gt;# try to re-parse every cert in the root stores as X.509 DER&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="nv"&gt;$stores&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;@(&lt;/span&gt;&lt;span class="s1"&gt;'Cert:\LocalMachine\Root'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;'Cert:\CurrentUser\Root'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s1"&gt;'Cert:\LocalMachine\CA'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="kr"&gt;foreach&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$s&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kr"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;$stores&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="kr"&gt;if&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;-not&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;Test-Path&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;$s&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kr"&gt;continue&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="n"&gt;Get-ChildItem&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;$s&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-ErrorAction&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;SilentlyContinue&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;ForEach-Object&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="kr"&gt;try&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="bp"&gt;$null&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;System.Security.Cryptography.X509Certificates.X509Certificate2&lt;/span&gt;&lt;span class="p"&gt;]::&lt;/span&gt;&lt;span class="n"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="bp"&gt;$_&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;RawData&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="kr"&gt;catch&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"BAD  {0}  {1}  {2}"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-f&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;$s&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="bp"&gt;$_&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;Thumbprint&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="bp"&gt;$_&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;Subject&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Whatever the &lt;code&gt;catch&lt;/code&gt; prints is your suspect. On a healthy machine you get nothing back, which is exactly what happened when I ran it here.&lt;/p&gt;

&lt;p&gt;Export before you delete anything:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$c&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Get-ChildItem&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Cert:\LocalMachine\Root\&lt;/span&gt;&lt;span class="err"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nx"&gt;THUMBPRINT&lt;/span&gt;&lt;span class="err"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="n"&gt;Export-Certificate&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Cert&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;$c&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-FilePath&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$&lt;/span&gt;&lt;span class="nn"&gt;env&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="nv"&gt;USERPROFILE&lt;/span&gt;&lt;span class="s2"&gt;\Desktop\suspect.cer"&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then remove it, either through &lt;code&gt;certlm.msc&lt;/code&gt; (Trusted Root Certification Authorities, find the thumbprint, delete) or:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Remove-Item&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Path&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Cert:\LocalMachine\Root\&lt;/span&gt;&lt;span class="err"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nx"&gt;THUMBPRINT&lt;/span&gt;&lt;span class="err"&gt;&amp;gt;&lt;/span&gt;&lt;span class="w"&gt;   &lt;/span&gt;&lt;span class="c"&gt;# admin shell&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One caveat that matters. If the cert turns out to be your company root for TLS inspection, do not delete it, re-import it properly. A half-imported corporate root will break internal TLS the moment it's gone, and now you have two problems instead of one.&lt;/p&gt;

&lt;p&gt;Where do malformed certs come from anyway? Usually a truncated import by an installer, an endpoint agent or AV product, an old tool that wrote DER where the store wanted PEM, or someone hand-copying a root out of a browser.&lt;/p&gt;

&lt;h2&gt;
  
  
  If you cannot touch the store, skip the scan
&lt;/h2&gt;

&lt;p&gt;For code you own, pass an explicit CA source. Any of &lt;code&gt;cafile&lt;/code&gt;, &lt;code&gt;capath&lt;/code&gt; or &lt;code&gt;cadata&lt;/code&gt; takes a different branch, so &lt;code&gt;load_default_certs&lt;/code&gt; never runs:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;ssl&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;certifi&lt;/span&gt;
&lt;span class="n"&gt;ctx&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;ssl&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;create_default_context&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;cafile&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;certifi&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;where&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;   &lt;span class="c1"&gt;# store is never scanned
&lt;/span&gt;
&lt;span class="c1"&gt;# from Lib/ssl.py:
#   if cafile or capath or cadata:
#       context.load_verify_locations(cafile, capath, cadata)
#   elif context.verify_mode != CERT_NONE:
#       context.load_default_certs(purpose)
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For tools you don't own, it's the same idea expressed as whatever config that tool reads. This is also the answer to the other half of the problem that kept showing up in those reports. Behind a TLS-inspecting proxy, exporting your corporate CA to one tool does nothing for the next tool, because each one has its own knob.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Tool&lt;/th&gt;
&lt;th&gt;Setting&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Python &lt;code&gt;ssl&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;SSL_CERT_FILE&lt;/code&gt;, &lt;code&gt;SSL_CERT_DIR&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;requests&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;REQUESTS_CA_BUNDLE&lt;/code&gt;, &lt;code&gt;CURL_CA_BUNDLE&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;pip&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;PIP_CERT&lt;/code&gt;, or &lt;code&gt;--cert&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Node.js&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;NODE_EXTRA_CA_CERTS&lt;/code&gt; (PEM, appended to the bundled roots at launch)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;npm&lt;/td&gt;
&lt;td&gt;&lt;code&gt;npm config set cafile &amp;lt;path&amp;gt;&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;curl&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;CURL_CA_BUNDLE&lt;/code&gt;, &lt;code&gt;--cacert&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;git&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;GIT_SSL_CAINFO&lt;/code&gt;, &lt;code&gt;git config --global http.sslCAInfo &amp;lt;path&amp;gt;&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;uv&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;SSL_CERT_FILE&lt;/code&gt;, &lt;code&gt;SSL_CERT_DIR&lt;/code&gt;, plus &lt;code&gt;UV_NATIVE_TLS&lt;/code&gt; if you want the platform store&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;AWS CLI / boto3&lt;/td&gt;
&lt;td&gt;&lt;code&gt;AWS_CA_BUNDLE&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;And please don't "fix" this with &lt;code&gt;verify=False&lt;/code&gt; or &lt;code&gt;curl -k&lt;/code&gt;. That hides the error and hands the problem to whoever reads your traffic next.&lt;/p&gt;

&lt;h2&gt;
  
  
  The bit worth remembering
&lt;/h2&gt;

&lt;p&gt;A shared trust store is a single point of failure. One bad byte in an entry you never put there takes down every process that asks for a default context, and the error lands in whichever tool happened to load first. The thing that crashes is rarely the thing that is broken.&lt;/p&gt;

&lt;p&gt;I can be wrong about your specific machine, but if the traceback points at &lt;code&gt;_load_windows_store_certs&lt;/code&gt;, run the scan above before you downgrade anything. Hope this helps.&lt;/p&gt;

</description>
      <category>python</category>
      <category>tutorial</category>
      <category>devops</category>
    </item>
    <item>
      <title>Method invocation is supported only on core types in this language mode: what it means and how to fix it</title>
      <dc:creator>MilkyWay008</dc:creator>
      <pubDate>Tue, 29 Sep 2026 02:50:39 +0000</pubDate>
      <link>https://dev.to/milkyway008/method-invocation-is-supported-only-on-core-types-in-this-language-mode-what-it-means-and-how-to-369j</link>
      <guid>https://dev.to/milkyway008/method-invocation-is-supported-only-on-core-types-in-this-language-mode-what-it-means-and-how-to-369j</guid>
      <description>&lt;p&gt;You run one command in your agent CLI. It works. Then a wall of red shows up anyway:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Cannot&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;invoke&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;the&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;method.&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Method&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;invocation&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;is&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;supported&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;only&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;on&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;core&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;types&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;in&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;this&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;language&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;mode.&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;FullyQualifiedErrorId&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;MethodInvocationNotSupportedInConstrainedLanguage&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Exit code 0. The thing you asked for actually happened. So what's complaining, and why does it keep complaining on every single command?&lt;/p&gt;

&lt;p&gt;I keep seeing this one in issue trackers, most recently for Copilot CLI on a domain-joined Windows 11 machine. The same error shows up in other tools too, anywhere a CLI shells out to PowerShell. If I remember correctly, here's the mechanism......&lt;/p&gt;

&lt;h2&gt;
  
  
  What's actually happening
&lt;/h2&gt;

&lt;p&gt;PowerShell doesn't decide on its own to restrict you. When Windows detects an application control policy, either AppLocker or WDAC (Microsoft now calls that App Control for Business), PowerShell starts the session in ConstrainedLanguage mode. It's documented in about_Language_Modes if you want the source.&lt;/p&gt;

&lt;p&gt;Constrained language isn't "no PowerShell". Cmdlets still run. Loops, ifs, functions, strings, all of that works. What changes is .NET method calls: they're only allowed on a short allow-list of core types, and everything else throws.&lt;/p&gt;

&lt;p&gt;The tool's wrapper script usually ends like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$host&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;SetShouldExit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;$LASTEXITCODE&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;$host&lt;/code&gt; is an InternalHost object, which isn't a core type, so that line fails. It's also the last line of the epilogue, which means it fails after your actual command already ran and returned 0. The cleanup path is complaining, not the work.&lt;/p&gt;

&lt;p&gt;One JetBrains user hit the same class of thing from a second call, &lt;code&gt;$Error.RemoveAt(0)&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Confirm it in one line
&lt;/h2&gt;

&lt;p&gt;Cmdlets aren't blocked under constrained language, so the diagnostics themselves work fine:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="bp"&gt;$ExecutionContext&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;SessionState&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;LanguageMode&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If that prints &lt;code&gt;ConstrainedLanguage&lt;/code&gt;, you found it. Then work out what's enforcing it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Get-AppLockerPolicy&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Effective&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Xml&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Non-empty output means AppLocker is the enforcer. For WDAC the logs are the tell. In Event Viewer:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Applications and Services Logs &amp;gt; Microsoft &amp;gt; Windows &amp;gt; AppLocker &amp;gt; MSI and Script: 8007 is a script blocked, 8006 is audit only, 8005 is allowed.&lt;/li&gt;
&lt;li&gt;CodeIntegrity &amp;gt; Operational: 3076 (audit) and 3077 (blocked) mean WDAC is enforcing.&lt;/li&gt;
&lt;li&gt;PowerShell &amp;gt; Operational: entries state that the session ran in ConstrainedLanguage.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What actually fixes it
&lt;/h2&gt;

&lt;p&gt;Two levels here, and I want to be careful about which one I'm putting my name behind.&lt;/p&gt;

&lt;p&gt;The real fix is a policy exception, and it is documented. Scripts a policy allows run in FullLanguage even when the session is constrained. So the tool's wrapper scripts need to be in the allow list: a WDAC file or signer rule covering the app's payload, or the Managed Installer option if it's deployed through Intune or ConfigMgr. On the AppLocker side, a script rule exception for the tool's path or publisher. That's a change request rather than a workaround, and I know it can take a week to land. Raise it anyway, because it's the only version of this that survives the next update.&lt;/p&gt;

&lt;p&gt;For today, stop routing through PowerShell. Most agent CLIs let you choose which shell their shell tool uses. Point it at &lt;code&gt;cmd.exe&lt;/code&gt; or &lt;code&gt;bash&lt;/code&gt; and the wrapper never runs. That's what people in the Copilot CLI thread reported working. One thing that does not work: setting VS Code's &lt;code&gt;terminal.integrated.automationProfile.windows&lt;/code&gt;. Someone tested it and it changed nothing, because the CLI spawns its own shell rather than going through the integrated terminal. Passing that on so you don't burn an afternoon on the same red herring.&lt;/p&gt;

&lt;h2&gt;
  
  
  What not to do
&lt;/h2&gt;

&lt;p&gt;You'll find these recommended. Don't.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Flipping &lt;code&gt;__PSLockDownPolicy&lt;/code&gt; under &lt;code&gt;HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Environment&lt;/code&gt; to turn constrained mode off.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;powershell -Version 2&lt;/code&gt;, or launching from a whitelisted directory to dodge the policy.&lt;/li&gt;
&lt;li&gt;Smuggling the command through an allowed helper with encoded arguments.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;All of those defeat application control, which is usually there because somebody's audit depends on it. Getting your machine quarantined over a red error message that wasn't even breaking anything is a bad trade. And if the error is pure noise and the command exits 0, letting it scroll past while your ticket sits in the queue is a legitimate option. It's ugly, but it's honest.&lt;/p&gt;

&lt;h2&gt;
  
  
  The general lesson
&lt;/h2&gt;

&lt;p&gt;If a PowerShell script works on your box and fails on a managed one, check the language mode before you rewrite any logic. Constrained language also hits &lt;code&gt;Add-Type&lt;/code&gt;, most COM object use, and plenty of method calls you'd never think twice about. The error text names the method it blocked, which is a decent tell that you're hitting the mode rather than a bug in your script.&lt;/p&gt;

&lt;p&gt;And if you maintain one of these CLIs, the fix on your side is small. Use the &lt;code&gt;exit&lt;/code&gt; keyword instead of &lt;code&gt;$host.SetShouldExit()&lt;/code&gt;, or guard that call behind a language-mode check. The Copilot CLI issue is still open with no merged fix that I could find, so in the meantime it's the shell switch plus a ticket.&lt;/p&gt;

&lt;p&gt;The language mode check takes ten seconds, so it's cheap to rule in or out. I could be wrong about the details for your particular tool, but I hope this saves somebody an afternoon.&lt;/p&gt;

</description>
      <category>tutorial</category>
      <category>devops</category>
      <category>ai</category>
    </item>
    <item>
      <title>MongoDB's 16MB document limit: how to find the doc that kills your agent run</title>
      <dc:creator>MilkyWay008</dc:creator>
      <pubDate>Sun, 27 Sep 2026 17:15:01 +0000</pubDate>
      <link>https://dev.to/milkyway008/mongodbs-16mb-document-limit-how-to-find-the-doc-that-kills-your-agent-run-2k68</link>
      <guid>https://dev.to/milkyway008/mongodbs-16mb-document-limit-how-to-find-the-doc-that-kills-your-agent-run-2k68</guid>
      <description>&lt;p&gt;One of the agents I look after keeps its state in MongoDB, and a few weeks back it started doing the thing I dislike most: nothing. No stack trace, no failed HTTP call to point at. The run just sat there while the UI kept spinning.&lt;/p&gt;

&lt;p&gt;There was an error, all right. It wasn't in the app, it was in the database.&lt;/p&gt;

&lt;p&gt;A Mastra user hit the same wall in issue #21412. Long DurableAgent runs, MongoDB storage, &lt;code&gt;workflow-snapshots&lt;/code&gt; collection. Their snapshot document grew past MongoDB's per-document size limit, the write got rejected, the snapshot flipped to &lt;code&gt;failed&lt;/code&gt;, the framework deleted it, and the stream had nowhere left to go. That thread closed as resolved in a later release, but the trap underneath it isn't Mastra's. It belongs to MongoDB, and it's waiting for anyone who stores a growing conversation inside a single document.&lt;/p&gt;

&lt;p&gt;So here's the limit, how to find the document about to cross it, and how I restructure before it happens.&lt;/p&gt;

&lt;h2&gt;
  
  
  The limit itself
&lt;/h2&gt;

&lt;p&gt;MongoDB's docs are blunt about this one. The maximum BSON document size is 16 mebibytes, 16777216 bytes, and it's described as a hard limit. The server reports the value as &lt;code&gt;maxBsonObjectSize&lt;/code&gt; in the &lt;code&gt;hello&lt;/code&gt; command:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nx"&gt;db&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;runCommand&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt; &lt;span class="na"&gt;hello&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="p"&gt;}).&lt;/span&gt;&lt;span class="nx"&gt;maxBsonObjectSize&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You can't raise it. No server flag, no &lt;code&gt;mongod&lt;/code&gt; option, no storage-engine setting, and no compression switch that changes the encoded size. The constant sits in the server source as &lt;code&gt;BSONObjMaxInternalSize&lt;/code&gt;. Once the encoded BSON crosses that line the whole write is rejected: the document isn't truncated, it isn't split, and nothing partial lands in the collection.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why agent state walks straight into it
&lt;/h2&gt;

&lt;p&gt;16MB sounds roomy until you look at what an agent appends every turn. The user message, the assistant message, every tool call, every tool result, and usually a copy of all of it inside the stored history. Tool results are the fat ones: a &lt;code&gt;read_file&lt;/code&gt; on a 400KB log, a JSON API response, an HTML page, a base64 blob.&lt;/p&gt;

&lt;p&gt;A few hundred turns of that and you're not at 16MB because somebody wrote bad code. You're there because the data is genuinely big, and nothing in the pipeline was ever told to stop growing.&lt;/p&gt;

&lt;p&gt;The failure is quieter than it should be. The snapshot write fails, the framework moves on, and the symptom you actually get to see is a stream that stops.&lt;/p&gt;

&lt;h2&gt;
  
  
  Finding the document before it kills a run
&lt;/h2&gt;

&lt;p&gt;Open &lt;code&gt;mongosh&lt;/code&gt; against the database and size the documents. &lt;code&gt;$bsonSize&lt;/code&gt; is the aggregation expression for exactly this, and it works on &lt;code&gt;$$ROOT&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nx"&gt;db&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;workflow_snapshots&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;aggregate&lt;/span&gt;&lt;span class="p"&gt;([&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;$project&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;sizeMB&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;$divide&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;$bsonSize&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;$$ROOT&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="mi"&gt;1024&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;1024&lt;/span&gt; &lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;$sort&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;sizeMB&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;$limit&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;])&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's your ten fattest documents. To see which field is eating the space, unwind the document into fields and size each one:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nx"&gt;db&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;workflow_snapshots&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;aggregate&lt;/span&gt;&lt;span class="p"&gt;([&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;$project&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;fields&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;$objectToArray&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;$$ROOT&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;$unwind&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;$fields&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;$project&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;field&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;$fields.k&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="na"&gt;bytes&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;$bsonSize&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;$fields.v&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;$sort&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;bytes&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;$limit&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;15&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;])&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The top of that list is usually what you'd expect: the message history, then whatever tool output got stored next to it. If you only want one document's size from the shell, &lt;code&gt;Object.bsonsize(doc)&lt;/code&gt; is the shortcut.&lt;/p&gt;

&lt;h2&gt;
  
  
  Fixing it
&lt;/h2&gt;

&lt;p&gt;Keep the growing array out of the document that has to stay small. Let the snapshot record hold state, not the transcript. Messages go in their own collection keyed by session, and the snapshot keeps a reference plus a turn count.&lt;/p&gt;

&lt;p&gt;Cap the array as you append to it. &lt;code&gt;$push&lt;/code&gt; with &lt;code&gt;$each&lt;/code&gt; and a negative &lt;code&gt;$slice&lt;/code&gt; keeps only the newest entries, atomically, in a single update:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nx"&gt;db&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;sessions&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;updateOne&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;sessionId&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;$push&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;history&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="na"&gt;$each&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt; &lt;span class="nx"&gt;message&lt;/span&gt; &lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="na"&gt;$slice&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;200&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A negative number means "keep the last N", per the &lt;code&gt;$slice&lt;/code&gt; modifier docs. &lt;code&gt;$slice&lt;/code&gt; also requires &lt;code&gt;$each&lt;/code&gt;. On its own it gets silently skipped, which is a fine way to spend an afternoon wondering why your array never shrinks.&lt;/p&gt;

&lt;p&gt;Move the blobs out. If a tool result is a file you never query into, store the blob and keep the id. GridFS is built for that: it splits a file into chunks across &lt;code&gt;fs.files&lt;/code&gt; and &lt;code&gt;fs.chunks&lt;/code&gt;, 255KiB each by default, which is how it sidesteps the 16MB cap. Its own docs say that if your files are all under 16MB, just store them in a document. So reach for it when the thing really is bigger than a document should be, not as a general message store.&lt;/p&gt;

&lt;p&gt;Set retention. TTL indexes delete whole documents once a date field ages out. The background task runs every 60 seconds, and TTL indexes are single-field only. Great for old sessions. Useless for trimming an array inside a live document.&lt;/p&gt;

&lt;h2&gt;
  
  
  What doesn't work
&lt;/h2&gt;

&lt;p&gt;Raising the limit. It's hard-coded, and searching for a flag wastes a day.&lt;/p&gt;

&lt;p&gt;A TTL index on an embedded array field, hoping it keeps the last N messages. TTL expires documents, not array elements, and it can't be compound.&lt;/p&gt;

&lt;p&gt;GridFS as your conversation store. It gets around the limit honestly, one chunk per document, but it's a file API. Rebuilding a chat from chunk documents is not a thing you want to be doing at 2am.&lt;/p&gt;

&lt;p&gt;Capped collections as a fix for a growing array. A capped collection is an append-only log, and it does nothing for the snapshot document that keeps getting updated.&lt;/p&gt;

&lt;p&gt;The named data-modeling patterns (bucket, subset, extended reference) are the textbook answer, and the bucket pattern is the one I'd look at for time-sliced history. Fair warning: the docs restructured recently and the old &lt;code&gt;building-patterns&lt;/code&gt; pages now 404, so I couldn't re-check the wording before writing this.&lt;/p&gt;

&lt;h2&gt;
  
  
  The honest part
&lt;/h2&gt;

&lt;p&gt;I haven't run this against Mastra itself, so I'm going on the reporter's account plus what the database does. The &lt;code&gt;$bsonSize&lt;/code&gt; pipeline is verified against MongoDB's operator docs. The per-field version is composed from documented stages and I haven't run that exact three-stage form end to end, so paste it into a copy first.&lt;/p&gt;

&lt;p&gt;I also couldn't confirm the exact error string a current server returns when a document overflows. It will name the 16777216-byte limit and it will land at write time. If you're grepping logs for a magic phrase, stop and size the documents instead. It's a two-minute query and it tells you the truth.&lt;/p&gt;

&lt;p&gt;Worth a look at your own state collection even if nothing has died yet. The document sitting at 12MB today is the one that takes a run down next week.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>tutorial</category>
      <category>devops</category>
      <category>javascript</category>
    </item>
    <item>
      <title>fetch-pack: invalid index-pack output, and a .git folder that ballooned to 6 GB</title>
      <dc:creator>MilkyWay008</dc:creator>
      <pubDate>Sat, 26 Sep 2026 17:19:05 +0000</pubDate>
      <link>https://dev.to/milkyway008/fetch-pack-invalid-index-pack-output-and-a-git-folder-that-ballooned-to-6-gb-1b3l</link>
      <guid>https://dev.to/milkyway008/fetch-pack-invalid-index-pack-output-and-a-git-folder-that-ballooned-to-6-gb-1b3l</guid>
      <description>&lt;p&gt;If you have ever wrapped &lt;code&gt;git fetch&lt;/code&gt; in a timeout, cancelled a clone halfway, or shut the laptop lid mid-pull, you may already be carrying this bug around. Nothing warns you. Git just starts acting strange weeks later.&lt;/p&gt;

&lt;p&gt;I ran into it while following a bug report on a self-updating app that ran &lt;code&gt;git fetch&lt;/code&gt; with a 10 second timeout and swallowed every error with &lt;code&gt;except Exception: pass&lt;/code&gt; (which is its own lesson). On the reporter's machine the repo had grown to 6 GB, update checks hung for 30+ seconds, and fetches were failing with a pile of different messages:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;code&gt;fetch-pack: invalid index-pack output&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;Could not read &amp;lt;sha&amp;gt;&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;invalid reflog entry&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;HTTP 429&lt;/code&gt; on &lt;code&gt;ls-remote&lt;/code&gt; (this one is usually a red herring)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Three different symptoms, one cause, and it is not corruption.......&lt;/p&gt;

&lt;h2&gt;
  
  
  What git leaves behind when you kill a fetch
&lt;/h2&gt;

&lt;p&gt;Git does not write an incoming pack straight to its final filename. &lt;code&gt;builtin/index-pack.c&lt;/code&gt; writes the pack to a temp path inside &lt;code&gt;.git/objects/pack/&lt;/code&gt; named &lt;code&gt;tmp_pack_XXXXXX&lt;/code&gt;, and only renames it to &lt;code&gt;pack-&amp;lt;hash&amp;gt;.pack&lt;/code&gt; after the transfer completes and the index has been built (&lt;code&gt;rename_tmp_packfile()&lt;/code&gt;). Index files get the same treatment as &lt;code&gt;tmp_idx_*&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Kill the process between those two moments and the rename never happens. You are left with &lt;code&gt;tmp_pack_XXXXXX&lt;/code&gt; in the pack directory, usually marked read-only (0444).&lt;/p&gt;

&lt;p&gt;Git will never treat that file as a pack. It is not indexed, not counted as a pack, and not cleaned up by itself. It just sits there taking up disk. In the report I was following, repeated failed update checks had pushed the repo to 6 GB. I reproduced it on Windows in a scratch repo: killed a clone mid-transfer and a 264 MB read-only &lt;code&gt;tmp_pack&lt;/code&gt; stayed behind.&lt;/p&gt;

&lt;p&gt;Git will show you these if you know which field to look at:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;$ &lt;/span&gt;git count-objects &lt;span class="nt"&gt;-vH&lt;/span&gt;
...
garbage: 1
size-garbage: 264.00 MiB
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;garbage&lt;/code&gt; and &lt;code&gt;size-garbage&lt;/code&gt; are where orphaned temp files land. I had never paid attention to that field either.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why gc doesn't clean it up
&lt;/h2&gt;

&lt;p&gt;This is the part that made me put the keyboard down for a minute. Plain &lt;code&gt;git gc&lt;/code&gt; does not remove them. Neither does &lt;code&gt;git repack -ad&lt;/code&gt;. &lt;code&gt;git fsck --full&lt;/code&gt; only prints a warning about them. I tested all three on scratch repos and the temp files survived every one.&lt;/p&gt;

&lt;p&gt;The reason is the prune grace period. The thing that actually deletes &lt;code&gt;tmp_*&lt;/code&gt; files under &lt;code&gt;objects/&lt;/code&gt; and &lt;code&gt;objects/pack/&lt;/code&gt; is &lt;code&gt;git prune&lt;/code&gt; (&lt;code&gt;remove_temporary_files()&lt;/code&gt; in &lt;code&gt;builtin/prune.c&lt;/code&gt;). But &lt;code&gt;git gc&lt;/code&gt; runs prune with &lt;code&gt;gc.pruneExpire&lt;/code&gt;, and that defaults to 2 weeks. A temp file written five minutes ago is younger than the grace period, so gc looks straight past it.&lt;/p&gt;

&lt;p&gt;So files created minutes ago get ignored, and files that get ignored stay forever if you keep re-running a failing fetch. That is the whole trap.&lt;/p&gt;

&lt;h2&gt;
  
  
  The fix
&lt;/h2&gt;

&lt;p&gt;Nothing here touches your commits, but do it when nothing else is using the repo.&lt;/p&gt;

&lt;p&gt;1st, confirm no git process is running. On Windows:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;tasklist | &lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-i&lt;/span&gt; git
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A live fetch plus a prune is the one combination that can genuinely damage a healthy repo.&lt;/p&gt;

&lt;p&gt;2nd, look at what you have:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;du&lt;/span&gt; &lt;span class="nt"&gt;-sh&lt;/span&gt; .git
find .git/objects/pack &lt;span class="nt"&gt;-name&lt;/span&gt; &lt;span class="s1"&gt;'tmp_*'&lt;/span&gt; &lt;span class="nt"&gt;-ls&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;3rd, check whether this is only orphans or something worse:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git fsck &lt;span class="nt"&gt;--full&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;4th, if fsck only reports garbage (no missing objects, no broken links), delete the orphans:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git prune &lt;span class="nt"&gt;--expire&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;now
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;git gc --prune=now&lt;/code&gt; does the same job. The &lt;code&gt;--prune=now&lt;/code&gt; is the part that matters; without it you get the 2 week grace period back and nothing gets removed.&lt;/p&gt;

&lt;p&gt;On Windows those temp files are read-only, so &lt;code&gt;del&lt;/code&gt; in cmd will refuse until you run &lt;code&gt;attrib -R .git\objects\pack\tmp_*&lt;/code&gt; first. In Git Bash, &lt;code&gt;rm -f .git/objects/pack/tmp_*&lt;/code&gt; is fine. PowerShell's &lt;code&gt;Remove-Item -Force&lt;/code&gt; works too. I tried all three.&lt;/p&gt;

&lt;p&gt;Finally, confirm and move on:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git count-objects &lt;span class="nt"&gt;-vH&lt;/span&gt;
git fetch
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;size-garbage&lt;/code&gt; should be 0 and the fetch should stop complaining.&lt;/p&gt;

&lt;h2&gt;
  
  
  When it is actually corruption
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;git fsck --full&lt;/code&gt; is the line in the sand. If it keeps reporting &lt;code&gt;missing blob/tree/commit&lt;/code&gt;, &lt;code&gt;broken link&lt;/code&gt;, or &lt;code&gt;invalid reflog entry &amp;lt;sha&amp;gt;&lt;/code&gt;, then objects your refs point at are gone, and pruning will not bring them back. That is the re-clone case:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone &lt;span class="nt"&gt;--depth&lt;/span&gt; 1 &amp;lt;url&amp;gt; newdir
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A warning about one common piece of advice: &lt;code&gt;git reflog expire --expire-unreachable=now --all&lt;/code&gt; gets recommended a lot here. It does silence reflog complaints, but it permanently drops your undo history. If all you have is orphaned temp files, you do not need it. Skip it.&lt;/p&gt;

&lt;p&gt;Two more things I would avoid. Do not run &lt;code&gt;git gc --prune=now --aggressive&lt;/code&gt; on a repo you have not fsck'd yet, and do not blame your network for a 429 on &lt;code&gt;ls-remote&lt;/code&gt; until the temp files are gone and you have retried once.&lt;/p&gt;

&lt;h2&gt;
  
  
  If you build tools that call git
&lt;/h2&gt;

&lt;p&gt;The app-side fix that went in for the bug I was following was a cleanup step on the fetch timeout path: look for stale &lt;code&gt;tmp_pack_*&lt;/code&gt;, remove the ones older than 10 minutes, and report "update check failed" instead of hanging silently. If your tool shells out to &lt;code&gt;git fetch&lt;/code&gt; with a timeout, that is worth copying. A timeout that kills the child and walks away leaves a small landmine for the next run, and the next run does not know about it.&lt;/p&gt;

&lt;p&gt;Your mileage will vary with repo size and git version (I tested on Windows with git 2.54). I have only seen this on machines where fetches were being killed rather than failing cleanly, so there may be other paths into the same state. But if &lt;code&gt;.git&lt;/code&gt; is enormous and fetches throw &lt;code&gt;invalid index-pack output&lt;/code&gt;, check the pack directory before you nuke anything.&lt;/p&gt;

&lt;p&gt;Source thread: &lt;a href="https://github.com/NousResearch/hermes-agent/issues/93732" rel="noopener noreferrer"&gt;NousResearch/hermes-agent#93732&lt;/a&gt;&lt;/p&gt;

</description>
      <category>devops</category>
      <category>tutorial</category>
      <category>automation</category>
    </item>
    <item>
      <title>Why your CLI says you're not logged in on a headless Linux server</title>
      <dc:creator>MilkyWay008</dc:creator>
      <pubDate>Fri, 25 Sep 2026 17:07:02 +0000</pubDate>
      <link>https://dev.to/milkyway008/why-your-cli-says-youre-not-logged-in-on-a-headless-linux-server-j1o</link>
      <guid>https://dev.to/milkyway008/why-your-cli-says-youre-not-logged-in-on-a-headless-linux-server-j1o</guid>
      <description>&lt;p&gt;You finish the OAuth flow on a fresh VPS. The browser says success. The CLI writes a token file with an expiry three weeks out. Then it hangs on "Verifying your account", or tells you that you are not logged in.&lt;/p&gt;

&lt;p&gt;If the token file is right there and still valid, you are probably not fighting the login. You are fighting the credential store the CLI looks in first, which on Linux is the desktop secret service. On a headless box that service is not running.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the CLI is actually asking for
&lt;/h2&gt;

&lt;p&gt;Most cross-platform CLIs (Gemini CLI, gh, gcloud, the agent CLIs, anything built on keytar or libsecret) do not keep OAuth tokens in a plain file. They hand them to the OS secret store:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Linux: libsecret, which talks to &lt;code&gt;org.freedesktop.secrets&lt;/code&gt; over D-Bus, implemented by gnome-keyring or KWallet&lt;/li&gt;
&lt;li&gt;macOS: Keychain&lt;/li&gt;
&lt;li&gt;Windows: Credential Manager&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;On a desktop this is invisible. Your login session starts a session bus and unlocks the keyring, so reads and writes just work.&lt;/p&gt;

&lt;p&gt;Over SSH as root on a cloud image, none of that exists. No session bus means &lt;code&gt;DBUS_SESSION_BUS_ADDRESS&lt;/code&gt; is empty. No daemon means nothing is registered on the bus. And even when gnome-keyring is installed, it is usually locked, because unlocking happens through PAM at a graphical login that never took place.&lt;/p&gt;

&lt;p&gt;The part that wastes an evening: when the keyring call throws, most CLIs catch the exception and print the same message they print for a missing token. "You are not logged in." You are logged in. The file is there. The process just cannot reach its credential store. I have seen that exact line in agent CLI logs sitting next to a token file with a valid future expiry.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three commands to confirm it
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$DBUS_SESSION_BUS_ADDRESS&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;     &lt;span class="c"&gt;# empty means no session bus&lt;/span&gt;
secret-tool lookup service &lt;span class="nb"&gt;test&lt;/span&gt;      &lt;span class="c"&gt;# not found / cannot connect means no secret service&lt;/span&gt;
timedatectl status                   &lt;span class="c"&gt;# check the clock while you are here&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If the first is empty and the second fails, you have found your problem.&lt;/p&gt;

&lt;h2&gt;
  
  
  The fix
&lt;/h2&gt;

&lt;p&gt;Install the pieces, then give the CLI a session bus with an unlocked keyring:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;sudo &lt;/span&gt;apt update
&lt;span class="nb"&gt;sudo &lt;/span&gt;apt &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nt"&gt;-y&lt;/span&gt; dbus-x11 gnome-keyring libsecret-tools
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Package names vary by distro, so a quick &lt;code&gt;apt-cache search secret-tool&lt;/code&gt; (or your distro's equivalent) is worth doing before you trust that list. On Fedora and RHEL the packages are &lt;code&gt;dbus-x11&lt;/code&gt;, &lt;code&gt;gnome-keyring&lt;/code&gt; and &lt;code&gt;libsecret&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Then run the login inside a private session bus, with the keyring unlocked from stdin:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;dbus-run-session &lt;span class="nt"&gt;--&lt;/span&gt; bash &lt;span class="nt"&gt;-c&lt;/span&gt; &lt;span class="s1"&gt;'
  printf %s "$KEYRING_PW" | gnome-keyring-daemon --unlock --components=secrets
  exec your-cli login
'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;dbus-run-session&lt;/code&gt; starts a bus for that one command tree. &lt;code&gt;gnome-keyring-daemon --unlock&lt;/code&gt; reads the password from stdin and exports &lt;code&gt;GNOME_KEYRING_CONTROL&lt;/code&gt; and &lt;code&gt;GNOME_KEYRING_PID&lt;/code&gt; for its children. The CLI then finds a working secret service, stores the token, and stops pretending you never logged in.&lt;/p&gt;

&lt;p&gt;Confirm the store is reachable:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;printf&lt;/span&gt; &lt;span class="s1"&gt;'hunter2'&lt;/span&gt; | secret-tool store &lt;span class="nt"&gt;--label&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nb"&gt;test &lt;/span&gt;svc key
secret-tool lookup svc key
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If lookup prints the value back, you are good.&lt;/p&gt;

&lt;p&gt;Two caveats I would rather say out loud. Running as root is the worst case: root's session does not get a PAM keyring unlock, &lt;code&gt;/run/user/0&lt;/code&gt; may have no bus at all, and some backends refuse root outright. Where you can, create a normal user, run &lt;code&gt;loginctl enable-linger &amp;lt;user&amp;gt;&lt;/code&gt;, and do the login as that user, so a systemd user session keeps a bus alive between SSH sessions. On a long-lived server you want that bus to outlive your SSH connection, otherwise the next cron run starts from scratch.&lt;/p&gt;

&lt;p&gt;If the host is disposable and you only need it working now, a keyring created with an empty password is an option. It is still a secret store, but it is one that anyone with the file can open. I would treat it as a last resort.&lt;/p&gt;

&lt;h2&gt;
  
  
  The other thing that breaks login right after a login: the clock
&lt;/h2&gt;

&lt;p&gt;Different cause, same symptom family. Refresh tokens are validated against server time. A VM or container whose clock is minutes off gets &lt;code&gt;invalid_grant&lt;/code&gt; on refresh even with a perfectly good keyring, which looks exactly like a successful OAuth login followed by an immediate auth failure. There is a gemini-cli issue on Ubuntu reporting &lt;code&gt;invalid_grant&lt;/code&gt; with exit code 41, where clock skew and stale credentials are among the usual suspects.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;timedatectl status
&lt;span class="nb"&gt;sudo &lt;/span&gt;timedatectl set-ntp &lt;span class="nb"&gt;true&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Sometimes there is a token env var and you can skip all of this
&lt;/h2&gt;

&lt;p&gt;Headless auth is common enough that plenty of tools ship an escape hatch:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;gh&lt;/code&gt;: &lt;code&gt;GH_TOKEN&lt;/code&gt; bypasses keyring storage entirely&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;gcloud&lt;/code&gt;: &lt;code&gt;--no-launch-browser&lt;/code&gt;, with &lt;code&gt;CLOUDSDK_CONFIG&lt;/code&gt; controlling where credentials live&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;docker&lt;/code&gt;: credential helpers, or a plain config file&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Thirty seconds with &lt;code&gt;your-cli --help | grep -i token&lt;/code&gt; is worth it before you install a keyring stack. If the tool documents a file-based credential store, use it instead.&lt;/p&gt;

&lt;h2&gt;
  
  
  What not to do
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Do not export an invented &lt;code&gt;DBUS_SESSION_BUS_ADDRESS&lt;/code&gt;. The bus has to exist. Pointing at a socket that is not there gives you a different error and a longer afternoon.&lt;/li&gt;
&lt;li&gt;Do not put the token or the keyring password in argv or a shell env var on a shared box. &lt;code&gt;/proc&lt;/code&gt; and CI logs are not a secret store.&lt;/li&gt;
&lt;li&gt;Do not assume a token file means an authenticated session. That assumption is the whole bug.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;None of this is exotic. It is a desktop assumption leaking into server tooling, and it costs people an evening because the error message points at the wrong thing. Worth a try: read the CLI's own log for the word keyring, secret, or token source before you change anything. If a keyring error is in there, the steps above are the path. If it is not, it may genuinely be a bad credential, and the auth logs are the next stop.&lt;/p&gt;

</description>
      <category>tutorial</category>
      <category>devops</category>
      <category>ai</category>
    </item>
    <item>
      <title>Your Windows app auto-updated and now it won't open: what 0x80073D02 means</title>
      <dc:creator>MilkyWay008</dc:creator>
      <pubDate>Thu, 24 Sep 2026 17:06:19 +0000</pubDate>
      <link>https://dev.to/milkyway008/your-windows-app-auto-updated-and-now-it-wont-open-what-0x80073d02-means-49be</link>
      <guid>https://dev.to/milkyway008/your-windows-app-auto-updated-and-now-it-wont-open-what-0x80073d02-means-49be</guid>
      <description>&lt;p&gt;A few weeks ago an app on my machine updated itself, quit, and then never came back. The desktop icon did nothing. Nothing showed up under Apps in Task Manager. The only clue was a dialog about apps needing to be closed and an error code: 0x80073D02.&lt;/p&gt;

&lt;p&gt;If you've run into this with Claude Desktop, Codex, or any other app that ships as an MSIX package, here's the short version: it's usually not a corrupted install, and reinstalling is often the slowest way out. It's a package swap that didn't finish, and you can usually clear it in a couple of minutes.&lt;/p&gt;

&lt;h2&gt;
  
  
  An MSIX update is a package swap, not a patch
&lt;/h2&gt;

&lt;p&gt;When one of these apps updates, Windows stages the new files, adds the package, then registers it for your user account.&lt;/p&gt;

&lt;p&gt;Staging and adding are file operations. Register is the step that touches your user's state, and it comes with a hard requirement: nothing from the old version can still be running. If something is, the deployment service returns 0x80073D02, which is ERROR_PACKAGES_IN_USE. Microsoft's wording is that the package's resources are "currently in use".&lt;/p&gt;

&lt;p&gt;That's why the failure shows up at the very end of an update that looked fine right up to the last step.&lt;/p&gt;

&lt;h2&gt;
  
  
  The updater quits the app, not everything the app started
&lt;/h2&gt;

&lt;p&gt;The shutdown API Windows gives packaged apps covers the processes belonging to the package. For Claude Desktop, that's the Claude.exe set. It does not cover the helpers the app launched that carry no package identity: a bundled CLI running out of %APPDATA%, node.exe processes from MCP servers, a native messaging host for browser integration, a Python process holding an stdio MCP connection.&lt;/p&gt;

&lt;p&gt;One surviving process is enough to pin the package and abort Register.&lt;/p&gt;

&lt;p&gt;The annoying part: those helpers don't show under the app's name in Task Manager's Processes tab, because that tab groups by app identity. Switch to the Details tab and sort by name.&lt;/p&gt;

&lt;p&gt;Packaged services cause the same thing. Some apps install one, and Claude Desktop's is called CoworkVMService. It's marked AUTO_START, so a repair flow that stops it to free the package gets it back a few seconds later, mid-servicing, and Register fails again. Disabling it isn't an option either: &lt;code&gt;Set-Service -StartupType Disabled&lt;/code&gt; returns Access denied, because the service's DACL only grants that right to the OS deployment service. That one is by design.&lt;/p&gt;

&lt;h2&gt;
  
  
  Finding what's holding the package
&lt;/h2&gt;

&lt;p&gt;Start with the package and where it lives:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Get-AppxPackage&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Name&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"*Claude*"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="n"&gt;Select-Object&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;PackageFullName&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;InstallLocation&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Status&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now the part that matters. &lt;code&gt;Get-Process -Name claude&lt;/code&gt; only matches on name, which is why it misses the leftovers. Match on executable path instead:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Get-CimInstance&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Win32_Process&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="n"&gt;Where-Object&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="bp"&gt;$_&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;ExecutablePath&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;-like&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$&lt;/span&gt;&lt;span class="nn"&gt;env&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="nv"&gt;LOCALAPPDATA&lt;/span&gt;&lt;span class="s2"&gt;\Packages\Claude_*"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="n"&gt;Select-Object&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;ProcessId&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;ExecutablePath&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And check whether a packaged service is in the picture:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Get-Service&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;CoworkVMService&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Select-Object&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;Status&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;StartType&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  The fix, cheapest attempt first
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Kill the leftovers, then relaunch.
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Get-Process&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;claude&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;cowork-svc&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-ErrorAction&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;SilentlyContinue&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Stop-Process&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Force&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If name matching misses (it often does), take the PIDs from the path query above and stop those. Several reporters said this alone sorted it, on the same machine, repeatedly.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Stop the service, kill, re-register the package yourself.
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Stop-Service&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;CoworkVMService&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Force&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-ErrorAction&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;SilentlyContinue&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="n"&gt;Get-Process&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;claude&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;cowork-svc&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-ErrorAction&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;SilentlyContinue&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Stop-Process&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Force&lt;/span&gt;&lt;span class="w"&gt;

&lt;/span&gt;&lt;span class="nv"&gt;$pkg&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Get-AppxPackage&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Name&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"*Claude*"&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="n"&gt;Add-AppxPackage&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-DisableDevelopmentMode&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Register&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nv"&gt;$pkg&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;InstallLocation&lt;/span&gt;&lt;span class="si"&gt;)&lt;/span&gt;&lt;span class="s2"&gt;\AppxManifest.xml"&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's roughly what Settings &amp;gt; Apps &amp;gt; Advanced options &amp;gt; Repair does under the hood, minus the part where AUTO_START brings the service back halfway through. One reporter who hit 0x80073D02 on every repair had this run clean on the first try.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;If the deployment path itself is wedged, reboot and reinstall.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;There's a variant where nothing of yours is holding anything: an orphaned per-package container still has a registry hive open, which surfaces as 0x80070020 (ERROR_SHARING_VIOLATION, 32) and the staged build never deploys. A reboot releases it. If the app is available from the vendor's download page, install it fresh after that reboot instead of fighting the in-app updater.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Last resort: remove and reinstall.
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;Get-AppxPackage&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-Name&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"*Claude*"&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="o"&gt;|&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="n"&gt;Remove-AppxPackage&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Remove-AppxPackage only removes the package for the account(s) you target. A package provisioned for all users needs Remove-AppxProvisionedPackage with elevation, which is a different cmdlet.&lt;/p&gt;

&lt;h2&gt;
  
  
  The other codes in this family
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Code&lt;/th&gt;
&lt;th&gt;Message you'll see&lt;/th&gt;
&lt;th&gt;What it usually means&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;0x80073D02&lt;/td&gt;
&lt;td&gt;apps need to be closed / resources in use&lt;/td&gt;
&lt;td&gt;Something from the old version is still running, often a packaged service&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0x80073CF6&lt;/td&gt;
&lt;td&gt;the package could not be registered&lt;/td&gt;
&lt;td&gt;Stale or duplicate package state; also shows up when the package sits on a non-C: drive&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0x80073CF9&lt;/td&gt;
&lt;td&gt;package installation failed&lt;/td&gt;
&lt;td&gt;Registration failed, and the inner error is the useful one&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0x80073D05&lt;/td&gt;
&lt;td&gt;error deleting previously existing application data&lt;/td&gt;
&lt;td&gt;A half-written install left state behind&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0x80073D28&lt;/td&gt;
&lt;td&gt;administrator privileges required&lt;/td&gt;
&lt;td&gt;A packaged service tried to register without elevation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;0x80070020 (32)&lt;/td&gt;
&lt;td&gt;the file is in use by another process&lt;/td&gt;
&lt;td&gt;A file or registry hive is held open; a reboot releases it&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  You can't run binaries out of WindowsApps
&lt;/h2&gt;

&lt;p&gt;This trips up anyone trying to work around a broken app by calling its bundled CLI directly. Files under C:\Program Files\WindowsApps are owned by the OS (TrustedInstaller) and locked down on purpose, so Access denied from your shell is the expected answer, not a permissions problem you can take ownership of.&lt;/p&gt;

&lt;p&gt;If the app needs an external CLI, install it yourself and point the app at it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="n"&gt;npm.cmd&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;install&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;-g&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;@&lt;/span&gt;&lt;span class="nx"&gt;openai/codex&lt;/span&gt;&lt;span class="err"&gt;@&lt;/span&gt;&lt;span class="nx"&gt;latest&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;Environment&lt;/span&gt;&lt;span class="p"&gt;]::&lt;/span&gt;&lt;span class="n"&gt;SetEnvironmentVariable&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"CODEX_CLI_PATH"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$&lt;/span&gt;&lt;span class="nn"&gt;env&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="nv"&gt;APPDATA&lt;/span&gt;&lt;span class="s2"&gt;\npm\codex.cmd"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"User"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One more drive-related trap, since it caught two reporters in a row: keep the package on C:. If Windows' "Where new content is saved" points at D:, some apps' internal path checks fail and the reinstall dies with 0x80073CF6.&lt;/p&gt;

&lt;h2&gt;
  
  
  Honest limits
&lt;/h2&gt;

&lt;p&gt;I can be wrong about your specific install, and this doesn't cover every case. Two things I couldn't verify well enough to promise:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Sometimes Register succeeds and the app still won't launch. That's the OS servicing path stuck rather than a process pinning the package, and a reboot is the next honest step.&lt;/li&gt;
&lt;li&gt;All of this is per-user. Clearing a packaged service for real may need elevation, and vendor dialogs will cheerfully say "administrator access is required" even when you already ran elevated. Trust the error code over the dialog.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So: when an MSIX app dies after updating itself, don't reinstall right away. Find what's still running under the package path and kill by path, not by name.&lt;/p&gt;

&lt;p&gt;Sources: the reports I leaned on are in anthropics/claude-code issues 89599, 88962, 89108 and 88500, and openai/codex issues 38843 and 40867. Microsoft's AppX troubleshooting page covers the deployment error codes and the read-only package volume.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>devops</category>
      <category>tutorial</category>
      <category>claude</category>
    </item>
    <item>
      <title>Windows won't boot but the disk is fine: rebuilding the EFI bootloader with diskpart and bcdboot</title>
      <dc:creator>MilkyWay008</dc:creator>
      <pubDate>Wed, 23 Sep 2026 17:17:17 +0000</pubDate>
      <link>https://dev.to/milkyway008/windows-wont-boot-but-the-disk-is-fine-rebuilding-the-efi-bootloader-with-diskpart-and-bcdboot-4h50</link>
      <guid>https://dev.to/milkyway008/windows-wont-boot-but-the-disk-is-fine-rebuilding-the-efi-bootloader-with-diskpart-and-bcdboot-4h50</guid>
      <description>&lt;p&gt;Two dead laptops landed on my bench this month with the same story. Power on, vendor logo, then "Boot Device Not Found", or just a black screen. Both drives were healthy, and every file the owners cared about was sitting right there when I mounted the disks from a USB stick.&lt;/p&gt;

&lt;p&gt;Windows didn't lose the data. It lost the small partition that tells the firmware where Windows lives.&lt;/p&gt;

&lt;p&gt;It's a FAT32 partition called the EFI System Partition, and it has no drive letter. Nobody notices until a cloning tool skips it, an image restore drops it, or a resize eats it. Then the machine looks bricked while it's completely fine underneath.&lt;/p&gt;

&lt;p&gt;Here's the fix I use. Fifteen minutes, and nothing you run here touches your files.&lt;/p&gt;

&lt;h2&gt;
  
  
  What actually boots a Windows PC
&lt;/h2&gt;

&lt;p&gt;The firmware doesn't go hunting for Windows. It reads a boot entry out of NVRAM and loads a file called &lt;code&gt;bootmgfw.efi&lt;/code&gt; from the EFI System Partition (ESP for short). Microsoft's partition docs put it plainly: the device must contain a system partition, and on GPT drives that's the ESP, formatted FAT32.&lt;/p&gt;

&lt;p&gt;Two separate things can break: the files inside the ESP, and the firmware's memory of where the ESP is. Most of the "but bcdboot said it worked" cases I've run into are the second one.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three checks before you type anything
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Is the drive visible in firmware setup at all? If your SSD isn't in the BIOS or UEFI storage list, stop. That's hardware, a cable, or a storage-mode change like RAID versus AHCI. Not a bootloader problem, and nothing below will help.&lt;/li&gt;
&lt;li&gt;Is the firmware in UEFI mode or legacy/CSM? A GPT disk won't boot through CSM on its own. Microsoft's Secure Boot guidance says it plainly: to use a CSM you may also need to reformat the drive as MBR and reinstall Windows. If a CMOS battery swap reset your board to legacy mode, you just found your answer.&lt;/li&gt;
&lt;li&gt;Is BitLocker on? Find the recovery key before you touch boot files, because changing them can trigger a recovery prompt on the next boot.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Get to a command prompt in WinRE
&lt;/h2&gt;

&lt;p&gt;Two routes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;From inside Windows: Settings &amp;gt; System &amp;gt; Recovery &amp;gt; Advanced startup &amp;gt; Restart now, then Troubleshoot &amp;gt; Advanced options &amp;gt; Command Prompt. Holding Shift while you click Restart from the power menu also works.&lt;/li&gt;
&lt;li&gt;From a USB stick: boot Windows install media, choose Repair your computer, then Troubleshoot &amp;gt; Advanced options &amp;gt; Command Prompt.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The X: drive you land on is the recovery environment's RAM disk, not your Windows install. Your Windows volume is somewhere else, often with no letter assigned.&lt;/p&gt;

&lt;h2&gt;
  
  
  Find the two partitions you care about
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight batchfile"&gt;&lt;code&gt;&lt;span class="nb"&gt;diskpart&lt;/span&gt;
&lt;span class="kd"&gt;list&lt;/span&gt; &lt;span class="kd"&gt;disk&lt;/span&gt;
&lt;span class="kd"&gt;select&lt;/span&gt; &lt;span class="kd"&gt;disk&lt;/span&gt; &lt;span class="m"&gt;0&lt;/span&gt;
&lt;span class="kd"&gt;list&lt;/span&gt; &lt;span class="kd"&gt;volume&lt;/span&gt;
&lt;span class="k"&gt;exit&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You want two entries: a big NTFS volume (Windows) and a small FAT32 volume with no letter, usually 100 to 300 MB. That small one is your ESP.&lt;/p&gt;

&lt;p&gt;Mount them with letters so the next commands have something to talk to:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight batchfile"&gt;&lt;code&gt;&lt;span class="nb"&gt;diskpart&lt;/span&gt;
&lt;span class="kd"&gt;select&lt;/span&gt; &lt;span class="kd"&gt;volume&lt;/span&gt; &lt;span class="m"&gt;3&lt;/span&gt;
&lt;span class="kd"&gt;assign&lt;/span&gt; &lt;span class="kd"&gt;letter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kd"&gt;S&lt;/span&gt;
&lt;span class="kd"&gt;select&lt;/span&gt; &lt;span class="kd"&gt;volume&lt;/span&gt; &lt;span class="m"&gt;4&lt;/span&gt;
&lt;span class="kd"&gt;assign&lt;/span&gt; &lt;span class="kd"&gt;letter&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kd"&gt;W&lt;/span&gt;
&lt;span class="k"&gt;exit&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Use the volume numbers you actually saw, not mine. Then check what you're looking at before writing:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight batchfile"&gt;&lt;code&gt;&lt;span class="nb"&gt;dir&lt;/span&gt; &lt;span class="kd"&gt;S&lt;/span&gt;:\EFI\Microsoft\Boot
&lt;span class="nb"&gt;dir&lt;/span&gt; &lt;span class="kd"&gt;W&lt;/span&gt;:\Windows\System32
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If &lt;code&gt;dir W:\Windows&lt;/code&gt; answers with File Not Found, you picked the wrong volume. Go back to &lt;code&gt;list volume&lt;/code&gt;. Guessing here is how people end up wiping the wrong partition.&lt;/p&gt;

&lt;h2&gt;
  
  
  Rebuild the boot files
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight batchfile"&gt;&lt;code&gt;&lt;span class="nb"&gt;bcdboot&lt;/span&gt; &lt;span class="kd"&gt;W&lt;/span&gt;:\Windows &lt;span class="na"&gt;/s &lt;/span&gt;&lt;span class="kd"&gt;S&lt;/span&gt;: &lt;span class="na"&gt;/f &lt;/span&gt;&lt;span class="kd"&gt;UEFI&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Expect &lt;code&gt;Boot files successfully created.&lt;/code&gt; That means bcdboot copied a fresh set of boot files from your Windows install onto the ESP and built a new BCD store there. It's the documented repair path: Microsoft describes bcdboot as the tool for repairing a corrupted system partition by recreating those files from the Windows partition.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why it can still refuse to boot
&lt;/h2&gt;

&lt;p&gt;This is the detail I wish someone had handed me years ago. Per the bcdboot docs: by default, bcdboot adds a firmware entry in NVRAM pointing at the Windows Boot Manager and puts it first in the boot list. But if you use the &lt;code&gt;/s&lt;/code&gt; option, that NVRAM entry is not created. bcdboot then relies on the firmware's default behavior, which by the UEFI 2.3.1 spec should open &lt;code&gt;\efi\boot\bootx64.efi&lt;/code&gt; from the ESP.&lt;/p&gt;

&lt;p&gt;So bcdboot can print success while your firmware still points at a stale entry from an old disk or a cloning tool. Two things fix that.&lt;/p&gt;

&lt;p&gt;First, give the firmware the fallback path it's looking for. Your ESP is already mounted:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight batchfile"&gt;&lt;code&gt;&lt;span class="nb"&gt;mkdir&lt;/span&gt; &lt;span class="kd"&gt;S&lt;/span&gt;:\EFI\Boot
&lt;span class="nb"&gt;copy&lt;/span&gt; &lt;span class="kd"&gt;S&lt;/span&gt;:\EFI\Microsoft\Boot\bootmgfw.efi &lt;span class="kd"&gt;S&lt;/span&gt;:\EFI\Boot\bootx64.efi
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then reboot into firmware setup and move "Windows Boot Manager" to the top of the boot order. From WinRE, &lt;code&gt;bcdedit /enum firmware&lt;/code&gt; shows the firmware entries, and running bcdboot again without &lt;code&gt;/s&lt;/code&gt; lets it create the NVRAM entry itself. Be careful with that one if the machine has more than one ESP, since without &lt;code&gt;/s&lt;/code&gt; you're trusting it to pick the right disk.&lt;/p&gt;

&lt;p&gt;If Secure Boot is in the picture, test one boot with it disabled to see whether the bootloader is even the problem, then turn it back on. Don't leave it off.&lt;/p&gt;

&lt;h2&gt;
  
  
  When the BCD store itself is missing
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight batchfile"&gt;&lt;code&gt;&lt;span class="kd"&gt;bootrec&lt;/span&gt; &lt;span class="na"&gt;/scanos
&lt;/span&gt;&lt;span class="kd"&gt;bootrec&lt;/span&gt; &lt;span class="na"&gt;/rebuildbcd
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;/scanos&lt;/code&gt; looks for Windows installations, &lt;code&gt;/rebuildbcd&lt;/code&gt; builds a store from whatever it finds. The older &lt;code&gt;/fixboot&lt;/code&gt; often just answers "Access is denied" on UEFI machines. I can't point you to an official page for that one, it's what I keep seeing, and my read is there's no legacy boot sector left for it to rewrite. Once bcdboot has written the ESP files you usually don't need bootrec.&lt;/p&gt;

&lt;h2&gt;
  
  
  What can't hurt your data, and what can
&lt;/h2&gt;

&lt;p&gt;Safe, because none of these writes to your files: &lt;code&gt;diskpart list&lt;/code&gt;, &lt;code&gt;select&lt;/code&gt;, &lt;code&gt;assign&lt;/code&gt;, &lt;code&gt;bcdboot&lt;/code&gt;, &lt;code&gt;bootrec&lt;/code&gt;, &lt;code&gt;bcdedit&lt;/code&gt;. Assigning a letter just puts a name on a volume that already exists.&lt;/p&gt;

&lt;p&gt;Never a first move: &lt;code&gt;diskpart clean&lt;/code&gt;, &lt;code&gt;format&lt;/code&gt;, &lt;code&gt;convert gpt&lt;/code&gt;, deleting volumes. If the drive is clicking, or the files matter more than the machine boots, clone it and repair the clone.&lt;/p&gt;

&lt;h2&gt;
  
  
  The short version
&lt;/h2&gt;

&lt;p&gt;USB stick, command prompt, &lt;code&gt;diskpart&lt;/code&gt; to find the FAT32 ESP and the Windows volume, &lt;code&gt;bcdboot W:\Windows /s S: /f UEFI&lt;/code&gt;, copy &lt;code&gt;bootmgfw.efi&lt;/code&gt; to &lt;code&gt;\EFI\Boot\bootx64.efi&lt;/code&gt;, set the boot order. That's the whole job.&lt;/p&gt;

&lt;p&gt;I've only done this on straight UEFI and GPT single-disk machines, so your mileage will vary on RAID, Storage Spaces, or a legacy/MBR setup. And if the firmware can't see the drive at all, the hardware comes first. But for the "bricked" laptop with a perfectly good disk...... this one has saved me a reinstall more than once.&lt;/p&gt;

</description>
      <category>tutorial</category>
      <category>devops</category>
      <category>productivity</category>
    </item>
    <item>
      <title>uv lock says "No solution found": how to read the dependency deadlock and break it</title>
      <dc:creator>MilkyWay008</dc:creator>
      <pubDate>Tue, 22 Sep 2026 17:07:21 +0000</pubDate>
      <link>https://dev.to/milkyway008/uv-lock-says-no-solution-found-how-to-read-the-dependency-deadlock-and-break-it-3g9h</link>
      <guid>https://dev.to/milkyway008/uv-lock-says-no-solution-found-how-to-read-the-dependency-deadlock-and-break-it-3g9h</guid>
      <description>&lt;p&gt;You add one library, run &lt;code&gt;uv lock&lt;/code&gt;, and the resolver prints 40 lines of "Because ... And because ..." and quits with a non-zero exit code. You did not change any pins. The project was fine yesterday.&lt;/p&gt;

&lt;p&gt;I have run into this class of failure a few times and it usually ends the same way: the resolver is right, and one package in the graph has a stale upper bound.&lt;/p&gt;

&lt;h2&gt;
  
  
  What that wall of text is saying
&lt;/h2&gt;

&lt;p&gt;The first line is the one that matters:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;× No solution found when resolving dependencies:
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Everything after it is a derivation chain. Each line is a requirement read out of some package's metadata, and the chain ends where two of them want different things.&lt;/p&gt;

&lt;p&gt;A real example, from a LangGraph issue (&lt;a href="https://github.com/langchain-ai/langgraph/issues/8352" rel="noopener noreferrer"&gt;langchain-ai/langgraph#8352&lt;/a&gt;): &lt;code&gt;langgraph-api&lt;/code&gt; 0.11.0 pins &lt;code&gt;opentelemetry-exporter-prometheus&amp;lt;0.59&lt;/code&gt;, which forces &lt;code&gt;opentelemetry-sdk&lt;/code&gt; into an old range. Meanwhile &lt;code&gt;pydantic-ai&lt;/code&gt; 2.x pulls &lt;code&gt;logfire&amp;gt;=4.35.0&lt;/code&gt;, which wants &lt;code&gt;opentelemetry-sdk&amp;gt;=1.39.0,&amp;lt;1.43.0&lt;/code&gt;. Those two ranges never overlap, so no set of versions satisfies both.&lt;/p&gt;

&lt;p&gt;Nobody is being unreasonable here. That 0.59 bound was probably correct when someone wrote it...... it just never got revisited. The person who filed the issue could not fix it either, because the metadata ships from a private source tree, so only the maintainers can widen the pin.&lt;/p&gt;

&lt;p&gt;Two commands make the chain readable:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;uv lock &lt;span class="nt"&gt;-v&lt;/span&gt;                            &lt;span class="c"&gt;# full derivation, as the resolver sees it&lt;/span&gt;
uv tree &lt;span class="nt"&gt;--invert&lt;/span&gt; opentelemetry-sdk    &lt;span class="c"&gt;# who wants which version&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Read the chain bottom-up, find the two packages that disagree, ignore the middle.&lt;/p&gt;

&lt;h2&gt;
  
  
  The silent downgrade is worse than the error
&lt;/h2&gt;

&lt;p&gt;In that same issue the resolver could quietly settle on a much older &lt;code&gt;langgraph-api&lt;/code&gt; (0.7.27 was observed), and &lt;code&gt;langgraph dev&lt;/code&gt; fell back to an older build without complaining. The app boots, on code you did not ask for, and the Studio upgrade banner stays unsatisfiable.&lt;/p&gt;

&lt;p&gt;That one costs people days, because nothing looks broken. A successful resolution is not proof you got what you wanted. Check the installed version, not the requested one.&lt;/p&gt;

&lt;h2&gt;
  
  
  Constraints cannot fix a deadlock (this is the part people get wrong)
&lt;/h2&gt;

&lt;p&gt;uv has two settings for this, both in the workspace-root &lt;code&gt;pyproject.toml&lt;/code&gt; under &lt;code&gt;[tool.uv]&lt;/code&gt;. They look similar and behave nothing alike.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;constraint-dependencies&lt;/code&gt; is additive. The docs describe constraints as restricting "the versions of dependencies that are selected during resolution", so they get intersected with whatever the packages already declared.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;override-dependencies&lt;/code&gt; is absolute. The docs say overrides "force selection of a specific version of a package, regardless of the version requested by any other package, and regardless of whether choosing that version would typically constitute an invalid resolution". The resolution docs call overrides the escape hatch for erroneous upper version bounds.&lt;/p&gt;

&lt;p&gt;So when the ranges are disjoint, a constraint can never save you. Constraints only narrow the space, and the space is already empty. This matters because the reflex is to reach for a constraint when the real tool is an override.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight toml"&gt;&lt;code&gt;&lt;span class="nn"&gt;[tool.uv]&lt;/span&gt;
&lt;span class="c"&gt;# stops the resolver from silently picking something ancient&lt;/span&gt;
&lt;span class="py"&gt;constraint-dependencies&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="py"&gt;["langgraph-api&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;0.10&lt;/span&gt;&lt;span class="err"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="s"&gt;"]&lt;/span&gt;&lt;span class="err"&gt;
&lt;/span&gt;
&lt;span class="c"&gt;# the deadlock breaker: deliberately contradicts a declared upper bound&lt;/span&gt;
&lt;span class="py"&gt;override-dependencies&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="py"&gt;["opentelemetry-exporter-prometheus&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;0.63&lt;/span&gt;&lt;span class="err"&gt;b&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="s"&gt;"]&lt;/span&gt;&lt;span class="err"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One placement gotcha: uv reads these from the workspace root only. Putting them in &lt;code&gt;uv.toml&lt;/code&gt;, or in a member package's pyproject, does nothing.&lt;/p&gt;

&lt;h2&gt;
  
  
  The order I would try things
&lt;/h2&gt;

&lt;p&gt;1st, pin the thing that moved back to the last version that worked. Boring, safe, usually enough to get moving today.&lt;/p&gt;

&lt;p&gt;2nd, add a floor constraint on whatever was being silently downgraded, so you find out about it instead of living with it.&lt;/p&gt;

&lt;p&gt;3rd, override the stale bound, then actually run the app. &lt;code&gt;uv lock&lt;/code&gt; exiting 0 only proves the graph resolves. The package that asked for the old SDK is now untested territory.&lt;/p&gt;

&lt;p&gt;4th, if the two libraries genuinely cannot coexist, give them separate environments. Ugly, but honest, and sometimes correct.&lt;/p&gt;

&lt;h2&gt;
  
  
  Leave a trail
&lt;/h2&gt;

&lt;p&gt;Keep the override in the file with a comment, a link to the upstream issue, and the date you added it. An override without a note turns into permanent mystery debt six months later. In this case the pin was widened on the 0.13.0 dev line (&lt;code&gt;&amp;lt;0.64,&amp;gt;=0.63b1&lt;/code&gt;, landing the SDK at 1.42.1) while stable still carried the old bound, so the plan is: wait for the release, then delete the block.&lt;/p&gt;

&lt;p&gt;If you use pip instead of uv, the constraint file (&lt;code&gt;pip install --dry-run -c constraints.txt&lt;/code&gt;) is additive in the same way, so it will not break a disjoint range either. Worth knowing before spending an hour on it.&lt;/p&gt;

&lt;p&gt;I can be wrong about your particular graph, but from what I have seen the fix is rarely a clever resolver flag. It is reading the chain, working out which bound is stale, and forcing that one thing while you wait for upstream...... then writing down why.&lt;/p&gt;

&lt;p&gt;One honest gap: I could not independently confirm whether stable &lt;code&gt;langgraph-api&lt;/code&gt; still carries &lt;code&gt;&amp;lt;0.59&lt;/code&gt; at the time of writing. The issue is open. Check it before copying the override.&lt;/p&gt;

</description>
      <category>python</category>
      <category>devops</category>
      <category>tutorial</category>
      <category>ai</category>
    </item>
    <item>
      <title>HTTP 400 from your own LLM endpoint? Check for content: null</title>
      <dc:creator>MilkyWay008</dc:creator>
      <pubDate>Mon, 21 Sep 2026 17:28:55 +0000</pubDate>
      <link>https://dev.to/milkyway008/http-400-from-your-own-llm-endpoint-check-for-content-null-1o46</link>
      <guid>https://dev.to/milkyway008/http-400-from-your-own-llm-endpoint-check-for-content-null-1o46</guid>
      <description>&lt;p&gt;Someone I was helping had a tidy little setup: a self-hosted OpenAI-compatible endpoint in front of their model, a coding agent pointed at it, months of smooth sailing. Then they bumped the agent from 7.3.42 to 7.3.44 and every request came back HTTP 400. Same endpoint, same key, same model. Only the client version changed.&lt;/p&gt;

&lt;p&gt;The error body was short, and it looked roughly like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="mi"&gt;400&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;Bad&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;Request&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"error"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"message"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Type validation failed"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
           &lt;/span&gt;&lt;span class="nl"&gt;"path"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"/messages/2/content"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
           &lt;/span&gt;&lt;span class="nl"&gt;"keyword"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"invalid_union"&lt;/span&gt;&lt;span class="p"&gt;}}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If you run a gateway in front of a local model (vLLM, LiteLLM, llama.cpp, Open WebUI, LM Studio, or your own FastAPI shim), this one will find you eventually. The error already tells you where to look, and the fix is usually a couple of lines on whichever side of the wire you control.&lt;/p&gt;

&lt;h2&gt;
  
  
  The field it names is the whole clue
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;/messages/2/content&lt;/code&gt; means the third message in the array, its &lt;code&gt;content&lt;/code&gt; field, and a union check that failed. Validators generated from a schema (pydantic, zod, ajv and friends) hand you the JSON path for free. Read the path first, then read the raw request, and stop guessing about the model.&lt;/p&gt;

&lt;p&gt;Log the outgoing body at the gateway (raw JSON, not a redacted summary), find that index, and look at the message. In this case it was an assistant message with tool calls and no text:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"role"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"assistant"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"content"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"tool_calls"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"call_1"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"function"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
     &lt;/span&gt;&lt;span class="nl"&gt;"function"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"read_file"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"arguments"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"{&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;path&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;: &lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;a.txt&lt;/span&gt;&lt;span class="se"&gt;\"&lt;/span&gt;&lt;span class="s2"&gt;}"&lt;/span&gt;&lt;span class="p"&gt;}}&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Nothing looks broken there. That's the trap.&lt;/p&gt;

&lt;h2&gt;
  
  
  What changed, and why neither side is crazy
&lt;/h2&gt;

&lt;p&gt;On the client side, Vercel's AI SDK changed tool-only assistant messages from &lt;code&gt;content: ""&lt;/code&gt; to &lt;code&gt;content: null&lt;/code&gt; on purpose. The commit message spells out why (&lt;a href="https://github.com/vercel/ai/commit/bfb756d839d435e3fd0afa22135c59d4587c79f2" rel="noopener noreferrer"&gt;vercel/ai@bfb756d8&lt;/a&gt;, April 2026): providers backed by AWS Bedrock reject an empty text block with &lt;code&gt;ValidationException: text content blocks must be non-empty&lt;/code&gt;, so the serializer became &lt;code&gt;content: text || null&lt;/code&gt;. A lot of coding agents sit on top of that SDK, which is why the change showed up in several places at once.&lt;/p&gt;

&lt;p&gt;On the server side, the endpoint's validator insists that an assistant &lt;code&gt;content&lt;/code&gt; is a string. Null isn't a string. Union check fails, request dies.&lt;/p&gt;

&lt;p&gt;OpenAI's own schema allows null, though. In the &lt;a href="https://github.com/openai/openai-openapi/blob/master/openapi.yaml" rel="noopener noreferrer"&gt;OpenAI OpenAPI spec&lt;/a&gt;, the assistant message defines &lt;code&gt;content&lt;/code&gt; as &lt;code&gt;string | array of parts | null&lt;/code&gt;, described as "The contents of the assistant message. Required unless &lt;code&gt;tool_calls&lt;/code&gt; or &lt;code&gt;function_call&lt;/code&gt; is specified." The only required property on that object is &lt;code&gt;role&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;So &lt;code&gt;content: null&lt;/code&gt; on a tool-call-only assistant message is legal, and the validator is the piece that's out of step with the protocol. "My schema says no" is a different sentence from "the spec says no."&lt;/p&gt;

&lt;h2&gt;
  
  
  Fixing it
&lt;/h2&gt;

&lt;p&gt;If you own the endpoint, normalize at the edge. Accept null when tool calls are present, and rewrite it into something your backend tolerates. A small piece of Starlette middleware is enough:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;starlette.middleware.base&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;BaseHTTPMiddleware&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;normalize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;m&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;messages&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[]):&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;role&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;assistant&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;tool_calls&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;content&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;m&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;content&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;""&lt;/span&gt;   &lt;span class="c1"&gt;# or " " if your backend also dislikes empty strings
&lt;/span&gt;    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;body&lt;/span&gt;

&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;NullContentFix&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;BaseHTTPMiddleware&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;dispatch&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;call_next&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;endswith&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;/chat/completions&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
            &lt;span class="n"&gt;payload&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
            &lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_body&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dumps&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;normalize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;)).&lt;/span&gt;&lt;span class="nf"&gt;encode&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nf"&gt;call_next&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That comment about empty strings isn't hypothetical. Continue's OpenAI converter does &lt;code&gt;message.content || " "&lt;/code&gt; with the literal comment "LM Studio (and other providers) don't accept empty content." Different clients picked different workarounds for the same edge case, which is a decent hint about how long this has been rattling around. If your backend is picky about empty strings too, coerce to a single space.&lt;/p&gt;

&lt;p&gt;If you don't own the endpoint, pin the client. The actual difference between working and broken was &lt;code&gt;@ai-sdk/openai-compatible&lt;/code&gt; 2.0.41 versus 2.0.48. A sibling tool still sitting on 2.0.41 was reported working against the same endpoint, which is a reminder that "it works in tool X" says nothing about your config. Pin it in the lockfile, or put your own proxy in the middle to rewrite null.&lt;/p&gt;

&lt;p&gt;And don't wait on the upstream fix. A normalizing pull request for the client was opened and closed without merging (&lt;a href="https://github.com/Kilo-Org/kilocode/issues/12331" rel="noopener noreferrer"&gt;kilocode#12331&lt;/a&gt;; the original report is &lt;a href="https://github.com/Kilo-Org/kilocode/issues/12330" rel="noopener noreferrer"&gt;kilocode#12330&lt;/a&gt;), so as of today it's on you to pin or normalize.&lt;/p&gt;

&lt;h2&gt;
  
  
  The same 400 in other disguises
&lt;/h2&gt;

&lt;p&gt;Once you've seen one message-shape mismatch, you spot them everywhere. The ones I keep running into:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Reasoning blocks that don't survive a round trip. Open WebUI dropped the Anthropic thinking-block signature when saving &lt;code&gt;reasoning_details&lt;/code&gt;, then replayed the unsigned block on the next turn. Anthropic rejects it with &lt;code&gt;Invalid signature in thinking block&lt;/code&gt;, so turn 2 dies every time, identically across five providers (&lt;a href="https://github.com/open-webui/open-webui/issues/27467" rel="noopener noreferrer"&gt;open-webui#27467&lt;/a&gt;). The fix PR is closed without merging, so stripping unsigned reasoning from prior assistant turns in a proxy is the workaround for now.&lt;/li&gt;
&lt;li&gt;Streaming reasoning where &lt;code&gt;delta.content&lt;/code&gt; isn't a string. n8n 2.36.8 with the Mistral Chat Model: content arrives as an array of thinking blocks, the node's schema wants a string, and you get &lt;code&gt;invalid_union&lt;/code&gt;. Routing through a proxy that strips thinking deltas worked (&lt;a href="https://github.com/n8n-io/n8n/issues/37352" rel="noopener noreferrer"&gt;n8n#37352&lt;/a&gt;, which n8n closed on Sep 8).&lt;/li&gt;
&lt;li&gt;Extra parameters the client started injecting. One client began sending &lt;code&gt;prompt_cache_breakpoint&lt;/code&gt; to a custom Responses endpoint and got &lt;code&gt;400 invalid_parameter&lt;/code&gt; on every request, reproducing on a clean VM (&lt;a href="https://github.com/Kilo-Org/kilocode/issues/13285" rel="noopener noreferrer"&gt;kilocode#13285&lt;/a&gt;). Strip the unknown field at the proxy, or pin the client back.&lt;/li&gt;
&lt;li&gt;Role remapping that breaks the model's chat template. LM Studio maps each developer message to its own system message, so the second one lands mid-prompt and Qwen's Jinja template throws &lt;code&gt;System message must be at the beginning&lt;/code&gt; (&lt;a href="https://github.com/lmstudio-ai/lmstudio-bug-tracker/issues/2298" rel="noopener noreferrer"&gt;lmstudio#2298&lt;/a&gt;). Consolidate developer messages client-side, or fall back to &lt;code&gt;/v1/chat/completions&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The habit that saves the evening
&lt;/h2&gt;

&lt;p&gt;When a proxy or router sits in the middle, the 400 is often produced by the translation layer, not the model. Test with tools disabled, try the same model through a different route, and read the JSON path in the error before touching anything else.&lt;/p&gt;

&lt;p&gt;Then decide who owns the field, fix the side you control, and pin the version of the thing that changed instead of the whole stack.&lt;/p&gt;

&lt;p&gt;And if you happen to be the one writing the validator: accept null on assistant messages that carry tool calls. It's in the spec.&lt;/p&gt;

&lt;p&gt;I've only hit this on a handful of stacks, so treat it as a starting point rather than gospel, and test it against your own gateway. I keep notes on this class of failure in a small KB repo, Windows-flavoured more often than not: &lt;a href="https://github.com/MilkyWay008/hermes-kb-hack-fix" rel="noopener noreferrer"&gt;hermes-kb-hack-fix&lt;/a&gt;. Hope it saves you an evening.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>tutorial</category>
      <category>devops</category>
    </item>
  </channel>
</rss>
