<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: David Yan</title>
    <description>The latest articles on DEV Community by David Yan (@bailorgana).</description>
    <link>https://dev.to/bailorgana</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3853023%2Fc23b54e7-fcb1-410b-98a6-d42a25027f2b.png</url>
      <title>DEV Community: David Yan</title>
      <link>https://dev.to/bailorgana</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/bailorgana"/>
    <language>en</language>
    <item>
      <title>Why ChatGPT/Codex Desktop Had No Window After an Update on Windows</title>
      <dc:creator>David Yan</dc:creator>
      <pubDate>Sun, 06 Sep 2026 09:41:03 +0000</pubDate>
      <link>https://dev.to/bailorgana/why-chatgptcodex-desktop-had-no-window-after-an-update-on-windows-4cn7</link>
      <guid>https://dev.to/bailorgana/why-chatgptcodex-desktop-had-no-window-after-an-update-on-windows-4cn7</guid>
      <description>&lt;p&gt;After a Windows update, ChatGPT/Codex Desktop may appear to start without ever showing a window. Task Manager shows several responsive &lt;code&gt;ChatGPT.exe&lt;/code&gt; processes, while &lt;code&gt;%LOCALAPPDATA%\OpenAI\Codex\runtimes\cua_node&lt;/code&gt; fills with &lt;code&gt;.staging-*&lt;/code&gt; directories.&lt;/p&gt;

&lt;p&gt;It is tempting to blame a crashed Electron renderer, corrupted user data, or broken MSIX permissions. In this investigation, none of those was the root cause. The application was synchronously copying thousands of Application Protected files out of WindowsApps before creating its UI. A normal file copy failed for each protected source file, and the client then retried that file through a slower byte-stream fallback.&lt;/p&gt;

&lt;p&gt;The short version is: &lt;strong&gt;Windows' MSIX protection was working as designed; the defect was in Codex Desktop's runtime relocation strategy and startup sequencing.&lt;/strong&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Environment: &lt;code&gt;OpenAI.Codex 26.901.6511.0&lt;/code&gt; on Windows 10 Enterprise build 19045. All machine names, user names, and SIDs in this article are anonymized.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Symptoms
&lt;/h2&gt;

&lt;p&gt;The post-update behavior combined several unusual signals:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Clicking the app produced no visible UI.&lt;/li&gt;
&lt;li&gt;Task Manager showed roughly five or six &lt;code&gt;ChatGPT.exe&lt;/code&gt; processes.&lt;/li&gt;
&lt;li&gt;The processes reported &lt;code&gt;Responding=True&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;MainWindowHandle&lt;/code&gt; was zero.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;EnumWindows&lt;/code&gt; found no matching top-level window.&lt;/li&gt;
&lt;li&gt;Every launch created another &lt;code&gt;.staging-&amp;lt;hash&amp;gt;-&amp;lt;random&amp;gt;&lt;/code&gt; directory.&lt;/li&gt;
&lt;li&gt;A staging directory contained &lt;code&gt;bin\node.exe&lt;/code&gt;, but not &lt;code&gt;bin\node_repl.exe&lt;/code&gt; or &lt;code&gt;manifest.json&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;After a long delay, the UI would sometimes appear without any other intervention.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That last observation mattered most. The application was not consistently crashing; it was blocked in a long startup phase that eventually completed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Start with evidence, not destructive repair
&lt;/h2&gt;

&lt;p&gt;The investigation deliberately avoided:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;uninstalling or reinstalling the app;&lt;/li&gt;
&lt;li&gt;resetting application data;&lt;/li&gt;
&lt;li&gt;deleting the whole &lt;code&gt;%LOCALAPPDATA%\OpenAI\Codex&lt;/code&gt; tree;&lt;/li&gt;
&lt;li&gt;taking ownership of WindowsApps;&lt;/li&gt;
&lt;li&gt;granting permissions to &lt;code&gt;Everyone&lt;/code&gt;;&lt;/li&gt;
&lt;li&gt;disabling Windows security features;&lt;/li&gt;
&lt;li&gt;deleting staging directories while they were still changing.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Microsoft documents WindowsApps as a protected MSIX package location. Installed package files are read-only at runtime, while application state is stored separately. That boundary is expected and should not be bypassed: &lt;a href="https://learn.microsoft.com/en-us/windows/msix/msix-containerization-overview" rel="noopener noreferrer"&gt;MSIX containerization overview&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Package and update state
&lt;/h2&gt;

&lt;p&gt;The installed package was:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Name: OpenAI.Codex
Version: 26.901.6511.0
PackageFamilyName: OpenAI.Codex_2p2nqsd0c76g0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Its apparent install path was:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;C:\Program Files\WindowsApps\OpenAI.Codex_26.901.6511.0_x64__2p2nqsd0c76g0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That directory was a junction to the real package volume:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;D:\WindowsApps\OpenAI.Codex_26.901.6511.0_x64__2p2nqsd0c76g0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The package manifest declared a full-trust desktop application and excluded &lt;code&gt;%LOCALAPPDATA%\OpenAI&lt;/code&gt; from filesystem virtualization. The startup path therefore involved a real relocation:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;D:\WindowsApps\...\app\resources\cua_node
    ↓ runtime relocation
%LOCALAPPDATA%\OpenAI\Codex\runtimes\cua_node\&amp;lt;hash&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Microsoft Store telemetry showed the update completing successfully with &lt;code&gt;HResult=0&lt;/code&gt;. Application, AppXDeployment, and AppModel logs contained no OpenAI.Codex deployment or activation errors, so AppX re-registration was not justified.&lt;/p&gt;

&lt;h2&gt;
  
  
  An incomplete staging folder is not automatically corrupt
&lt;/h2&gt;

&lt;p&gt;Inspection of the installed &lt;code&gt;app.asar&lt;/code&gt; showed a relocation sequence equivalent to this pseudocode:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;staging&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;mkdtemp&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;runtimeRoot&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;/.staging-&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;hash&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;-`&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="nf"&gt;copyDirectoryRecursively&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;sourceRuntime&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;staging&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="nf"&gt;renameSync&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;staging&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;finalHashDirectory&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="nf"&gt;cleanupOldRuntimeAndMatchingStaging&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The copy is depth-first. &lt;code&gt;node.exe&lt;/code&gt; is encountered relatively early, while &lt;code&gt;node_repl.exe&lt;/code&gt;, npm launchers, and the root &lt;code&gt;manifest.json&lt;/code&gt; arrive near the end.&lt;/p&gt;

&lt;p&gt;This snapshot alone does not prove corruption:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;bin\node.exe       exists
bin\node_repl.exe  missing
manifest.json       missing
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The correct questions are:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Is the file count still increasing?&lt;/li&gt;
&lt;li&gt;Is the staging directory's modification time moving?&lt;/li&gt;
&lt;li&gt;Is the main process still alive?&lt;/li&gt;
&lt;li&gt;Is there an explicit relocation error?&lt;/li&gt;
&lt;li&gt;Does the final content-addressed directory eventually appear via rename?&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The decisive evidence: normal copies from WindowsApps failed
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;cipher /c&lt;/code&gt; showed that the packaged executables carried the &lt;code&gt;Encrypted&lt;/code&gt; attribute and that ordinary user code could not retrieve their key information. The relocated files under LocalAppData were normal, unencrypted files.&lt;/p&gt;

&lt;p&gt;A controlled test used the same bundled Node.js runtime to copy the package's 426-byte manifest into an isolated temporary directory:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"copyFileSync"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"error"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"code"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"UNKNOWN"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"errno"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;-4094&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"syscall"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"copyfile"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"ms"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;88&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Reading the source bytes and writing a new destination succeeded immediately:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"readWriteFallback"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"ok"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"ms"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Windows error 6000 maps to:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;The specified file could not be encrypted.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Codex Desktop 26.901.6511.0 already contains a compatibility fallback for both Windows errno 6000 and Node/libuv's &lt;code&gt;UNKNOWN / -4094&lt;/code&gt;. When &lt;code&gt;copyFileSync&lt;/code&gt; fails, it retries with &lt;code&gt;readFileSync&lt;/code&gt; followed by &lt;code&gt;writeFileSync&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The performance bug is the granularity: &lt;strong&gt;every protected file pays for a failed normal copy before the fallback runs.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The runtime contained 4,684 files. At 88 milliseconds of failed-copy overhead per file:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;4,684 × 88 ms ≈ 412 seconds ≈ 6.9 minutes
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Add actual reads and writes, hashing, directory creation, and the final rename, and the measured relocation time was about nine minutes. That matched the delayed UI almost exactly.&lt;/p&gt;

&lt;p&gt;This also ruled out &lt;code&gt;EPERM&lt;/code&gt;. The locally reproduced error was &lt;code&gt;UNKNOWN / -4094&lt;/code&gt;, and byte-stream reads remained successful.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why repeated clicks created multiple staging folders
&lt;/h2&gt;

&lt;p&gt;AppModel events recorded several application activations within a short period. A fresh process could:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;check for the final hash directory;&lt;/li&gt;
&lt;li&gt;see that it was missing or incomplete;&lt;/li&gt;
&lt;li&gt;create a new random staging directory;&lt;/li&gt;
&lt;li&gt;restart the entire 338 MB copy.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The in-memory relocation cache is process-local, and a later process does not resume an incomplete staging directory. Repeated launches or early termination can therefore multiply both I/O and per-file failure overhead.&lt;/p&gt;

&lt;p&gt;The official OpenAI repository contains several closely matching public bug reports:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/openai/codex/issues/42501" rel="noopener noreferrer"&gt;26.901.1978.0: no UI, incomplete staging, and missing node_repl.exe&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/openai/codex/issues/40843" rel="noopener noreferrer"&gt;MSIX relocation failure with UNKNOWN/-4094 and ERROR_ENCRYPTION_FAILED&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/openai/codex/issues/41654" rel="noopener noreferrer"&gt;Repeated Application Protected relocation regressions across updates&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These reports point to the Windows relocation path in the client, not an ordinary ACL problem on one machine.&lt;/p&gt;

&lt;h2&gt;
  
  
  Proving that the runtime finalized correctly
&lt;/h2&gt;

&lt;p&gt;One launch was left running. The runtime wrote files continuously from 16:25:03 until 16:34:17, then atomically appeared as:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;%LOCALAPPDATA%\OpenAI\Codex\runtimes\cua_node\b474a88d5d105afa
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Source and destination were compared:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Check&lt;/th&gt;
&lt;th&gt;Package source&lt;/th&gt;
&lt;th&gt;Final runtime&lt;/th&gt;
&lt;th&gt;Result&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;File count&lt;/td&gt;
&lt;td&gt;4,684&lt;/td&gt;
&lt;td&gt;4,684&lt;/td&gt;
&lt;td&gt;Match&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Total bytes&lt;/td&gt;
&lt;td&gt;337,646,562&lt;/td&gt;
&lt;td&gt;337,646,562&lt;/td&gt;
&lt;td&gt;Match&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Missing relative paths&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;Pass&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Extra relative paths&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;Pass&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Size mismatches&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;Pass&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;SHA-256 hashes also matched for the three files used to identify and validate the runtime:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;manifest.json
bin\node.exe
bin\node_repl.exe
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The manifest described the expected Windows x64 runtime:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"platform"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"windows"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"arch"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"x64"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"target"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"windows-x64"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"node_version"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"24.19.0"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"node_path"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"bin/node.exe"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"node_modules"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"bin/node_modules"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"node_repl_path"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"bin/node_repl.exe"&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Node.js, Node REPL, and the CUA helper were readable or executable, and the important executables had valid signatures. Existing configuration now referenced the current hash for Node, Node REPL, node_modules, and the computer-use helper.&lt;/p&gt;

&lt;p&gt;After finalization, the client removed matching failed staging folders and the previous runtime. No application directory was manually deleted or overwritten.&lt;/p&gt;

&lt;h2&gt;
  
  
  Process-tree verification
&lt;/h2&gt;

&lt;p&gt;Once relocation finished, the process tree progressed beyond the basic Chromium bootstrap processes:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;ChatGPT.exe                 main process
├─ ChatGPT.exe              crashpad / utility / GPU / renderer workers
├─ ChatGPT.exe              network and storage services
└─ codex.exe                app-server
   ├─ node_repl.exe
   ├─ codex-code-mode-host.exe
   └─ codex-command-runner.exe
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Some renderer and utility processes use stricter sandbox tokens, so a separate diagnostic account cannot always read their complete command lines. That is expected process isolation. A visible UI, high-memory renderer children, a running app-server, and an active Node REPL confirmed that both rendering and backend startup were healthy.&lt;/p&gt;

&lt;h2&gt;
  
  
  Cold-start validation
&lt;/h2&gt;

&lt;p&gt;The app was fully exited and launched again:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The new main process started at 17:06:42.&lt;/li&gt;
&lt;li&gt;The Codex app-server started at 17:06:47.&lt;/li&gt;
&lt;li&gt;The UI appeared within the normal startup window.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;cua_node&lt;/code&gt; still contained only &lt;code&gt;b474a88d5d105afa&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;The staging count remained zero.&lt;/li&gt;
&lt;li&gt;The runtime's last-write time remained 16:34:17.&lt;/li&gt;
&lt;li&gt;None of the 4,684 files was recopied.&lt;/li&gt;
&lt;li&gt;Application, AppX, and AppModel error counts remained zero.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This proved that the local state was repaired and that later launches reused the finalized runtime correctly.&lt;/p&gt;

&lt;h2&gt;
  
  
  Interpreting &lt;code&gt;codex doctor&lt;/code&gt;
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;codex doctor&lt;/code&gt; was also run. CLI installation, runtime, disk, Git, and search-tool checks passed.&lt;/p&gt;

&lt;p&gt;Desktop, authentication, and connectivity warnings came from running the command under an isolated diagnostic identity with restricted network access. That identity could not inspect the primary user's AppX registration or private package LocalCache. Those warnings described the diagnostic sandbox, not the running desktop application.&lt;/p&gt;

&lt;p&gt;Diagnostic output must always be interpreted together with its user identity, inherited environment, filesystem access, and network policy.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why reinstalling or changing permissions was the wrong fix
&lt;/h2&gt;

&lt;p&gt;The evidence showed that:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;the package payload was complete;&lt;/li&gt;
&lt;li&gt;the Store update succeeded;&lt;/li&gt;
&lt;li&gt;package and executable signatures were valid;&lt;/li&gt;
&lt;li&gt;LocalAppData permissions were correct;&lt;/li&gt;
&lt;li&gt;there was no &lt;code&gt;EPERM&lt;/code&gt;, code-integrity block, quarantine, or application crash;&lt;/li&gt;
&lt;li&gt;relocation could finish and atomically finalize.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Reinstalling, resetting data, changing WindowsApps ACLs, or disabling security controls would not remove the per-file fallback cost. Those actions could instead erase login state, damage package integrity, or weaken system security.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to do after the next update
&lt;/h2&gt;

&lt;p&gt;If a future release keeps the same &lt;code&gt;cua_node&lt;/code&gt; content hash, the existing runtime will be reused.&lt;/p&gt;

&lt;p&gt;If the hash changes, the first launch may remain slow until OpenAI changes the relocation implementation:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Launch the app once after updating.&lt;/li&gt;
&lt;li&gt;Do not repeatedly click the app icon.&lt;/li&gt;
&lt;li&gt;Do not terminate the process while staging file counts and timestamps are still moving.&lt;/li&gt;
&lt;li&gt;Allow roughly 10–15 minutes for first-time materialization.&lt;/li&gt;
&lt;li&gt;Investigate further only if staging stops changing for an extended period or a concrete error is logged.&lt;/li&gt;
&lt;li&gt;Never delete a content-addressed runtime that has already passed integrity checks.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;A robust client-side fix should:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;use a decrypted-destination copy mode immediately for WindowsApps sources;&lt;/li&gt;
&lt;li&gt;avoid making every file fail once before falling back;&lt;/li&gt;
&lt;li&gt;ship the many small files as an archive and extract it once;&lt;/li&gt;
&lt;li&gt;move materialization off the UI-critical startup path or show progress;&lt;/li&gt;
&lt;li&gt;add an inter-process relocation lock and resumable staging;&lt;/li&gt;
&lt;li&gt;surface the failing file, operation, and Win32 error in the UI.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Final takeaway
&lt;/h2&gt;

&lt;p&gt;The failure looked like “Electron started but never created a renderer.” In reality, the renderer was waiting behind a synchronous runtime relocation that incurred 4,684 Application Protected copy failures.&lt;/p&gt;

&lt;p&gt;No reinstall, application reset, ownership change, or manual runtime overwrite was necessary. Once one relocation completed, Codex created the correct content-addressed directory, cleaned its staging residue, and reused the result on the next cold start.&lt;/p&gt;

&lt;p&gt;For similar incidents, build a timeline before deleting anything: &lt;strong&gt;Did the package update succeed? Is staging still growing? Are the source files protected? What is the actual copy error? Did the final directory appear atomically? Does a cold start reuse it?&lt;/strong&gt; That evidence separates expected Windows security behavior from the client bug that actually needs fixing.&lt;/p&gt;

</description>
      <category>windows</category>
      <category>codex</category>
      <category>debugging</category>
      <category>msix</category>
    </item>
    <item>
      <title>When AI Leaks Internal Tags: Debugging a 3-Layer Streaming Architecture Bug</title>
      <dc:creator>David Yan</dc:creator>
      <pubDate>Fri, 03 Apr 2026 09:17:32 +0000</pubDate>
      <link>https://dev.to/bailorgana/when-ai-leaks-internal-tags-debugging-a-3-layer-streaming-architecture-bug-ig4</link>
      <guid>https://dev.to/bailorgana/when-ai-leaks-internal-tags-debugging-a-3-layer-streaming-architecture-bug-ig4</guid>
      <description>&lt;p&gt;As an SDET testing AI applications, I recently encountered a bizarre issue in the OpenClaw Gateway UI. Instead of normal conversational text, the AI assistant started spitting out raw internal directive tags like &lt;code&gt;[[reply_to:&amp;lt;&lt;/code&gt; and &lt;code&gt;[[reply&lt;/code&gt; directly into the chat interface.&lt;/p&gt;

&lt;p&gt;These tags are designed for internal message routing and should be silently stripped by the system before reaching the user. At first glance, it looked like a simple "dumb LLM" problem. But diving deeper, I uncovered a fascinating architectural trap: a perfect storm of &lt;strong&gt;three distinct bugs across the backend stream, the UI state logic, and the defense-in-depth strategy&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Here is how I debugged and fixed this cascading failure.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Investigation: Hunting the Leak
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Round 1: The Code Check
&lt;/h3&gt;

&lt;p&gt;My first instinct was to check the stripping logic. The backend used a standard Regex &lt;code&gt;REPLY_TAG_RE&lt;/code&gt; to find and remove fully closed tags like &lt;code&gt;[[reply_to:123]]&lt;/code&gt;. I wrote a quick test script, and the Regex worked perfectly on complete tags. So why were they leaking?&lt;/p&gt;

&lt;h3&gt;
  
  
  Round 2: The Smoking Gun in the Session Logs
&lt;/h3&gt;

&lt;p&gt;I bypassed the UI and checked the raw &lt;code&gt;.jsonl&lt;/code&gt; session logs containing the LLM's raw output. I found the exact payload for the failing prompt:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"role"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"assistant"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"content"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[{&lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"text"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"text"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"[[reply_to_current]]&lt;/span&gt;&lt;span class="se"&gt;\n\n&lt;/span&gt;&lt;span class="s2"&gt;[[reply_to:&amp;lt;id&amp;gt;]]&lt;/span&gt;&lt;span class="se"&gt;\n\n&lt;/span&gt;&lt;span class="s2"&gt;...(repeats 100 times)...&lt;/span&gt;&lt;span class="se"&gt;\n\n&lt;/span&gt;&lt;span class="s2"&gt;[[reply_to:&amp;lt;"&lt;/span&gt;&lt;span class="p"&gt;}],&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"stopReason"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"length"&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Two massive clues here:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;The &lt;code&gt;stopReason&lt;/code&gt; was &lt;code&gt;"length"&lt;/code&gt; (truncated by maxTokens), not normal completion.&lt;/li&gt;
&lt;li&gt;The model had hallucinated and repeated the system prompt instructions until it ran out of tokens, leaving the final tag incomplete (&lt;code&gt;[[reply_to:&amp;lt;&lt;/code&gt;).&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  Round 3: The Streaming Epiphany
&lt;/h3&gt;

&lt;p&gt;Then it hit me. LLMs don't output text all at once; they stream token by token. &lt;br&gt;
When the model generates &lt;code&gt;[[reply_to:123]]&lt;/code&gt;, the data flow looks like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;code&gt;Hello&lt;/code&gt; (No tag -&amp;gt; Safe)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;Hello [[re&lt;/code&gt; (Regex fails to match -&amp;gt; &lt;strong&gt;LEAKED to UI&lt;/strong&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;Hello [[reply_to:&lt;/code&gt; (Regex fails to match -&amp;gt; &lt;strong&gt;LEAKED to UI&lt;/strong&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;Hello [[reply_to:123]]&lt;/code&gt; (Regex matches -&amp;gt; Stripped -&amp;gt; Safe)&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The backend was broadcasting the "growing," incomplete tags to the frontend because the Regex only looked for fully closed brackets.&lt;/p&gt;
&lt;h2&gt;
  
  
  The Root Cause: A 3-Layered Trap
&lt;/h2&gt;

&lt;p&gt;This wasn't just a regex failure. It was a combination of three isolated flaws that created an unrecoverable state:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Layer 1: The Streaming Leak (Backend)&lt;/strong&gt; The &lt;code&gt;stripInlineDirectiveTagsForDisplay()&lt;/code&gt; function only removed closed tags. Intermediate fragments during the Server-Sent Events (SSE) stream slipped right through.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Layer 2: The UI State Logic Trap (Frontend)&lt;/strong&gt;&lt;br&gt;
Inside the frontend controller, the chat stream state update had a fatal assumption:&lt;br&gt;
&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;   &lt;span class="k"&gt;if &lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="nx"&gt;current&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nx"&gt;next&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="nx"&gt;current&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;length&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
     &lt;span class="nx"&gt;state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;chatStream&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;next&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
   &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;The UI assumed text &lt;em&gt;only gets longer&lt;/em&gt;. So, when the backend leaked &lt;code&gt;Hello [[reply&lt;/code&gt; (15 chars), the UI saved it. But when the backend finally received the full tag, stripped it, and sent the clean &lt;code&gt;Hello&lt;/code&gt; (6 chars), the UI rejected the update because 6 &amp;lt; 15! The dirty tag became permanently stuck in the UI.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Layer 3: No Defense in Depth (Frontend)&lt;/strong&gt;
The frontend completely trusted the backend to strip the tags and had no localized sanitization function for assistant messages. &lt;/li&gt;
&lt;/ol&gt;
&lt;h2&gt;
  
  
  The Fix
&lt;/h2&gt;

&lt;p&gt;To solve this, I implemented fixes across the stack:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. Catching Partial Streams (Backend)&lt;/strong&gt;&lt;br&gt;
I added a &lt;code&gt;PARTIAL_REPLY_TAG_RE&lt;/code&gt; regex specifically to target unclosed tags at the very end of the string stream:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;PARTIAL_REPLY_TAG_RE&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sr"&gt;/&lt;/span&gt;&lt;span class="se"&gt;\[\[\s&lt;/span&gt;&lt;span class="sr"&gt;*reply&lt;/span&gt;&lt;span class="se"&gt;(?:&lt;/span&gt;&lt;span class="sr"&gt;_to&lt;/span&gt;&lt;span class="se"&gt;(?:&lt;/span&gt;&lt;span class="sr"&gt;_current|:&lt;/span&gt;&lt;span class="se"&gt;\s&lt;/span&gt;&lt;span class="sr"&gt;*&lt;/span&gt;&lt;span class="se"&gt;[^\]\n]&lt;/span&gt;&lt;span class="sr"&gt;*&lt;/span&gt;&lt;span class="se"&gt;)?)?\s&lt;/span&gt;&lt;span class="sr"&gt;*$/i&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Now, strings like &lt;code&gt;[[reply&lt;/code&gt; are stripped in real-time before broadcasting.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Implementing Defense in Depth (Frontend)&lt;/strong&gt;&lt;br&gt;
I updated the frontend &lt;code&gt;processMessageText()&lt;/code&gt; function to independently run the stripping utility, ensuring that even if a dirty payload somehow bypasses the gateway, the UI sanitizes it before rendering.&lt;/p&gt;

&lt;h2&gt;
  
  
  The SDET Takeaway
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Streaming architectures break traditional parsing:&lt;/strong&gt; When dealing with SSE or WebSockets in AI apps, you must account for intermediate "growing" states. A regex that works on static text will often fail on a stream.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Text length is a dangerous state metric:&lt;/strong&gt; Never assume an LLM's output string will monotonically increase in length. Formatting, redaction, or tag stripping will shrink the string, breaking &lt;code&gt;next.length &amp;gt;= current.length&lt;/code&gt; update logic.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Session Logs are your best friend:&lt;/strong&gt; When the UI misbehaves, don't guess. Go straight to the raw JSON logs. A &lt;code&gt;stopReason: "length"&lt;/code&gt; is a massive red flag.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;By applying defense-in-depth, we ensured that this UI bug is gone for good.&lt;/p&gt;

</description>
      <category>debugging</category>
      <category>javascript</category>
      <category>ai</category>
      <category>testing</category>
    </item>
    <item>
      <title>Debugging a CLI Tool Blindspot: Why "Reload" Commands Don't Always Reload Everything</title>
      <dc:creator>David Yan</dc:creator>
      <pubDate>Thu, 02 Apr 2026 09:36:05 +0000</pubDate>
      <link>https://dev.to/bailorgana/debugging-a-cli-tool-blindspot-why-reload-commands-dont-always-reload-everything-55g8</link>
      <guid>https://dev.to/bailorgana/debugging-a-cli-tool-blindspot-why-reload-commands-dont-always-reload-everything-55g8</guid>
      <description>&lt;p&gt;As developers, we love automation tools and CLI wrappers. They save us time, abstract away complex configurations, and make our workflows smoother. But what happens when the tool designed to manage your configuration lies to you?&lt;/p&gt;

&lt;p&gt;Recently, I encountered a fascinating edge case while managing API keys for my Claude Code environment using a third-party CLI helper (&lt;code&gt;@z_ai/coding-helper&lt;/code&gt;). &lt;/p&gt;

&lt;p&gt;This is a story about "split-brain" configurations, digging into &lt;code&gt;node_modules&lt;/code&gt; to find the truth, and why you should never blindly trust a CLI's success message.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Ghostly "429 Plan Expired" Error
&lt;/h2&gt;

&lt;p&gt;My setup involved using &lt;strong&gt;Claude Code&lt;/strong&gt; powered by a specific API plan (GLM Coding Plan). After my initial API key expired, I renewed my subscription, got a new key, and ran the standard command provided by the CLI helper to reload my credentials:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;chelper auth reload claude
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I even ran the built-in health check (&lt;code&gt;chelper doctor&lt;/code&gt;), and everything returned a perfect string of green checkmarks. Path? Valid. API Key &amp;amp; Network? Valid. Plan? Active.&lt;/p&gt;

&lt;p&gt;However, the moment I tried to invoke a Model Context Protocol (MCP) tool—specifically, the &lt;code&gt;web-reader&lt;/code&gt; tool—the agent instantly crashed with this error:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="err"&gt;Error:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;MCP&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;error&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;-429&lt;/span&gt;&lt;span class="err"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"error"&lt;/span&gt;&lt;span class="p"&gt;:{&lt;/span&gt;&lt;span class="nl"&gt;"code"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;1309&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="nl"&gt;"message"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="s2"&gt;"Your GLM Coding Plan has expired, please renew to restore access."&lt;/span&gt;&lt;span class="p"&gt;}}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Wait a minute. The health check passed, standard chats worked perfectly, but MCP tools were screaming that my account was expired. &lt;/p&gt;

&lt;h2&gt;
  
  
  The Investigation: Spot the Difference
&lt;/h2&gt;

&lt;p&gt;Since the UI and the CLI tool were giving me conflicting information, I bypassed them and went straight to the underlying configuration files. &lt;/p&gt;

&lt;p&gt;I checked the two places where credentials could theoretically be stored:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt; &lt;strong&gt;The CLI helper's main config (&lt;code&gt;~/.chelper/config.yaml&lt;/code&gt;):&lt;/strong&gt; This contained my &lt;strong&gt;brand new, valid API key&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt; &lt;strong&gt;Claude Code's MCP config (&lt;code&gt;~/.claude.json&lt;/code&gt;):&lt;/strong&gt; Looking at the &lt;code&gt;mcpServers&lt;/code&gt; object, I checked the HTTP headers. &lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Boom. The &lt;code&gt;Authorization: Bearer&lt;/code&gt; header for the MCP tools was still using the &lt;strong&gt;old, expired API key&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;We had a "split-brain" scenario. The MCP tools were reading from a stale configuration file.&lt;/p&gt;

&lt;h2&gt;
  
  
  Root Cause Analysis: Diving into &lt;code&gt;node_modules&lt;/code&gt;
&lt;/h2&gt;

&lt;p&gt;Why didn't &lt;code&gt;chelper auth reload&lt;/code&gt; update the MCP config? To find out, I opened up the CLI's source code located in &lt;code&gt;node_modules/@z_ai/coding-helper/dist/lib/claude-code-manager.js&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;I tracked down the exact function responsible for reloading the configuration:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="nf"&gt;loadGLMConfig&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;plan&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nx"&gt;apiKey&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// 1. Ensure onboarding is completed&lt;/span&gt;
    &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;ensureOnboardingCompleted&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;

    &lt;span class="c1"&gt;// 2. Clean up shell environment variables&lt;/span&gt;
    &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;cleanupShellEnvVars&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;

    &lt;span class="c1"&gt;// 3. Update ANTHROPIC_AUTH_TOKEN in settings.json&lt;/span&gt;
    &lt;span class="k"&gt;this&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;saveSettings&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;glmConfig&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The flaw in the architecture was immediately obvious. &lt;/p&gt;

&lt;p&gt;Claude Code actually splits its configuration into two distinct files:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;~/.claude/settings.json&lt;/code&gt; (Used for standard Claude API calls)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;~/.claude.json&lt;/code&gt; (Used to register MCP servers and their auth headers)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The developer of the CLI tool only programmed &lt;code&gt;this.saveSettings()&lt;/code&gt; to update the standard API token. The tool &lt;strong&gt;completely ignored&lt;/strong&gt; the MCP server configurations. As a result, any HTTP-based MCP tools registered previously were left stranded with expired credentials.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Fix
&lt;/h2&gt;

&lt;p&gt;If you are dealing with a similar misbehaving CLI wrapper, you can't rely on its &lt;code&gt;reload&lt;/code&gt; command. You have a few options:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Option 1: The Nuclear Option (Recommended)&lt;/strong&gt;&lt;br&gt;
Run the initialization command again from scratch. This usually forces the tool to overwrite all files completely.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx @z_ai/coding-helper init
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Option 2: The Python Automation Fix&lt;/strong&gt;&lt;br&gt;
If you have heavily customized your &lt;code&gt;mcpServers&lt;/code&gt; and don't want to reset everything, I wrote a quick Python script to automatically hunt down HTTP-based MCP tools in your config and update their headers securely:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;

&lt;span class="n"&gt;config_path&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;expanduser&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;~/.claude.json&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;new_key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;YOUR_NEW_VALID_API_KEY&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;

&lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="nf"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;config_path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;r&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;config&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;load&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# Iterate through all configured MCP servers
&lt;/span&gt;    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;mcp_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;mcp_config&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;config&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;mcpServers&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{}).&lt;/span&gt;&lt;span class="nf"&gt;items&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
        &lt;span class="c1"&gt;# Update only those that use HTTP headers for authentication
&lt;/span&gt;        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;headers&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;config&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;mcpServers&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{}).&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mcp_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{}):&lt;/span&gt;
            &lt;span class="n"&gt;config&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;mcpServers&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="n"&gt;mcp_name&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;headers&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;Authorization&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;Bearer &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;new_key&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;
            &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Updated auth token for MCP tool: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;mcp_name&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="nf"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;config_path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;w&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;dump&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;config&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;indent&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;All MCP keys updated successfully!&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nb"&gt;FileNotFoundError&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Error: ~/.claude.json configuration file not found.&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  The SDET Takeaway
&lt;/h2&gt;

&lt;p&gt;This is why Software Development Engineers in Test (SDETs) don't just rely on UI green lights or standard CLI outputs. &lt;/p&gt;

&lt;p&gt;When an automation tool tells you "Success!" but the system behaves as if it failed, there is almost always a state synchronization issue underneath. The abstraction layers designed to help us can quickly become blindspots.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Golden Rule of Debugging Toolchains:&lt;/strong&gt; When system behavior contradicts configuration state, stop trusting the management tools and start reading the raw &lt;code&gt;.json&lt;/code&gt; and &lt;code&gt;.yaml&lt;/code&gt; files.&lt;/p&gt;

</description>
      <category>debugging</category>
      <category>cli</category>
      <category>javascript</category>
      <category>testing</category>
    </item>
    <item>
      <title>Debugging a 400 Error: How a Silent API Gateway Update Broke My LLM Agent</title>
      <dc:creator>David Yan</dc:creator>
      <pubDate>Wed, 01 Apr 2026 02:42:09 +0000</pubDate>
      <link>https://dev.to/bailorgana/debugging-a-400-error-how-a-silent-api-gateway-update-broke-my-llm-agent-1ho2</link>
      <guid>https://dev.to/bailorgana/debugging-a-400-error-how-a-silent-api-gateway-update-broke-my-llm-agent-1ho2</guid>
      <description>&lt;p&gt;As an SDET (Software Development Engineer in Test), I spend a lot of time breaking things. But there is a special kind of frustration when an environment that worked perfectly yesterday suddenly throws a fatal error today, despite &lt;strong&gt;zero changes&lt;/strong&gt; to your local code or configuration.&lt;/p&gt;

&lt;p&gt;This is the story of how a silent, server-side API validation update completely broke my local AI agent workflow, and how I debugged it by diving into the raw JSON payloads.&lt;/p&gt;

&lt;p&gt;If you are building AI agents using third-party LLM gateways or Model Context Protocol (MCP) tools, this debugging journey might save you hours of pulling your hair out.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Setup &amp;amp; The Incident
&lt;/h2&gt;

&lt;p&gt;I was using &lt;strong&gt;Claude Code CLI (v2.1.69)&lt;/strong&gt;, but instead of routing it to Anthropic’s official API, I pointed the base URL to &lt;strong&gt;Zhipu AI's Anthropic-compatible endpoint&lt;/strong&gt; (&lt;code&gt;https://open.bigmodel.cn/api/anthropic&lt;/code&gt;), powered by their &lt;code&gt;glm-5.1&lt;/code&gt; model. This is a common, cost-effective architecture for developers testing local AI agents.&lt;/p&gt;

&lt;p&gt;Everything was running smoothly. I was using MCP tools for image analysis and web searching. Then, suddenly, the agent flatlined.&lt;/p&gt;

&lt;p&gt;When I prompted the agent to do a simple task that required tools (e.g., &lt;em&gt;"Check what files are in the current directory"&lt;/em&gt;), it instantly crashed with a &lt;code&gt;400 Bad Request&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="err"&gt;API&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;Error:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;400&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"error"&lt;/span&gt;&lt;span class="p"&gt;:{&lt;/span&gt;&lt;span class="nl"&gt;"code"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="s2"&gt;"1214"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="nl"&gt;"message"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="s2"&gt;"messages[4].content[0].type类型错误"&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="nl"&gt;"request_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="s2"&gt;"20260330140839a4b83527b21b4c46"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;em&gt;(Translation of the error: "Type error in messages[4].content[0].type")&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The most bizarre part? Standard text chats worked perfectly. But the moment the agent tried to touch &lt;strong&gt;any&lt;/strong&gt; MCP tool, it triggered the &lt;code&gt;1214&lt;/code&gt; loop of death.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Investigation: Hunting the Ghost in the Machine
&lt;/h2&gt;

&lt;p&gt;My first instinct was to blame context limit limits or a corrupted local cache. I cleared the &lt;code&gt;.claude&lt;/code&gt; hidden directories and reset the session. The error persisted. &lt;/p&gt;

&lt;p&gt;It was time to stop guessing and look at the raw network traffic.&lt;/p&gt;

&lt;p&gt;By inspecting the exact payload Claude Code was sending to the API right before the crash, I found the culprit hidden deep within the Anthropic protocol's message array:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"role"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"user"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"content"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"tool_result"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"tool_use_id"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"call_xxx"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"content"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"tool_reference"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"tool_name"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Bash"&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  The "Aha!" Moment
&lt;/h3&gt;

&lt;p&gt;Claude Code v2.x relies heavily on a feature called &lt;code&gt;ToolSearch&lt;/code&gt;. Because it supports dynamic MCP tools, it doesn't load everything into the context window at once (they are marked as &lt;code&gt;deferred&lt;/code&gt;). Instead, it searches for the right tool and then alerts the LLM using a highly specific, internal Anthropic content block: &lt;code&gt;"type": "tool_reference"&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The Zhipu API gateway, built to &lt;em&gt;mimic&lt;/em&gt; the Anthropic protocol, simply &lt;strong&gt;did not know what &lt;code&gt;tool_reference&lt;/code&gt; was&lt;/strong&gt;. &lt;/p&gt;

&lt;h2&gt;
  
  
  The Root Cause Analysis (RCA)
&lt;/h2&gt;

&lt;p&gt;But wait—if the gateway didn't support it, &lt;strong&gt;why did it work perfectly just hours ago?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I checked my terminal logs and constructed a timeline:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;March 30, 04:06 AM:&lt;/strong&gt; Last successful &lt;code&gt;ToolSearch&lt;/code&gt; invocation.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;March 30, 06:05 AM:&lt;/strong&gt; First appearance of the &lt;code&gt;1214&lt;/code&gt; error.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;The Conclusion:&lt;/strong&gt; The API provider deployed a silent backend update during that 2-hour window. &lt;/p&gt;

&lt;p&gt;Previously, their gateway likely had a lenient validation policy: &lt;em&gt;If you see a JSON field or &lt;code&gt;type&lt;/code&gt; you don't recognize, just ignore it.&lt;/em&gt; However, the new update introduced &lt;strong&gt;strict Schema Validation&lt;/strong&gt;. The moment the gateway encountered the unsupported &lt;code&gt;tool_reference&lt;/code&gt; type, it rejected the entire payload with a 400 error.&lt;/p&gt;

&lt;p&gt;This single, undocumented API regression caused a catastrophic failure downstream:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;code&gt;ToolSearch&lt;/code&gt; failed completely.&lt;/li&gt;
&lt;li&gt;All deferred MCP tools became undiscoverable.&lt;/li&gt;
&lt;li&gt;Image analysis MCPs (&lt;code&gt;mcp__zai-mcp-server__analyze_image&lt;/code&gt;) were rendered completely useless.&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  The Workaround
&lt;/h2&gt;

&lt;p&gt;Until the API provider updates their compatibility layer to support the &lt;code&gt;tool_reference&lt;/code&gt; block, the only way to unblock the workflow is to surgically disable the feature generating the payload.&lt;/p&gt;

&lt;p&gt;If you are facing this exact issue, you can temporarily bypass it by modifying your &lt;code&gt;~/.claude/settings.json&lt;/code&gt; to disable tool searching:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"env"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"ANTHROPIC_BASE_URL"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"https://open.bigmodel.cn/api/anthropic"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"ANTHROPIC_AUTH_TOKEN"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"your_api_key"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"ENABLE_TOOL_SEARCH"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"0"&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Trade-off:&lt;/strong&gt; This stops the 400 errors, but you will lose access to dynamic MCP tool discovery and image analysis capabilities.&lt;/p&gt;

&lt;h2&gt;
  
  
  The SDET Takeaway
&lt;/h2&gt;

&lt;p&gt;This incident is a textbook example of why building applications on top of "API masquerading/compatibility layers" is inherently fragile. &lt;/p&gt;

&lt;p&gt;When you sit between a rapidly iterating client (Claude Code) and a third-party gateway trying to reverse-engineer its protocol, you are at the mercy of both sides. A minor schema update on the client, or a strict validation patch on the server, will snap the integration in half.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The ultimate lesson?&lt;/strong&gt; When the UI says "Unknown Error," don't just reboot. Grab your network inspector, dive into the raw JSON payloads, and follow the data. The truth is always in the Schema.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Have you encountered similar protocol mismatches while building LLM agents? Let me know in the comments!&lt;/em&gt;&lt;/p&gt;

</description>
      <category>debugging</category>
      <category>api</category>
      <category>sdet</category>
      <category>mcp</category>
    </item>
  </channel>
</rss>
