<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: YMori</title>
    <description>The latest articles on DEV Community by YMori (@yasumorishima).</description>
    <link>https://dev.to/yasumorishima</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3761588%2F10324d86-be69-483f-90af-7c8f8eb80cf1.png</url>
      <title>DEV Community: YMori</title>
      <link>https://dev.to/yasumorishima</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/yasumorishima"/>
    <language>en</language>
    <item>
      <title>M5 StackChan pairing fails with No devices found — the factory firmware is the cause</title>
      <dc:creator>YMori</dc:creator>
      <pubDate>Sat, 25 Jul 2026 09:00:55 +0000</pubDate>
      <link>https://dev.to/yasumorishima/m5-stackchan-pairing-fails-with-no-devices-found-the-factory-firmware-is-the-cause-387e</link>
      <guid>https://dev.to/yasumorishima/m5-stackchan-pairing-fails-with-no-devices-found-the-factory-firmware-is-the-cause-387e</guid>
      <description>&lt;h2&gt;
  
  
  TL;DR
&lt;/h2&gt;

&lt;p&gt;I bought the official M5Stack StackChan (&lt;code&gt;M5STACK-K151&lt;/code&gt;), and the "StackChan World" mobile app refused to finish setup: after selecting the device, it showed &lt;strong&gt;&lt;code&gt;No devices found&lt;/code&gt;&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The cause was &lt;strong&gt;not Bluetooth permissions and not an app bug — the factory firmware was simply too old&lt;/strong&gt;. After flashing the latest official firmware over USB, pairing went through on the first attempt.&lt;/p&gt;

&lt;p&gt;What makes this worth writing about is that the situation is &lt;strong&gt;a closed loop you cannot escape on the device itself&lt;/strong&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;New firmware arrives via OTA&lt;/li&gt;
&lt;li&gt;OTA needs Wi-Fi&lt;/li&gt;
&lt;li&gt;Wi-Fi setup needs app pairing&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Pairing fails on the old firmware&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So &lt;strong&gt;flashing over USB is the only way out&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Environment and dates
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Product&lt;/td&gt;
&lt;td&gt;M5StackChan AI Desktop Robot (ESP32-S3) / &lt;code&gt;M5STACK-K151&lt;/code&gt; (official assembled unit)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Purchased&lt;/td&gt;
&lt;td&gt;2026-07-24 (Switch Science, Japan)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Setup attempted&lt;/td&gt;
&lt;td&gt;2026-07-25&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Factory firmware on the unit&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;1.2.4&lt;/strong&gt; (released 2026-04-20)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Latest official firmware that day&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;1.4.4&lt;/strong&gt; (released 2026-07-13)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Host&lt;/td&gt;
&lt;td&gt;Windows 11&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Phone&lt;/td&gt;
&lt;td&gt;Android&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;The dates matter here.&lt;/strong&gt; This product launched in 2026-05 and both the firmware and the app are still being updated frequently. &lt;strong&gt;The firmware version that ships with a unit depends on when you buy it&lt;/strong&gt;, so whether you hit this problem depends on timing. If you buy stock that has been sitting for a few months, you are a good candidate for the same trap.&lt;/p&gt;

&lt;h2&gt;
  
  
  The symptom
&lt;/h2&gt;

&lt;p&gt;The setup flow is:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Power on, pass the servo test&lt;/li&gt;
&lt;li&gt;The device shows a 12-digit ID&lt;/li&gt;
&lt;li&gt;In the app: "Add a new StackChan" → scan for nearby devices&lt;/li&gt;
&lt;li&gt;Pick the entry matching the ID on the device screen&lt;/li&gt;
&lt;li&gt;Set device name → AI agent → Wi-Fi&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;strong&gt;Step 4 immediately produced &lt;code&gt;No devices found&lt;/code&gt;&lt;/strong&gt;, and step 5 was never reachable.&lt;/p&gt;

&lt;h2&gt;
  
  
  What did not help
&lt;/h2&gt;

&lt;p&gt;The message talks about devices not being found, so of course I attacked discovery first. None of this changed anything:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Granting Bluetooth and location permissions to the app (verified from the OS settings, not just the in-app prompt)&lt;/li&gt;
&lt;li&gt;Turning location services on (BLE scanning on Android is entangled with location)&lt;/li&gt;
&lt;li&gt;Force-stopping and restarting the app, rebooting the phone&lt;/li&gt;
&lt;li&gt;Rebooting the device, returning to the ID screen before scanning&lt;/li&gt;
&lt;li&gt;Toggling Bluetooth off and on&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;One thing I learned the hard way: &lt;strong&gt;do not pair from the OS Bluetooth settings screen.&lt;/strong&gt; If you bond the device at the OS level, it gets registered as a known device and &lt;strong&gt;disappears from the app's "find a new device" list&lt;/strong&gt;. I did exactly this while "just checking", which made the picture even muddier. If you already did it, remove the pairing.&lt;/p&gt;

&lt;h2&gt;
  
  
  The serial log told a completely different story
&lt;/h2&gt;

&lt;p&gt;I stopped guessing about permissions and captured the USB serial log. When you connect the unit to a PC, the CoreS3 (ESP32-S3) native USB shows up as a serial port (&lt;code&gt;VID_303A&lt;/code&gt; / &lt;code&gt;PID_1001&lt;/code&gt;, USB-Serial/JTAG).&lt;/p&gt;

&lt;p&gt;Here is what the log showed while the app was "not finding" anything:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;I (111140) NimBLE: connection established;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;status&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0
&lt;span class="go"&gt;I (111160) NimBLE:  peer_ota_addr_type=1 peer_ota_addr=
I (111160) NimBLE: 63:a2:04:c7:fa:5d
[info] [WifiConfigServer] app Connected
&lt;/span&gt;&lt;span class="gp"&gt;I (112260) NimBLE: Stack-Chan characteristic write;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;conn_handle&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;1 &lt;span class="nv"&gt;attr_handle&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;22
&lt;span class="go"&gt;I (112260) NimBLE: Config data received (42 bytes): {"cmd":"handshake","data":"1784964126892"}
&lt;/span&gt;&lt;span class="gp"&gt;I (112420) NimBLE: GATT procedure initiated: notify;&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="gp"&gt;I (112430) NimBLE: notify_tx event;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;conn_handle&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;1 &lt;span class="nv"&gt;attr_handle&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;22 &lt;span class="nv"&gt;status&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0 &lt;span class="nv"&gt;is_indication&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0
&lt;span class="go"&gt;I (112440) NimBLE: Config notification sent
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Reading that:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The BLE connection was established&lt;/strong&gt; (&lt;code&gt;status=0&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The app was connected&lt;/strong&gt; (&lt;code&gt;app Connected&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The app sent a handshake&lt;/strong&gt; (&lt;code&gt;{"cmd":"handshake"}&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The device answered&lt;/strong&gt; (&lt;code&gt;Config notification sent&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;And then &lt;strong&gt;nothing more arrives from the app&lt;/strong&gt; — not even a disconnect&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So &lt;code&gt;No devices found&lt;/code&gt; did not describe reality. The process was not failing at &lt;strong&gt;discovery&lt;/strong&gt;; it was failing at &lt;strong&gt;verification of the response the device sent back&lt;/strong&gt;. Trusting that message is what sent me down the permissions rabbit hole.&lt;/p&gt;

&lt;p&gt;For confirmation I read the app side too. The firmware, the app, and the server for StackChan are all published in &lt;a href="https://github.com/m5stack/StackChan" rel="noopener noreferrer"&gt;m5stack/StackChan&lt;/a&gt; — the Flutter app source lives in the same repository. The device-selection screen has a 30-second connect timeout and a 20-second verification timeout, with failure messages such as &lt;code&gt;Device did not return encryption data.&lt;/code&gt; and &lt;code&gt;Device verification decryption failed.&lt;/code&gt;, and it registers the device with the server only after verification succeeds.&lt;/p&gt;

&lt;h2&gt;
  
  
  Root cause: firmware version, plus a deadlock
&lt;/h2&gt;

&lt;p&gt;The boot log prints the version:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;I (690) app_init: Project name:     stack-chan
I (696) app_init: App version:      1.2.4
I (700) app_init: Compile time:     Apr 15 2026 09:18:33
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;1.2.4&lt;/strong&gt;, while the latest official build that day was &lt;strong&gt;1.4.4&lt;/strong&gt; — nine versions apart.&lt;/p&gt;

&lt;p&gt;And that is where the loop from the top of this post bites: OTA needs Wi-Fi, Wi-Fi needs pairing, pairing fails on old firmware. &lt;strong&gt;USB is the only remaining path.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Note that &lt;strong&gt;building the firmware yourself does not help&lt;/strong&gt;. The handshake implementation in the public repository (&lt;code&gt;firmware/main/hal/utils/secret_logic/secret_logic.cpp&lt;/code&gt;) is a stub whose token function returns a fixed string. It is declared as a weak symbol, and the real implementation is linked in at official build time. &lt;strong&gt;Only the official binary can satisfy the app's verification step.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The fix: flash the official firmware over USB, without M5Burner
&lt;/h2&gt;

&lt;p&gt;The official flashing tool is M5Burner (a GUI), but &lt;strong&gt;the firmware distribution API it uses is public&lt;/strong&gt;, so the whole thing can be done from the CLI. No Python required either.&lt;/p&gt;

&lt;h3&gt;
  
  
  1. List the official firmware versions
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="s2"&gt;"https://m5burner-api.m5stack.com/api/firmware"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  | jq &lt;span class="nt"&gt;-r&lt;/span&gt; &lt;span class="s1"&gt;'.[] | select(.name=="StackChan-UserDemo") | .versions[] | "\(.version)  \(.published_at)  \(.file)"'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The factory firmware is named &lt;strong&gt;&lt;code&gt;StackChan-UserDemo&lt;/code&gt;&lt;/strong&gt; (author: M5Stack):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;V1.2.4   2026-04-20  3c8ffe6be0ca26375d836e10e06e3609.bin
V1.4.3   2026-07-02  fb75fa818e63b7ee6b0d35eba308f386.bin
V1.4.4   2026-07-13  790e3fcde496020aa7f188153b23e6f0.bin
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Older versions stay available, so &lt;strong&gt;rolling back is possible too&lt;/strong&gt; — which makes updating a lot less scary.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Download the binary
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-sL&lt;/span&gt; &lt;span class="s2"&gt;"https://m5burner-cdn.m5stack.com/firmware/790e3fcde496020aa7f188153b23e6f0.bin"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-o&lt;/span&gt; stackchan-v1.4.4.bin
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If the first byte is &lt;code&gt;e9&lt;/code&gt; (the ESP image magic), it is a merged full image to be written at offset &lt;code&gt;0x0&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;od&lt;/span&gt; &lt;span class="nt"&gt;-An&lt;/span&gt; &lt;span class="nt"&gt;-tx1&lt;/span&gt; &lt;span class="nt"&gt;-N&lt;/span&gt; 1 stackchan-v1.4.4.bin
&lt;span class="c"&gt;#  e9&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  3. Get esptool (no Python needed)
&lt;/h3&gt;

&lt;p&gt;Espressif ships standalone executables:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;gh api repos/espressif/esptool/releases/latest &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--jq&lt;/span&gt; &lt;span class="s1"&gt;'.assets[] | select(.name|test("windows-amd64")) | .browser_download_url'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  4. Verify communication with a read-only command first
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;\esptool.exe&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;--port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;COM3&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;flash-id&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;Detecting chip type... ESP32-S3
Chip type:          ESP32-S3 (QFN56) (revision v0.2)
USB mode:           USB-Serial/JTAG
Detected flash size: 16MB
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;No manual download mode was necessary&lt;/strong&gt; — esptool resets the chip through USB-Serial/JTAG. If it cannot connect, hold the reset button for about 2 seconds and release it once the internal green LED lights up.&lt;/p&gt;

&lt;h3&gt;
  
  
  5. Flash
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;\esptool.exe&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;--port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;COM3&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;write-flash&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;0x0&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;stackchan-v1.4.4.bin&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;Wrote 12783792 bytes (3185393 compressed) at 0x00000000 in 47.7 seconds
Verifying written data...
Hash of data verified.
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;About 50 seconds for just under 12.8 MB.&lt;/p&gt;

&lt;h3&gt;
  
  
  6. Confirm the version
&lt;/h3&gt;

&lt;p&gt;Look for &lt;code&gt;app_init: App version: 1.4.4&lt;/code&gt; in the boot log. With that in place, pairing succeeded on the first try, and I went straight through AI agent setup, Wi-Fi setup, and an actual voice conversation.&lt;/p&gt;

&lt;h2&gt;
  
  
  Other traps worth knowing
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Opening the serial port reboots the device
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;rst:0x15 (USB_UART_CHIP_RESET),boot:0x2b (SPI_FAST_FLASH_BOOT)
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;You cannot capture logs and operate the device at the same time.&lt;/strong&gt; Opening the port in the middle of the setup wizard sends you back to the start (it did for me). The flip side is useful: if you want the boot log from the very first line, open the port and then trigger a reset.&lt;/p&gt;

&lt;h3&gt;
  
  
  There is no interactive serial console
&lt;/h3&gt;

&lt;p&gt;Sending commands produces no response, and the write itself blocks. Treat the serial port as read-only.&lt;/p&gt;

&lt;h3&gt;
  
  
  Do not power both USB-C ports at once
&lt;/h3&gt;

&lt;p&gt;The unit has a USB-C port on the CoreS3 and another on the heel (base). 5V fed into the CoreS3 side is routed to the base through the M-BUS &lt;code&gt;BUS_5V&lt;/code&gt; line, so &lt;strong&gt;powering both ports puts two supplies on the same rail&lt;/strong&gt;. The official documentation's "try powering from both ports" means one at a time. The heel port is the recommended one for power and flashing.&lt;/p&gt;

&lt;h3&gt;
  
  
  Servo zero positions survive a full flash erase
&lt;/h3&gt;

&lt;p&gt;I backed up NVS before flashing, but the same zero positions were read back afterwards (&lt;code&gt;[ScsServo] id: 1 get zero pos: 460 from settings&lt;/code&gt;). &lt;strong&gt;Those values appear to live in the servos themselves&lt;/strong&gt;, so a full-flash rewrite does not lose them.&lt;/p&gt;

&lt;h2&gt;
  
  
  A separate warning about charging
&lt;/h2&gt;

&lt;p&gt;The firmware sets the AXP2101 charge current to &lt;strong&gt;700mA&lt;/strong&gt; at boot:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cpp"&gt;&lt;code&gt;&lt;span class="k"&gt;auto&lt;/span&gt; &lt;span class="n"&gt;ret&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;setChargerConstantCurr&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;XPOWERS_AXP2101_CHG_CUR_700MA&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The ESP32-S3 enumerates as a USB 2.0 device, so &lt;strong&gt;a PC USB port gives you 500mA at most&lt;/strong&gt;. Combined with the system draw (LCD, camera, servos, radio) that exceeds the input limit, and &lt;strong&gt;charging never starts from a PC port&lt;/strong&gt;. Use a USB charger with headroom at 5V.&lt;/p&gt;

&lt;p&gt;If the servos do not report their current position (&lt;code&gt;[ScsServo] ignore invalid current pos: -1&lt;/code&gt;), read that as another sign of insufficient power.&lt;/p&gt;

&lt;h2&gt;
  
  
  Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;No devices found&lt;/code&gt; &lt;strong&gt;did not describe what was happening&lt;/strong&gt;. The BLE connection and the handshake both succeeded; the failure was in response verification&lt;/li&gt;
&lt;li&gt;The real cause was &lt;strong&gt;old factory firmware&lt;/strong&gt;, fixed by flashing the latest official build over USB&lt;/li&gt;
&lt;li&gt;OTA ↔ Wi-Fi ↔ pairing form a loop, so &lt;strong&gt;USB flashing is the only exit&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Do not over-trust the error string — capture the log.&lt;/strong&gt; Every minute I spent re-checking permissions would have been saved by one serial capture&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Scripts and the full write-up are in the repository:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/yasumorishima/stackchan-lab" rel="noopener noreferrer"&gt;https://github.com/yasumorishima/stackchan-lab&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  References
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Official docs: &lt;a href="https://docs.m5stack.com/en/stackchan" rel="noopener noreferrer"&gt;https://docs.m5stack.com/en/stackchan&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Firmware / app / server source: &lt;a href="https://github.com/m5stack/StackChan" rel="noopener noreferrer"&gt;https://github.com/m5stack/StackChan&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>m5stack</category>
      <category>esp32</category>
      <category>iot</category>
      <category>ble</category>
    </item>
    <item>
      <title>M5 StackChan pairing fails with \"No devices found\"? The factory firmware is the cause</title>
      <dc:creator>YMori</dc:creator>
      <pubDate>Sat, 25 Jul 2026 08:48:29 +0000</pubDate>
      <link>https://dev.to/yasumorishima/m5-stackchan-pairing-fails-with-no-devices-found-the-factory-firmware-is-the-cause-1od8</link>
      <guid>https://dev.to/yasumorishima/m5-stackchan-pairing-fails-with-no-devices-found-the-factory-firmware-is-the-cause-1od8</guid>
      <description>&lt;h2&gt;
  
  
  TL;DR
&lt;/h2&gt;

&lt;p&gt;I bought the official M5Stack StackChan (&lt;code&gt;M5STACK-K151&lt;/code&gt;), and the "StackChan World" mobile app refused to finish setup: after selecting the device, it showed &lt;strong&gt;&lt;code&gt;No devices found&lt;/code&gt;&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The cause was &lt;strong&gt;not Bluetooth permissions and not an app bug — the factory firmware was simply too old&lt;/strong&gt;. After flashing the latest official firmware over USB, pairing went through on the first attempt.&lt;/p&gt;

&lt;p&gt;What makes this worth writing about is that the situation is &lt;strong&gt;a closed loop you cannot escape on the device itself&lt;/strong&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;New firmware arrives via OTA&lt;/li&gt;
&lt;li&gt;OTA needs Wi-Fi&lt;/li&gt;
&lt;li&gt;Wi-Fi setup needs app pairing&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Pairing fails on the old firmware&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So &lt;strong&gt;flashing over USB is the only way out&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Environment and dates
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Product&lt;/td&gt;
&lt;td&gt;M5StackChan AI Desktop Robot (ESP32-S3) / &lt;code&gt;M5STACK-K151&lt;/code&gt; (official assembled unit)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Purchased&lt;/td&gt;
&lt;td&gt;2026-07-24 (Switch Science, Japan)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Setup attempted&lt;/td&gt;
&lt;td&gt;2026-07-25&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Factory firmware on the unit&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;1.2.4&lt;/strong&gt; (released 2026-04-20)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Latest official firmware that day&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;1.4.4&lt;/strong&gt; (released 2026-07-13)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Host&lt;/td&gt;
&lt;td&gt;Windows 11&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Phone&lt;/td&gt;
&lt;td&gt;Android&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;The dates matter here.&lt;/strong&gt; This product launched in 2026-05 and both the firmware and the app are still being updated frequently. &lt;strong&gt;The firmware version that ships with a unit depends on when you buy it&lt;/strong&gt;, so whether you hit this problem depends on timing. If you buy stock that has been sitting for a few months, you are a good candidate for the same trap.&lt;/p&gt;

&lt;h2&gt;
  
  
  The symptom
&lt;/h2&gt;

&lt;p&gt;The setup flow is:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Power on, pass the servo test&lt;/li&gt;
&lt;li&gt;The device shows a 12-digit ID&lt;/li&gt;
&lt;li&gt;In the app: "Add a new StackChan" → scan for nearby devices&lt;/li&gt;
&lt;li&gt;Pick the entry matching the ID on the device screen&lt;/li&gt;
&lt;li&gt;Set device name → AI agent → Wi-Fi&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;strong&gt;Step 4 immediately produced &lt;code&gt;No devices found&lt;/code&gt;&lt;/strong&gt;, and step 5 was never reachable.&lt;/p&gt;

&lt;h2&gt;
  
  
  What did not help
&lt;/h2&gt;

&lt;p&gt;The message talks about devices not being found, so of course I attacked discovery first. None of this changed anything:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Granting Bluetooth and location permissions to the app (verified from the OS settings, not just the in-app prompt)&lt;/li&gt;
&lt;li&gt;Turning location services on (BLE scanning on Android is entangled with location)&lt;/li&gt;
&lt;li&gt;Force-stopping and restarting the app, rebooting the phone&lt;/li&gt;
&lt;li&gt;Rebooting the device, returning to the ID screen before scanning&lt;/li&gt;
&lt;li&gt;Toggling Bluetooth off and on&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;One thing I learned the hard way: &lt;strong&gt;do not pair from the OS Bluetooth settings screen.&lt;/strong&gt; If you bond the device at the OS level, it gets registered as a known device and &lt;strong&gt;disappears from the app's "find a new device" list&lt;/strong&gt;. I did exactly this while "just checking", which made the picture even muddier. If you already did it, remove the pairing.&lt;/p&gt;

&lt;h2&gt;
  
  
  The serial log told a completely different story
&lt;/h2&gt;

&lt;p&gt;I stopped guessing about permissions and captured the USB serial log. When you connect the unit to a PC, the CoreS3 (ESP32-S3) native USB shows up as a serial port (&lt;code&gt;VID_303A&lt;/code&gt; / &lt;code&gt;PID_1001&lt;/code&gt;, USB-Serial/JTAG).&lt;/p&gt;

&lt;p&gt;Here is what the log showed while the app was "not finding" anything:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;I (111140) NimBLE: connection established;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;status&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0
&lt;span class="go"&gt;I (111160) NimBLE:  peer_ota_addr_type=1 peer_ota_addr=
I (111160) NimBLE: 63:a2:04:c7:fa:5d
[info] [WifiConfigServer] app Connected
&lt;/span&gt;&lt;span class="gp"&gt;I (112260) NimBLE: Stack-Chan characteristic write;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;conn_handle&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;1 &lt;span class="nv"&gt;attr_handle&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;22
&lt;span class="go"&gt;I (112260) NimBLE: Config data received (42 bytes): {"cmd":"handshake","data":"1784964126892"}
&lt;/span&gt;&lt;span class="gp"&gt;I (112420) NimBLE: GATT procedure initiated: notify;&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="gp"&gt;I (112430) NimBLE: notify_tx event;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nv"&gt;conn_handle&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;1 &lt;span class="nv"&gt;attr_handle&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;22 &lt;span class="nv"&gt;status&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0 &lt;span class="nv"&gt;is_indication&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0
&lt;span class="go"&gt;I (112440) NimBLE: Config notification sent
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Reading that:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The BLE connection was established&lt;/strong&gt; (&lt;code&gt;status=0&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The app was connected&lt;/strong&gt; (&lt;code&gt;app Connected&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The app sent a handshake&lt;/strong&gt; (&lt;code&gt;{"cmd":"handshake"}&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The device answered&lt;/strong&gt; (&lt;code&gt;Config notification sent&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;And then &lt;strong&gt;nothing more arrives from the app&lt;/strong&gt; — not even a disconnect&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So &lt;code&gt;No devices found&lt;/code&gt; did not describe reality. The process was not failing at &lt;strong&gt;discovery&lt;/strong&gt;; it was failing at &lt;strong&gt;verification of the response the device sent back&lt;/strong&gt;. Trusting that message is what sent me down the permissions rabbit hole.&lt;/p&gt;

&lt;p&gt;For confirmation I read the app side too. The firmware, the app, and the server for StackChan are all published in &lt;a href="https://github.com/m5stack/StackChan" rel="noopener noreferrer"&gt;m5stack/StackChan&lt;/a&gt; — the Flutter app source lives in the same repository. The device-selection screen has a 30-second connect timeout and a 20-second verification timeout, with failure messages such as &lt;code&gt;Device did not return encryption data.&lt;/code&gt; and &lt;code&gt;Device verification decryption failed.&lt;/code&gt;, and it registers the device with the server only after verification succeeds.&lt;/p&gt;

&lt;h2&gt;
  
  
  Root cause: firmware version, plus a deadlock
&lt;/h2&gt;

&lt;p&gt;The boot log prints the version:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;I (690) app_init: Project name:     stack-chan
I (696) app_init: App version:      1.2.4
I (700) app_init: Compile time:     Apr 15 2026 09:18:33
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;1.2.4&lt;/strong&gt;, while the latest official build that day was &lt;strong&gt;1.4.4&lt;/strong&gt; — nine versions apart.&lt;/p&gt;

&lt;p&gt;And that is where the loop from the top of this post bites: OTA needs Wi-Fi, Wi-Fi needs pairing, pairing fails on old firmware. &lt;strong&gt;USB is the only remaining path.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Note that &lt;strong&gt;building the firmware yourself does not help&lt;/strong&gt;. The handshake implementation in the public repository (&lt;code&gt;firmware/main/hal/utils/secret_logic/secret_logic.cpp&lt;/code&gt;) is a stub whose token function returns a fixed string. It is declared as a weak symbol, and the real implementation is linked in at official build time. &lt;strong&gt;Only the official binary can satisfy the app's verification step.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The fix: flash the official firmware over USB, without M5Burner
&lt;/h2&gt;

&lt;p&gt;The official flashing tool is M5Burner (a GUI), but &lt;strong&gt;the firmware distribution API it uses is public&lt;/strong&gt;, so the whole thing can be done from the CLI. No Python required either.&lt;/p&gt;

&lt;h3&gt;
  
  
  1. List the official firmware versions
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="s2"&gt;"https://m5burner-api.m5stack.com/api/firmware"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  | jq &lt;span class="nt"&gt;-r&lt;/span&gt; &lt;span class="s1"&gt;'.[] | select(.name=="StackChan-UserDemo") | .versions[] | "\(.version)  \(.published_at)  \(.file)"'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The factory firmware is named &lt;strong&gt;&lt;code&gt;StackChan-UserDemo&lt;/code&gt;&lt;/strong&gt; (author: M5Stack):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;V1.2.4   2026-04-20  3c8ffe6be0ca26375d836e10e06e3609.bin
V1.4.3   2026-07-02  fb75fa818e63b7ee6b0d35eba308f386.bin
V1.4.4   2026-07-13  790e3fcde496020aa7f188153b23e6f0.bin
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Older versions stay available, so &lt;strong&gt;rolling back is possible too&lt;/strong&gt; — which makes updating a lot less scary.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Download the binary
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-sL&lt;/span&gt; &lt;span class="s2"&gt;"https://m5burner-cdn.m5stack.com/firmware/790e3fcde496020aa7f188153b23e6f0.bin"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-o&lt;/span&gt; stackchan-v1.4.4.bin
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If the first byte is &lt;code&gt;e9&lt;/code&gt; (the ESP image magic), it is a merged full image to be written at offset &lt;code&gt;0x0&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;od&lt;/span&gt; &lt;span class="nt"&gt;-An&lt;/span&gt; &lt;span class="nt"&gt;-tx1&lt;/span&gt; &lt;span class="nt"&gt;-N&lt;/span&gt; 1 stackchan-v1.4.4.bin
&lt;span class="c"&gt;#  e9&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  3. Get esptool (no Python needed)
&lt;/h3&gt;

&lt;p&gt;Espressif ships standalone executables:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;gh api repos/espressif/esptool/releases/latest &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--jq&lt;/span&gt; &lt;span class="s1"&gt;'.assets[] | select(.name|test("windows-amd64")) | .browser_download_url'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  4. Verify communication with a read-only command first
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;\esptool.exe&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;--port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;COM3&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;flash-id&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;Detecting chip type... ESP32-S3
Chip type:          ESP32-S3 (QFN56) (revision v0.2)
USB mode:           USB-Serial/JTAG
Detected flash size: 16MB
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;No manual download mode was necessary&lt;/strong&gt; — esptool resets the chip through USB-Serial/JTAG. If it cannot connect, hold the reset button for about 2 seconds and release it once the internal green LED lights up.&lt;/p&gt;

&lt;h3&gt;
  
  
  5. Flash
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight powershell"&gt;&lt;code&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;\esptool.exe&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nt"&gt;--port&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;COM3&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;write-flash&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;0x0&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nx"&gt;stackchan-v1.4.4.bin&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;Wrote 12783792 bytes (3185393 compressed) at 0x00000000 in 47.7 seconds
Verifying written data...
Hash of data verified.
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;About 50 seconds for just under 12.8 MB.&lt;/p&gt;

&lt;h3&gt;
  
  
  6. Confirm the version
&lt;/h3&gt;

&lt;p&gt;Look for &lt;code&gt;app_init: App version: 1.4.4&lt;/code&gt; in the boot log. With that in place, pairing succeeded on the first try, and I went straight through AI agent setup, Wi-Fi setup, and an actual voice conversation.&lt;/p&gt;

&lt;h2&gt;
  
  
  Other traps worth knowing
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Opening the serial port reboots the device
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;rst:0x15 (USB_UART_CHIP_RESET),boot:0x2b (SPI_FAST_FLASH_BOOT)
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;You cannot capture logs and operate the device at the same time.&lt;/strong&gt; Opening the port in the middle of the setup wizard sends you back to the start (it did for me). The flip side is useful: if you want the boot log from the very first line, open the port and then trigger a reset.&lt;/p&gt;

&lt;h3&gt;
  
  
  There is no interactive serial console
&lt;/h3&gt;

&lt;p&gt;Sending commands produces no response, and the write itself blocks. Treat the serial port as read-only.&lt;/p&gt;

&lt;h3&gt;
  
  
  Do not power both USB-C ports at once
&lt;/h3&gt;

&lt;p&gt;The unit has a USB-C port on the CoreS3 and another on the heel (base). 5V fed into the CoreS3 side is routed to the base through the M-BUS &lt;code&gt;BUS_5V&lt;/code&gt; line, so &lt;strong&gt;powering both ports puts two supplies on the same rail&lt;/strong&gt;. The official documentation's "try powering from both ports" means one at a time. The heel port is the recommended one for power and flashing.&lt;/p&gt;

&lt;h3&gt;
  
  
  Servo zero positions survive a full flash erase
&lt;/h3&gt;

&lt;p&gt;I backed up NVS before flashing, but the same zero positions were read back afterwards (&lt;code&gt;[ScsServo] id: 1 get zero pos: 460 from settings&lt;/code&gt;). &lt;strong&gt;Those values appear to live in the servos themselves&lt;/strong&gt;, so a full-flash rewrite does not lose them.&lt;/p&gt;

&lt;h2&gt;
  
  
  A separate warning about charging
&lt;/h2&gt;

&lt;p&gt;The firmware sets the AXP2101 charge current to &lt;strong&gt;700mA&lt;/strong&gt; at boot:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight cpp"&gt;&lt;code&gt;&lt;span class="k"&gt;auto&lt;/span&gt; &lt;span class="n"&gt;ret&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;setChargerConstantCurr&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;XPOWERS_AXP2101_CHG_CUR_700MA&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The ESP32-S3 enumerates as a USB 2.0 device, so &lt;strong&gt;a PC USB port gives you 500mA at most&lt;/strong&gt;. Combined with the system draw (LCD, camera, servos, radio) that exceeds the input limit, and &lt;strong&gt;charging never starts from a PC port&lt;/strong&gt;. Use a USB charger with headroom at 5V.&lt;/p&gt;

&lt;p&gt;If the servos do not report their current position (&lt;code&gt;[ScsServo] ignore invalid current pos: -1&lt;/code&gt;), read that as another sign of insufficient power.&lt;/p&gt;

&lt;h2&gt;
  
  
  Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;No devices found&lt;/code&gt; &lt;strong&gt;did not describe what was happening&lt;/strong&gt;. The BLE connection and the handshake both succeeded; the failure was in response verification&lt;/li&gt;
&lt;li&gt;The real cause was &lt;strong&gt;old factory firmware&lt;/strong&gt;, fixed by flashing the latest official build over USB&lt;/li&gt;
&lt;li&gt;OTA ↔ Wi-Fi ↔ pairing form a loop, so &lt;strong&gt;USB flashing is the only exit&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Do not over-trust the error string — capture the log.&lt;/strong&gt; Every minute I spent re-checking permissions would have been saved by one serial capture&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Scripts and the full write-up are in the repository:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://github.com/yasumorishima/stackchan-lab" rel="noopener noreferrer"&gt;https://github.com/yasumorishima/stackchan-lab&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  References
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;Official docs: &lt;a href="https://docs.m5stack.com/en/stackchan" rel="noopener noreferrer"&gt;https://docs.m5stack.com/en/stackchan&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Firmware / app / server source: &lt;a href="https://github.com/m5stack/StackChan" rel="noopener noreferrer"&gt;https://github.com/m5stack/StackChan&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>m5stack</category>
      <category>esp32</category>
      <category>iot</category>
      <category>ble</category>
    </item>
    <item>
      <title>My Raspberry Pi 5 Kept Vanishing From the Network — Diagnosing With journalctl and Fixing It With a Watchdog</title>
      <dc:creator>YMori</dc:creator>
      <pubDate>Tue, 23 Jun 2026 14:33:34 +0000</pubDate>
      <link>https://dev.to/yasumorishima/my-raspberry-pi-5-kept-vanishing-from-the-network-diagnosing-with-journalctl-and-fixing-it-with-a-1m5p</link>
      <guid>https://dev.to/yasumorishima/my-raspberry-pi-5-kept-vanishing-from-the-network-diagnosing-with-journalctl-and-fixing-it-with-a-1m5p</guid>
      <description>&lt;h2&gt;
  
  
  The symptom
&lt;/h2&gt;

&lt;p&gt;I run a headless Raspberry Pi 5 (over Wi-Fi) via SSH and Tailscale. Every so often it would &lt;strong&gt;stay powered on (LED lit) but completely vanish from the network&lt;/strong&gt;:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;SSH over LAN → timeout&lt;/li&gt;
&lt;li&gt;SSH over Tailscale → timeout&lt;/li&gt;
&lt;li&gt;RDP → timeout&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;And it kept recurring — every time I "fixed" it by power-cycling. Here is how I actually diagnosed it and set up auto-recovery.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1: &lt;code&gt;tailscale status&lt;/code&gt; tells you device-side vs your-side
&lt;/h2&gt;

&lt;p&gt;When SSH hangs, it's tempting to assume "the Pi died." But the problem can be &lt;strong&gt;on your side&lt;/strong&gt; (your PC's Wi-Fi / VPN). Rule that out first.&lt;/p&gt;

&lt;p&gt;If you use Tailscale, &lt;code&gt;tailscale status&lt;/code&gt; is the fastest check:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;100.x.y.z   raspberrypi   user@   linux   active; relay "..."; offline, last seen 1h ago, tx 124956 rx 0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The key is the peer line: &lt;strong&gt;&lt;code&gt;offline, last seen 1h ago&lt;/code&gt;&lt;/strong&gt;. This comes from Tailscale's coordination server — it's a &lt;strong&gt;third-party fact, independent of your own SSH attempt&lt;/strong&gt;.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;peer &lt;code&gt;offline&lt;/code&gt; → the Pi really did drop off the network&lt;/li&gt;
&lt;li&gt;peer &lt;code&gt;active&lt;/code&gt;/online → it's &lt;em&gt;your&lt;/em&gt; side that's broken (your VPN/network)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;code&gt;last seen&lt;/code&gt; also tells you &lt;strong&gt;when&lt;/strong&gt; it dropped. &lt;code&gt;tx ... rx 0&lt;/code&gt; means "you're sending, but getting zero bytes back."&lt;/p&gt;

&lt;p&gt;Confirm your own side is healthy too via &lt;code&gt;tailscale status --json&lt;/code&gt; → &lt;code&gt;BackendState=Running&lt;/code&gt; / &lt;code&gt;Self.Online=true&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 2: After recovery, read &lt;code&gt;journalctl -b -1&lt;/code&gt; (the &lt;em&gt;previous&lt;/em&gt; boot)
&lt;/h2&gt;

&lt;p&gt;This is the part people miss. While the Pi is off the network you obviously can't pull its logs — you read them after it's back. But &lt;strong&gt;the moment you reboot, the "current boot" is brand new&lt;/strong&gt;, so &lt;code&gt;journalctl&lt;/code&gt; (default) and &lt;code&gt;dmesg&lt;/code&gt; won't contain the moment it dropped.&lt;/p&gt;

&lt;p&gt;The drop is in the &lt;strong&gt;previous boot&lt;/strong&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# tail of the previous boot&lt;/span&gt;
journalctl &lt;span class="nt"&gt;-b&lt;/span&gt; &lt;span class="nt"&gt;-1&lt;/span&gt; &lt;span class="nt"&gt;--no-pager&lt;/span&gt; | &lt;span class="nb"&gt;tail&lt;/span&gt; &lt;span class="nt"&gt;-30&lt;/span&gt;

&lt;span class="c"&gt;# only wlan0 / network-related lines&lt;/span&gt;
journalctl &lt;span class="nt"&gt;-b&lt;/span&gt; &lt;span class="nt"&gt;-1&lt;/span&gt; &lt;span class="nt"&gt;--no-pager&lt;/span&gt; | &lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-iE&lt;/span&gt; &lt;span class="s2"&gt;"wlan0|cfg80211|brcmfmac|LinkChange|network is unreachable"&lt;/span&gt; | &lt;span class="nb"&gt;tail&lt;/span&gt; &lt;span class="nt"&gt;-40&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;What I found, in order:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;19:57  avahi-daemon: Withdrawing address record ... on wlan0   # IPv6 address flapping
20:08  tailscaled: LinkChange: major, rebinding ... rebind-reason=[time-jumped(13m50s),ips-changed,protocols-changed]
22:06  tailscaled: ... connect: network is unreachable          # fully down
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The important part: &lt;strong&gt;the logs kept flowing the whole time&lt;/strong&gt;. So this was not a full OS freeze or OOM — only the &lt;strong&gt;wlan0 / network layer dropped&lt;/strong&gt; while the OS stayed alive (no &lt;code&gt;oom&lt;/code&gt;, no &lt;code&gt;panic&lt;/code&gt; at the tail of &lt;code&gt;journalctl -b -1&lt;/code&gt;).&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;I couldn't pin the &lt;em&gt;root trigger&lt;/em&gt; (why wlan0 dropped) from the logs — there was no explicit driver crash line, and &lt;code&gt;vcgencmd get_throttled&lt;/code&gt; read &lt;code&gt;0x0&lt;/code&gt; after recovery (no undervoltage flag). So from here it's a &lt;strong&gt;symptomatic fix for wlan0 drops in general&lt;/strong&gt;.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  The fix: a NetworkManager watchdog
&lt;/h2&gt;

&lt;p&gt;If "the OS is alive but only the network is down," then a cron job that watches for it and restarts networking can auto-recover. Recent Pi OS uses NetworkManager (&lt;code&gt;resolv.conf&lt;/code&gt; says &lt;code&gt;# Generated by NetworkManager&lt;/code&gt;), so restart that.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;/home/pi/.local/bin/net-watchdog.sh&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;#!/usr/bin/env bash&lt;/span&gt;
&lt;span class="c"&gt;# If the gateway is unreachable, restart NetworkManager to bring wlan0 back.&lt;/span&gt;
&lt;span class="nv"&gt;GATEWAY&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;192.168.1.1   &lt;span class="c"&gt;# replace with your router IP&lt;/span&gt;
&lt;span class="nv"&gt;LOG&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;/home/pi/logs/net-watchdog.log
&lt;span class="k"&gt;if &lt;/span&gt;ping &lt;span class="nt"&gt;-c&lt;/span&gt; 3 &lt;span class="nt"&gt;-W&lt;/span&gt; 3 &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$GATEWAY&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt;/dev/null 2&amp;gt;&amp;amp;1&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;then
  &lt;/span&gt;&lt;span class="nb"&gt;exit &lt;/span&gt;0
&lt;span class="k"&gt;fi
&lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;date&lt;/span&gt; &lt;span class="nt"&gt;-Is&lt;/span&gt;&lt;span class="si"&gt;)&lt;/span&gt;&lt;span class="s2"&gt; gateway &lt;/span&gt;&lt;span class="nv"&gt;$GATEWAY&lt;/span&gt;&lt;span class="s2"&gt; unreachable, restarting NetworkManager"&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$LOG&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;
systemctl restart NetworkManager
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;/etc/cron.d/net-watchdog&lt;/code&gt; (every 3 minutes, as root):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;*/3 * * * * root /home/pi/.local/bin/net-watchdog.sh
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Install:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;chmod&lt;/span&gt; +x /home/pi/.local/bin/net-watchdog.sh
&lt;span class="nb"&gt;mkdir&lt;/span&gt; &lt;span class="nt"&gt;-p&lt;/span&gt; /home/pi/logs
&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"*/3 * * * * root /home/pi/.local/bin/net-watchdog.sh"&lt;/span&gt; | &lt;span class="nb"&gt;sudo tee&lt;/span&gt; /etc/cron.d/net-watchdog
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Verify it does NOT restart when the network is fine
&lt;/h3&gt;

&lt;p&gt;You don't want spurious restarts. Run it once by hand while the network is healthy:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;/home/pi/.local/bin/net-watchdog.sh
&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="nv"&gt;$?&lt;/span&gt;        &lt;span class="c"&gt;# → 0&lt;/span&gt;
&lt;span class="nb"&gt;cat&lt;/span&gt; /home/pi/logs/net-watchdog.log   &lt;span class="c"&gt;# → no such file (= it did not restart anything)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;While &lt;code&gt;ping&lt;/code&gt; succeeds, nothing is logged and &lt;code&gt;systemctl restart&lt;/code&gt; never runs. It only acts when the network is actually down.&lt;/p&gt;

&lt;h2&gt;
  
  
  Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;"Powered on but gone from the network" is usually a &lt;strong&gt;network-layer drop&lt;/strong&gt; — the OS is still alive.&lt;/li&gt;
&lt;li&gt;Fastest triage: &lt;strong&gt;&lt;code&gt;tailscale status&lt;/code&gt; (third-party offline record) → after recovery, &lt;code&gt;journalctl -b -1&lt;/code&gt; (the previous boot)&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;A &lt;strong&gt;gateway-ping → restart-NetworkManager watchdog&lt;/strong&gt; in cron auto-recovers within minutes, as long as the OS keeps running.&lt;/li&gt;
&lt;li&gt;Caveat: a cron-driven watchdog can't help if the OS itself stops executing.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I still haven't pinned the root cause, but at least I'm out of the "power-cycle it by hand every time" loop.&lt;/p&gt;

&lt;h2&gt;
  
  
  Follow-up: a &lt;em&gt;selective&lt;/em&gt; outage the watchdog couldn't see — and watchdog v2
&lt;/h2&gt;

&lt;p&gt;About two weeks after deploying the watchdog above, the Pi died in a &lt;strong&gt;completely different way&lt;/strong&gt; — and since the gateway ping still succeeded, the watchdog never fired.&lt;/p&gt;

&lt;h3&gt;
  
  
  The symptom (nothing like last time)
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;SSH (over Tailscale) worked fine&lt;/li&gt;
&lt;li&gt;Ping to the gateway worked fine&lt;/li&gt;
&lt;li&gt;But DNS resolution was completely dead (&lt;code&gt;getent hosts github.com&lt;/code&gt; failed)&lt;/li&gt;
&lt;li&gt;Behavior differed &lt;em&gt;per destination&lt;/em&gt;:

&lt;ul&gt;
&lt;li&gt;Google properties returned HTTPS 200&lt;/li&gt;
&lt;li&gt;GitHub / Cloudflare (1.1.1.1) / everything else → TCP connect timeout&lt;/li&gt;
&lt;li&gt;IPv6 was entirely dead&lt;/li&gt;
&lt;/ul&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;With "only Google works," it &lt;em&gt;looks&lt;/em&gt; like an ISP or router outage. Spoiler: it was &lt;strong&gt;the Pi itself again — a wedged wlan0 Wi-Fi client state&lt;/strong&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  Triage 1: hit the same destinations from another device on the same router (the decisive check)
&lt;/h3&gt;

&lt;p&gt;From a PC on the same Wi-Fi, GitHub and everything else worked normally. → &lt;strong&gt;The router and the ISP are innocent; the problem is Pi-specific.&lt;/strong&gt; Skipping this check would have sent me down the useless "reboot the router" path.&lt;/p&gt;

&lt;h3&gt;
  
  
  Triage 2: probe each DNS server directly over UDP
&lt;/h3&gt;

&lt;p&gt;No &lt;code&gt;dig&lt;/code&gt; or &lt;code&gt;nslookup&lt;/code&gt; on the Pi? A few lines of python3 can send a raw DNS query:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;socket&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;struct&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;q&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;server&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="c1"&gt;# minimal DNS A query for github.com
&lt;/span&gt;    &lt;span class="n"&gt;pkt&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;struct&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;pack&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;&amp;gt;HHHHHH&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x1234&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x0100&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;part&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="sa"&gt;b&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;github com&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;split&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
        &lt;span class="n"&gt;pkt&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="nf"&gt;bytes&lt;/span&gt;&lt;span class="p"&gt;([&lt;/span&gt;&lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;part&lt;/span&gt;&lt;span class="p"&gt;)])&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;part&lt;/span&gt;
    &lt;span class="n"&gt;pkt&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="sa"&gt;b&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="se"&gt;\x00&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;struct&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;pack&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;&amp;gt;HH&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;s&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;socket&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;socket&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;socket&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;AF_INET&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;socket&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;SOCK_DGRAM&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;settimeout&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sendto&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;pkt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;server&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;53&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;_&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;recvfrom&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;512&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;OK answers=&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;struct&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;unpack&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;&amp;gt;H&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="si"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;])[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;
    &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nb"&gt;Exception&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;FAIL &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nf"&gt;type&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="n"&gt;__name__&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;
    &lt;span class="k"&gt;finally&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;srv&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;192.168.1.1&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;100.100.100.100&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;8.8.8.8&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;1.1.1.1&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]:&lt;/span&gt;
    &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;srv&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nf"&gt;q&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;srv&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Result: &lt;code&gt;router=FAIL / Tailscale MagicDNS=FAIL / 8.8.8.8=OK / 1.1.1.1=FAIL&lt;/code&gt;. So DNS wasn't "down" — &lt;strong&gt;traffic to specific destinations was down&lt;/strong&gt;, and DNS failure was just one symptom. &lt;code&gt;ip route get&lt;/code&gt; and the firewall were both normal.&lt;/p&gt;

&lt;h3&gt;
  
  
  The fix: a &lt;em&gt;full&lt;/em&gt; wlan0 reconnect (&lt;code&gt;reapply&lt;/code&gt; is not enough)
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;nmcli dev reapply wlan0&lt;/code&gt; (re-applying the config) did &lt;strong&gt;not&lt;/strong&gt; fix it. What fixed it was a full &lt;strong&gt;disconnect → connect&lt;/strong&gt;. Since SSH itself rides on wlan0, detach the command so it survives your own disconnection:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;sudo nohup &lt;/span&gt;bash &lt;span class="nt"&gt;-c&lt;/span&gt; &lt;span class="s1"&gt;'nmcli dev disconnect wlan0; sleep 3; nmcli dev connect wlan0'&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt;/dev/null 2&amp;gt;&amp;amp;1 &amp;amp;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;About 20 seconds later everything was back: all destinations 200, and the router DNS that had "looked dead" was answering again. The DNS death, the selective timeouts, and the IPv6 blackout were all symptoms of one root cause: the wedged wlan0 client state.&lt;/p&gt;

&lt;h3&gt;
  
  
  watchdog v2: also probe external connectivity
&lt;/h3&gt;

&lt;p&gt;Gateway ping alone misses this failure mode, so I added a second tier:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;#!/bin/bash&lt;/span&gt;
&lt;span class="c"&gt;# net-watchdog v2: two-tier self-healing&lt;/span&gt;
&lt;span class="c"&gt;# tier1: gateway unreachable -&amp;gt; restart NetworkManager&lt;/span&gt;
&lt;span class="c"&gt;# tier2: gateway OK but external connectivity mostly dead (&amp;lt;2 of 4 probes) -&amp;gt; full wlan0 reconnect&lt;/span&gt;
&lt;span class="nv"&gt;LOG&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;/home/pi/logs/net-watchdog.log
&lt;span class="nv"&gt;STATE&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;/home/pi/logs/net-watchdog.last-reconnect
&lt;span class="nv"&gt;GW&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;192.168.1.1   &lt;span class="c"&gt;# replace with your router IP&lt;/span&gt;
&lt;span class="nb"&gt;mkdir&lt;/span&gt; &lt;span class="nt"&gt;-p&lt;/span&gt; /home/pi/logs

&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt; ping &lt;span class="nt"&gt;-c3&lt;/span&gt; &lt;span class="nt"&gt;-W3&lt;/span&gt; &lt;span class="nv"&gt;$GW&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt;/dev/null 2&amp;gt;&amp;amp;1&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;then
  &lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;date&lt;/span&gt; &lt;span class="nt"&gt;-Is&lt;/span&gt;&lt;span class="si"&gt;)&lt;/span&gt;&lt;span class="s2"&gt; gateway unreachable -&amp;gt; restart NetworkManager"&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="nv"&gt;$LOG&lt;/span&gt;
  systemctl restart NetworkManager
  &lt;span class="nb"&gt;exit &lt;/span&gt;0
&lt;span class="k"&gt;fi

&lt;/span&gt;&lt;span class="nv"&gt;alive&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0
&lt;span class="k"&gt;for &lt;/span&gt;url &lt;span class="k"&gt;in &lt;/span&gt;https://www.google.com https://github.com https://1.1.1.1 https://www.yahoo.co.jp&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;do
  &lt;/span&gt;curl &lt;span class="nt"&gt;-4&lt;/span&gt; &lt;span class="nt"&gt;-sS&lt;/span&gt; &lt;span class="nt"&gt;--max-time&lt;/span&gt; 5 &lt;span class="nt"&gt;-o&lt;/span&gt; /dev/null &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$url&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; 2&amp;gt;/dev/null &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nv"&gt;alive&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;$((&lt;/span&gt;alive+1&lt;span class="k"&gt;))&lt;/span&gt;
&lt;span class="k"&gt;done

if&lt;/span&gt; &lt;span class="o"&gt;[&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$alive&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="nt"&gt;-lt&lt;/span&gt; 2 &lt;span class="o"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;then
  &lt;/span&gt;&lt;span class="nv"&gt;now&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;date&lt;/span&gt; +%s&lt;span class="si"&gt;)&lt;/span&gt;
  &lt;span class="nv"&gt;last&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;cat&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$STATE&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; 2&amp;gt;/dev/null &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nb"&gt;echo &lt;/span&gt;0&lt;span class="si"&gt;)&lt;/span&gt;
  &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="o"&gt;[&lt;/span&gt; &lt;span class="k"&gt;$((&lt;/span&gt;now &lt;span class="o"&gt;-&lt;/span&gt; last&lt;span class="k"&gt;))&lt;/span&gt; &lt;span class="nt"&gt;-lt&lt;/span&gt; 1800 &lt;span class="o"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;then
    &lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;date&lt;/span&gt; &lt;span class="nt"&gt;-Is&lt;/span&gt;&lt;span class="si"&gt;)&lt;/span&gt;&lt;span class="s2"&gt; external dead (alive=&lt;/span&gt;&lt;span class="nv"&gt;$alive&lt;/span&gt;&lt;span class="s2"&gt;/4) but reconnected &amp;lt;30min ago, skip"&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="nv"&gt;$LOG&lt;/span&gt;
    &lt;span class="nb"&gt;exit &lt;/span&gt;0
  &lt;span class="k"&gt;fi
  &lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$now&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$STATE&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;
  &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;date&lt;/span&gt; &lt;span class="nt"&gt;-Is&lt;/span&gt;&lt;span class="si"&gt;)&lt;/span&gt;&lt;span class="s2"&gt; gateway OK but external dead (alive=&lt;/span&gt;&lt;span class="nv"&gt;$alive&lt;/span&gt;&lt;span class="s2"&gt;/4) -&amp;gt; wlan0 reconnect"&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="nv"&gt;$LOG&lt;/span&gt;
  nmcli dev disconnect wlan0
  &lt;span class="nb"&gt;sleep &lt;/span&gt;3
  nmcli dev connect wlan0
&lt;span class="k"&gt;fi
&lt;/span&gt;&lt;span class="nb"&gt;exit &lt;/span&gt;0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Design notes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;2 of 4 independent probes alive = healthy&lt;/strong&gt; — a single site being down never triggers a reconnect&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;https://1.1.1.1&lt;/code&gt; is IP-literal, so this probe &lt;strong&gt;still works when DNS is dead&lt;/strong&gt;
&lt;/li&gt;
&lt;li&gt;Reconnects have a &lt;strong&gt;30-minute cooldown&lt;/strong&gt; (no flapping)&lt;/li&gt;
&lt;li&gt;As with v1: run it by hand while healthy and confirm it does nothing before wiring it into cron&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Follow-up takeaways
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;A gateway-ping watchdog is blind to "selective outage caused by a wedged Wi-Fi client state"&lt;/li&gt;
&lt;li&gt;"Only some sites work" &lt;em&gt;looks&lt;/em&gt; like an ISP problem but &lt;strong&gt;can be your own device&lt;/strong&gt;. The fastest triage is hitting the same destinations from another device on the same LAN&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;nmcli dev reapply&lt;/code&gt; and a full disconnect/connect are different things — escalate to the full reconnect when reapply doesn't cut it&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>raspberrypi</category>
      <category>linux</category>
      <category>networking</category>
      <category>homelab</category>
    </item>
    <item>
      <title>I Had Claude Fable 5 Review My Indie Diary App Before It Got Pulled — The Bugs It Caught</title>
      <dc:creator>YMori</dc:creator>
      <pubDate>Mon, 15 Jun 2026 13:00:53 +0000</pubDate>
      <link>https://dev.to/yasumorishima/i-had-claude-fable-5-review-my-indie-diary-app-before-it-got-pulled-the-bugs-it-caught-8ca</link>
      <guid>https://dev.to/yasumorishima/i-had-claude-fable-5-review-my-indie-diary-app-before-it-got-pulled-the-bugs-it-caught-8ca</guid>
      <description>&lt;h2&gt;
  
  
  Intro
&lt;/h2&gt;

&lt;p&gt;I had Claude Fable 5 review and fix my Flutter diary app, &lt;strong&gt;Daily Diary&lt;/strong&gt; (live on Google Play), before a release. This is a record of that.&lt;/p&gt;

&lt;p&gt;Fable 5 was pulled worldwide just three days after launch (June 9 -&amp;gt; June 12, 2026) by a US Commerce Department export-control directive — but I got this pass in right before it went dark. Instead of generic "it caught what other models missed" claims, I'm pasting the &lt;strong&gt;actual diffs&lt;/strong&gt; from my repo's commit history.&lt;/p&gt;

&lt;h2&gt;
  
  
  The app: Daily Diary
&lt;/h2&gt;

&lt;p&gt;A simple offline diary app.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;📱 &lt;strong&gt;Offline-first&lt;/strong&gt;: data stays on the device, no cloud&lt;/li&gt;
&lt;li&gt;🌐 &lt;strong&gt;5 languages&lt;/strong&gt;: Japanese / English / Chinese / Korean / Spanish&lt;/li&gt;
&lt;li&gt;🌙 Dark mode, 📅 calendar with mood indicators, 📊 stats (streaks, mood trends), 🔍 full-text search, 🎲 random past entry, 💾 JSON export/import&lt;/li&gt;
&lt;li&gt;Google Play: &lt;a href="https://play.google.com/store/apps/details?id=com.diary.daily" rel="noopener noreferrer"&gt;https://play.google.com/store/apps/details?id=com.diary.daily&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Stack: Flutter 3.x / Dart, Provider, Hive (local NoSQL), Flutter intl.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I asked for
&lt;/h2&gt;

&lt;p&gt;Not "rewrite everything" — more like &lt;em&gt;"find and fix what's questionable quality-wise"&lt;/em&gt;: resolve &lt;code&gt;flutter analyze&lt;/code&gt; warnings (there were 47), set up tests, add CI, update dependencies.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. BuildContext used across an async gap
&lt;/h2&gt;

&lt;p&gt;Several spots touched &lt;code&gt;BuildContext&lt;/code&gt; (&lt;code&gt;Navigator&lt;/code&gt;, localization) after an &lt;code&gt;await&lt;/code&gt;. If the widget is disposed during the await, that throws. Fable inserted &lt;code&gt;if (!mounted) return;&lt;/code&gt; right after the await:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight diff"&gt;&lt;code&gt;   final db = DatabaseService();
   final existingEntry = await db.getEntryByDate(_selectedDate);
&lt;span class="gi"&gt;+  if (!mounted) return;
&lt;/span&gt;&lt;span class="err"&gt;
&lt;/span&gt;   final result = await Navigator.push(
     context,
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You rarely hit this in normal use, so it's easy to miss even when testing yourself.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Hoisting the l10n lookup before the await
&lt;/h2&gt;

&lt;p&gt;In the import handler &lt;code&gt;_importData&lt;/code&gt;, &lt;code&gt;AppLocalizations.of(context)&lt;/code&gt; (localized strings) was looked up repeatedly &lt;em&gt;after&lt;/em&gt; awaits. It got hoisted to a single lookup before the first await, with a &lt;code&gt;mounted&lt;/code&gt; guard before showing the confirmation dialog:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight diff"&gt;&lt;code&gt;       _isImporting = true;
     });
&lt;span class="gi"&gt;+
+    final l10n = AppLocalizations.of(context)!;
&lt;/span&gt;&lt;span class="err"&gt;
&lt;/span&gt;     try {
       final result = await FilePicker...
       ...
       if (file.bytes == null) {
&lt;span class="gd"&gt;-        final l10n = AppLocalizations.of(context)!;   // looked up after await
&lt;/span&gt;         throw Exception(l10n.fileReadError);
       }
       ...
&lt;span class="gd"&gt;-      final l10n = AppLocalizations.of(context)!;
&lt;/span&gt;&lt;span class="gi"&gt;+      if (!mounted) return;                            // guard before dialog
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  3. The test that tested nothing
&lt;/h2&gt;

&lt;p&gt;The leftover template &lt;code&gt;widget_test.dart&lt;/code&gt; referenced a non-existent &lt;code&gt;MyApp&lt;/code&gt; and didn't even compile (= effectively zero tests). It was removed and replaced with &lt;strong&gt;12 real tests&lt;/strong&gt; for the model and Hive persistence:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;removed test/widget_test.dart             (broken template)
added  test/diary_entry_test.dart   +49   (model serialization, etc.)
added  test/database_service_test.dart +92 (Hive-backed save/load)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;"You think you have tests, you don't" is the scariest state to be in, so fixing this mattered most.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. 47 analyzer warnings to zero
&lt;/h2&gt;

&lt;p&gt;The headline one was the &lt;code&gt;Color.withOpacity()&lt;/code&gt; deprecation (&lt;code&gt;withValues&lt;/code&gt; is now preferred for color precision). 27 spots replaced:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight diff"&gt;&lt;code&gt;&lt;span class="gd"&gt;-  color: Colors.white.withOpacity(0.1),
&lt;/span&gt;&lt;span class="gi"&gt;+  color: Colors.white.withValues(alpha: 0.1),
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Plus removing the deprecated &lt;code&gt;ColorScheme.background / onBackground&lt;/code&gt;, and migrating &lt;code&gt;RadioListTile&lt;/code&gt; (&lt;code&gt;groupValue / onChanged&lt;/code&gt; removed) to the &lt;code&gt;RadioGroup&lt;/code&gt; ancestor API. &lt;code&gt;dart fix --apply&lt;/code&gt; handled the mechanical ones; API migrations were done by hand.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Major dependency bumps
&lt;/h2&gt;

&lt;p&gt;Bumped the main dependencies, with two breaking API changes:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight diff"&gt;&lt;code&gt;&lt;span class="gd"&gt;-  google_mobile_ads: ^4.0.0
&lt;/span&gt;&lt;span class="gi"&gt;+  google_mobile_ads: ^9.0.0
&lt;/span&gt;&lt;span class="gd"&gt;-  share_plus: ^7.2.1
-  file_picker: ^8.1.4
&lt;/span&gt;&lt;span class="gi"&gt;+  share_plus: ^12.0.2
+  file_picker: ^11.0.2
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;share_plus&lt;/strong&gt;: &lt;code&gt;Share.shareXFiles(...)&lt;/code&gt; -&amp;gt; &lt;code&gt;SharePlus.instance.share(ShareParams(...))&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight diff"&gt;&lt;code&gt;&lt;span class="gd"&gt;-  await Share.shareXFiles(
-    [XFile(file.path)],
-    subject: fileName,
&lt;/span&gt;&lt;span class="gi"&gt;+  await SharePlus.instance.share(
+    ShareParams(
+      files: [XFile(file.path)],
+      subject: fileName,
+    ),
&lt;/span&gt;   );
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;file_picker&lt;/strong&gt;: the instance API &lt;code&gt;FilePicker.platform.pickFiles(...)&lt;/code&gt; became a static &lt;code&gt;FilePicker.pickFiles(...)&lt;/code&gt;.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight diff"&gt;&lt;code&gt;&lt;span class="gd"&gt;-  final result = await FilePicker.platform.pickFiles(
&lt;/span&gt;&lt;span class="gi"&gt;+  final result = await FilePicker.pickFiles(
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;On the pins: share_plus 13 conflicts with file_picker 11, and file_picker 12 is beta-only — so I settled on &lt;strong&gt;share_plus 12 / file_picker 11&lt;/strong&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  6. Added CI
&lt;/h2&gt;

&lt;p&gt;A new &lt;code&gt;flutter-ci.yml&lt;/code&gt; runs &lt;code&gt;flutter analyze&lt;/code&gt; + tests on push/PR, so the project can't slip back into the "broken test" state.&lt;/p&gt;

&lt;h2&gt;
  
  
  How it felt as a solo dev
&lt;/h2&gt;

&lt;p&gt;Honestly, what Fable caught wasn't flashy features — it was the class of things I systematically don't check on my own:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;async-gap mounted -&amp;gt; only crashes on certain timing&lt;/li&gt;
&lt;li&gt;a broken test -&amp;gt; you think you have tests, you don't&lt;/li&gt;
&lt;li&gt;a pile of deprecations -&amp;gt; works now, breaks later&lt;/li&gt;
&lt;li&gt;stale dependencies -&amp;gt; major bumps get postponed out of fear of breaking changes&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I can write the features myself, but "only fails under specific conditions," "are the tests actually testing anything," and "deprecated-API + dependency housekeeping" tend to slip without a dedicated reviewer. One thorough pass on that was genuinely useful for a side project.&lt;/p&gt;

&lt;p&gt;There are limits too: device-UI-dependent behavior (ads, export/import sharing) can't be verified in CI — I still test that on a real device. Concept, design, device testing, store submission, and final review stay my job.&lt;/p&gt;

&lt;p&gt;It's a shame it went dark after three days, but I'm glad I got this pass in before it did.&lt;/p&gt;

&lt;p&gt;Daily Diary is live on Google Play — give it a try:&lt;/p&gt;

&lt;p&gt;📱 &lt;a href="https://play.google.com/store/apps/details?id=com.diary.daily" rel="noopener noreferrer"&gt;https://play.google.com/store/apps/details?id=com.diary.daily&lt;/a&gt;&lt;/p&gt;

</description>
      <category>flutter</category>
      <category>claude</category>
      <category>ai</category>
      <category>indiedev</category>
    </item>
    <item>
      <title>transform: translateY(0) Breaks position: fixed — A Hidden Trap in SPA Animations</title>
      <dc:creator>YMori</dc:creator>
      <pubDate>Mon, 06 Apr 2026 09:55:09 +0000</pubDate>
      <link>https://dev.to/yasumorishima/transform-translatey0-breaks-position-fixed-a-hidden-trap-in-spa-animations-2hed</link>
      <guid>https://dev.to/yasumorishima/transform-translatey0-breaks-position-fixed-a-hidden-trap-in-spa-animations-2hed</guid>
      <description>&lt;h2&gt;
  
  
  The Bug
&lt;/h2&gt;

&lt;p&gt;One day I got this bug report on my Next.js site:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Clicking a photo near the bottom of the gallery opens a lightbox, but it's completely black. Scroll up and the image is there.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;A &lt;code&gt;position: fixed; inset: 0&lt;/code&gt; overlay was not covering the viewport — it was stuck at the top of the page. Browser bug? No. This is CSS working exactly as specified.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to Reproduce
&lt;/h2&gt;

&lt;p&gt;Two ingredients:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;An ancestor element with &lt;code&gt;transform&lt;/code&gt; set&lt;/strong&gt; (even &lt;code&gt;translateY(0)&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A descendant with &lt;code&gt;position: fixed&lt;/code&gt;&lt;/strong&gt;
&lt;/li&gt;
&lt;/ol&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight css"&gt;&lt;code&gt;&lt;span class="c"&gt;/* Page transition animation */&lt;/span&gt;
&lt;span class="k"&gt;@keyframes&lt;/span&gt; &lt;span class="n"&gt;page-enter&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nt"&gt;from&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;opacity&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="m"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nl"&gt;transform&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;translateY&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="m"&gt;12px&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="nt"&gt;to&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;opacity&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="m"&gt;1&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nl"&gt;transform&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;translateY&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="m"&gt;0&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="c"&gt;/* The culprit */&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nc"&gt;.page-enter&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;animation&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;page-enter&lt;/span&gt; &lt;span class="m"&gt;0.35s&lt;/span&gt; &lt;span class="n"&gt;ease&lt;/span&gt; &lt;span class="nb"&gt;both&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="c"&gt;/* both = keeps final values */&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight tsx"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Lightbox (descendant of .page-enter)&lt;/span&gt;
&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nt"&gt;div&lt;/span&gt; &lt;span class="na"&gt;className&lt;/span&gt;&lt;span class="p"&gt;=&lt;/span&gt;&lt;span class="s"&gt;"fixed inset-0 z-50 bg-black/90"&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;&lt;/span&gt;
  &lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nt"&gt;img&lt;/span&gt; &lt;span class="na"&gt;src&lt;/span&gt;&lt;span class="p"&gt;=&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nx"&gt;photo&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;url&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt; &lt;span class="p"&gt;/&amp;gt;&lt;/span&gt;
&lt;span class="p"&gt;&amp;lt;/&lt;/span&gt;&lt;span class="nt"&gt;div&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Near the top of the page, everything looks fine. Scroll down and open the lightbox — it renders at the &lt;strong&gt;top of the ancestor element&lt;/strong&gt;, not the viewport.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why This Happens — The CSS Spec
&lt;/h2&gt;

&lt;p&gt;From &lt;a href="https://developer.mozilla.org/en-US/docs/Web/CSS/position#fixed" rel="noopener noreferrer"&gt;MDN's &lt;code&gt;position: fixed&lt;/code&gt; documentation&lt;/a&gt;:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;The element is positioned relative to the initial containing block established by the viewport, &lt;strong&gt;except when one of its ancestors has a &lt;code&gt;transform&lt;/code&gt;, &lt;code&gt;perspective&lt;/code&gt;, or &lt;code&gt;filter&lt;/code&gt; property set to something other than &lt;code&gt;none&lt;/code&gt;&lt;/strong&gt;.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Ancestor's &lt;code&gt;transform&lt;/code&gt;
&lt;/th&gt;
&lt;th&gt;
&lt;code&gt;fixed&lt;/code&gt; is relative to&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;
&lt;code&gt;none&lt;/code&gt; or unset&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;viewport&lt;/strong&gt; (expected)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;translateY(0)&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;that ancestor&lt;/strong&gt; (broken)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;translateY(12px)&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;that ancestor&lt;/strong&gt; (broken)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;&lt;code&gt;translateY(0)&lt;/code&gt; is not the same as no transform.&lt;/strong&gt; It's a transform that moves nothing — but the CSS engine still creates a new containing block.&lt;/p&gt;

&lt;h2&gt;
  
  
  The &lt;code&gt;animation-fill-mode: both&lt;/code&gt; Trap
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight css"&gt;&lt;code&gt;&lt;span class="nc"&gt;.page-enter&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nl"&gt;animation&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;page-enter&lt;/span&gt; &lt;span class="m"&gt;0.35s&lt;/span&gt; &lt;span class="n"&gt;ease&lt;/span&gt; &lt;span class="nb"&gt;both&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;both&lt;/code&gt; (&lt;code&gt;forwards&lt;/code&gt; + &lt;code&gt;backwards&lt;/code&gt;) keeps the final keyframe values &lt;strong&gt;after the animation ends&lt;/strong&gt;. So &lt;code&gt;transform: translateY(0)&lt;/code&gt; persists for the lifetime of the element.&lt;/p&gt;

&lt;p&gt;The same applies to JavaScript inline styles:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight tsx"&gt;&lt;code&gt;&lt;span class="c1"&gt;// IntersectionObserver fadeIn component&lt;/span&gt;
&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nt"&gt;div&lt;/span&gt; &lt;span class="na"&gt;style&lt;/span&gt;&lt;span class="p"&gt;=&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;transform&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;visible&lt;/span&gt; &lt;span class="p"&gt;?&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;translateY(0)&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;translateY(16px)&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="c1"&gt;// After visible=true, translateY(0) stays forever&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;&lt;/span&gt;
  &lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nx"&gt;children&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;&amp;lt;/&lt;/span&gt;&lt;span class="nt"&gt;div&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Blast Radius
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Every &lt;code&gt;fixed&lt;/code&gt; descendant&lt;/strong&gt; of a &lt;code&gt;transform&lt;/code&gt;-bearing ancestor is affected:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Lightboxes / modals&lt;/li&gt;
&lt;li&gt;Toast notifications&lt;/li&gt;
&lt;li&gt;Cookie consent banners&lt;/li&gt;
&lt;li&gt;PWA install prompts&lt;/li&gt;
&lt;li&gt;Progress bars&lt;/li&gt;
&lt;li&gt;Scroll-to-top buttons&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Bottom navs and sticky headers may not visibly break (they sit at viewport edges), but they are technically affected too.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Fix
&lt;/h2&gt;

&lt;h3&gt;
  
  
  1. Use &lt;code&gt;transform: none&lt;/code&gt; (Most Important)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight css"&gt;&lt;code&gt;&lt;span class="k"&gt;@keyframes&lt;/span&gt; &lt;span class="n"&gt;page-enter&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nt"&gt;from&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;opacity&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="m"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nl"&gt;transform&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;translateY&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="m"&gt;12px&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="nt"&gt;to&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nl"&gt;opacity&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="m"&gt;1&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nl"&gt;transform&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;none&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="c"&gt;/* Not translateY(0) */&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight tsx"&gt;&lt;code&gt;&lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nt"&gt;div&lt;/span&gt; &lt;span class="na"&gt;style&lt;/span&gt;&lt;span class="p"&gt;=&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;transform&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;visible&lt;/span&gt; &lt;span class="p"&gt;?&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;none&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;translateY(16px)&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;transform: none&lt;/code&gt; means "no transform is applied" — no containing block is created.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Use &lt;code&gt;createPortal&lt;/code&gt; to Escape the DOM Tree (Defensive)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight tsx"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nx"&gt;createPortal&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;react-dom&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;Lightbox&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;createPortal&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="p"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nt"&gt;div&lt;/span&gt; &lt;span class="na"&gt;className&lt;/span&gt;&lt;span class="p"&gt;=&lt;/span&gt;&lt;span class="s"&gt;"fixed inset-0 z-50 bg-black/90"&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;&lt;/span&gt;
      &lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="cm"&gt;/* ... */&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;&amp;lt;/&lt;/span&gt;&lt;span class="nt"&gt;div&lt;/span&gt;&lt;span class="p"&gt;&amp;gt;,&lt;/span&gt;
    &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;body&lt;/span&gt; &lt;span class="c1"&gt;// Renders at body root — immune to ancestor CSS&lt;/span&gt;
  &lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;No matter what ancestors do, the overlay is not affected. This is a best practice for any viewport-covering overlay.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Do Both (Recommended)
&lt;/h3&gt;

&lt;p&gt;Fix the root cause with &lt;code&gt;transform: none&lt;/code&gt;, and add &lt;code&gt;createPortal&lt;/code&gt; as defense-in-depth. If someone later adds a new &lt;code&gt;transform&lt;/code&gt; ancestor, overlays still work.&lt;/p&gt;

&lt;h2&gt;
  
  
  Summary
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Don't&lt;/th&gt;
&lt;th&gt;Do&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Use &lt;code&gt;translateY(0)&lt;/code&gt; as animation end value&lt;/td&gt;
&lt;td&gt;Use &lt;code&gt;transform: none&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Render &lt;code&gt;fixed&lt;/code&gt; overlays deep in the DOM tree&lt;/td&gt;
&lt;td&gt;Use &lt;code&gt;createPortal(document.body)&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Add animations without checking &lt;code&gt;fixed&lt;/code&gt; elements&lt;/td&gt;
&lt;td&gt;Audit &lt;code&gt;fixed&lt;/code&gt; descendants when adding &lt;code&gt;transform&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;&lt;code&gt;translateY(0)&lt;/code&gt; and &lt;code&gt;none&lt;/code&gt; look identical but behave differently.&lt;/strong&gt; Miss this spec detail and every overlay on your site breaks the moment you add a page transition animation.&lt;/p&gt;

</description>
      <category>css</category>
      <category>nextjs</category>
      <category>react</category>
      <category>webdev</category>
    </item>
    <item>
      <title>NPB 2021 Backtest: Could a Bayesian Model Predict Last-Place-to-Champion?</title>
      <dc:creator>YMori</dc:creator>
      <pubDate>Tue, 24 Mar 2026 02:51:07 +0000</pubDate>
      <link>https://dev.to/yasumorishima/npb-2021-backtest-could-a-bayesian-model-predict-last-place-to-champion-3kl2</link>
      <guid>https://dev.to/yasumorishima/npb-2021-backtest-could-a-bayesian-model-predict-last-place-to-champion-3kl2</guid>
      <description>&lt;h2&gt;
  
  
  Introduction
&lt;/h2&gt;

&lt;p&gt;In a &lt;a href="https://dev.to/yasumorishima/adding-bayesian-ensemble-monte-carlo-to-an-npb-prediction-app-58po"&gt;previous article&lt;/a&gt;, I added Bayesian integration to my NPB prediction system. The 8-year backtest showed "97% probability of beating Marcel." But how did it perform in the &lt;strong&gt;worst year for predictions&lt;/strong&gt;?&lt;/p&gt;

&lt;p&gt;2021 was NPB's biggest upset: both &lt;strong&gt;Yakult (CL)&lt;/strong&gt; and &lt;strong&gt;Orix (PL)&lt;/strong&gt; went from last place to champions. I ran a full backtest with &lt;strong&gt;25 new foreign players individually projected&lt;/strong&gt; using FanGraphs and Baseball Savant data.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;GitHub&lt;/strong&gt;: &lt;a href="https://github.com/yasumorishima/npb-2021-backtest" rel="noopener noreferrer"&gt;npb-2021-backtest&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Main model&lt;/strong&gt;: &lt;a href="https://github.com/yasumorishima/npb-prediction" rel="noopener noreferrer"&gt;npb-prediction&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Team Standings: Predicted vs Actual
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Central League
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Team&lt;/th&gt;
&lt;th&gt;Actual&lt;/th&gt;
&lt;th&gt;Bayes (no foreign)&lt;/th&gt;
&lt;th&gt;Bayes (with foreign)&lt;/th&gt;
&lt;th&gt;Foreign Effect&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Yakult&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;73W (1st)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;69.5W (4th)&lt;/td&gt;
&lt;td&gt;70.7W (4th)&lt;/td&gt;
&lt;td&gt;+1.2W (Santana, Osuna)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Hanshin&lt;/td&gt;
&lt;td&gt;77W (2nd)&lt;/td&gt;
&lt;td&gt;72.8W (2nd)&lt;/td&gt;
&lt;td&gt;72.6W (2nd)&lt;/td&gt;
&lt;td&gt;-0.2W&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Giants&lt;/td&gt;
&lt;td&gt;61W (3rd)&lt;/td&gt;
&lt;td&gt;83.1W (1st)&lt;/td&gt;
&lt;td&gt;84.3W (1st)&lt;/td&gt;
&lt;td&gt;+1.2W (Smoak, Thames)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h3&gt;
  
  
  Pacific League
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Team&lt;/th&gt;
&lt;th&gt;Actual&lt;/th&gt;
&lt;th&gt;Bayes (no foreign)&lt;/th&gt;
&lt;th&gt;Bayes (with foreign)&lt;/th&gt;
&lt;th&gt;Foreign Effect&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Orix&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;70W (1st)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;64.5W (6th)&lt;/td&gt;
&lt;td&gt;62.0W (6th)&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;-2.5W (worse)&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SoftBank&lt;/td&gt;
&lt;td&gt;60W (4th)&lt;/td&gt;
&lt;td&gt;77.6W (1st)&lt;/td&gt;
&lt;td&gt;76.2W (1st)&lt;/td&gt;
&lt;td&gt;-1.4W&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;MAE: 10.4 wins → 10.7 wins.&lt;/strong&gt; Foreign player predictions slightly worsened accuracy.&lt;/p&gt;

&lt;h2&gt;
  
  
  Foreign Player Predictions vs Actual
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Accurate Predictions (average MLB players)
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Player&lt;/th&gt;
&lt;th&gt;Team&lt;/th&gt;
&lt;th&gt;Pred OPS&lt;/th&gt;
&lt;th&gt;Actual OPS&lt;/th&gt;
&lt;th&gt;Diff&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Kevin Cron&lt;/td&gt;
&lt;td&gt;Carp&lt;/td&gt;
&lt;td&gt;.703&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;.701&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;-.002&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Jose Osuna&lt;/td&gt;
&lt;td&gt;Swallows&lt;/td&gt;
&lt;td&gt;.683&lt;/td&gt;
&lt;td&gt;.694&lt;/td&gt;
&lt;td&gt;+.011&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cy Sneed&lt;/td&gt;
&lt;td&gt;Swallows&lt;/td&gt;
&lt;td&gt;ERA 3.53&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;ERA 3.41&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;-0.12&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h3&gt;
  
  
  Major Misses (extreme players)
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Player&lt;/th&gt;
&lt;th&gt;Team&lt;/th&gt;
&lt;th&gt;Pred OPS&lt;/th&gt;
&lt;th&gt;Actual OPS&lt;/th&gt;
&lt;th&gt;Diff&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Mike Gerber&lt;/td&gt;
&lt;td&gt;Dragons&lt;/td&gt;
&lt;td&gt;.862&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;.352&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;-.510&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Mel Rojas Jr.&lt;/td&gt;
&lt;td&gt;Tigers&lt;/td&gt;
&lt;td&gt;.867&lt;/td&gt;
&lt;td&gt;.663&lt;/td&gt;
&lt;td&gt;-.204&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Domingo Santana&lt;/td&gt;
&lt;td&gt;Swallows&lt;/td&gt;
&lt;td&gt;.713&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;.877&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;+.164&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Gerber&lt;/strong&gt; had an MLB wOBA of .127 (49.3% K rate) — the model over-regressed toward the mean, predicting .862 OPS when the actual was .352.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Santana&lt;/strong&gt; was predicted from 84 PA in 2020 (COVID-shortened). His career .757 OPS would have been more predictive.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Actually Drove the 2021 Standings
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Yakult's Championship Run
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Player&lt;/th&gt;
&lt;th&gt;2020&lt;/th&gt;
&lt;th&gt;2021&lt;/th&gt;
&lt;th&gt;Change&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Tetsuto Yamada&lt;/td&gt;
&lt;td&gt;OPS .766&lt;/td&gt;
&lt;td&gt;OPS .885&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;+.119&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Domingo Santana&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;OPS .877&lt;/td&gt;
&lt;td&gt;New signing&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Noboru Shimizu&lt;/td&gt;
&lt;td&gt;ERA 3.54&lt;/td&gt;
&lt;td&gt;ERA 2.39&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;-1.15&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h3&gt;
  
  
  Orix's Championship Run
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Player&lt;/th&gt;
&lt;th&gt;2020&lt;/th&gt;
&lt;th&gt;2021&lt;/th&gt;
&lt;th&gt;Change&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Yutaro Sugimoto&lt;/td&gt;
&lt;td&gt;OPS .695&lt;/td&gt;
&lt;td&gt;OPS .931&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;+.236&lt;/strong&gt; (HR King at 31)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Hiroya Miyagi&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;ERA 2.51 (147IP)&lt;/td&gt;
&lt;td&gt;20-year-old, 13 wins&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Yoshinobu Yamamoto&lt;/td&gt;
&lt;td&gt;ERA 2.20&lt;/td&gt;
&lt;td&gt;ERA 1.39&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;-0.81&lt;/strong&gt; (Sawamura Award)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Sugimoto and Miyagi's breakouts were impossible to predict from past data.&lt;/strong&gt; This is a structural change, not a statistical fluctuation.&lt;/p&gt;

&lt;h3&gt;
  
  
  Giants Collapse (Predicted 84.3W → Actual 61W)
&lt;/h3&gt;

&lt;p&gt;Sugano (ERA 1.97→3.19), Sakamoto (OPS .844→.657), Maru (OPS .899→.775) — three stars declining simultaneously. The Bayesian model &lt;strong&gt;trusted their skill metrics&lt;/strong&gt; and predicted even higher than Marcel.&lt;/p&gt;

&lt;h2&gt;
  
  
  Key Findings
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Average MLB players predicted well&lt;/strong&gt; (Cron .703 vs .701 actual)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Extreme players over-regressed&lt;/strong&gt; (Gerber .862 vs .352) → need regression limits&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Single-year small samples mislead&lt;/strong&gt; (Santana's 84 PA in 2020) → use career stats&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Bad MLB pitchers stay bad in NPB&lt;/strong&gt; (Sparkman 6.02→3.88 pred→6.88 actual)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;2021 was driven by Japanese player breakouts&lt;/strong&gt;, not foreign players&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;Individual foreign player projections improve accuracy for "average" players but carry risk for extreme cases. In 2021, Japanese player breakouts and collapses determined the standings — foreign player predictions had minimal impact (MAE +0.3 wins).&lt;/p&gt;

&lt;p&gt;This is a personal hobby project. There may be oversights in data collection and verification.&lt;/p&gt;

&lt;h2&gt;
  
  
  Data Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://baseball-data.com" rel="noopener noreferrer"&gt;Baseball Data Freak&lt;/a&gt; — NPB player stats&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://npb.jp" rel="noopener noreferrer"&gt;NPB Official&lt;/a&gt; — Official records&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.fangraphs.com" rel="noopener noreferrer"&gt;FanGraphs&lt;/a&gt; — MLB wOBA/K%/BB%&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://baseballsavant.mlb.com" rel="noopener noreferrer"&gt;Baseball Savant&lt;/a&gt; — MLB Statcast&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.baseball-reference.com" rel="noopener noreferrer"&gt;Baseball Reference&lt;/a&gt; — MLB/MiLB stats&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>baseball</category>
      <category>python</category>
      <category>bayesian</category>
      <category>datascience</category>
    </item>
    <item>
      <title>Adding Bayesian Ensemble + Monte Carlo to an NPB Prediction App</title>
      <dc:creator>YMori</dc:creator>
      <pubDate>Mon, 23 Mar 2026 21:48:12 +0000</pubDate>
      <link>https://dev.to/yasumorishima/adding-bayesian-ensemble-monte-carlo-to-an-npb-prediction-app-58po</link>
      <guid>https://dev.to/yasumorishima/adding-bayesian-ensemble-monte-carlo-to-an-npb-prediction-app-58po</guid>
      <description>&lt;h2&gt;
  
  
  Introduction
&lt;/h2&gt;

&lt;p&gt;I've been running a personal NPB (Japanese pro baseball) prediction app:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Dashboard&lt;/strong&gt;: &lt;a href="https://npb-prediction.streamlit.app/" rel="noopener noreferrer"&gt;npb-prediction.streamlit.app&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;GitHub&lt;/strong&gt;: &lt;a href="https://github.com/yasumorishima/npb-prediction" rel="noopener noreferrer"&gt;npb-prediction&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;It used Marcel projections (3-year weighted average) and ML (XGBoost/LightGBM). Decent, but I wanted better accuracy. After adding Bayesian corrections, the predicted standings changed significantly.&lt;/p&gt;

&lt;h2&gt;
  
  
  Terms
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Term&lt;/th&gt;
&lt;th&gt;Meaning&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Marcel&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Predict next year from weighted average of past 3 years&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Bayesian&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Combine prior knowledge with data. Gives uncertainty estimates&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;CI&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Credible interval — range where the true value falls with 80%/95% probability&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;OPS&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;On-base + Slugging. Overall batting metric&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;ERA&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Earned Run Average. Runs allowed per 9 innings&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;MAE&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Mean Absolute Error. Average prediction miss. Lower = better&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Problems with the Previous Approach
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Problem 1: All Foreign Players Treated as "Average"
&lt;/h3&gt;

&lt;p&gt;Marcel needs 3 years of NPB data. First-year foreign players have none, so all 24 of them were treated as league-average. Dalbec (Giants, .355 wOBA in MLB) and Hummel (BayStars, .240 wOBA) were calculated identically.&lt;/p&gt;

&lt;h3&gt;
  
  
  Problem 2: Skill Metrics Ignored
&lt;/h3&gt;

&lt;p&gt;Marcel averages past results directly. Two players with OPS .800 might have very different K% and BB% profiles, which affects how stable their performance will be next year.&lt;/p&gt;

&lt;h3&gt;
  
  
  Problem 3: No Uncertainty
&lt;/h3&gt;

&lt;p&gt;"Maki's OPS: .812" gives no sense of how much it might vary. The difference between .750-.870 and .790-.830 matters a lot for team projections.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Changed with Bayesian Integration
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Foreign Players: Average → Individual Predictions
&lt;/h3&gt;

&lt;p&gt;Built a model to convert MLB/KBO stats to NPB projections. For example, a .350 wOBA MLB hitter maps to approximately &lt;code&gt;.350 × 1.235 = .432&lt;/code&gt; NPB-equivalent wOBA.&lt;/p&gt;

&lt;p&gt;All 24 players' names and prior-league stats were individually web-verified (guessing English names from katakana is surprisingly error-prone).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Foreign hitter examples:&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Player&lt;/th&gt;
&lt;th&gt;Team&lt;/th&gt;
&lt;th&gt;Prior wOBA&lt;/th&gt;
&lt;th&gt;NPB Pred OPS&lt;/th&gt;
&lt;th&gt;80% CI&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Sano&lt;/td&gt;
&lt;td&gt;Dragons&lt;/td&gt;
&lt;td&gt;.370&lt;/td&gt;
&lt;td&gt;.760&lt;/td&gt;
&lt;td&gt;.632–.889&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Seymour&lt;/td&gt;
&lt;td&gt;Buffaloes&lt;/td&gt;
&lt;td&gt;.365&lt;/td&gt;
&lt;td&gt;.735&lt;/td&gt;
&lt;td&gt;.607–.863&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Dalbec&lt;/td&gt;
&lt;td&gt;Giants&lt;/td&gt;
&lt;td&gt;.355&lt;/td&gt;
&lt;td&gt;.725&lt;/td&gt;
&lt;td&gt;.577–.884&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Hummel&lt;/td&gt;
&lt;td&gt;BayStars&lt;/td&gt;
&lt;td&gt;.240&lt;/td&gt;
&lt;td&gt;.694&lt;/td&gt;
&lt;td&gt;.530–.849&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Foreign pitcher examples:&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Player&lt;/th&gt;
&lt;th&gt;Team&lt;/th&gt;
&lt;th&gt;Prior ERA&lt;/th&gt;
&lt;th&gt;NPB Pred ERA&lt;/th&gt;
&lt;th&gt;80% CI&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Quijada&lt;/td&gt;
&lt;td&gt;Swallows&lt;/td&gt;
&lt;td&gt;3.26&lt;/td&gt;
&lt;td&gt;2.76&lt;/td&gt;
&lt;td&gt;1.28–4.24&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Hjelle&lt;/td&gt;
&lt;td&gt;Buffaloes&lt;/td&gt;
&lt;td&gt;3.90&lt;/td&gt;
&lt;td&gt;3.34&lt;/td&gt;
&lt;td&gt;1.05–5.59&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cox&lt;/td&gt;
&lt;td&gt;BayStars&lt;/td&gt;
&lt;td&gt;8.86&lt;/td&gt;
&lt;td&gt;3.36&lt;/td&gt;
&lt;td&gt;1.82–4.85&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Players with poor prior-league stats get pulled toward league average (Bayesian regression effect), but with wider CIs = lower confidence.&lt;/p&gt;

&lt;h3&gt;
  
  
  Japanese Players: K%/BB%/BABIP Corrections
&lt;/h3&gt;

&lt;p&gt;Three models combined into a final prediction:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Weight&lt;/th&gt;
&lt;th&gt;Notes&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Marcel&lt;/td&gt;
&lt;td&gt;35%&lt;/td&gt;
&lt;td&gt;Strong baseline, especially for pitcher ERA&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Bayesian correction&lt;/td&gt;
&lt;td&gt;40%&lt;/td&gt;
&lt;td&gt;K%/BB%/BABIP/age adjustment on top of Marcel&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ML&lt;/td&gt;
&lt;td&gt;25%&lt;/td&gt;
&lt;td&gt;XGBoost/LightGBM&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Did Accuracy Improve?
&lt;/h2&gt;

&lt;p&gt;8-year backtest (2018–2025, predict each year and compare to actual):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;Marcel MAE&lt;/th&gt;
&lt;th&gt;Bayesian MAE&lt;/th&gt;
&lt;th&gt;Improvement prob.&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Hitter wOBA&lt;/td&gt;
&lt;td&gt;0.05023&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;0.04980&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;97.1%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Pitcher ERA&lt;/td&gt;
&lt;td&gt;1.23008&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;1.22241&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;97.1%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Small improvement, but consistent — &lt;strong&gt;97% probability of beating Marcel across 8 years&lt;/strong&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  Historical Marcel Accuracy for Context
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;Overall (8 years × 12 teams = 96 team-years):&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;Value&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Wins MAE&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;6.4 wins&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Avg rank error&lt;/td&gt;
&lt;td&gt;1.42 positions&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Exact rank rate&lt;/td&gt;
&lt;td&gt;18%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Within 1 rank&lt;/td&gt;
&lt;td&gt;65%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Recent examples of Marcel misses:&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Year&lt;/th&gt;
&lt;th&gt;Team&lt;/th&gt;
&lt;th&gt;Actual&lt;/th&gt;
&lt;th&gt;Predicted&lt;/th&gt;
&lt;th&gt;Miss&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;2025&lt;/td&gt;
&lt;td&gt;Swallows (CL)&lt;/td&gt;
&lt;td&gt;57W (6th)&lt;/td&gt;
&lt;td&gt;72W (4th)&lt;/td&gt;
&lt;td&gt;+15&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2024&lt;/td&gt;
&lt;td&gt;SoftBank (PL)&lt;/td&gt;
&lt;td&gt;91W (1st)&lt;/td&gt;
&lt;td&gt;75W (2nd)&lt;/td&gt;
&lt;td&gt;-16&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2024&lt;/td&gt;
&lt;td&gt;Buffaloes (PL)&lt;/td&gt;
&lt;td&gt;63W (5th)&lt;/td&gt;
&lt;td&gt;78W (1st)&lt;/td&gt;
&lt;td&gt;+15&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Patterns:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Overestimates bottom teams, underestimates top teams (regression to mean)&lt;/li&gt;
&lt;li&gt;Can't predict collapses (2024 Buffaloes: defending champions → 5th place)&lt;/li&gt;
&lt;li&gt;Foreign player impact not captured when all treated as average&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  How Did the 2026 Standings Change?
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Central League — Tigers Runaway Disappears, 4-Team Deadlock
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Team&lt;/th&gt;
&lt;th&gt;Marcel&lt;/th&gt;
&lt;th&gt;Bayesian&lt;/th&gt;
&lt;th&gt;Diff&lt;/th&gt;
&lt;th&gt;P(Pennant)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Tigers&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;80.1W (1st)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;71.5W (1st)&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;-8.6&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;26.0%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Giants&lt;/td&gt;
&lt;td&gt;70.7W (3rd)&lt;/td&gt;
&lt;td&gt;71.1W (2nd)&lt;/td&gt;
&lt;td&gt;+0.4&lt;/td&gt;
&lt;td&gt;20.2%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Dragons&lt;/td&gt;
&lt;td&gt;68.8W (5th)&lt;/td&gt;
&lt;td&gt;71.0W (3rd)&lt;/td&gt;
&lt;td&gt;+2.2&lt;/td&gt;
&lt;td&gt;21.2%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;BayStars&lt;/td&gt;
&lt;td&gt;71.3W (2nd)&lt;/td&gt;
&lt;td&gt;70.7W (4th)&lt;/td&gt;
&lt;td&gt;-0.6&lt;/td&gt;
&lt;td&gt;20.2%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Carp&lt;/td&gt;
&lt;td&gt;70.4W (4th)&lt;/td&gt;
&lt;td&gt;69.1W (5th)&lt;/td&gt;
&lt;td&gt;-1.3&lt;/td&gt;
&lt;td&gt;12.3%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Swallows&lt;/td&gt;
&lt;td&gt;64.3W (6th)&lt;/td&gt;
&lt;td&gt;61.2W (6th)&lt;/td&gt;
&lt;td&gt;-3.1&lt;/td&gt;
&lt;td&gt;0.1%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Tigers dropped from 80.1W to 71.5W (-8.6).&lt;/strong&gt; Skill corrections pulled them down. Giants at 71.1W even after losing Okamoto to MLB. &lt;strong&gt;Four teams within 0.8 wins&lt;/strong&gt; — Tigers 26%, Dragons 21%, Giants 20%, BayStars 20%. Swallows at 61.2W (78% last place) after Murakami's MLB departure.&lt;/p&gt;

&lt;h3&gt;
  
  
  Pacific League — Lions Surge
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Team&lt;/th&gt;
&lt;th&gt;Marcel&lt;/th&gt;
&lt;th&gt;Bayesian&lt;/th&gt;
&lt;th&gt;Diff&lt;/th&gt;
&lt;th&gt;P(Pennant)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Hawks&lt;/td&gt;
&lt;td&gt;80.5W (1st)&lt;/td&gt;
&lt;td&gt;81.3W (1st)&lt;/td&gt;
&lt;td&gt;+0.8&lt;/td&gt;
&lt;td&gt;47.9%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Fighters&lt;/td&gt;
&lt;td&gt;76.8W (2nd)&lt;/td&gt;
&lt;td&gt;79.1W (2nd)&lt;/td&gt;
&lt;td&gt;+2.3&lt;/td&gt;
&lt;td&gt;27.2%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Buffaloes&lt;/td&gt;
&lt;td&gt;73.8W (3rd)&lt;/td&gt;
&lt;td&gt;77.5W (3rd)&lt;/td&gt;
&lt;td&gt;+3.7&lt;/td&gt;
&lt;td&gt;17.6%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Lions&lt;/td&gt;
&lt;td&gt;68.6W (4th)&lt;/td&gt;
&lt;td&gt;74.9W (4th)&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;+6.3&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;7.1%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Eagles&lt;/td&gt;
&lt;td&gt;65.5W (5th)&lt;/td&gt;
&lt;td&gt;66.7W (5th)&lt;/td&gt;
&lt;td&gt;+1.2&lt;/td&gt;
&lt;td&gt;0.1%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Marines&lt;/td&gt;
&lt;td&gt;67.1W (6th)&lt;/td&gt;
&lt;td&gt;64.9W (6th)&lt;/td&gt;
&lt;td&gt;-2.2&lt;/td&gt;
&lt;td&gt;0.1%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Lions +6.3 wins&lt;/strong&gt; — foreign player projections offsetting Imai's MLB departure.&lt;/p&gt;

&lt;h2&gt;
  
  
  Summary
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Problem&lt;/th&gt;
&lt;th&gt;Before&lt;/th&gt;
&lt;th&gt;After&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Foreign players&lt;/td&gt;
&lt;td&gt;All league-average&lt;/td&gt;
&lt;td&gt;24 individual projections from prior-league stats&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Skill metrics&lt;/td&gt;
&lt;td&gt;Not used&lt;/td&gt;
&lt;td&gt;K%/BB%/BABIP corrections on Marcel&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Uncertainty&lt;/td&gt;
&lt;td&gt;None (point estimates)&lt;/td&gt;
&lt;td&gt;80%/95% credible intervals on every prediction&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Team standings&lt;/td&gt;
&lt;td&gt;Single number&lt;/td&gt;
&lt;td&gt;10,000 Monte Carlo sims with pennant probabilities&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Accuracy&lt;/td&gt;
&lt;td&gt;Marcel MAE 0.050&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;0.0498&lt;/strong&gt; (97% probability of improvement)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The accuracy gain is modest, but "foreign players are no longer invisible," "MLB departures are reflected," and "every prediction comes with uncertainty" meaningfully changed the standings picture. The CL went from "Tigers runaway" to a four-team deadlock.&lt;/p&gt;

&lt;h3&gt;
  
  
  Caveat: Data Limitations
&lt;/h3&gt;

&lt;p&gt;During this work, I discovered that &lt;strong&gt;players who moved to MLB (Murakami, Okamoto)&lt;/strong&gt; were still included in the team simulation — the roster filter only existed in the Streamlit display layer, not in the CSV generation pipeline. Fixed and regenerated, but &lt;strong&gt;there may be other oversights I haven't caught.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;This is a personal project without professional-grade QA. The data is best treated as automated model output, not authoritative predictions.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Dashboard&lt;/strong&gt;: &lt;a href="https://npb-prediction.streamlit.app/" rel="noopener noreferrer"&gt;npb-prediction.streamlit.app&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;GitHub&lt;/strong&gt;: &lt;a href="https://github.com/yasumorishima/npb-prediction" rel="noopener noreferrer"&gt;github.com/yasumorishima/npb-prediction&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Data Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://baseball-data.com" rel="noopener noreferrer"&gt;Baseball Data Freak&lt;/a&gt; — NPB player stats&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://npb.jp" rel="noopener noreferrer"&gt;NPB Official&lt;/a&gt; — Official records&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>baseball</category>
      <category>python</category>
      <category>bayesian</category>
      <category>datascience</category>
    </item>
    <item>
      <title>Adding Bayesian Ensemble + Monte Carlo to an NPB Prediction System</title>
      <dc:creator>YMori</dc:creator>
      <pubDate>Mon, 23 Mar 2026 20:55:48 +0000</pubDate>
      <link>https://dev.to/yasumorishima/adding-bayesian-ensemble-monte-carlo-to-an-npb-prediction-system-2fl1</link>
      <guid>https://dev.to/yasumorishima/adding-bayesian-ensemble-monte-carlo-to-an-npb-prediction-system-2fl1</guid>
      <description>&lt;h2&gt;
  
  
  Introduction
&lt;/h2&gt;

&lt;p&gt;In a previous article, I documented my journey adding Bayesian regression (Stan/Ridge) to my NPB (Japanese pro baseball) prediction system.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Previous article&lt;/strong&gt;: &lt;a href="https://dev.to/shogaku/beyond-marcel-adding-bayesian-regression-to-npb-baseball-predictions-a-15-step-journey-37a0"&gt;Beyond Marcel: Adding Bayesian Regression to NPB Predictions&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;That work lived in a separate experiment repository (&lt;a href="https://github.com/yasumorishima/npb-bayes-projection" rel="noopener noreferrer"&gt;npb-bayes-projection&lt;/a&gt;). This article covers adding those pieces into the main app — a 7-phase process that touched 19 files and added 4,087 lines.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;GitHub&lt;/strong&gt;: &lt;a href="https://github.com/yasumorishima/npb-prediction" rel="noopener noreferrer"&gt;npb-prediction&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Live dashboard&lt;/strong&gt;: &lt;a href="https://npb-prediction.streamlit.app/" rel="noopener noreferrer"&gt;npb-prediction.streamlit.app&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Before: Point Estimates Only
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Marcel (3-year weighted avg) → ML (XGBoost/LightGBM)
    ↓                              ↓
  Point estimate               Point estimate
    ↓
Pythagorean Win% → Team standings
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Problems:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;No uncertainty quantification&lt;/li&gt;
&lt;li&gt;24 new foreign players treated as league-average (wRAA=0)&lt;/li&gt;
&lt;li&gt;Marcel and ML run independently — no ensemble&lt;/li&gt;
&lt;li&gt;Team standings are a single number with no confidence interval&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  After: Bayesian Ensemble + Monte Carlo
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Layer 1: Marcel (unchanged)
    ↓
Layer 2: Stan Bayesian correction
  - Japanese: Ridge correction via K%/BB%/BABIP/age
  - Foreign: Prior-league stats × league-specific conversion (Stan v2)
    ↓
Layer 3: ML (XGBoost/LightGBM)
    ↓
Layer 4: BMA (Bayesian Model Averaging)
  - Marcel 35% + Stan 40% + ML 25%
  - 80%/95% credible intervals on every prediction
    ↓
Monte Carlo 10,000 draws → Team win distributions
  - P(pennant) / P(Climax Series) / P(last place)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  The 7 Phases
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Phase 1: Japanese Player Bayesian Inference
&lt;/h3&gt;

&lt;p&gt;The key design decision: &lt;strong&gt;Stan does not run at inference time.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;cmdstanpy is heavy to install and won't fit on a Raspberry Pi 5 (4GB RAM). Instead, I pre-compute posterior parameters into &lt;code&gt;posteriors.json&lt;/code&gt; during training (in GitHub Actions), then sample with NumPy at runtime.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# posteriors.json structure (hitter example)
&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;japanese_hitter&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;beta&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mf"&gt;0.152&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mf"&gt;0.089&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mf"&gt;0.245&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mf"&gt;0.003&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;sigma_residual&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mf"&gt;0.06215&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;feature_names&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;K_pct&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;BB_pct&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;BABIP&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;age_from_peak&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;# Runtime sampling (milliseconds, not seconds)
&lt;/span&gt;&lt;span class="n"&gt;z&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;features&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;scaler_mean&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="n"&gt;scaler_std&lt;/span&gt;
&lt;span class="n"&gt;correction&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;beta&lt;/span&gt; &lt;span class="o"&gt;@&lt;/span&gt; &lt;span class="n"&gt;z&lt;/span&gt;
&lt;span class="n"&gt;samples&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;marcel_value&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;correction&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;rng&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;normal&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;sigma&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;size&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;5000&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;ci_80&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;percentile&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;samples&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;90&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Phase 2: Foreign Player Stan v2 Predictions
&lt;/h3&gt;

&lt;p&gt;The most labor-intensive phase. I had to web-verify all 24 foreign players individually:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Katakana name → correct English name&lt;/li&gt;
&lt;li&gt;Origin league (MLB / KBO / independent)&lt;/li&gt;
&lt;li&gt;Most recent season stats&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Lesson learned: Never guess English names from katakana.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Over 10 of my initial 28 guesses were wrong:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;NPB Name&lt;/th&gt;
&lt;th&gt;Initial Guess&lt;/th&gt;
&lt;th&gt;Correct&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Dalbec&lt;/td&gt;
&lt;td&gt;Spencer Torkelson&lt;/td&gt;
&lt;td&gt;Bobby Dalbec&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Jerry&lt;/td&gt;
&lt;td&gt;Sean Gerry&lt;/td&gt;
&lt;td&gt;Sean Hjelle&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Lucas&lt;/td&gt;
&lt;td&gt;Josh Lucas&lt;/td&gt;
&lt;td&gt;Easton Lucas&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;I also misidentified 4 Japanese draft picks (with katakana names) as foreign players. The rule: &lt;strong&gt;verify every single entry via web search before committing.&lt;/strong&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Phase 3: Monte Carlo Team Simulation
&lt;/h3&gt;

&lt;p&gt;Player-level uncertainty propagates to team-level through 10,000 independent simulations:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;sim&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;10000&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;team&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;teams&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;rs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;sum&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;sample_hitter_runs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;h&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;h&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;team&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;hitters&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;ra&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;sum&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;sample_pitcher_runs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;team&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pitchers&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;rs&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ra&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;apply_park_factor&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;rs&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ra&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;team&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;wins&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;team&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="n"&gt;sim&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;143&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;rs&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="mf"&gt;1.83&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;rs&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="mf"&gt;1.83&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;ra&lt;/span&gt;&lt;span class="o"&gt;**&lt;/span&gt;&lt;span class="mf"&gt;1.83&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Foreign players get 1.5x sigma (wider uncertainty since they have no NPB data).&lt;/p&gt;

&lt;h3&gt;
  
  
  Phase 5: API Integration
&lt;/h3&gt;

&lt;p&gt;Three new FastAPI endpoints:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Endpoint&lt;/th&gt;
&lt;th&gt;Description&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/predict/hitter/{name}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Bayesian OPS + 80%/95% CI (added to existing)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/predict/foreign/{name}&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Foreign player Stan v2 projections (new)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;/standings/simulation&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Monte Carlo team standings (new)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h3&gt;
  
  
  Phase 6: Streamlit Integration
&lt;/h3&gt;

&lt;p&gt;The largest phase — added ~370 lines to the 1,669-line &lt;code&gt;streamlit_app.py&lt;/code&gt;:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Bayesian CI bars&lt;/strong&gt; on existing prediction pages (Plotly overlay bars for 80%/95% intervals)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Team Simulation page&lt;/strong&gt; (new) — fan chart + probability table&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Foreign Players page&lt;/strong&gt; (new) — prior-league stats + NPB projection with CI&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  Phase 7: BigQuery Integration
&lt;/h3&gt;

&lt;p&gt;Added 8 tables (25 → 33 total): Bayesian predictions, foreign player data, simulation results, and conversion factors.&lt;/p&gt;

&lt;h2&gt;
  
  
  Technical Decisions
&lt;/h2&gt;

&lt;h3&gt;
  
  
  posteriors.json vs. cmdstanpy at runtime
&lt;/h3&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;posteriors.json&lt;/th&gt;
&lt;th&gt;cmdstanpy runtime&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Inference speed&lt;/td&gt;
&lt;td&gt;NumPy only (ms)&lt;/td&gt;
&lt;td&gt;Stan call (seconds)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Memory&lt;/td&gt;
&lt;td&gt;Few KB&lt;/td&gt;
&lt;td&gt;Hundreds of MB&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Updates&lt;/td&gt;
&lt;td&gt;Annual retraining via GitHub Actions&lt;/td&gt;
&lt;td&gt;Fit every time&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;For a system running on RPi5 with 4GB RAM, this was the only viable option. With annual data updates, there's no need to re-fit on every request.&lt;/p&gt;

&lt;h3&gt;
  
  
  BMA Weight Rationale
&lt;/h3&gt;

&lt;p&gt;Marcel 35% + Stan 40% + ML 25% was determined by 8-year LOO-CV:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Stan correction improved Marcel 97.1% of the time (bootstrap)&lt;/li&gt;
&lt;li&gt;ML matched Marcel on hitter OPS but underperformed on pitcher ERA&lt;/li&gt;
&lt;li&gt;The 3-model BMA was more robust than any single model&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  The Full-Width Space Trap
&lt;/h3&gt;

&lt;p&gt;Marcel CSVs used full-width spaces (U+3000) in player names while sabermetrics CSVs used half-width spaces. This caused 237 of 463 players to fail matching until I normalized with a &lt;code&gt;player_join&lt;/code&gt; column.&lt;/p&gt;

&lt;h2&gt;
  
  
  Results
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;Value&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;New files&lt;/td&gt;
&lt;td&gt;12 (2 Python + 10 data)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Modified files&lt;/td&gt;
&lt;td&gt;7&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Lines added&lt;/td&gt;
&lt;td&gt;+4,087&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;BigQuery tables&lt;/td&gt;
&lt;td&gt;25 → 33&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Streamlit pages&lt;/td&gt;
&lt;td&gt;7 → 9&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Foreign players individually projected&lt;/td&gt;
&lt;td&gt;0 → 24&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The system moved from point estimates to probability distributions. "The Giants have a 42.6% chance of winning the pennant" is more useful than "The Giants are projected to win 74 games."&lt;/p&gt;

&lt;h2&gt;
  
  
  Takeaways
&lt;/h2&gt;

&lt;p&gt;Moving experiment code into an app has its own challenges, distinct from the experiments themselves:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Data quality matters more than model quality.&lt;/strong&gt; Incorrect foreign player names/stats would have propagated through the entire pipeline&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Design for your runtime constraints.&lt;/strong&gt; posteriors.json lets a 4GB RPi5 do Bayesian inference&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Uncertainty visualization needs thought.&lt;/strong&gt; CI bars, fan charts, and probability tables each communicate different aspects of the same distributions&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Phase 4 (automated Stan retraining pipeline) remains for next season. But the prediction system now runs Bayesian ensemble predictions end-to-end, from individual players to team championship probabilities.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Dashboard&lt;/strong&gt;: &lt;a href="https://npb-prediction.streamlit.app/" rel="noopener noreferrer"&gt;npb-prediction.streamlit.app&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;GitHub&lt;/strong&gt;: &lt;a href="https://github.com/yasumorishima/npb-prediction" rel="noopener noreferrer"&gt;github.com/yasumorishima/npb-prediction&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Data Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://baseball-data.com" rel="noopener noreferrer"&gt;Baseball Data Freak&lt;/a&gt; — NPB player stats&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://npb.jp" rel="noopener noreferrer"&gt;NPB Official&lt;/a&gt; — Official records&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>baseball</category>
      <category>python</category>
      <category>bayesian</category>
      <category>datascience</category>
    </item>
    <item>
      <title>5 Pitfalls of Grafana + BigQuery — When Your Dashboard Shows Nothing</title>
      <dc:creator>YMori</dc:creator>
      <pubDate>Sun, 22 Mar 2026 05:37:11 +0000</pubDate>
      <link>https://dev.to/yasumorishima/5-pitfalls-of-grafana-bigquery-when-your-dashboard-shows-nothing-35nl</link>
      <guid>https://dev.to/yasumorishima/5-pitfalls-of-grafana-bigquery-when-your-dashboard-shows-nothing-35nl</guid>
      <description>&lt;h2&gt;
  
  
  Introduction
&lt;/h2&gt;

&lt;p&gt;I built 7 Grafana dashboards (70+ panels) on Grafana Cloud with BigQuery as the data source. Along the way, I hit multiple issues where queries returned data through the API but panels showed nothing in the UI.&lt;/p&gt;

&lt;p&gt;Here are the 5 pitfalls I encountered and how to fix them. Verified on Grafana 13 + BigQuery datasource plugin.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Non-ASCII Column Aliases Need Backticks
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Symptom
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;Syntax error: Illegal input character&lt;/code&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Cause
&lt;/h3&gt;

&lt;p&gt;If you use non-ASCII characters (e.g., Japanese, Chinese) in column aliases, they must be wrapped in backticks.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- Fails&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;team&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="err"&gt;チーム&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;HR&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="err"&gt;本塁打&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="p"&gt;...&lt;/span&gt;

&lt;span class="c1"&gt;-- Works&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;team&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="nv"&gt;`チーム`&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;HR&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="nv"&gt;`本塁打`&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="p"&gt;...&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This also applies to mixed ASCII + non-ASCII aliases like &lt;code&gt;K率&lt;/code&gt; and references in &lt;code&gt;GROUP BY&lt;/code&gt; / &lt;code&gt;ORDER BY&lt;/code&gt; clauses.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. BigQuery Datasource Doesn't Support &lt;code&gt;format: "time_series"&lt;/code&gt;
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Symptom
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;error unmarshaling query JSON to the Query Model: invalid format value: time_series&lt;/code&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Fix
&lt;/h3&gt;

&lt;p&gt;Always use &lt;code&gt;format: "table"&lt;/code&gt;. For time series data, return a &lt;code&gt;TIMESTAMP&lt;/code&gt; column named &lt;code&gt;time&lt;/code&gt; — Grafana auto-detects it.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="k"&gt;CAST&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;date&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="nb"&gt;TIMESTAMP&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="nb"&gt;time&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="p"&gt;...&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  3. Historical Data in Timeseries Panels Shows "Data outside time range"
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Symptom
&lt;/h3&gt;

&lt;p&gt;Panel displays "Data outside time range" with a "Zoom to data" button.&lt;/p&gt;

&lt;h3&gt;
  
  
  Cause
&lt;/h3&gt;

&lt;p&gt;Timeseries panels filter by the dashboard time range (e.g., "Last 6 hours"). Historical data from 2015–2025 falls outside this range.&lt;/p&gt;

&lt;h3&gt;
  
  
  Fix
&lt;/h3&gt;

&lt;p&gt;Use &lt;strong&gt;barchart panels&lt;/strong&gt; for historical aggregations. Return the year as a string:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="k"&gt;CAST&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;year&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;STRING&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="nb"&gt;year&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="p"&gt;...&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  4. Extra fieldConfig Properties Can Break Barchart Rendering
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Symptom
&lt;/h3&gt;

&lt;p&gt;Barchart panel is completely blank. No error message. Query returns data when tested directly.&lt;/p&gt;

&lt;h3&gt;
  
  
  Cause
&lt;/h3&gt;

&lt;p&gt;In Grafana 13, adding &lt;code&gt;color&lt;/code&gt;, &lt;code&gt;decimals&lt;/code&gt;, &lt;code&gt;unit&lt;/code&gt;, or &lt;code&gt;custom.axisLabel&lt;/code&gt; to &lt;code&gt;fieldConfig.defaults&lt;/code&gt; can silently prevent barchart rendering.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="err"&gt;//&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;Broken&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;—&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;renders&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;nothing&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="nl"&gt;"fieldConfig"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"defaults"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"color"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"fixedColor"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"#5470c6"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"mode"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"fixed"&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"decimals"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"unit"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"none"&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;

&lt;/span&gt;&lt;span class="err"&gt;//&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;Works&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="nl"&gt;"fieldConfig"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"defaults"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{},&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"overrides"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Start with minimal config, verify it renders, then add properties one at a time.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Panels Inside Expanded Row's &lt;code&gt;panels&lt;/code&gt; Array Are Invisible
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Symptom
&lt;/h3&gt;

&lt;p&gt;Panels exist in the dashboard JSON but don't appear in the UI.&lt;/p&gt;

&lt;h3&gt;
  
  
  Cause
&lt;/h3&gt;

&lt;p&gt;Grafana row panels have two modes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Collapsed (&lt;code&gt;collapsed: true&lt;/code&gt;)&lt;/strong&gt;: child panels stored in the row's &lt;code&gt;panels&lt;/code&gt; array&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Expanded (&lt;code&gt;collapsed: false&lt;/code&gt;)&lt;/strong&gt;: child panels must be &lt;strong&gt;top-level siblings&lt;/strong&gt; after the row. The row's &lt;code&gt;panels&lt;/code&gt; array must be empty.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If &lt;code&gt;collapsed: false&lt;/code&gt; but the &lt;code&gt;panels&lt;/code&gt; array still contains panels, those panels are invisible.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="err"&gt;//&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;Broken&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;—&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;panels&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;inside&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;expanded&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;row&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;are&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;hidden&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"row"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"collapsed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"panels"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[{&lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"barchart"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"title"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Hidden Panel"&lt;/span&gt;&lt;span class="p"&gt;}]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;

&lt;/span&gt;&lt;span class="err"&gt;//&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;Fixed&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;—&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;panels&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;at&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;top&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;level&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;after&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;the&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="err"&gt;row&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"row"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"collapsed"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"panels"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[]}&lt;/span&gt;&lt;span class="err"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="nl"&gt;"type"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"barchart"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="nl"&gt;"title"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"Visible Panel"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Also check &lt;code&gt;gridPos.y&lt;/code&gt; — if a panel's Y position is above its row header, it won't appear in the expected section.&lt;/p&gt;

&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;Grafana + BigQuery is a powerful combination, but building dashboards via the API exposes issues you'd never encounter through the UI editor. The hardest to debug: "query is correct but panel is blank." Hope this saves you some time.&lt;/p&gt;

</description>
      <category>grafana</category>
      <category>bigquery</category>
      <category>gcp</category>
      <category>datavisualization</category>
    </item>
    <item>
      <title>Moving an NPB Prediction System to BigQuery — BQML and Cloud Run on the Free Tier</title>
      <dc:creator>YMori</dc:creator>
      <pubDate>Sun, 22 Mar 2026 00:34:23 +0000</pubDate>
      <link>https://dev.to/yasumorishima/moving-an-npb-prediction-system-to-bigquery-bqml-and-cloud-run-on-the-free-tier-4lb4</link>
      <guid>https://dev.to/yasumorishima/moving-an-npb-prediction-system-to-bigquery-bqml-and-cloud-run-on-the-free-tier-4lb4</guid>
      <description>&lt;h2&gt;
  
  
  Background
&lt;/h2&gt;

&lt;p&gt;I've been running an NPB (Japanese professional baseball) player performance prediction project for over a year.&lt;/p&gt;

&lt;p&gt;→ Previous articles:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://dev.to/yasunorim/why-marcel-beat-lightgbm-building-an-npb-player-performance-prediction-system-2ln4"&gt;Why Marcel Beat LightGBM: Building an NPB Player Performance Prediction System&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://dev.to/yasunorim/annual-auto-retraining-for-npb-baseball-predictions-with-github-actions-30ln"&gt;Annual Auto-Retraining for NPB Baseball Predictions with GitHub Actions&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The setup was: GitHub Actions fetches data → trains models → saves CSVs → Streamlit displays results. Data lived in CSVs, the API ran on a Raspberry Pi 5 Docker container, and analysis was done in local Python.&lt;/p&gt;

&lt;p&gt;I added Google BigQuery to centralize the data, run SQL analysis, compare BQML accuracy against Python ML, and deploy the API to Cloud Run. Everything fits within GCP's free tier.&lt;/p&gt;

&lt;p&gt;→ &lt;strong&gt;GitHub&lt;/strong&gt;: &lt;a href="https://github.com/yasumorishima/npb-prediction" rel="noopener noreferrer"&gt;https://github.com/yasumorishima/npb-prediction&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Why BigQuery
&lt;/h2&gt;

&lt;p&gt;Pain points with the CSV-based setup:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Full re-fetch every run&lt;/strong&gt; — The annual pipeline re-downloads all data from scratch. No incremental updates&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cross-analysis was tedious&lt;/strong&gt; — JOINing hitter stats with park factors meant writing pandas merge code every time&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Wanted SQL access&lt;/strong&gt; — Quick queries like "wRC+ TOP 10" or "age curve peak" required writing Python each time&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Wanted to try BQML&lt;/strong&gt; — How far can SQL-only ML go compared to Python?&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  Architecture
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;GitHub Actions (Annual Pipeline)
  ├── Data fetch (baseball-data.com / npb.jp)
  ├── Marcel projections
  ├── ML projections (XGBoost / LightGBM)
  ├── load_to_bq.py → BigQuery 25 tables
  ├── bqml_train.py → BQML 4 models
  └── Cloud Run deploy (on master merge)

BigQuery (npb dataset)
  ├── Raw data: 15 tables
  ├── Predictions: 4 tables
  ├── Metrics: 6 tables
  ├── BQML: 4 models
  └── Analysis views: 10

Display layer
  ├── Streamlit Cloud (dashboard)
  ├── Cloud Run API (serverless)
  └── Raspberry Pi 5 API (always-on)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Loading Data to BigQuery
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;load_to_bq.py&lt;/code&gt; loads CSV files into BigQuery.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;RAW_TABLE_MAP&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;npb_hitters_2015_2025.csv&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;raw_hitters&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;npb_pitchers_2015_2025.csv&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;raw_pitchers&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;npb_batting_detailed_2015_2025.csv&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;raw_batting_detailed&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;npb_sabermetrics_2015_2025.csv&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;sabermetrics&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="c1"&gt;# ... 25 tables
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;NPB data has column names like &lt;code&gt;K%&lt;/code&gt;, &lt;code&gt;BB%&lt;/code&gt;, &lt;code&gt;HR/9&lt;/code&gt; which BigQuery doesn't accept. The loader sanitizes them:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;new&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;new&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;%&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;_pct&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;new&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;new&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;replace&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;/&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;_per_&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;new&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;re&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sub&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;r&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;[^a-zA-Z0-9_]&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;_&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;new&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;All tables use &lt;code&gt;WRITE_TRUNCATE&lt;/code&gt; (full replace) on each run, so schema changes are handled automatically.&lt;/p&gt;

&lt;h2&gt;
  
  
  BQML: ML with SQL Only
&lt;/h2&gt;

&lt;p&gt;BigQuery ML lets you build features with SQL window functions and train models with &lt;code&gt;CREATE MODEL&lt;/code&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  Training View (Feature Engineering)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;OR&lt;/span&gt; &lt;span class="k"&gt;REPLACE&lt;/span&gt; &lt;span class="k"&gt;VIEW&lt;/span&gt; &lt;span class="nv"&gt;`npb.v_batter_train`&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt;
&lt;span class="k"&gt;WITH&lt;/span&gt; &lt;span class="n"&gt;base&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;player&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;season&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;OPS&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;wOBA&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;K_pct&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;BB_pct&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;Age&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;PA&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;...&lt;/span&gt;
  &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="nv"&gt;`npb.raw_hitters`&lt;/span&gt;
  &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;PA&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="mi"&gt;100&lt;/span&gt;
&lt;span class="p"&gt;),&lt;/span&gt;
&lt;span class="n"&gt;lagged&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="k"&gt;SELECT&lt;/span&gt;
    &lt;span class="n"&gt;player&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;season&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;LAG&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;OPS&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;OVER&lt;/span&gt; &lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;OPS_y1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;LAG&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;wOBA&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;OVER&lt;/span&gt; &lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;wOBA_y1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;LAG&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;OPS&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;OVER&lt;/span&gt; &lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;OPS_y2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;LAG&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;OPS&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;OVER&lt;/span&gt; &lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;LAG&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;OPS&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;OVER&lt;/span&gt; &lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;OPS_delta&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;LAG&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;Age&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;OVER&lt;/span&gt; &lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;27&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;age_from_peak&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;POW&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;LAG&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;Age&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;OVER&lt;/span&gt; &lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;27&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;age_sq&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;OPS&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="n"&gt;target_ops&lt;/span&gt;
  &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;base&lt;/span&gt;
  &lt;span class="k"&gt;WINDOW&lt;/span&gt; &lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;PARTITION&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;player&lt;/span&gt; &lt;span class="k"&gt;ORDER&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;season&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;lagged&lt;/span&gt; &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;OPS_y1&lt;/span&gt; &lt;span class="k"&gt;IS&lt;/span&gt; &lt;span class="k"&gt;NOT&lt;/span&gt; &lt;span class="k"&gt;NULL&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The same lag features, deltas, and age curves I had in Python, reimplemented as SQL window functions.&lt;/p&gt;

&lt;h3&gt;
  
  
  Model Training
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;OR&lt;/span&gt; &lt;span class="k"&gt;REPLACE&lt;/span&gt; &lt;span class="n"&gt;MODEL&lt;/span&gt; &lt;span class="nv"&gt;`npb.bqml_batter_ops`&lt;/span&gt;
&lt;span class="k"&gt;OPTIONS&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="n"&gt;model_type&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'BOOSTED_TREE_REGRESSOR'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;input_label_cols&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s1"&gt;'target_ops'&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
  &lt;span class="n"&gt;max_iterations&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;learn_rate&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="mi"&gt;05&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;early_stop&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;TRUE&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;AS&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;OPS_y1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;wOBA_y1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;K_pct_y1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;BB_pct_y1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
       &lt;span class="n"&gt;age_from_peak&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;age_sq&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;OPS_delta&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;...&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="nv"&gt;`npb.v_batter_train`&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;4 models total:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Target&lt;/th&gt;
&lt;th&gt;Type&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;bqml_batter_ops&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Next-year OPS&lt;/td&gt;
&lt;td&gt;Boosted Tree&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;bqml_batter_ops_linear&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Next-year OPS&lt;/td&gt;
&lt;td&gt;Linear Regression&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;bqml_pitcher_era&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Next-year ERA&lt;/td&gt;
&lt;td&gt;Boosted Tree&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;bqml_pitcher_era_linear&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Next-year ERA&lt;/td&gt;
&lt;td&gt;Linear Regression&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  BQML vs Python ML Accuracy
&lt;/h2&gt;

&lt;p&gt;Same data, same evaluation period, MAE comparison.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Batter OPS MAE (lower is better)&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;MAE&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;BQML Boosted Tree&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;.0642&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Python (XGBoost)&lt;/td&gt;
&lt;td&gt;.063&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Python (LightGBM)&lt;/td&gt;
&lt;td&gt;.066&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Marcel&lt;/td&gt;
&lt;td&gt;.063&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Pitcher ERA MAE (lower is better)&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;MAE&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;BQML Boosted Tree&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;.909&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Python (XGBoost)&lt;/td&gt;
&lt;td&gt;.93&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Python (LightGBM)&lt;/td&gt;
&lt;td&gt;.92&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Marcel&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;.78&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;BQML performed comparably to Python ML. For pitcher ERA, both fall short of Marcel (0.78) — an ongoing challenge for ML approaches.&lt;/p&gt;

&lt;p&gt;BQML uses more features (park factors, DIPS metrics, Marcel weighted averages), which may contribute to its Boosted Tree performance.&lt;/p&gt;

&lt;h2&gt;
  
  
  Analysis Views
&lt;/h2&gt;

&lt;p&gt;10 views for my own analysis use:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;View&lt;/th&gt;
&lt;th&gt;Purpose&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;v_batter_trend&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Player OPS/wOBA trends by season&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;v_pitcher_trend&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Player ERA/WHIP trends + FIP approximation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;v_team_pythagorean&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Team win% vs Pythagorean expectation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;v_sabermetrics_leaders&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;wRC+ leaderboard by season&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;v_marcel_accuracy&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Marcel historical accuracy validation&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;v_age_curve&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;NPB-wide age curve (OPS × age)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;v_park_effects&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Park factor impact analysis&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;v_data_coverage&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Season-by-season data coverage&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;v_data_quality&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Per-table NULL/missing value summary&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;For example, checking "2025 wRC+ TOP 10" or "age curve peak" now takes SQL instead of writing pandas code.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- Example query from my environment&lt;/span&gt;
&lt;span class="k"&gt;SELECT&lt;/span&gt; &lt;span class="n"&gt;player&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;team&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;season&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;wRC_plus&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;wOBA&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;OPS&lt;/span&gt;
&lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="nv"&gt;`npb.v_sabermetrics_leaders`&lt;/span&gt;
&lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;season&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;2025&lt;/span&gt;
&lt;span class="k"&gt;ORDER&lt;/span&gt; &lt;span class="k"&gt;BY&lt;/span&gt; &lt;span class="n"&gt;wrc_rank&lt;/span&gt;
&lt;span class="k"&gt;LIMIT&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Cloud Run Deployment
&lt;/h2&gt;

&lt;p&gt;Deployed the existing FastAPI to Cloud Run.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight docker"&gt;&lt;code&gt;&lt;span class="k"&gt;FROM&lt;/span&gt;&lt;span class="s"&gt; python:3.12-slim&lt;/span&gt;
&lt;span class="k"&gt;COPY&lt;/span&gt;&lt;span class="s"&gt; requirements.txt .&lt;/span&gt;
&lt;span class="k"&gt;RUN &lt;/span&gt;pip &lt;span class="nb"&gt;install&lt;/span&gt; &lt;span class="nt"&gt;--no-cache-dir&lt;/span&gt; &lt;span class="nt"&gt;-r&lt;/span&gt; requirements.txt
&lt;span class="k"&gt;COPY&lt;/span&gt;&lt;span class="s"&gt; . .&lt;/span&gt;
&lt;span class="k"&gt;CMD&lt;/span&gt;&lt;span class="s"&gt; ["uvicorn", "api:app", "--host", "0.0.0.0", "--port", "${PORT:-8080}"]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Merging to master triggers automatic deployment via Artifact Registry.&lt;/p&gt;

&lt;p&gt;The same API runs on both the Raspberry Pi 5 Docker container and Cloud Run.&lt;/p&gt;

&lt;h2&gt;
  
  
  Free Tier Usage
&lt;/h2&gt;

&lt;p&gt;Everything runs within GCP's free tier.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Resource&lt;/th&gt;
&lt;th&gt;Free Tier&lt;/th&gt;
&lt;th&gt;Usage&lt;/th&gt;
&lt;th&gt;% Used&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Storage&lt;/td&gt;
&lt;td&gt;10 GB/mo&lt;/td&gt;
&lt;td&gt;~5 MB&lt;/td&gt;
&lt;td&gt;0.05%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Queries&lt;/td&gt;
&lt;td&gt;1 TB/mo&lt;/td&gt;
&lt;td&gt;~22 GB&lt;/td&gt;
&lt;td&gt;2.2%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cloud Run&lt;/td&gt;
&lt;td&gt;2M requests/mo&lt;/td&gt;
&lt;td&gt;minimal&lt;/td&gt;
&lt;td&gt;≈0%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Daily BigQuery usage monitoring with projected month-end pace is sent to Discord.&lt;/p&gt;

&lt;h2&gt;
  
  
  GitHub Actions Pipeline
&lt;/h2&gt;

&lt;p&gt;The annual pipeline (&lt;code&gt;annual_update.yml&lt;/code&gt;) now includes BigQuery loading, BQML training, and Cloud Run deployment.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Step 1: fetch_npb_data.py       → Scrape hitter/pitcher stats
Step 2: fetch_npb_detailed.py   → Detailed batting stats (for wOBA)
Step 3: pythagorean.py          → Standings + Pythagorean win%
Step 4: sabermetrics.py         → wOBA/wRC+/wRAA calculation
Step 5: marcel_projection.py    → Marcel projections
Step 6: ml_projection.py        → ML projections + model save
Step 7: git commit &amp;amp; push       → Auto-commit data/
Step 8: load_to_bq.py           → Load all data to BigQuery  ← NEW
Step 9: bqml_train.py           → BQML train &amp;amp; evaluate      ← NEW
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;BQML steps use &lt;code&gt;continue-on-error: true&lt;/code&gt;, so BigQuery issues don't break the Python ML pipeline.&lt;/p&gt;

&lt;h2&gt;
  
  
  Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;BQML accuracy was comparable to Python. Writing features as SQL window functions takes getting used to, but views make them reusable&lt;/li&gt;
&lt;li&gt;Analysis views are quietly useful. SQL replaces pandas for routine queries&lt;/li&gt;
&lt;li&gt;At ~40,000 rows, free tier usage is negligible&lt;/li&gt;
&lt;li&gt;Having the API on both Cloud Run and RPi5 means one can go down without losing service&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Related Articles
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://dev.to/yasunorim/why-marcel-beat-lightgbm-building-an-npb-player-performance-prediction-system-2ln4"&gt;Why Marcel Beat LightGBM: Building an NPB Player Performance Prediction System&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://dev.to/yasunorim/annual-auto-retraining-for-npb-baseball-predictions-with-github-actions-30ln"&gt;Annual Auto-Retraining for NPB Baseball Predictions with GitHub Actions&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>bigquery</category>
      <category>gcp</category>
      <category>python</category>
      <category>baseball</category>
    </item>
    <item>
      <title>Monitoring the Strait of Hormuz Blockade with Open AIS Data and a Raspberry Pi</title>
      <dc:creator>YMori</dc:creator>
      <pubDate>Sun, 15 Mar 2026 23:57:21 +0000</pubDate>
      <link>https://dev.to/yasumorishima/monitoring-the-strait-of-hormuz-blockade-with-open-ais-data-and-a-raspberry-pi-45jp</link>
      <guid>https://dev.to/yasumorishima/monitoring-the-strait-of-hormuz-blockade-with-open-ais-data-and-a-raspberry-pi-45jp</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Data scope disclaimer&lt;/strong&gt;: All data in this article comes from &lt;a href="https://aisstream.io/" rel="noopener noreferrer"&gt;aisstream.io&lt;/a&gt;'s &lt;strong&gt;terrestrial AIS receivers&lt;/strong&gt;. Coverage in open water (mid-strait) is limited; satellite AIS would provide a more complete picture. All figures are from &lt;strong&gt;mid-March 2026&lt;/strong&gt; and the situation is evolving daily.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  What This Is
&lt;/h2&gt;

&lt;p&gt;In March 2026, shipping through the Strait of Hormuz — through which roughly 20% of the world's oil passes — was reported to be severely restricted. I built a monitoring system to observe this using free AIS (Automatic Identification System) data and a Raspberry Pi 5.&lt;/p&gt;

&lt;p&gt;This post covers the system architecture, the analytics pipeline, and what the data shows within the limitations of terrestrial AIS coverage.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Repository&lt;/strong&gt;: &lt;a href="https://github.com/yasumorishima/hormuz-ship-tracker" rel="noopener noreferrer"&gt;yasumorishima/hormuz-ship-tracker&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fb50c5uerjb0do6z2goxb.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fb50c5uerjb0do6z2goxb.png" alt="Persian Gulf vessel distribution (mid-March 2026) — traffic concentrated around UAE coast, strait center nearly empty"&gt;&lt;/a&gt;&lt;br&gt;
&lt;em&gt;Auto-generated snapshot (every 6 hours). Shows gate line positions, transit IN/OUT stats, and vessel type distribution. Note the concentration around UAE ports and the near-empty strait center.&lt;/em&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  AIS Data
&lt;/h2&gt;

&lt;p&gt;AIS is a maritime safety system where vessels automatically broadcast their position, speed, course, name, and type over VHF radio. It's mandatory for international vessels over 300 gross tonnage.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://aisstream.io/" rel="noopener noreferrer"&gt;aisstream.io&lt;/a&gt; aggregates terrestrial AIS receiver data worldwide and streams it via a free WebSocket API. This is the data source for this project.&lt;/p&gt;
&lt;h2&gt;
  
  
  Architecture
&lt;/h2&gt;


&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;aisstream.io (WebSocket)
  → Collector (AIS receiver + land filter + SQLite)
  → Analytics Engine (gate-line transit detection + vessel classification)
  → FastAPI + Leaflet.js + Chart.js (dashboard)
  → matplotlib (6-hourly snapshot → GitHub auto-push)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;


&lt;p&gt;Two Docker containers run 24/7 on a Raspberry Pi 5: the main collector/API and a snapshot cron job.&lt;/p&gt;
&lt;h2&gt;
  
  
  What the Data Shows
&lt;/h2&gt;
&lt;h3&gt;
  
  
  67% Anchored Ratio (mid-March 2026)
&lt;/h3&gt;

&lt;p&gt;Of ~290 monitored vessels, about 67% were stationary (speed &amp;lt; 0.5 knots). In a typical port area, this ratio is usually around 30–40%. The elevated value is notable.&lt;/p&gt;
&lt;h3&gt;
  
  
  35 Vessels Waiting 6+ Hours (mid-March 2026)
&lt;/h3&gt;

&lt;p&gt;Vessels that haven't moved for over 6 hours are counted as the "waiting fleet." About 35 vessels met this criterion, with 11 stuck for over 24 hours.&lt;/p&gt;

&lt;p&gt;Waiting fleet flags (estimated from MMSI MID):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Flag&lt;/th&gt;
&lt;th&gt;Count&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Panama&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Marshall Islands&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;UAE&lt;/td&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Kuwait&lt;/td&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Others&lt;/td&gt;
&lt;td&gt;1 each&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Panama and Marshall Islands are open registries — commonly used by large commercial ships and tankers. Seven tankers were among the waiting fleet.&lt;/p&gt;
&lt;h3&gt;
  
  
  Near-Zero Strait Transits on Terrestrial AIS (mid-March 2026)
&lt;/h3&gt;

&lt;p&gt;A virtual gate line across the narrowest point of the Strait of Hormuz detects vessel crossings automatically. Only 1 transit was detected in 24 hours.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Important caveat&lt;/strong&gt;: this only reflects what aisstream.io's &lt;strong&gt;terrestrial AIS receivers&lt;/strong&gt; can capture. Coverage in mid-strait open water is limited. News reports indicate some vessels (Turkish, Indian, Saudi-flagged) have been allowed limited passage — these may not appear in terrestrial AIS data. &lt;strong&gt;"No data" does not equal "no ships."&lt;/strong&gt; This caveat applies to all figures in this article.&lt;/p&gt;
&lt;h3&gt;
  
  
  Traffic Concentrated Around UAE Coast (mid-March 2026)
&lt;/h3&gt;

&lt;p&gt;Most data clusters around Dubai, Jebel Ali, and Fujairah. Three gate lines capture port approach traffic:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Gate&lt;/th&gt;
&lt;th&gt;Inbound&lt;/th&gt;
&lt;th&gt;Outbound&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Dubai / Jebel Ali Approach&lt;/td&gt;
&lt;td&gt;20&lt;/td&gt;
&lt;td&gt;9&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Fujairah Approach&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;7&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Strait of Hormuz&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Dubai inbound significantly exceeds outbound. Fujairah shows only outbound traffic — likely vessels departing after bunkering (refueling).&lt;/p&gt;
&lt;h2&gt;
  
  
  Technical Implementation
&lt;/h2&gt;
&lt;h3&gt;
  
  
  Gate-Line Transit Detection
&lt;/h3&gt;

&lt;p&gt;Virtual gate lines (line segments) are defined at the strait and port approaches. For each vessel, consecutive position reports are checked for intersection with each gate using computational geometry:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;segments_intersect&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p4&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;d1&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;cross_product&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;d2&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;cross_product&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;d3&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;cross_product&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p3&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;d4&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;cross_product&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;p1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;p4&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="nf"&gt;if &lt;/span&gt;&lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="n"&gt;d1&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;d2&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;d1&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;d2&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; \
       &lt;span class="p"&gt;((&lt;/span&gt;&lt;span class="n"&gt;d3&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;d4&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;d3&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;d4&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)):&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="bp"&gt;True&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Direction (INBOUND/OUTBOUND) is determined by the sign of the cross product relative to the gate vector. Same-vessel crossings within 6 hours are deduplicated.&lt;/p&gt;

&lt;h3&gt;
  
  
  Data-Driven Situation Assessment
&lt;/h3&gt;

&lt;p&gt;All dashboard text is auto-generated from data patterns. The system classifies the situation level based on strait transits, anchored ratio, and waiting fleet size:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;strait_transits&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;anchored_pct&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;40&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;level&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;critical&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;title&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Strait Transit Suspended&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="k"&gt;elif&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="n"&gt;strait_transits&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;level&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;elevated&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;title&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Limited Strait Transit&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="k"&gt;else&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;level&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;normal&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;title&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Monitoring Active&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;When conditions normalize, the UI automatically shifts to normal mode — no hardcoded crisis messaging.&lt;/p&gt;

&lt;h3&gt;
  
  
  MMSI → Flag Mapping
&lt;/h3&gt;

&lt;p&gt;Since aisstream.io's metadata doesn't reliably include country codes, flags are derived from the first 3 digits of the 9-digit MMSI number (Maritime Identification Digits). The system maps 100+ MIDs to countries.&lt;/p&gt;

&lt;h3&gt;
  
  
  Destination Normalization
&lt;/h3&gt;

&lt;p&gt;AIS destination fields are free-text and wildly inconsistent (DUBAI, AE DXB, AEDXB, DMC DUBAI, etc.). Over 40 variants are mapped to canonical port names.&lt;/p&gt;

&lt;h2&gt;
  
  
  4-Day Data Analysis Update (March 18)
&lt;/h2&gt;

&lt;p&gt;After 4 days of continuous collection (43,000+ position records, 384 unique vessels), several new insights emerged.&lt;/p&gt;

&lt;h3&gt;
  
  
  Traffic Density Heatmap
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F8vjes67l2xoum38yc7gr.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F8vjes67l2xoum38yc7gr.png" alt="Traffic Density Heatmap"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Left: Full Gulf hexbin density. Right: Zoomed strait with AIS dead zone. Bottom: Port area, flag state, and vessel type breakdowns.&lt;/em&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;Value&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Clean positions&lt;/td&gt;
&lt;td&gt;36,000&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Anomalous (filtered)&lt;/td&gt;
&lt;td&gt;7,300 (17%)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Unique vessels&lt;/td&gt;
&lt;td&gt;384&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Strait crossings confirmed&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Dubai / Jebel Ali gate crossings&lt;/td&gt;
&lt;td&gt;61&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h3&gt;
  
  
  Timelapse — 24 Hours of Vessel Movement
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fx3ro914woqzxiop2dfjy.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fx3ro914woqzxiop2dfjy.gif" alt="Vessel Movement Timelapse"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;24-hour vessel movement animation. Positions are linearly interpolated between data points, with land-crossing prevention.&lt;/em&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  AIS Data Quality: What the Anomalies Actually Are
&lt;/h3&gt;

&lt;p&gt;About 17% of positions contained anomalous data. Two distinct patterns were identified:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Anomaly&lt;/th&gt;
&lt;th&gt;Count&lt;/th&gt;
&lt;th&gt;Cause&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Speed = 102.3 kn&lt;/td&gt;
&lt;td&gt;~3,200&lt;/td&gt;
&lt;td&gt;AIS protocol "not available" sentinel (10-bit 0x3FF)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Speed 40–99 kn&lt;/td&gt;
&lt;td&gt;~4,100&lt;/td&gt;
&lt;td&gt;Coastal receiver decode errors&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The ~48 kn cluster was particularly interesting: on 2026-03-16 at 07:00 UTC, 4 vessels simultaneously appeared at the same coordinates in the strait with identical speeds. This was a single receiver malfunction — no ships were actually there. These anomalies had produced 41 false transit detections, which were eliminated by filtering positions with speed &amp;gt;= 40 kn.&lt;/p&gt;

&lt;p&gt;The dashboard now shows anomalous vessels with red dashed markers and a "DATA QUALITY WARNING" popup.&lt;/p&gt;

&lt;h3&gt;
  
  
  Browser-Based Replay
&lt;/h3&gt;

&lt;p&gt;The &lt;code&gt;/replay&lt;/code&gt; endpoint provides a Leaflet.js animated replay with play/pause, speed control (0.25x–16x), timeline scrubbing, and keyboard shortcuts.&lt;/p&gt;

&lt;h2&gt;
  
  
  Limitations
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Terrestrial AIS coverage&lt;/strong&gt;: Free aisstream.io data comes from shore-based receivers. Open-water coverage (mid-strait) is limited&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;AIS speed 102.3 knots&lt;/strong&gt;: The "not available" sentinel value (0x3FF). Must be filtered&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Speed 40–99 kn receiver glitches&lt;/strong&gt;: Coastal receiver decode errors produce phantom positions. Transit detection filters speed &amp;gt;= 40 kn&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Collection period&lt;/strong&gt;: Ongoing collection. Longer-term trend analysis requires further accumulation&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Summary
&lt;/h2&gt;

&lt;p&gt;Using aisstream.io's free API and a Raspberry Pi 5, this system continuously collects and analyzes vessel traffic across the entire Persian Gulf. After 4 days, 43,000+ positions have been collected, with heatmap visualization, timelapse animation, and data quality analysis fully implemented.&lt;/p&gt;

&lt;p&gt;Statistics are auto-updated every 6 hours.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;a href="https://github.com/yasumorishima/hormuz-ship-tracker/blob/master/docs/STATS.md" rel="noopener noreferrer"&gt;Live Statistics (auto-updated)&lt;/a&gt;&lt;/strong&gt; / &lt;strong&gt;&lt;a href="https://github.com/yasumorishima/hormuz-ship-tracker" rel="noopener noreferrer"&gt;Repository&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Data source: &lt;a href="https://aisstream.io/" rel="noopener noreferrer"&gt;aisstream.io&lt;/a&gt; / Land polygons: &lt;a href="https://www.naturalearthdata.com/" rel="noopener noreferrer"&gt;Natural Earth&lt;/a&gt;&lt;/p&gt;

</description>
      <category>python</category>
      <category>raspberrypi</category>
      <category>docker</category>
      <category>maritime</category>
    </item>
    <item>
      <title>I Built a WBC Quarterfinal Scouting App with MLB Statcast Data</title>
      <dc:creator>YMori</dc:creator>
      <pubDate>Fri, 13 Mar 2026 16:58:40 +0000</pubDate>
      <link>https://dev.to/yasumorishima/i-built-a-wbc-quarterfinal-scouting-app-with-mlb-statcast-data-2k49</link>
      <guid>https://dev.to/yasumorishima/i-built-a-wbc-quarterfinal-scouting-app-with-mlb-statcast-data-2k49</guid>
      <description>&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;A Streamlit scouting dashboard for the WBC 2026 Quarterfinal: Japan vs Venezuela.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;App&lt;/strong&gt;: &lt;a href="https://wbc-qf-jpn-ven.streamlit.app/" rel="noopener noreferrer"&gt;https://wbc-qf-jpn-ven.streamlit.app/&lt;/a&gt;&lt;br&gt;
&lt;strong&gt;GitHub&lt;/strong&gt;: &lt;a href="https://github.com/yasumorishima/wbc-scouting" rel="noopener noreferrer"&gt;https://github.com/yasumorishima/wbc-scouting&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;For the pool round, I built 30 team-level dashboards (20 teams). But quarterfinals are head-to-head matchups — you want to know "which pitch type is effective against this batter?" and "which zone has the highest opponent BA against this pitcher?" in one place.&lt;/p&gt;
&lt;h2&gt;
  
  
  5-Tab Structure
&lt;/h2&gt;
&lt;h3&gt;
  
  
  🎯 Tab 1: Matchup Preview
&lt;/h3&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F5vdm9hs313p3q42cq5d5.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F5vdm9hs313p3q42cq5d5.png" alt="Predicted Lineup Table"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Venezuela's predicted starting lineup (9 batters) table, an alert for Machado (NPB player, no Statcast data), and a bench/pinch-hit candidates table.&lt;/p&gt;

&lt;p&gt;Each batter expands into a full scouting report:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;6 key metrics (AVG/OBP/SLG/OPS/K%/BB%) with MLB average comparison&lt;/li&gt;
&lt;li&gt;Radar chart (5-axis, MLB average line overlay)&lt;/li&gt;
&lt;li&gt;Zone heatmaps (3x3, 5x5) — BA and xwOBA by zone, split by vs LHP/RHP&lt;/li&gt;
&lt;li&gt;Spray charts — split by vs LHP/RHP&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fb37i7l3zkyz08nyufm40.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fb37i7l3zkyz08nyufm40.png" alt="Spray Charts (vs LHP / vs RHP)"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Platoon splits (OPS/AVG/K%/BB% side by side)&lt;/li&gt;
&lt;li&gt;Pitching plan — overall + vs LHP + vs RHP. Auto-generated from pitch type whiff rates, zone-level BA, count-split OPS, and platoon data&lt;/li&gt;
&lt;li&gt;Defensive positioning — auto-generated from spray angle, ground ball rate, and exit velocity, split by pitcher handedness&lt;/li&gt;
&lt;li&gt;Pitch type performance table (BA, SLG, Whiff%, Chase%)&lt;/li&gt;
&lt;li&gt;Count-based performance (color-coded: green=hitter ahead, red=behind, amber=even)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;At the bottom, there's a full analysis section for the starting pitcher (Ranger Suárez, LHP) with hitting approach (as LHB/RHB), arsenal table, movement chart, location heatmaps, platoon splits, and pitch selection by count — all in collapsible expanders.&lt;/p&gt;
&lt;h3&gt;
  
  
  📋 Tab 2: Game Plan
&lt;/h3&gt;

&lt;p&gt;Statcast data organized by game phase:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F5pj7v6hvxee1uzuxfv26.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F5pj7v6hvxee1uzuxfv26.png" alt="Team Weakness Analysis"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Team weakness detection&lt;/strong&gt; — batters with K% ≥ 22.4% (MLB avg), BB% &amp;lt; 8.3%, or platoon OPS gap ≥ 80 pts, auto-extracted with player names and values&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Innings 1-3 vs Suárez (starter)&lt;/strong&gt; — Batting: SP's K%/BB%/Whiff%/velocity and pitch mix. Pitching: per-batter AVG/K%/BB% grouped by lineup position (#1-3, #4-6, #7-9)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Innings 4-5 (2nd time through or bullpen transition)&lt;/strong&gt; — Batting: bridge reliever stats. Pitching: MLB league-wide trend (opp OPS rises 15-20% on 2nd time through) plus batter classification by K% and BB%&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Innings 6+ (high-leverage)&lt;/strong&gt; — Batting: closer/setup K%/Whiff%/Chase%/velocity with pitcher type classification. Pitching: platoon matchup data for batters with significant splits, full per-batter stat line&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Pinch-hit candidates&lt;/strong&gt; — bench player AVG/OPS/K%&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Every piece of text is driven by MLB Statcast numbers only. No coaching instructions — just data.&lt;/p&gt;
&lt;h3&gt;
  
  
  ⚔️ Tab 3: Lineup Scouting
&lt;/h3&gt;

&lt;p&gt;Team batting radar chart at the top (AVG/OBP/SLG/K%/BB%, 5-axis, MLB average line overlay). Below that, a full roster table and a dropdown selector for individual player analysis (metrics, scouting summary, pitching plan, defensive positioning, radar chart, zone heatmaps, spray charts, etc.).&lt;/p&gt;
&lt;h3&gt;
  
  
  🎱 Tab 4: Starting Pitcher Analysis
&lt;/h3&gt;

&lt;p&gt;Ranger Suárez's pitching data. Metric cards (avg velocity, avg spin, whiff%, chase%, put away%, opp avg, etc.) and scouting summary, plus collapsible expanders for:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Hitting approach (as LHB / as RHB)&lt;/li&gt;
&lt;li&gt;Arsenal table (velocity mph/km/h, break, whiff%, put away%) + movement chart&lt;/li&gt;
&lt;li&gt;Pitch location heatmap + platoon splits&lt;/li&gt;
&lt;li&gt;Pitch selection by count (donut charts) + count-based performance&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F0ew8r4dl1ux5glvlyou8.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F0ew8r4dl1ux5glvlyou8.png" alt="Pitch Selection by Count"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;h3&gt;
  
  
  🔥 Tab 5: Bullpen Scouting
&lt;/h3&gt;

&lt;p&gt;Bullpen overview (all relievers' ERA, K%, velocity in one info box), then a dropdown selector for individual reliever analysis. Same structure as Tab 4 (metric cards, scouting summary, hitting approach, arsenal, heatmaps, count analysis).&lt;/p&gt;
&lt;h2&gt;
  
  
  Technical Highlights
&lt;/h2&gt;
&lt;h3&gt;
  
  
  Dynamic text generation from raw Statcast data
&lt;/h3&gt;

&lt;p&gt;Six generator functions compute per-player analysis from pitch-by-pitch data:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Function&lt;/th&gt;
&lt;th&gt;Purpose&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;generate_player_summary()&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Batter scouting summary (strengths/weaknesses)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;generate_pitcher_summary()&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Pitcher scouting summary&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;generate_pitching_plan()&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;How to pitch to a batter (pitch types, zones, counts, platoon)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;generate_hitting_plan()&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;How to hit a pitcher (hittable pitches, zones, counts)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;generate_defensive_positioning()&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Infield/outfield shift recommendation from spray data&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;generate_sp_pitch_analysis()&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Starting pitcher's pitch-by-pitch analysis&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Each function calculates stats from raw Statcast data and outputs only items that cross statistical thresholds:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# Example: identify the pitch type with highest opponent BA
&lt;/span&gt;&lt;span class="n"&gt;hittable&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;sorted&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;p&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;pt_stats&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;p&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ba&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;lambda&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ba&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="n"&gt;reverse&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;hittable&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;hittable&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ba&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;=&lt;/span&gt; &lt;span class="mf"&gt;0.250&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;h&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;hittable&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;lines&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;- **Highest opp BA pitch:** &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;h&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;label&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
        &lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt; (BA .&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nf"&gt;int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;h&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;ba&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="si"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;03&lt;/span&gt;&lt;span class="n"&gt;d&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;)&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  MLB average as baseline for every stat
&lt;/h3&gt;

&lt;p&gt;A raw number like "SLG .476" is meaningless without context. Every stat shows the MLB average alongside it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;K% 28.3% (MLB avg 22.4%)
BB% 6.1% (MLB avg 8.3%)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Handedness-aware zone names
&lt;/h3&gt;

&lt;p&gt;"Inside" and "outside" flip depending on batter handedness. &lt;code&gt;_zone_names_for_bats()&lt;/code&gt; automatically adjusts zone labels so "inside high" is always correct relative to the batter's stance.&lt;/p&gt;

&lt;h3&gt;
  
  
  Glossary built into every section
&lt;/h3&gt;

&lt;p&gt;Every stat has a &lt;strong&gt;?&lt;/strong&gt; tooltip (Streamlit's &lt;code&gt;help&lt;/code&gt; parameter) showing its definition and MLB average. Count displays include a reading guide ("Balls-Strikes" format) with color legend (🟢 hitter ahead, 🔴 hitter behind, 🟡 even).&lt;/p&gt;

&lt;h2&gt;
  
  
  Data Source
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://baseballsavant.mlb.com/" rel="noopener noreferrer"&gt;Baseball Savant&lt;/a&gt; Statcast data (2024-2025 MLB regular season)&lt;/li&gt;
&lt;li&gt;Retrieved via &lt;a href="https://github.com/jldbc/pybaseball" rel="noopener noreferrer"&gt;pybaseball&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Related
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://dev.to/shogaku/i-built-a-wbc-2026-scouting-dashboard-with-mlb-statcast-data-3k3j"&gt;I Built a WBC 2026 Scouting Dashboard with MLB Statcast Data&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>baseball</category>
      <category>python</category>
      <category>streamlit</category>
      <category>datascience</category>
    </item>
  </channel>
</rss>
