<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Musa Nayyer</title>
    <description>The latest articles on DEV Community by Musa Nayyer (@muzasio).</description>
    <link>https://dev.to/muzasio</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3889552%2Fea50db40-183c-4528-abe1-823df040d8ab.png</url>
      <title>DEV Community: Musa Nayyer</title>
      <link>https://dev.to/muzasio</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/muzasio"/>
    <language>en</language>
    <item>
      <title>Store-and-Forward Explained: Inside a Router, From the Book to Linux</title>
      <dc:creator>Musa Nayyer</dc:creator>
      <pubDate>Wed, 21 Oct 2026 19:56:44 +0000</pubDate>
      <link>https://dev.to/muzasio/store-and-forward-explained-inside-a-router-from-the-book-to-linux-4flj</link>
      <guid>https://dev.to/muzasio/store-and-forward-explained-inside-a-router-from-the-book-to-linux-4flj</guid>
      <description>&lt;p&gt;Most of us studied circuit and packet switching as course material. Here is what it actually looks like inside a router.&lt;/p&gt;

&lt;p&gt;The textbook defines store-and-forward in one sentence: a packet switch "must receive the entire packet before it can begin to transmit the first bit of the packet onto the outbound link" (Kurose &amp;amp; Ross, &lt;em&gt;Computer Networking: A Top-Down Approach&lt;/em&gt;, 7th ed., section 1.3.1). That sentence leaves three questions open:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Why&lt;/strong&gt; does the router wait?&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;What&lt;/strong&gt; does it do with the packet while it holds it?&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;What&lt;/strong&gt; does the waiting cost?&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;strong&gt;Scope:&lt;/strong&gt; the router here is a &lt;strong&gt;Linux machine with IP forwarding on&lt;/strong&gt;, because its source code is open. Hardware routers follow the same rules (RFC 1812 for IPv4) in specialized chips. Linux fast paths are left out. Kernel code is from mainline 7.3; function names can change between versions. Test system: Linux 7.2.7 (CachyOS), iproute2 7.2.0.&lt;/p&gt;

&lt;p&gt;The parts build on one another:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Store: nothing leaves until the last bit arrives&lt;/li&gt;
&lt;li&gt;Why wait: the error check is at the end&lt;/li&gt;
&lt;li&gt;Forward: what Linux does with the frame&lt;/li&gt;
&lt;li&gt;The four small edits at every hop&lt;/li&gt;
&lt;li&gt;The cost: N × L/R&lt;/li&gt;
&lt;li&gt;Why your data is cut into small packets&lt;/li&gt;
&lt;li&gt;Not every box waits: cut-through switches&lt;/li&gt;
&lt;/ol&gt;




&lt;h2&gt;
  
  
  Part 1: Store: Nothing Leaves Until the Last Bit Arrives
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fivqw4yzbyt3cv6xzdlys.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fivqw4yzbyt3cv6xzdlys.png" alt="A router buffer filling up, then sending" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The question
&lt;/h3&gt;

&lt;p&gt;When the first bits of a packet reach a router, when does the router start sending them on?&lt;/p&gt;

&lt;h3&gt;
  
  
  What I found
&lt;/h3&gt;

&lt;p&gt;Only after the last bit arrives. The router keeps incoming bits in a buffer until the &lt;strong&gt;whole&lt;/strong&gt; packet is in, then sends it on the next link.&lt;/p&gt;

&lt;p&gt;The time to push one packet onto a link is its &lt;strong&gt;transmission delay&lt;/strong&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;transmission delay = L / R

L = packet size in bits
R = link rate in bits per second
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt; (round numbers, not a measurement): a 1500-byte packet on a 100 Mbps link.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;1500 bytes × 8 = 12,000 bits
12,000 bits / 100,000,000 bits per second = 0.00012 s = 120 µs
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The book's example: one router between two hosts, propagation delay ignored. The source needs L/R to send the packet, and the router needs another L/R because it starts only after the last bit. Total: &lt;strong&gt;2L/R&lt;/strong&gt;, not L/R.&lt;/p&gt;

&lt;p&gt;Precision note: L here is the 1500-byte IP packet. Real Ethernet adds 18 bytes of frame (14-byte header, 4-byte FCS), plus an 8-byte preamble and a minimum 12-byte gap between frames. With those, 120 µs becomes about 123 µs. The examples keep 1500 bytes for round numbers.&lt;/p&gt;

&lt;h3&gt;
  
  
  Where the "store" happens on Linux
&lt;/h3&gt;

&lt;p&gt;Storing starts in the network card. A typical NIC receives the frame, checks it (Part 2), and copies it to a ring buffer in RAM through DMA. The driver then hands the kernel the complete frame in an &lt;code&gt;sk_buff&lt;/code&gt; (socket buffer). By the time routing code runs, the whole frame is already in memory.&lt;/p&gt;

&lt;h3&gt;
  
  
  Key idea
&lt;/h3&gt;

&lt;p&gt;A router never forwards half a packet. Every link costs at least one full L/R, because the next device cannot start until the last bit has arrived.&lt;/p&gt;




&lt;h2&gt;
  
  
  Part 2: Why Wait? The Error Check Is at the End
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fxh3dp7k107kzb59zvmiv.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fxh3dp7k107kzb59zvmiv.png" alt="An Ethernet frame with the FCS at the end" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The question
&lt;/h3&gt;

&lt;p&gt;Waiting costs time. Why not send bits on as soon as they arrive?&lt;/p&gt;

&lt;h3&gt;
  
  
  What I found
&lt;/h3&gt;

&lt;p&gt;One reason is the frame layout. An Ethernet frame ends with a 4-byte &lt;strong&gt;FCS&lt;/strong&gt; (Frame Check Sequence), a &lt;strong&gt;CRC-32&lt;/strong&gt; value. The sender calculates it over the frame, from the destination MAC to the end of the data (including any padding), and sends it &lt;strong&gt;last&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The receiving card runs the same calculation on the bytes it got and compares:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Same value:&lt;/strong&gt; no error detected. The frame moves on.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Different value:&lt;/strong&gt; some bits were damaged. The card &lt;strong&gt;discards&lt;/strong&gt; the frame.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Ethernet sends nothing back in either case: no acknowledgement on a pass, no negative acknowledgement on a fail (book, section 6.4.2). A bad frame just disappears, and higher layers such as TCP must notice the gap.&lt;/p&gt;

&lt;p&gt;Because the FCS comes last, it can only be checked after the &lt;strong&gt;last byte&lt;/strong&gt; arrives. Waiting stops a damaged frame at the first hop instead of carrying it across the network.&lt;/p&gt;

&lt;p&gt;A second check happens at the IP layer: RFC 1812 (section 5.2.2) says a router &lt;strong&gt;must&lt;/strong&gt; verify the IPv4 header checksum before processing a packet and &lt;strong&gt;must&lt;/strong&gt; silently discard it if the checksum is wrong. The book counts bit-error checks as part of a router's processing delay (section 1.4.1).&lt;/p&gt;

&lt;h3&gt;
  
  
  Checking it on Linux
&lt;/h3&gt;

&lt;p&gt;&lt;strong&gt;FCS errors&lt;/strong&gt; are counted per interface. Two &lt;code&gt;-s&lt;/code&gt; flags show the detailed error columns (&lt;code&gt;enp7s0&lt;/code&gt; is the wired interface):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;ip &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="nb"&gt;link &lt;/span&gt;show dev enp7s0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Output, trimmed to the receive lines:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;    RX:  bytes packets errors dropped  missed   mcast
    3943946473 2833648      0       0       0     465
    RX errors:  length    crc   frame    fifo overrun
                     0      0       0       0       0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;crc&lt;/code&gt; column counts frames with a bad FCS, as reported by the driver. It shows zero, but what it counts depends on the driver. The &lt;code&gt;r8169&lt;/code&gt; driver (&lt;code&gt;drivers/net/ethernet/realtek/r8169_main.c&lt;/code&gt;), for example, counts only errored frames the card hands over, and with the default settings below, the card may drop them first. Some drivers also show hardware CRC counters in &lt;code&gt;ethtool -S &amp;lt;interface&amp;gt;&lt;/code&gt;; &lt;code&gt;r8169&lt;/code&gt; has none named "crc".&lt;/p&gt;

&lt;p&gt;Two card settings show what reaches the kernel:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;ethtool &lt;span class="nt"&gt;-k&lt;/span&gt; enp7s0 | &lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-E&lt;/span&gt; &lt;span class="s1"&gt;'rx-fcs|rx-all'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;rx-fcs: off
rx-all: off
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;From the kernel documentation (&lt;code&gt;netdev-features&lt;/code&gt;): &lt;code&gt;rx-fcs&lt;/code&gt; asks the card to "append the Ethernet Frame Checksum (FCS) to the end of the skb data", and &lt;code&gt;rx-all&lt;/code&gt; asks it to "receive all possible frames, including errored frames (such as bad FCS, etc)". Both are off, the normal setting: frames with a bad FCS never reach the network stack, and good frames reach it without their FCS. With &lt;code&gt;r8169&lt;/code&gt;, the driver trims the 4 FCS bytes (&lt;code&gt;pkt_size -= ETH_FCS_LEN&lt;/code&gt; in &lt;code&gt;r8169_main.c&lt;/code&gt;). That is why &lt;code&gt;tcpdump&lt;/code&gt; normally shows no FCS, on either the receiving or the sending side.&lt;/p&gt;

&lt;p&gt;The kernel counts &lt;strong&gt;IPv4 header checksum errors&lt;/strong&gt; itself. In &lt;code&gt;ip_rcv_core()&lt;/code&gt; (&lt;code&gt;net/ipv4/ip_input.c&lt;/code&gt;), a failed &lt;code&gt;ip_fast_csum()&lt;/code&gt; raises two counters before the packet is dropped:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;nstat &lt;span class="nt"&gt;-az&lt;/span&gt; IpExtInCsumErrors IpInHdrErrors
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;#kernel
IpInHdrErrors                   0                  0.0
IpExtInCsumErrors               0                  0.0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Key idea
&lt;/h3&gt;

&lt;p&gt;The error check sits at the end of the frame. A store-and-forward device must wait for it, and in return a damaged frame stops at the first hop.&lt;/p&gt;




&lt;h2&gt;
  
  
  Part 3: Forward: What Linux Does With the Frame
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwj5t5bryienwan582vnv.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fwj5t5bryienwan582vnv.png" alt="The Linux IPv4 forwarding path" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The question
&lt;/h3&gt;

&lt;p&gt;Once the frame is stored and checked, what happens to it inside the router?&lt;/p&gt;

&lt;h3&gt;
  
  
  What I found
&lt;/h3&gt;

&lt;p&gt;Linux only forwards packets when IP forwarding is on:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;sysctl net.ipv4.ip_forward
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;net.ipv4.ip_forward = 1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;1&lt;/code&gt;: the machine forwards packets not addressed to it. &lt;code&gt;0&lt;/code&gt;: it does not.&lt;/p&gt;

&lt;p&gt;On a desktop, &lt;code&gt;1&lt;/code&gt; is common when Docker is installed. Docker's documentation says it "enables the &lt;code&gt;sysctl&lt;/code&gt; settings &lt;code&gt;net.ipv4.ip_forward&lt;/code&gt; and &lt;code&gt;net.ipv6.conf.all.forwarding&lt;/code&gt;" if they are not already enabled when it starts (except with its experimental nftables backend). So a Linux machine running Docker with its default setup already forwards packets, like a small router.&lt;/p&gt;

&lt;p&gt;With forwarding on, one IPv4 packet takes this path (simplified; firewall hooks and fast paths left out unless noted):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Step&lt;/th&gt;
&lt;th&gt;Where&lt;/th&gt;
&lt;th&gt;What happens&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;NIC&lt;/td&gt;
&lt;td&gt;Receives the whole frame, checks the FCS, copies it to RAM (Part 1 and 2)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;&lt;code&gt;eth_type_trans()&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Reads the Ethernet header, then moves the packet's start pointer past its 14 bytes (&lt;code&gt;skb_pull&lt;/code&gt;). The old header is not erased, just no longer part of the packet data&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;&lt;code&gt;ip_rcv_core()&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Basic IPv4 checks, including the header checksum&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;Route lookup&lt;/td&gt;
&lt;td&gt;The destination is not this machine, so the route's input function is set to &lt;code&gt;ip_forward()&lt;/code&gt; (&lt;code&gt;net/ipv4/route.c&lt;/code&gt;)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;&lt;code&gt;ip_forward()&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;TTL and MTU checks, makes room in front of the packet for the new link-layer header, lowers the TTL&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;FORWARD hook&lt;/td&gt;
&lt;td&gt;Firewall rules in the forward chain (iptables or nftables) run here&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;7&lt;/td&gt;
&lt;td&gt;&lt;code&gt;ip_finish_output2()&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Finds the next hop's neighbor entry and writes the new Ethernet header&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;8&lt;/td&gt;
&lt;td&gt;Driver and NIC&lt;/td&gt;
&lt;td&gt;Hands the frame to the card, which normally adds the new FCS as it transmits&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The core of &lt;code&gt;ip_forward()&lt;/code&gt; (&lt;code&gt;net/ipv4/ip_forward.c&lt;/code&gt;), shortened (&lt;code&gt;...&lt;/code&gt; marks removed lines):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight c"&gt;&lt;code&gt;    &lt;span class="cm"&gt;/*
     *  According to the RFC, we must first decrease the TTL field. If
     *  that reaches zero, we must reply an ICMP control message telling
     *  that the packet's lifetime expired.
     */&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ip_hdr&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;skb&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;ttl&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;goto&lt;/span&gt; &lt;span class="n"&gt;too_many_hops&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="p"&gt;...&lt;/span&gt;
    &lt;span class="n"&gt;__IP_INC_STATS&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;net&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;IPSTATS_MIB_OUTFORWDATAGRAMS&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;...&lt;/span&gt;
    &lt;span class="cm"&gt;/* We are about to mangle packet. Copy it! */&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;skb_cow&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;skb&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;LL_RESERVED_SPACE&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;rt&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;dst&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;dev&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt;&lt;span class="n"&gt;rt&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;dst&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;header_len&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="k"&gt;goto&lt;/span&gt; &lt;span class="n"&gt;drop&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="n"&gt;iph&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;ip_hdr&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;skb&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="cm"&gt;/* Decrease ttl after skb cow done */&lt;/span&gt;
    &lt;span class="n"&gt;ip_decrease_ttl&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;iph&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;...&lt;/span&gt;
&lt;span class="n"&gt;too_many_hops&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;
    &lt;span class="cm"&gt;/* Tell the sender its packet died... */&lt;/span&gt;
    &lt;span class="n"&gt;__IP_INC_STATS&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;net&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;IPSTATS_MIB_INHDRERRORS&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="n"&gt;icmp_send&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;skb&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ICMP_TIME_EXCEEDED&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ICMP_EXC_TTL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Three details matter here:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;TTL is checked before it is lowered.&lt;/strong&gt; A packet arriving with TTL 1 would leave with TTL 0, so it is dropped here and the sender gets ICMP Time Exceeded, as RFC 1812 (section 5.3.1) requires.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;skb_cow()&lt;/code&gt; prepares the packet for editing.&lt;/strong&gt; It ensures the router has its own writable copy and enough headroom for the outgoing link-layer header (&lt;code&gt;LL_RESERVED_SPACE&lt;/code&gt;), the same headroom the kernel uses when building a packet on the sending side.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The new Ethernet header is usually copied, not built.&lt;/strong&gt; Linux caches a ready-made link-layer header for each neighbor (next hop). &lt;code&gt;neigh_hh_output()&lt;/code&gt; (&lt;code&gt;include/net/neighbour.h&lt;/code&gt;) copies it into the headroom, right in front of the IP header. If the neighbor's MAC address is unknown, ARP resolves it first.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The MTU check: if the packet is larger than the next link's MTU and has the IPv4 "Don't Fragment" (DF) bit set, &lt;code&gt;ip_forward()&lt;/code&gt; drops it and sends back ICMP "Fragmentation Needed". Without DF, the router may fragment it on the way out.&lt;/p&gt;

&lt;h3&gt;
  
  
  Checking it on Linux
&lt;/h3&gt;

&lt;p&gt;These kernel counters match the code above:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;nstat &lt;span class="nt"&gt;-az&lt;/span&gt; IpForwDatagrams IpInHdrErrors IcmpOutTimeExcds
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Output:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;#kernel
IpInHdrErrors                   0                  0.0
IpForwDatagrams                 30                 0.0
IcmpOutTimeExcds                0                  0.0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The counters start at zero at boot: 30 packets forwarded since then, none dropped for an expired TTL.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;IpForwDatagrams&lt;/code&gt; rises in &lt;code&gt;ip_forward()&lt;/code&gt; after a packet passes the TTL and policy checks.&lt;/li&gt;
&lt;li&gt;An expired TTL counts under &lt;code&gt;IpInHdrErrors&lt;/code&gt; (this path has no separate "TTL expired" counter); the ICMP reply counts under &lt;code&gt;IcmpOutTimeExcds&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Key idea
&lt;/h3&gt;

&lt;p&gt;Forwarding is a short, fixed list of steps: check, look up the route, lower the TTL, attach a new link-layer header, send. Each step is a few lines of kernel code you can read.&lt;/p&gt;




&lt;h2&gt;
  
  
  Part 4: The Four Small Edits at Every Hop
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fdidgx3i3lfhoj1zw7rd9.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fdidgx3i3lfhoj1zw7rd9.png" alt="The frame in and the frame out, with four edits" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The question
&lt;/h3&gt;

&lt;p&gt;When the frame leaves the router, how much of it has changed?&lt;/p&gt;

&lt;h3&gt;
  
  
  What I found
&lt;/h3&gt;

&lt;p&gt;Very little. For plain IPv4 routing, four things change:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;#&lt;/th&gt;
&lt;th&gt;What changes&lt;/th&gt;
&lt;th&gt;Why&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;
&lt;strong&gt;TTL goes down by 1&lt;/strong&gt; (for example, 64 becomes 63)&lt;/td&gt;
&lt;td&gt;Required by RFC 1812, so a packet caught in a routing loop cannot circulate forever&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;IPv4 header checksum is updated&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;The checksum covers the TTL, so the old value no longer matches&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;New Ethernet header&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Destination MAC = next hop, source MAC = router's outgoing interface. MACs change at every hop; IP addresses do not&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;New FCS&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;It covers the new header, so it must be calculated again&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;Not touched&lt;/strong&gt; in plain routing: the source and destination IP addresses, the TCP header, and your data.&lt;/p&gt;

&lt;p&gt;A detail in edit 2: the book says the IPv4 header checksum "needed to be recomputed at every router" (section 4.3.5). Linux does not re-add the whole header; it updates the checksum &lt;strong&gt;incrementally&lt;/strong&gt; (&lt;code&gt;include/net/ip.h&lt;/code&gt;):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight c"&gt;&lt;code&gt;&lt;span class="k"&gt;static&lt;/span&gt; &lt;span class="kr"&gt;inline&lt;/span&gt;
&lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="nf"&gt;ip_decrease_ttl&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="n"&gt;iphdr&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;iph&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;u32&lt;/span&gt; &lt;span class="n"&gt;check&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;__force&lt;/span&gt; &lt;span class="n"&gt;u32&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="n"&gt;iph&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;check&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="n"&gt;check&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;__force&lt;/span&gt; &lt;span class="n"&gt;u32&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="n"&gt;htons&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mh"&gt;0x0100&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="n"&gt;iph&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;check&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;__force&lt;/span&gt; &lt;span class="n"&gt;__sum16&lt;/span&gt;&lt;span class="p"&gt;)(&lt;/span&gt;&lt;span class="n"&gt;check&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;check&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;=&lt;/span&gt;&lt;span class="mh"&gt;0xFFFF&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="o"&gt;--&lt;/span&gt;&lt;span class="n"&gt;iph&lt;/span&gt;&lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;&lt;span class="n"&gt;ttl&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;TTL is the high byte of a 16-bit header word (TTL + Protocol), so lowering TTL by 1 lowers that word by &lt;code&gt;0x0100&lt;/code&gt;. The checksum is the ones' complement of the sum, so it goes &lt;strong&gt;up&lt;/strong&gt; by &lt;code&gt;0x0100&lt;/code&gt;; &lt;code&gt;check + (check&amp;gt;=0xFFFF)&lt;/code&gt; handles the end-around carry. RFC 1812 (section 4.2.2.5) allows this incremental update, and RFC 1624 describes the method.&lt;/p&gt;

&lt;h3&gt;
  
  
  Exceptions worth knowing
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;NAT:&lt;/strong&gt; a home router doing NAT also changes the source IP address and port, and with them the IP and TCP/UDP checksums.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Fragmentation:&lt;/strong&gt; in IPv4, a router may split a packet that is too large for the next link (see Part 3).&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;QoS and congestion marks:&lt;/strong&gt; some routers rewrite the DSCP value or set the ECN bits in the IP header.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;IPv6&lt;/strong&gt; is simpler: the Hop Limit replaces the TTL, there is &lt;strong&gt;no header checksum&lt;/strong&gt; (RFC 8200), and routers never fragment IPv6 packets; only the source does. So an IPv6 router makes three of the four edits.&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  TTL and traceroute
&lt;/h3&gt;

&lt;p&gt;When the TTL runs out, RFC 1812 says the router must drop the packet and send ICMP Time Exceeded to the source. &lt;code&gt;traceroute&lt;/code&gt; relies on this: its man page says it starts its probes "with a ttl of one and increase[s] by one" and listens for the Time Exceeded replies. Each reply reveals one router's address on the path.&lt;/p&gt;

&lt;p&gt;Not every router answers. Linux rate-limits these ICMP messages by default (&lt;code&gt;net.ipv4.icmp_ratelimit&lt;/code&gt; and &lt;code&gt;net.ipv4.icmp_ratemask&lt;/code&gt;), and some networks filter them. That is one reason traceroute sometimes shows &lt;code&gt;* * *&lt;/code&gt; for a hop.&lt;/p&gt;

&lt;h3&gt;
  
  
  Key idea
&lt;/h3&gt;

&lt;p&gt;The router keeps your packet and rebuilds the frame around it. The IP packet inside changes in only two small fields (TTL and checksum), unless NAT or another feature is involved.&lt;/p&gt;




&lt;h2&gt;
  
  
  Part 5: The Cost: N × L/R
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fce7107aemcxadfh7yqvl.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fce7107aemcxadfh7yqvl.png" alt="Delay over three links" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The question
&lt;/h3&gt;

&lt;p&gt;If every router waits for the whole packet, how does the waiting add up across a path?&lt;/p&gt;

&lt;h3&gt;
  
  
  What I found
&lt;/h3&gt;

&lt;p&gt;The book (section 1.3.1, equation 1.1) gives the delay for one packet over &lt;strong&gt;N&lt;/strong&gt; links of rate &lt;strong&gt;R&lt;/strong&gt;, with N - 1 routers in between:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;d(end-to-end) = N × L / R
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This ignores propagation, queuing and processing delay. Each link adds one full L/R, because each router waits for the last bit.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt; (same round numbers, three links):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;3 × 12,000 bits / 100 Mbps = 3 × 120 µs = 360 µs
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Where this fits among the four delays
&lt;/h3&gt;

&lt;p&gt;Section 1.4.1 of the book splits the delay at each node into four parts:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Delay&lt;/th&gt;
&lt;th&gt;What it is&lt;/th&gt;
&lt;th&gt;Typical size (book)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Processing&lt;/td&gt;
&lt;td&gt;Examine the header, decide where to send it, check for bit errors&lt;/td&gt;
&lt;td&gt;microseconds or less in high-speed routers&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Queuing&lt;/td&gt;
&lt;td&gt;Wait behind other packets for the output link&lt;/td&gt;
&lt;td&gt;microseconds to milliseconds, depends on traffic&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Transmission&lt;/td&gt;
&lt;td&gt;Push all bits onto the link: L/R&lt;/td&gt;
&lt;td&gt;microseconds to milliseconds&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Propagation&lt;/td&gt;
&lt;td&gt;Travel time along the link: distance / speed&lt;/td&gt;
&lt;td&gt;milliseconds in wide-area networks&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Store-and-forward makes the &lt;strong&gt;transmission&lt;/strong&gt; delay repeat at every hop; the other three come on top.&lt;/p&gt;

&lt;p&gt;Transmission and propagation are easy to mix up. A &lt;strong&gt;made-up example&lt;/strong&gt;: on a 1,000 km fiber link, signals travel at about 2 × 10⁸ m/s (the book gives 2 × 10⁸ to 3 × 10⁸ m/s), so propagation takes 1,000,000 m ÷ (2 × 10⁸ m/s) = &lt;strong&gt;5 ms&lt;/strong&gt;. Pushing our 1500-byte packet onto that link at 100 Mbps takes &lt;strong&gt;120 µs&lt;/strong&gt;. Transmission depends on packet size and link rate; propagation depends on distance.&lt;/p&gt;

&lt;p&gt;Compare circuit switching (book, section 1.3.2): with a reserved end-to-end circuit, "the transmission time is independent of the number of links". Packet switching pays per hop instead, and in return the links are shared.&lt;/p&gt;

&lt;h3&gt;
  
  
  Key idea
&lt;/h3&gt;

&lt;p&gt;Store-and-forward makes the L/R delay repeat on every link. On top of it come processing, queuing and propagation.&lt;/p&gt;




&lt;h2&gt;
  
  
  Part 6: Why Your Data Is Cut Into Small Packets
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyymizv8fd4215cdl026c.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyymizv8fd4215cdl026c.png" alt="One huge packet compared with 1000 small ones" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The question
&lt;/h3&gt;

&lt;p&gt;If every hop waits for the whole packet, what happens when a packet is very large?&lt;/p&gt;

&lt;h3&gt;
  
  
  What I found
&lt;/h3&gt;

&lt;p&gt;The wait grows with L. Smaller packets let the links work &lt;strong&gt;at the same time&lt;/strong&gt;: while packet 1 is on link 2, packet 2 is already on link 1. This is called &lt;strong&gt;pipelining&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The book's example: three packets, one router. The destination has all three after &lt;strong&gt;4L/R&lt;/strong&gt;, not 6L/R. The book leaves the general case (P packets, N links) as an exercise. By the same reasoning, the first packet arrives after N × L/R and each further packet one L/R later:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;d = (N + P - 1) × L / R
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt; (same 3 links at 100 Mbps, 1.5 MB to send, headers ignored):&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;How it is sent&lt;/th&gt;
&lt;th&gt;Calculation&lt;/th&gt;
&lt;th&gt;Time&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;One &lt;strong&gt;imaginary&lt;/strong&gt; 1.5 MB packet&lt;/td&gt;
&lt;td&gt;3 × (12,000,000 bits / 100 Mbps)&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;360 ms&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;1000 packets of 1500 bytes&lt;/td&gt;
&lt;td&gt;(3 + 1000 - 1) × 120 µs&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;≈ 120 ms&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The 1.5 MB packet is &lt;strong&gt;imaginary&lt;/strong&gt;: the 16-bit IPv4 Total Length field caps a packet at 65,535 bytes, and the usual 1500-byte Ethernet MTU allows at most 1500 bytes of IP packet per frame.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;With real headers:&lt;/strong&gt; a 1500-byte IPv4/TCP packet without options carries 1460 bytes of data (1500 - 20 - 20). Sending 1,500,000 bytes then takes 1028 packets: (3 + 1028 - 1) × 120 µs ≈ &lt;strong&gt;123.6 ms&lt;/strong&gt; (Ethernet framing still ignored, as in Part 1). Same conclusion.&lt;/p&gt;

&lt;p&gt;Small packets have a cost too: each one carries its own headers, so very small packets spend more of the link on headers, and each one costs the router some processing. The book's end-of-chapter problems P31 and P33 explore this trade-off, with and without header overhead.&lt;/p&gt;

&lt;h3&gt;
  
  
  Key idea
&lt;/h3&gt;

&lt;p&gt;Cutting data into packets lets the links of a path work in parallel. The store-and-forward wait then applies to one small packet at a time instead of the whole message.&lt;/p&gt;




&lt;h2&gt;
  
  
  Part 7: Not Every Box Waits
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fo07e8ifdh7a5phez67pe.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fo07e8ifdh7a5phez67pe.png" alt="Store-and-forward compared with cut-through" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The question
&lt;/h3&gt;

&lt;p&gt;Does every network device store and forward?&lt;/p&gt;

&lt;h3&gt;
  
  
  What I found
&lt;/h3&gt;

&lt;p&gt;The book describes both routers and Ethernet switches as store-and-forward packet switches (sections 1.3.1 and 6.4.2), and the Linux path above is store-and-forward too.&lt;/p&gt;

&lt;p&gt;Some Ethernet switches use a shortcut called &lt;strong&gt;cut-through&lt;/strong&gt;: they start sending a frame as soon as they have read the destination address and chosen the output port. The destination MAC is the first field of the Ethernet header, so this can happen long before the frame has fully arrived. The trade-off:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Store-and-forward&lt;/th&gt;
&lt;th&gt;Cut-through&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Starts sending&lt;/td&gt;
&lt;td&gt;After the whole frame has arrived&lt;/td&gt;
&lt;td&gt;After the destination address is read&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;FCS checked before sending&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;td&gt;No, the end has not arrived yet&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Damaged frame&lt;/td&gt;
&lt;td&gt;Dropped at this switch&lt;/td&gt;
&lt;td&gt;Can be passed on to the next device&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Delay added&lt;/td&gt;
&lt;td&gt;At least L/R per hop&lt;/td&gt;
&lt;td&gt;Lower: it does not wait for the rest of the frame&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A cut-through switch must still buffer a frame when the output port is busy with another frame, or when the output port is faster than the input port (otherwise it would run out of bits to send).&lt;/p&gt;

&lt;h3&gt;
  
  
  Key idea
&lt;/h3&gt;

&lt;p&gt;Store-and-forward trades a little delay for a check on every hop. Cut-through trades that check for lower delay. Routers, as described in the book and implemented in Linux, store and forward.&lt;/p&gt;




&lt;h2&gt;
  
  
  Summary
&lt;/h2&gt;

&lt;p&gt;Store-and-forward in three lines:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Why&lt;/strong&gt; it waits: the FCS sits at the end of the frame, and RFC 1812 requires a valid IP header checksum before processing.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;What&lt;/strong&gt; it changes: TTL, header checksum, Ethernet header and FCS, and nothing else in plain IPv4 routing.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;What&lt;/strong&gt; it costs: one L/R per link, which is why data travels as many small packets.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;One detail stands out: the book says the checksum is recomputed at every router, but &lt;code&gt;ip_decrease_ttl()&lt;/code&gt; shows it can be a small incremental update instead of a full recalculation.&lt;/p&gt;

&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Kurose, J. &amp;amp; Ross, K., &lt;em&gt;Computer Networking: A Top-Down Approach&lt;/em&gt;, 7th ed.: sections 1.3.1 (store-and-forward, equation 1.1), 1.3.2 (circuit switching example), 1.4.1 (four types of delay), 4.3.1 (IPv4 datagram format), 4.3.5 (IPv6), 6.4.2 (Ethernet frame and CRC); end-of-chapter problems P31 and P33 (chapter 1)&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rfc-editor.org/rfc/rfc1812" rel="noopener noreferrer"&gt;RFC 1812&lt;/a&gt;: Requirements for IP Version 4 Routers (sections 4.2.2.5, 5.2.2, 5.3.1)&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rfc-editor.org/rfc/rfc1624" rel="noopener noreferrer"&gt;RFC 1624&lt;/a&gt;: Computation of the Internet Checksum via Incremental Update&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rfc-editor.org/rfc/rfc791" rel="noopener noreferrer"&gt;RFC 791&lt;/a&gt;: Internet Protocol&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rfc-editor.org/rfc/rfc792" rel="noopener noreferrer"&gt;RFC 792&lt;/a&gt;: Internet Control Message Protocol&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://www.rfc-editor.org/rfc/rfc8200" rel="noopener noreferrer"&gt;RFC 8200&lt;/a&gt;: Internet Protocol, Version 6 (IPv6) Specification&lt;/li&gt;
&lt;li&gt;Linux source (mainline 7.3): &lt;a href="https://github.com/torvalds/linux/blob/master/drivers/net/ethernet/realtek/r8169_main.c" rel="noopener noreferrer"&gt;&lt;code&gt;drivers/net/ethernet/realtek/r8169_main.c&lt;/code&gt;&lt;/a&gt;, &lt;a href="https://github.com/torvalds/linux/blob/master/net/ipv4/ip_forward.c" rel="noopener noreferrer"&gt;&lt;code&gt;net/ipv4/ip_forward.c&lt;/code&gt;&lt;/a&gt;, &lt;a href="https://github.com/torvalds/linux/blob/master/net/ipv4/ip_input.c" rel="noopener noreferrer"&gt;&lt;code&gt;net/ipv4/ip_input.c&lt;/code&gt;&lt;/a&gt;, &lt;a href="https://github.com/torvalds/linux/blob/master/net/ipv4/ip_output.c" rel="noopener noreferrer"&gt;&lt;code&gt;net/ipv4/ip_output.c&lt;/code&gt;&lt;/a&gt;, &lt;a href="https://github.com/torvalds/linux/blob/master/net/ipv4/route.c" rel="noopener noreferrer"&gt;&lt;code&gt;net/ipv4/route.c&lt;/code&gt;&lt;/a&gt;, &lt;a href="https://github.com/torvalds/linux/blob/master/include/net/ip.h" rel="noopener noreferrer"&gt;&lt;code&gt;include/net/ip.h&lt;/code&gt;&lt;/a&gt;, &lt;a href="https://github.com/torvalds/linux/blob/master/include/net/neighbour.h" rel="noopener noreferrer"&gt;&lt;code&gt;include/net/neighbour.h&lt;/code&gt;&lt;/a&gt;, &lt;a href="https://github.com/torvalds/linux/blob/master/include/linux/etherdevice.h" rel="noopener noreferrer"&gt;&lt;code&gt;include/linux/etherdevice.h&lt;/code&gt;&lt;/a&gt;, &lt;a href="https://github.com/torvalds/linux/blob/master/include/linux/skbuff.h" rel="noopener noreferrer"&gt;&lt;code&gt;include/linux/skbuff.h&lt;/code&gt;&lt;/a&gt;, &lt;a href="https://github.com/torvalds/linux/blob/master/net/ethernet/eth.c" rel="noopener noreferrer"&gt;&lt;code&gt;net/ethernet/eth.c&lt;/code&gt;&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Linux kernel documentation: &lt;a href="https://docs.kernel.org/networking/ip-sysctl.html" rel="noopener noreferrer"&gt;IP sysctl&lt;/a&gt; (&lt;code&gt;icmp_ratelimit&lt;/code&gt;, &lt;code&gt;icmp_ratemask&lt;/code&gt;), &lt;a href="https://docs.kernel.org/networking/netdev-features.html" rel="noopener noreferrer"&gt;netdev features&lt;/a&gt; (&lt;code&gt;rx-fcs&lt;/code&gt;, &lt;code&gt;rx-all&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Docker documentation: &lt;a href="https://docs.docker.com/engine/network/packet-filtering-firewalls/" rel="noopener noreferrer"&gt;Packet filtering and firewalls&lt;/a&gt; (IP forwarding)&lt;/li&gt;
&lt;li&gt;Wikipedia: &lt;a href="https://en.wikipedia.org/wiki/Cut-through_switching" rel="noopener noreferrer"&gt;Cut-through switching&lt;/a&gt;, which cites Cisco's white paper &lt;em&gt;Cut-Through and Store-and-Forward Ethernet Switching for Low-Latency Environments&lt;/em&gt; (C11-465436)&lt;/li&gt;
&lt;li&gt;Man pages: &lt;a href="https://man7.org/linux/man-pages/man8/ip-link.8.html" rel="noopener noreferrer"&gt;&lt;code&gt;ip-link(8)&lt;/code&gt;&lt;/a&gt;, &lt;a href="https://man7.org/linux/man-pages/man8/nstat.8.html" rel="noopener noreferrer"&gt;&lt;code&gt;nstat(8)&lt;/code&gt;&lt;/a&gt;, &lt;code&gt;ethtool(8)&lt;/code&gt;, &lt;code&gt;traceroute(8)&lt;/code&gt; (traceroute 2.1.6)&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>networking</category>
      <category>linux</category>
      <category>kernel</category>
      <category>computerscience</category>
    </item>
    <item>
      <title>Learning TCP/IP From the Linux Side: Spec, Code, and Wire</title>
      <dc:creator>Musa Nayyer</dc:creator>
      <pubDate>Tue, 13 Oct 2026 19:47:52 +0000</pubDate>
      <link>https://dev.to/muzasio/learning-tcpip-from-the-linux-side-spec-code-and-wire-5fp1</link>
      <guid>https://dev.to/muzasio/learning-tcpip-from-the-linux-side-spec-code-and-wire-5fp1</guid>
      <description>&lt;p&gt;My networking notes used to be full of layer names, header diagrams, and TCP flags. I knew the terminology, but when someone asked what happens after calling &lt;code&gt;send()&lt;/code&gt;, my explanation stopped at "the data goes down the stack."&lt;/p&gt;

&lt;p&gt;I changed my approach by connecting three things:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;The spec:&lt;/strong&gt; what the RFC defines.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;The code:&lt;/strong&gt; how Linux or C headers represent it.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;The wire:&lt;/strong&gt; what &lt;code&gt;tcpdump&lt;/code&gt; or Wireshark captures.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;This post applies that approach to four exercises, moving from packet construction to TCP security.&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;From application data to the network&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Layer 2 vs Layer 3&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;One TCP header, three views&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;How TCP validates RST packets&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The commands are generally safe on a normal Linux machine. Commands requiring elevated privileges are identified.&lt;/p&gt;




&lt;h2&gt;
  
  
  Part 1: From Application Data to the Network
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fdy5shp7fi1u701k4ui39.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fdy5shp7fi1u701k4ui39.png" alt="network overview inside application" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The question
&lt;/h3&gt;

&lt;p&gt;When an application sends bytes, which components add the headers, and where are those headers represented in memory?&lt;/p&gt;

&lt;h3&gt;
  
  
  What I found
&lt;/h3&gt;

&lt;p&gt;When an application calls &lt;code&gt;send()&lt;/code&gt; or &lt;code&gt;write()&lt;/code&gt; on a TCP socket, the operation enters the kernel. The networking stack processes the data using structures such as &lt;code&gt;sk_buff&lt;/code&gt;, defined in &lt;code&gt;include/linux/skbuff.h&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;An &lt;code&gt;sk_buff&lt;/code&gt; can provide space before packet data (&lt;strong&gt;headroom&lt;/strong&gt;) and after it (&lt;strong&gt;tailroom&lt;/strong&gt;). Networking layers can use this space to add headers without necessarily copying the payload into a new buffer at every layer.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Step&lt;/th&gt;
&lt;th&gt;Who does it&lt;/th&gt;
&lt;th&gt;What gets added&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;td&gt;Application&lt;/td&gt;
&lt;td&gt;Payload (your bytes)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;TCP&lt;/td&gt;
&lt;td&gt;TCP header, 20 bytes minimum (more with options)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;IP&lt;/td&gt;
&lt;td&gt;IP header, 20 bytes for IPv4 without options&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;4&lt;/td&gt;
&lt;td&gt;Ethernet / neighbor layer&lt;/td&gt;
&lt;td&gt;14 byte Ethernet header (MACs + EtherType)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;NIC driver&lt;/td&gt;
&lt;td&gt;Hands the buffer to the card, usually through DMA&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;6&lt;/td&gt;
&lt;td&gt;NIC hardware&lt;/td&gt;
&lt;td&gt;Adds the FCS (CRC) and puts bits on the medium&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;SA typical Ethernet frame looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[ Ethernet ][ IP ][ TCP ][ payload ][ FCS ]
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The exact operations depend on the networking path and enabled features.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why it matters
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Headroom:&lt;/strong&gt; insufficient space may require the kernel to expand or reallocate the buffer.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;FCS:&lt;/strong&gt; normal host captures often omit it because the NIC adds it after the capture point.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Offloads:&lt;/strong&gt; checksum and segmentation offloads can make captured packets differ from their final on-wire form.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The packet diagram describes the format; the kernel and NIC determine how it is constructed and transmitted.&lt;/p&gt;




&lt;h2&gt;
  
  
  Part 2: Layer 2 vs Layer 3, Seen From Linux
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0wnxl9b3mi8y140vn0an.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F0wnxl9b3mi8y140vn0an.png" alt="layer 2 vs layer 3" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The question
&lt;/h3&gt;

&lt;p&gt;If IP addresses identify hosts, why do we also need MAC addresses?&lt;/p&gt;

&lt;h3&gt;
  
  
  What I found
&lt;/h3&gt;

&lt;p&gt;They serve different purposes:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;Layer 2 (Ethernet)&lt;/th&gt;
&lt;th&gt;Layer 3 (IP)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Address&lt;/td&gt;
&lt;td&gt;MAC address&lt;/td&gt;
&lt;td&gt;IP address&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Scope&lt;/td&gt;
&lt;td&gt;One local link&lt;/td&gt;
&lt;td&gt;End to end, across networks&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Unit&lt;/td&gt;
&lt;td&gt;Frame&lt;/td&gt;
&lt;td&gt;Packet&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Device&lt;/td&gt;
&lt;td&gt;Switch&lt;/td&gt;
&lt;td&gt;Router&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Linux command&lt;/td&gt;
&lt;td&gt;&lt;code&gt;ip neigh&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;ip route&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The &lt;code&gt;via&lt;/code&gt; field identifies the selected next hop.&lt;/p&gt;

&lt;p&gt;Next, inspect neighbor mappings:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;ip neigh show
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;192.168.1.1 dev wlan0 lladdr aa:bb:cc:dd:ee:01 REACHABLE
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;On IPv4 Ethernet networks, ARP commonly resolves an IP address to a MAC address.&lt;/p&gt;

&lt;p&gt;Capture the traffic to inspect both layers:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;sudo &lt;/span&gt;tcpdump &lt;span class="nt"&gt;-n&lt;/span&gt; &lt;span class="nt"&gt;-e&lt;/span&gt; &lt;span class="nt"&gt;-i&lt;/span&gt; wlan0 host 203.0.113.20
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;-e&lt;/code&gt; option displays link-layer information. For traffic routed through the gateway, the Ethernet destination MAC should correspond to the gateway, while the IP destination remains &lt;code&gt;203.0.113.20&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The distinction:&lt;/strong&gt; &lt;code&gt;ip route&lt;/code&gt; reveals the next-hop decision; &lt;code&gt;ip neigh&lt;/code&gt; helps identify the corresponding link-layer address.&lt;/p&gt;




&lt;h2&gt;
  
  
  Part 3: One TCP Header, Three Views
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fg1wv5wnkuk0nbsu6mdbx.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fg1wv5wnkuk0nbsu6mdbx.png" alt="Three views" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The question
&lt;/h3&gt;

&lt;p&gt;Does the TCP header diagram in a textbook correspond to what the kernel and &lt;code&gt;tcpdump&lt;/code&gt; actually use?&lt;/p&gt;

&lt;h3&gt;
  
  
  View 1: The spec
&lt;/h3&gt;

&lt;p&gt;The current base TCP specification is &lt;strong&gt;RFC 9293&lt;/strong&gt; (2022), which obsoleted the original RFC 793. The header format is described in section 3.1:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt; 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Source Port          |       Destination Port        |
|                        Sequence Number                        |
|                    Acknowledgment Number                      |
| Offset| Rsrvd |   Flags       |            Window             |
|           Checksum            |         Urgent Pointer        |
|                    Options (variable length)                  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Each row represents 32 bits. The flags are &lt;code&gt;CWR ECE URG ACK PSH RST SYN FIN&lt;/code&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  View 2: The code
&lt;/h3&gt;

&lt;p&gt;On Linux systems using glibc, the TCP header is declared in &lt;code&gt;&amp;lt;netinet/tcp.h&amp;gt;&lt;/code&gt; as &lt;code&gt;struct tcphdr&lt;/code&gt;. A shortened representation looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight c"&gt;&lt;code&gt;&lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="n"&gt;tcphdr&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kt"&gt;uint16_t&lt;/span&gt; &lt;span class="n"&gt;source&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kt"&gt;uint16_t&lt;/span&gt; &lt;span class="n"&gt;dest&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kt"&gt;uint32_t&lt;/span&gt; &lt;span class="n"&gt;seq&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kt"&gt;uint32_t&lt;/span&gt; &lt;span class="n"&gt;ack_seq&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kt"&gt;uint16_t&lt;/span&gt; &lt;span class="n"&gt;doff&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;   &lt;span class="cm"&gt;/* data offset */&lt;/span&gt;
    &lt;span class="kt"&gt;uint16_t&lt;/span&gt; &lt;span class="n"&gt;syn&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="cm"&gt;/* ... other flag bits ... */&lt;/span&gt;
    &lt;span class="kt"&gt;uint16_t&lt;/span&gt; &lt;span class="n"&gt;window&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kt"&gt;uint16_t&lt;/span&gt; &lt;span class="n"&gt;check&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="kt"&gt;uint16_t&lt;/span&gt; &lt;span class="n"&gt;urg_ptr&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The actual declaration includes additional flags and handles bit-field ordering according to byte order. Linux also has kernel-side TCP definitions, and other systems may use different field names.&lt;/p&gt;

&lt;p&gt;The fields map to the RFC concepts: &lt;code&gt;doff&lt;/code&gt; is the data offset, &lt;code&gt;check&lt;/code&gt; the checksum, and &lt;code&gt;urg_ptr&lt;/code&gt; the urgent pointer.&lt;/p&gt;

&lt;h3&gt;
  
  
  View 3: The wire
&lt;/h3&gt;

&lt;p&gt;Capture TCP traffic on port 443:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;sudo &lt;/span&gt;tcpdump &lt;span class="nt"&gt;-n&lt;/span&gt; &lt;span class="nt"&gt;-S&lt;/span&gt; &lt;span class="nt"&gt;-i&lt;/span&gt; any tcp port 443
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;code&gt;-n&lt;/code&gt; displays numeric addresses and ports instead of resolving names.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;code&gt;-S&lt;/code&gt; displays &lt;strong&gt;absolute&lt;/strong&gt; sequence numbers rather than relative ones.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A SYN packet might appear like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;IP 192.0.2.10.51514 &amp;gt; 198.51.100.20.443: Flags [S], seq 1520943817, win 64240, options [mss 1460,...], length 0
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The displayed fields can now be related to the C declaration and the RFC:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;tcpdump&lt;/th&gt;
&lt;th&gt;struct tcphdr&lt;/th&gt;
&lt;th&gt;RFC 9293&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;51514&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;source&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Source Port&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;443&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;dest&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Destination Port&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;seq 1520943817&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;seq&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Sequence Number&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;Flags [S]&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;syn&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;SYN flag&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;win 64240&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;&lt;code&gt;window&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Window&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;options [mss 1460]&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;follows &lt;code&gt;doff&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;Options&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Common flag notation includes &lt;code&gt;[S]&lt;/code&gt; for SYN, &lt;code&gt;[S.]&lt;/code&gt; for SYN+ACK, &lt;code&gt;[P.]&lt;/code&gt; for PSH+ACK, &lt;code&gt;[F.]&lt;/code&gt; for FIN+ACK, and &lt;code&gt;[R]&lt;/code&gt; for RST.&lt;/p&gt;

&lt;h3&gt;
  
  
  The checksum gotcha
&lt;/h3&gt;

&lt;p&gt;When I first ran &lt;code&gt;tcpdump -v&lt;/code&gt; on the sending host, some outgoing packets appeared to have incorrect TCP checksums.&lt;/p&gt;

&lt;p&gt;A common cause is &lt;strong&gt;checksum offload&lt;/strong&gt;. The kernel may leave checksum calculation to the NIC, while the capture occurs before the final checksum is written. The packet can therefore appear invalid in the capture even though the NIC calculates the correct checksum for transmission.&lt;/p&gt;

&lt;p&gt;You can inspect the relevant interface features with:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;ethtool &lt;span class="nt"&gt;-k&lt;/span&gt; wlan0 | &lt;span class="nb"&gt;grep &lt;/span&gt;checksum
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The capture point matters: captured packets may not yet reflect their final on-wire representation.&lt;/p&gt;




&lt;h2&gt;
  
  
  Part 4: How TCP Knows an RST Is Legitimate
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgj5svj8pey6mo67ee350.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgj5svj8pey6mo67ee350.png" alt="RST" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The question
&lt;/h3&gt;

&lt;p&gt;An RST can terminate a TCP connection. How does the receiver decide whether to accept one?&lt;/p&gt;

&lt;h3&gt;
  
  
  Sequence numbers as a defense
&lt;/h3&gt;

&lt;p&gt;TCP tracks bytes using sequence numbers. Two relevant receiver variables are:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;code&gt;RCV.NXT&lt;/code&gt;: the next sequence number expected.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;code&gt;RCV.WND&lt;/code&gt;: the receive window, describing the currently acceptable sequence-number range.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Under the original RFC 793 rules, an RST could be accepted if its sequence number fell within the receive window. A blind attacker therefore did not necessarily need to guess the exact expected number.&lt;/p&gt;

&lt;h3&gt;
  
  
  RFC 5961 tightened the check
&lt;/h3&gt;

&lt;p&gt;RFC 5961, section 3, introduced stricter RST validation. Suppose &lt;code&gt;RCV.NXT = 5000&lt;/code&gt; and the receive window is &lt;code&gt;1000&lt;/code&gt;.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;RST seq&lt;/th&gt;
&lt;th&gt;Where it falls&lt;/th&gt;
&lt;th&gt;What the receiver does&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;5000&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Exactly &lt;code&gt;RCV.NXT&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;Reset the connection&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;5400&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Inside the window, not exact&lt;/td&gt;
&lt;td&gt;Send a &lt;strong&gt;challenge ACK&lt;/strong&gt;, keep the connection&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;9000&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Outside the window&lt;/td&gt;
&lt;td&gt;Drop it silently&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Instead of immediately accepting an in-window RST with a non-exact sequence number, the receiver sends a &lt;strong&gt;challenge ACK&lt;/strong&gt; reflecting its current TCP state.&lt;/p&gt;

&lt;p&gt;This makes blind, off-path reset attacks harder because the attacker must produce a more precise packet without observing the connection. The sequence-number space contains (2^{32}), or about 4.29 billion, possible values, although real security also depends on protocol state, window size, implementation details, and the attacker's information.&lt;/p&gt;

&lt;h3&gt;
  
  
  Checking it on Linux
&lt;/h3&gt;

&lt;p&gt;Linux implements challenge-ACK handling. You can inspect the corresponding counter with:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;nstat &lt;span class="nt"&gt;-az&lt;/span&gt; TcpExtTCPChallengeACK
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Related settings include:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;sysctl net.ipv4.tcp_challenge_ack_limit
sysctl net.ipv4.tcp_invalid_ratelimit
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;CVE-2016-5696 demonstrated that Linux's shared challenge-ACK rate limit could leak information useful for inferring TCP connection state. Subsequent kernel changes addressed this side channel by changing how the limit was applied.&lt;/p&gt;

&lt;p&gt;This illustrates why security mechanisms must also be assessed for information leaks introduced by their implementations.&lt;/p&gt;

&lt;h3&gt;
  
  
  What to look for in Wireshark
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Filter resets with &lt;code&gt;tcp.flags.reset == 1&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Disable relative sequence numbers or add &lt;code&gt;tcp.seq_raw&lt;/code&gt; to inspect raw values.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Compare the RST sequence number with the receiver's expected sequence.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Look for a challenge ACK following an in-window RST that fails the stricter check. Wireshark's analysis labels depend on the observed exchange.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;An RST followed by an ACK while the connection continues may be consistent with challenge-ACK handling, but the surrounding packets are needed to establish what happened.&lt;/p&gt;

&lt;h3&gt;
  
  
  Blind vs on-path attackers
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Blind (off-path):&lt;/strong&gt; cannot observe the connection's traffic and must infer or guess sequence numbers. RFC 5961 makes certain blind reset attacks significantly harder.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;On-path:&lt;/strong&gt; can observe traffic and may know the relevant sequence numbers. Sequence-number checks alone do not authenticate against this attacker.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For deployments needing TCP-level authentication, &lt;strong&gt;TCP Authentication Option (TCP-AO)&lt;/strong&gt;, defined in RFC 5925, can authenticate segments using a shared key. A forged RST lacking the required authentication value should fail validation when TCP-AO is correctly configured and enforced.&lt;/p&gt;

&lt;p&gt;TCP-AO succeeds the older TCP MD5 signature option (RFC 2385), is relevant to sessions such as BGP, and gained Linux kernel support in version 6.7.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;TLS does not solve this problem:&lt;/strong&gt; TCP handles connection resets below the TLS layer, so TLS alone does not authenticate TCP RST packets.&lt;/p&gt;

&lt;h3&gt;
  
  
  Key idea
&lt;/h3&gt;

&lt;p&gt;Sequence-number checks help defend against blind attackers. When protection against attackers who can observe the traffic is required, TCP-level authentication, such as TCP-AO where supported and configured, addresses a different part of the threat model.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Changed in How I Learn
&lt;/h2&gt;

&lt;p&gt;These exercises exposed gaps in my earlier notes. I knew an RST terminates a connection, but had not investigated how TCP decides to accept one. I understood the difference between IP and MAC addresses, but had not examined a packet where the two destinations referred to different machines.&lt;/p&gt;

&lt;p&gt;The method is straightforward:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Read the RFC:&lt;/strong&gt; focus on the relevant section.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Find the implementation:&lt;/strong&gt; inspect a C header, kernel source file, or sysctl.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Observe the behavior:&lt;/strong&gt; use &lt;code&gt;tcpdump&lt;/code&gt;, Wireshark, &lt;code&gt;ip&lt;/code&gt;, &lt;code&gt;ss&lt;/code&gt;, or &lt;code&gt;nstat&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Investigate differences:&lt;/strong&gt; unexpected output can reveal details about the implementation or capture point.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Connecting the specification, implementation, and observed traffic gives you a practical way to test and deepen your understanding.&lt;/p&gt;

&lt;h2&gt;
  
  
  References
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;RFC 9293: Transmission Control Protocol (TCP)&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;RFC 5961: Improving TCP's Robustness to Blind In-Window Attacks&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;RFC 5925: The TCP Authentication Option&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;RFC 2385: Protection of BGP Sessions via the TCP MD5 Signature Option&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Linux source: &lt;code&gt;include/linux/skbuff.h&lt;/code&gt;, &lt;code&gt;net/ipv4/tcp_input.c&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;glibc: &lt;code&gt;&amp;lt;netinet/tcp.h&amp;gt;&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;code&gt;man 8 ip-route&lt;/code&gt;, &lt;code&gt;man 8 ip-neighbour&lt;/code&gt;, &lt;code&gt;man 8 tcpdump&lt;/code&gt;, &lt;code&gt;man 8 nstat&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>networking</category>
      <category>linux</category>
      <category>tcp</category>
      <category>cybersecurity</category>
    </item>
    <item>
      <title>Containers Are Isolated, But They Still Share the Kernel</title>
      <dc:creator>Musa Nayyer</dc:creator>
      <pubDate>Wed, 07 Oct 2026 19:49:11 +0000</pubDate>
      <link>https://dev.to/muzasio/containers-are-isolated-but-they-still-share-the-kernel-j5c</link>
      <guid>https://dev.to/muzasio/containers-are-isolated-but-they-still-share-the-kernel-j5c</guid>
      <description>&lt;p&gt;&lt;strong&gt;Namespaces, capabilities, seccomp, and two 2026 kernel vulnerabilities that show where Docker isolation ends&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Containers are lightweight because they do &lt;strong&gt;not&lt;/strong&gt; run their own kernel.&lt;/p&gt;

&lt;p&gt;They can have their own filesystem, process tree, network stack, and other isolated resources, but underneath all of that, containerized processes still use the &lt;strong&gt;host Linux kernel&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;That one fact is extremely important for container security.&lt;/p&gt;

&lt;p&gt;In 2026, two Linux kernel vulnerabilities made this boundary particularly interesting:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;CVE-2026-31431 ("Copy Fail")&lt;/strong&gt; showed how a vulnerable kernel interface reachable from a container could affect resources outside that container.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;CVE-2026-80521&lt;/strong&gt; demonstrated a container escape involving the Linux kernel's &lt;code&gt;AF_UNIX&lt;/code&gt; subsystem.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This does &lt;strong&gt;not&lt;/strong&gt; mean Docker is unsafe or that containers should not be used.&lt;/p&gt;

&lt;p&gt;The important lesson is simpler:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Container isolation is ultimately enforced by the host kernel.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Understanding that boundary helps explain both the strengths and limitations of container security.&lt;/p&gt;




&lt;h2&gt;
  
  
  Containers Are Not Small Virtual Machines
&lt;/h2&gt;

&lt;p&gt;A virtual machine runs its own guest operating system and its own kernel.&lt;/p&gt;

&lt;p&gt;A container does not.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fk4y2jng7kup4qj5k0x15.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fk4y2jng7kup4qj5k0x15.png" alt="vm instances" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;A container is better understood as a group of Linux processes running with restricted views and permissions provided by the kernel.&lt;/p&gt;

&lt;p&gt;Those processes still make system calls to the host kernel.&lt;/p&gt;

&lt;p&gt;This is one of the reasons containers are so lightweight. There is no separate guest kernel to boot for every container.&lt;/p&gt;

&lt;p&gt;But it also creates an important security relationship:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;The host Linux kernel is part of the trusted computing base for containers.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;If a kernel vulnerability allows an attacker to bypass one of the mechanisms enforcing isolation, the container boundary can potentially be weakened or crossed.&lt;/p&gt;

&lt;p&gt;Virtual machines are not completely immune to this problem either. Hypervisors have also had escape vulnerabilities.&lt;/p&gt;

&lt;p&gt;The broader principle is that &lt;strong&gt;every isolation mechanism has a trusted computing base&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;For conventional containers, a particularly important part of that trusted base is the Linux kernel.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Actually Isolates a Container?
&lt;/h2&gt;

&lt;p&gt;Docker uses several Linux security mechanisms. The most important ones include:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Namespaces&lt;/li&gt;
&lt;li&gt;cgroups&lt;/li&gt;
&lt;li&gt;Capabilities&lt;/li&gt;
&lt;li&gt;seccomp&lt;/li&gt;
&lt;li&gt;AppArmor or SELinux&lt;/li&gt;
&lt;li&gt;User namespaces&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Each solves a different problem.&lt;/p&gt;

&lt;h2&gt;
  
  
  Namespaces: What Can the Process See?
&lt;/h2&gt;

&lt;p&gt;Linux namespaces provide processes with isolated views of system resources.&lt;/p&gt;

&lt;p&gt;For example:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Namespace&lt;/th&gt;
&lt;th&gt;Main purpose&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;PID&lt;/td&gt;
&lt;td&gt;Process visibility&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Mount&lt;/td&gt;
&lt;td&gt;Filesystem mount view&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Network&lt;/td&gt;
&lt;td&gt;Network interfaces and ports&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;UTS&lt;/td&gt;
&lt;td&gt;Hostname&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;IPC&lt;/td&gt;
&lt;td&gt;Inter-process communication&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;User&lt;/td&gt;
&lt;td&gt;User and group ID mappings&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cgroup&lt;/td&gt;
&lt;td&gt;Cgroup hierarchy view&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Time&lt;/td&gt;
&lt;td&gt;Certain process-visible clocks&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A simple way to remember namespaces is:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Namespaces control what a process can see.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;A container can therefore have its own process list, filesystem view, and network environment even though those resources ultimately exist on the same kernel.&lt;/p&gt;




&lt;h2&gt;
  
  
  cgroups: How Much Can It Use?
&lt;/h2&gt;

&lt;p&gt;Control groups, or &lt;strong&gt;cgroups&lt;/strong&gt;, deal with resource management.&lt;/p&gt;

&lt;p&gt;They can control or account for resources such as:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;CPU&lt;/li&gt;
&lt;li&gt;Memory&lt;/li&gt;
&lt;li&gt;Number of processes&lt;/li&gt;
&lt;li&gt;I/O&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So the basic mental model is:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Namespaces
    ↓
What can this process see?

cgroups
    ↓
How much can this process use?
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;These mechanisms complement each other rather than solving the same problem.&lt;/p&gt;




&lt;h2&gt;
  
  
  Capabilities: What Privileged Operations Are Allowed?
&lt;/h2&gt;

&lt;p&gt;Linux does not treat root as one giant permission.&lt;/p&gt;

&lt;p&gt;Instead, many privileged operations are divided into &lt;strong&gt;capabilities&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Examples include:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;CAP_NET_ADMIN
CAP_SYS_ADMIN
CAP_SYS_MODULE
CAP_SYS_PTRACE
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Docker normally starts containers with a reduced set of capabilities.&lt;/p&gt;

&lt;p&gt;That means a process can have UID 0 — effectively "root" inside the container — without automatically having every privileged operation available to host root.&lt;/p&gt;

&lt;p&gt;Administrators can also explicitly add or remove capabilities:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nt"&gt;--cap-add&lt;/span&gt;
&lt;span class="nt"&gt;--cap-drop&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For application containers, dropping capabilities that are not required is generally preferable to giving the container additional privileges.&lt;/p&gt;




&lt;h2&gt;
  
  
  User Namespaces: Is Container Root Also Host Root?
&lt;/h2&gt;

&lt;p&gt;User namespaces can map the container's UID 0 to an unprivileged UID on the host.&lt;/p&gt;

&lt;p&gt;Conceptually:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Container
UID 0
  ↓
Host
UID 100000
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Docker supports this through user namespace remapping, and rootless Docker provides another approach to reducing host-level privilege.&lt;/p&gt;

&lt;p&gt;However, these protections depend on configuration.&lt;/p&gt;

&lt;p&gt;On a normal Docker configuration without user namespace remapping, UID 0 inside the container can map directly to UID 0 on the host.&lt;/p&gt;

&lt;p&gt;That does &lt;strong&gt;not&lt;/strong&gt; mean the container automatically has unrestricted host access. Other controls such as capabilities, namespaces, seccomp, and LSM policies still apply.&lt;/p&gt;

&lt;p&gt;But it does mean there is no additional UID-remapping boundary in that configuration.&lt;/p&gt;




&lt;h2&gt;
  
  
  seccomp: Which System Calls Can the Container Make?
&lt;/h2&gt;

&lt;p&gt;A containerized application constantly interacts with the kernel through &lt;strong&gt;system calls&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;seccomp can filter these calls.&lt;/p&gt;

&lt;p&gt;Conceptually:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Container process
       |
       | system call
       ↓
   seccomp filter
      /      \
   ALLOW    DENY
      |
      ↓
 Linux kernel
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Docker's default seccomp profile restricts access to a number of system calls.&lt;/p&gt;

&lt;p&gt;This is useful because reducing the available system-call surface can make some kernel vulnerabilities harder to reach.&lt;/p&gt;

&lt;p&gt;But there is an important limitation:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;seccomp is not a complete sandbox.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;It reduces the kernel interface available to a process. It does not make the remaining kernel code vulnerability-free.&lt;/p&gt;

&lt;p&gt;That distinction becomes especially important when looking at real kernel vulnerabilities.&lt;/p&gt;




&lt;h2&gt;
  
  
  AppArmor and SELinux
&lt;/h2&gt;

&lt;p&gt;Linux Security Modules, or &lt;strong&gt;LSMs&lt;/strong&gt;, provide another layer of mandatory access control.&lt;/p&gt;

&lt;p&gt;On systems using AppArmor, Docker can apply profiles such as &lt;code&gt;docker-default&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Other distributions commonly use SELinux.&lt;/p&gt;

&lt;p&gt;The overall security model therefore looks roughly like this:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fx0fu6ar9ahe112mdc26k.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fx0fu6ar9ahe112mdc26k.png" alt="Isolation layer" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Notice where the model ends:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;the Linux kernel.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The kernel ultimately enforces these isolation mechanisms.&lt;/p&gt;

&lt;p&gt;That is why kernel vulnerabilities can become container-security problems.&lt;/p&gt;




&lt;h2&gt;
  
  
  Case Study 1: Copy Fail — CVE-2026-31431
&lt;/h2&gt;

&lt;p&gt;One useful example is &lt;strong&gt;CVE-2026-31431&lt;/strong&gt;, known as "Copy Fail."&lt;/p&gt;

&lt;p&gt;The vulnerability affected the Linux kernel's &lt;code&gt;algif_aead&lt;/code&gt; component of the &lt;code&gt;AF_ALG&lt;/code&gt; userspace cryptographic interface.&lt;/p&gt;

&lt;p&gt;The important point for containers is not simply that the kernel contained a vulnerability.&lt;/p&gt;

&lt;p&gt;It is that a &lt;strong&gt;default container could reach the vulnerable interface&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;At a high level:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Container
    ↓
AF_ALG interface
    ↓
Linux kernel
    ↓
algif_aead
    ↓
CVE-2026-31431
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;According to Docker's technical analysis, the vulnerability could allow an unprivileged user with access to an &lt;code&gt;AF_ALG&lt;/code&gt; socket to perform controlled writes to the page cache.&lt;/p&gt;

&lt;p&gt;The page cache is a kernel-managed resource rather than something isolated per container.&lt;/p&gt;

&lt;p&gt;That means the consequences are not necessarily limited to the container in which the vulnerable code executes.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Important Concept: Reachability
&lt;/h2&gt;

&lt;p&gt;This is one of the most useful ideas in container security.&lt;/p&gt;

&lt;p&gt;A vulnerable kernel function is not automatically reachable from every container.&lt;/p&gt;

&lt;p&gt;The practical question is:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Is vulnerable code present?
        ↓
Can the container reach it?
        ↓
Can the attacker trigger it?
        ↓
What resources can be affected?
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For Copy Fail, the relevant interface was &lt;code&gt;AF_ALG&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Docker therefore introduced additional security-profile restrictions to reduce container access to that interface.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fukwaz1b1xrvzvlas8y8q.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fukwaz1b1xrvzvlas8y8q.png" alt="copy fail relaibility" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;But there is an important distinction:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;A Docker security-profile mitigation does not repair vulnerable kernel code.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;A kernel update fixes the underlying vulnerability.&lt;/p&gt;

&lt;p&gt;A security-profile change can instead reduce the ability of containers to reach the vulnerable functionality.&lt;/p&gt;

&lt;p&gt;Both approaches can be valuable.&lt;/p&gt;




&lt;h2&gt;
  
  
  Case Study 2: CVE-2026-80521 and AF_UNIX
&lt;/h2&gt;

&lt;p&gt;A second example is &lt;strong&gt;CVE-2026-80521&lt;/strong&gt;, involving the Linux kernel's &lt;code&gt;AF_UNIX&lt;/code&gt; subsystem.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;AF_UNIX&lt;/code&gt; sockets are widely used for local inter-process communication.&lt;/p&gt;

&lt;p&gt;The published research described a use-after-free involving the kernel's &lt;code&gt;AF_UNIX&lt;/code&gt; garbage-collection mechanism and demonstrated a container escape.&lt;/p&gt;

&lt;p&gt;This example is interesting for a different reason.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;AF_ALG&lt;/code&gt; is a relatively specialized interface.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;AF_UNIX&lt;/code&gt;, on the other hand, is deeply integrated into normal Linux applications.&lt;/p&gt;

&lt;p&gt;That creates a different defensive challenge.&lt;/p&gt;

&lt;p&gt;You can potentially block access to a specialized interface.&lt;/p&gt;

&lt;p&gt;It is much harder to remove access to a kernel subsystem that ordinary applications depend on.&lt;/p&gt;

&lt;h3&gt;
  
  
  A necessary distinction
&lt;/h3&gt;

&lt;p&gt;It would be misleading to summarize this as:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"Containers can always be escaped."&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;That is far too broad.&lt;/p&gt;

&lt;p&gt;A more accurate statement is:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;A kernel vulnerability in functionality reachable from a container can potentially undermine the isolation mechanisms that container runtimes depend on.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Whether a particular system is exploitable depends on factors such as:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Kernel version&lt;/li&gt;
&lt;li&gt;Whether the vulnerable code is present&lt;/li&gt;
&lt;li&gt;Whether the relevant interface is reachable&lt;/li&gt;
&lt;li&gt;Container configuration&lt;/li&gt;
&lt;li&gt;Capabilities&lt;/li&gt;
&lt;li&gt;seccomp&lt;/li&gt;
&lt;li&gt;LSM policies&lt;/li&gt;
&lt;li&gt;User namespace configuration&lt;/li&gt;
&lt;li&gt;Available mitigations&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So the vulnerability demonstrates a limitation of &lt;strong&gt;shared-kernel isolation&lt;/strong&gt;. It does not mean every container on every Linux system is automatically vulnerable.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Does "Container Escape" Actually Mean?
&lt;/h2&gt;

&lt;p&gt;A container escape occurs when code running inside a container gains access beyond the intended container boundary.&lt;/p&gt;

&lt;p&gt;Depending on the vulnerability and configuration, that could mean access to:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Host files&lt;/li&gt;
&lt;li&gt;Host processes&lt;/li&gt;
&lt;li&gt;Host kernel resources&lt;/li&gt;
&lt;li&gt;The container runtime&lt;/li&gt;
&lt;li&gt;Other workloads&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Conceptually:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;NORMAL

Container A  ──X──&amp;gt; Host
Container A  ──X──&amp;gt; Container B


AFTER A SUCCESSFUL ESCAPE

Container A ───────&amp;gt; Host
                       ├── Host filesystem
                       ├── Host processes
                       ├── Container runtime
                       └── Other workloads
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;"Escape" is not one fixed outcome.&lt;/p&gt;

&lt;p&gt;The actual impact depends heavily on the host configuration and what the attacker can access after crossing the intended boundary.&lt;/p&gt;

&lt;p&gt;That is why saying simply &lt;strong&gt;"the container was escaped"&lt;/strong&gt; does not tell the entire story.&lt;/p&gt;




&lt;h2&gt;
  
  
  A Small Hands-On Demonstration
&lt;/h2&gt;

&lt;p&gt;I tested several of these concepts on my own CachyOS system using:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;CachyOS&lt;/li&gt;
&lt;li&gt;Linux kernel 7.2.7-1-cachyos&lt;/li&gt;
&lt;li&gt;Docker Engine 29.8.2&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The following demonstrations do not exploit a vulnerability. They simply show how container isolation works.&lt;/p&gt;

&lt;h2&gt;
  
  
  Check Docker's Security Configuration
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;docker info &lt;span class="nt"&gt;--format&lt;/span&gt; &lt;span class="s2"&gt;"{{.SecurityOptions}} cgroup v{{.CgroupVersion}}"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;My system reported:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[name=seccomp,profile=builtin name=cgroupns] cgroup v2
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This showed that Docker was using its built-in seccomp profile and cgroup namespaces.&lt;/p&gt;

&lt;p&gt;AppArmor was not enabled on this system.&lt;/p&gt;

&lt;p&gt;That is an important reminder:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Container security depends partly on the host configuration.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  Compare Container and Host Processes
&lt;/h2&gt;

&lt;p&gt;Start a container:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;docker run &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="nt"&gt;--rm&lt;/span&gt; &lt;span class="nt"&gt;--name&lt;/span&gt; nslab alpine &lt;span class="nb"&gt;sleep &lt;/span&gt;30
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then inspect the process from the host:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;ps &lt;span class="nt"&gt;-o&lt;/span&gt; user,pid,cmd &lt;span class="nt"&gt;-C&lt;/span&gt; &lt;span class="nb"&gt;sleep&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The host can see the process.&lt;/p&gt;

&lt;p&gt;Inside a normal container, however:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;docker run &lt;span class="nt"&gt;--rm&lt;/span&gt; alpine ps
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;the container sees its own PID namespace rather than the host's complete process list.&lt;/p&gt;

&lt;p&gt;This is a simple demonstration of what namespaces actually provide:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;the process still exists on the host, but the container has a restricted view of it.&lt;/strong&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  See What Happens With &lt;code&gt;--privileged&lt;/code&gt;
&lt;/h2&gt;

&lt;p&gt;Now compare a normal container with a privileged container:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;docker run &lt;span class="nt"&gt;--rm&lt;/span&gt; &lt;span class="nt"&gt;--privileged&lt;/span&gt; alpine &lt;span class="se"&gt;\&lt;/span&gt;
  sh &lt;span class="nt"&gt;-c&lt;/span&gt; &lt;span class="s1"&gt;'grep -E "^(CapEff|NoNewPrivs|Seccomp)" /proc/self/status'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A privileged container receives substantially broader privileges and disables the normal seccomp restrictions shown in the test.&lt;/p&gt;

&lt;p&gt;This illustrates an important point:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;"It runs in a container" does not tell you the security posture of that container.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;You also need to know how it was configured.&lt;/p&gt;




&lt;h2&gt;
  
  
  Kubernetes: The Same Kernel Boundary Still Exists
&lt;/h2&gt;

&lt;p&gt;Kubernetes does not create a second kernel for every Pod.&lt;/p&gt;

&lt;p&gt;Pods ultimately run through a container runtime on a node and therefore share that node's Linux kernel.&lt;/p&gt;

&lt;p&gt;However, Kubernetes security settings can significantly change the isolation provided to workloads.&lt;/p&gt;

&lt;p&gt;For example, Kubernetes supports explicitly requesting the runtime's default seccomp profile:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;securityContext&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;seccompProfile&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;RuntimeDefault&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Other settings worth paying attention to include:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;hostPID
hostNetwork
hostIPC
hostPath
privileged
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;These features can intentionally weaken parts of the normal isolation model.&lt;/p&gt;

&lt;p&gt;So instead of asking:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"Does Kubernetes isolate my container?"&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;a better question is:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;"Which isolation mechanisms are actually enabled for this workload and this node?"&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;




&lt;h2&gt;
  
  
  Practical Hardening
&lt;/h2&gt;

&lt;p&gt;The goal is not to make containers impossible to use.&lt;/p&gt;

&lt;p&gt;The goal is to reduce unnecessary privileges and keep the underlying host secure.&lt;/p&gt;

&lt;h2&gt;
  
  
  Host
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Keep the host kernel patched.&lt;/li&gt;
&lt;li&gt;Reboot when an updated kernel needs to become active.&lt;/li&gt;
&lt;li&gt;Follow your distribution's security advisories.&lt;/li&gt;
&lt;li&gt;Stay on a supported kernel series.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Docker
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Keep Docker Engine updated.&lt;/li&gt;
&lt;li&gt;Avoid &lt;code&gt;--privileged&lt;/code&gt; for ordinary workloads.&lt;/li&gt;
&lt;li&gt;Drop unnecessary capabilities.&lt;/li&gt;
&lt;li&gt;Run applications as non-root users where possible.&lt;/li&gt;
&lt;li&gt;Consider &lt;code&gt;no-new-privileges&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Keep the default seccomp profile enabled.&lt;/li&gt;
&lt;li&gt;Avoid &lt;code&gt;seccomp=unconfined&lt;/code&gt; unless there is a documented reason.&lt;/li&gt;
&lt;li&gt;Consider rootless Docker or user namespace remapping.&lt;/li&gt;
&lt;li&gt;Avoid mounting &lt;code&gt;/var/run/docker.sock&lt;/code&gt; unless it is genuinely required.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Kubernetes
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Use &lt;code&gt;seccompProfile: RuntimeDefault&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Consider enabling &lt;code&gt;seccompDefault&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Use &lt;code&gt;runAsNonRoot: true&lt;/code&gt;.&lt;/li&gt;
&lt;li&gt;Set &lt;code&gt;allowPrivilegeEscalation: false&lt;/code&gt; where appropriate.&lt;/li&gt;
&lt;li&gt;Drop unnecessary capabilities.&lt;/li&gt;
&lt;li&gt;Avoid privileged containers.&lt;/li&gt;
&lt;li&gt;Avoid unnecessary host namespace sharing.&lt;/li&gt;
&lt;li&gt;Keep worker-node kernels patched.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  What About Untrusted Workloads?
&lt;/h2&gt;

&lt;p&gt;Not every workload needs the same isolation strength.&lt;/p&gt;

&lt;p&gt;A normal web application running in a container has a different threat model from:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Multi-tenant infrastructure&lt;/li&gt;
&lt;li&gt;Customer-submitted code&lt;/li&gt;
&lt;li&gt;Untrusted CI jobs&lt;/li&gt;
&lt;li&gt;Generated-code execution&lt;/li&gt;
&lt;li&gt;Security research sandboxes&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For highly untrusted workloads, stronger isolation technologies may be appropriate.&lt;/p&gt;

&lt;p&gt;Examples include:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Firecracker&lt;/li&gt;
&lt;li&gt;Kata Containers&lt;/li&gt;
&lt;li&gt;gVisor&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These approaches introduce their own trade-offs in complexity, compatibility, and performance.&lt;/p&gt;

&lt;p&gt;The goal is not to declare containers obsolete.&lt;/p&gt;

&lt;p&gt;The goal is to choose an isolation boundary appropriate for the workload.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Defenders Can Actually Monitor
&lt;/h2&gt;

&lt;p&gt;Patching is the first line of defense.&lt;/p&gt;

&lt;p&gt;Detection is useful when prevention fails or when someone changes the security configuration unexpectedly.&lt;/p&gt;

&lt;p&gt;On container hosts, useful signals can include unexpected:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;code&gt;setns&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;unshare&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Mount operations&lt;/li&gt;
&lt;li&gt;Kernel module loading&lt;/li&gt;
&lt;li&gt;Access to container runtime sockets&lt;/li&gt;
&lt;li&gt;Privileged processes&lt;/li&gt;
&lt;li&gt;Changes to sensitive host files&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In Kubernetes, defenders can also watch for unexpected:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;code&gt;privileged: true&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;hostPID&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;hostNetwork&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;hostIPC&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;hostPath&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Capability additions&lt;/li&gt;
&lt;li&gt;Suspicious &lt;code&gt;kubectl exec&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;Changes to Pod security settings&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Useful evidence sources include:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;code&gt;auditd&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;journald&lt;/li&gt;
&lt;li&gt;Container runtime logs&lt;/li&gt;
&lt;li&gt;Kubernetes audit logs&lt;/li&gt;
&lt;li&gt;Kernel logs&lt;/li&gt;
&lt;li&gt;eBPF-based runtime telemetry&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Tools such as Falco and Tetragon can also provide runtime visibility into process and syscall activity.&lt;/p&gt;

&lt;p&gt;For container escapes, &lt;strong&gt;node-level visibility is particularly important&lt;/strong&gt; because the boundary being attacked is ultimately enforced by the node's kernel.&lt;/p&gt;




&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;Containers remain one of the most useful ways to package and run software.&lt;/p&gt;

&lt;p&gt;The important security lesson is not that containers are inherently unsafe.&lt;/p&gt;

&lt;p&gt;It is that their isolation is implemented through &lt;strong&gt;kernel mechanisms&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;The simplified model is:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Application
    ↓
Container configuration
    ↓
Capabilities / user identity
    ↓
seccomp / AppArmor / SELinux
    ↓
Linux kernel
    ↓
Hardware / virtualization boundary
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The 2026 vulnerabilities discussed here illustrate two different sides of that boundary.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;CVE-2026-31431 (Copy Fail)&lt;/strong&gt; showed how restricting access to a vulnerable kernel interface can reduce container exposure.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;CVE-2026-80521&lt;/strong&gt; demonstrated the harder case: when vulnerable functionality belongs to a subsystem that ordinary applications depend on, completely removing access to that functionality may not be practical.&lt;/p&gt;

&lt;p&gt;The practical defense is therefore layered:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Reduce container privileges.&lt;/li&gt;
&lt;li&gt;Restrict unnecessary kernel interfaces.&lt;/li&gt;
&lt;li&gt;Use seccomp and LSM policies.&lt;/li&gt;
&lt;li&gt;Keep the host kernel patched.&lt;/li&gt;
&lt;li&gt;Monitor the node, not only the application.&lt;/li&gt;
&lt;li&gt;Use stronger isolation when the workload is genuinely untrusted.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;And the key idea to remember is simple:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;A container can isolate an application from the host only as well as the kernel enforcing that isolation.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>linux</category>
      <category>docker</category>
      <category>kubernetes</category>
      <category>containers</category>
    </item>
    <item>
      <title># LibreWolf Stopped Playing Videos After the FFmpeg 9 Update</title>
      <dc:creator>Musa Nayyer</dc:creator>
      <pubDate>Wed, 30 Sep 2026 14:21:18 +0000</pubDate>
      <link>https://dev.to/muzasio/-librewolf-stopped-playing-videos-after-the-ffmpeg-9-update-kg4</link>
      <guid>https://dev.to/muzasio/-librewolf-stopped-playing-videos-after-the-ffmpeg-9-update-kg4</guid>
      <description>&lt;p&gt;After a recent system update on Arch Linux, video playback in LibreWolf suddenly stopped working.&lt;/p&gt;

&lt;p&gt;At first, it wasn't obvious what had changed. I went through the usual possibilities, including hardware acceleration, VPN/DNS issues, browser cache, and other playback-related settings.&lt;/p&gt;

&lt;p&gt;Eventually, the timing pointed toward an FFmpeg update.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Problem
&lt;/h2&gt;

&lt;p&gt;LibreWolf was no longer able to play videos correctly after the system update.&lt;/p&gt;

&lt;p&gt;The browser reported:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;NS_ERROR_DOM_MEDIA_METADATA_ERR
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The important part was that normal browsing still worked. The failure was specific to media playback.&lt;/p&gt;

&lt;p&gt;That left several possible places to investigate.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Tried
&lt;/h2&gt;

&lt;p&gt;I started with the usual browser-side checks.&lt;/p&gt;

&lt;p&gt;I checked hardware acceleration and browser-related settings to make sure the problem wasn't caused by a configuration change inside LibreWolf.&lt;/p&gt;

&lt;p&gt;I also looked at the VPN and DNS configuration because the problem initially happened while the VPN was being used.&lt;/p&gt;

&lt;p&gt;The OpenVPN logs showed:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Initialization Sequence Completed
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;So the VPN connection itself was completing successfully.&lt;/p&gt;

&lt;p&gt;There was also a &lt;code&gt;resolvconf&lt;/code&gt; warning:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;resolvconf: signature mismatch: /etc/resolv.conf
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;I checked the resolver state and manually ran:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;sudo &lt;/span&gt;resolvconf &lt;span class="nt"&gt;-u&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That completed without producing the previous lock error, but it did not fix LibreWolf's media playback.&lt;/p&gt;

&lt;p&gt;At that point, the network configuration was looking less likely to be the cause.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F66w1teifycuggb5pkkvu.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F66w1teifycuggb5pkkvu.png" alt="roubleshooting flow showing LibreWolf video playback failure, FFmpeg 8 to 9 update, compatibility investigation, and ffmpeg4.4 workaround" width="800" height="800"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Found
&lt;/h2&gt;

&lt;p&gt;The investigation eventually moved toward the multimedia libraries used by LibreWolf.&lt;/p&gt;

&lt;p&gt;The important change in the system update was an FFmpeg major-version update from 8 to 9.&lt;/p&gt;

&lt;p&gt;That timing stood out because FFmpeg major releases can involve changes to the libraries applications depend on. If an application or package hasn't been updated for the new library interface yet, compatibility problems can occur.&lt;/p&gt;

&lt;p&gt;That matched what I was seeing.&lt;/p&gt;

&lt;p&gt;The problem was no longer looking like a VPN, DNS, browser-cache, or ordinary browser-settings issue.&lt;/p&gt;

&lt;p&gt;The FFmpeg update provided a much stronger lead.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Compatibility Problem
&lt;/h2&gt;

&lt;p&gt;The evidence pointed toward a compatibility problem between the updated FFmpeg libraries and the LibreWolf build installed on my system.&lt;/p&gt;

&lt;p&gt;The important distinction is that this does &lt;strong&gt;not&lt;/strong&gt; mean that FFmpeg 9 itself was broken.&lt;/p&gt;

&lt;p&gt;The problem was the transition between major library versions and whether the application using those libraries was ready for that change.&lt;/p&gt;

&lt;p&gt;In my case, the installed LibreWolf package had not yet caught up with the transition.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Foyguf7akj7xpyutuq9nk.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Foyguf7akj7xpyutuq9nk.png" alt="Diagram showing LibreWolf using FFmpeg libraries for media decoding and the role of ABI compatibility between application and multimedia libraries" width="800" height="800"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The Workaround
&lt;/h2&gt;

&lt;p&gt;I tested the compatibility theory by installing the older FFmpeg compatibility package:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;sudo &lt;/span&gt;pacman &lt;span class="nt"&gt;-S&lt;/span&gt; ffmpeg4.4
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;ffmpeg4.4&lt;/code&gt; package provides the older FFmpeg 4.4 libraries alongside the current FFmpeg package.&lt;/p&gt;

&lt;p&gt;After installing it, LibreWolf was able to play videos again.&lt;/p&gt;

&lt;p&gt;That gave me a much stronger connection between the FFmpeg transition and the playback failure.&lt;/p&gt;

&lt;h2&gt;
  
  
  Verification
&lt;/h2&gt;

&lt;p&gt;The verification was simple.&lt;/p&gt;

&lt;p&gt;I opened the same video that had previously failed.&lt;/p&gt;

&lt;p&gt;After installing &lt;code&gt;ffmpeg4.4&lt;/code&gt;, playback worked again.&lt;/p&gt;

&lt;p&gt;That made the compatibility explanation much more convincing than the earlier theories around VPN, DNS, hardware acceleration, or browser cache.&lt;/p&gt;

&lt;p&gt;The important sequence was:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;FFmpeg update
      ↓
LibreWolf media playback fails
      ↓
ffmpeg4.4 installed
      ↓
Media playback works again
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This was also consistent with a recent Arch Linux forum report from August 2026 where LibreWolf video playback broke after the FFmpeg 8 → 9 update and &lt;code&gt;ffmpeg4.4&lt;/code&gt; was used as a temporary workaround until a LibreWolf update resolved the issue.&lt;/p&gt;

&lt;h2&gt;
  
  
  Trade-offs
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;ffmpeg4.4&lt;/code&gt; is a workaround, not necessarily the permanent solution.&lt;/p&gt;

&lt;p&gt;The better long-term solution is for the affected application/package to support the newer FFmpeg libraries.&lt;/p&gt;

&lt;p&gt;That means the compatibility package may no longer be necessary once LibreWolf and its dependencies have caught up with the newer FFmpeg version.&lt;/p&gt;

&lt;p&gt;I would therefore treat the older package as a temporary compatibility measure rather than something that should automatically be kept forever.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Learned
&lt;/h2&gt;

&lt;p&gt;A major multimedia-library update can break an application even when the application itself appears to be working normally.&lt;/p&gt;

&lt;p&gt;The useful clue here was the timing.&lt;/p&gt;

&lt;p&gt;Video playback stopped working immediately after a significant system-library update, while the rest of the browser continued working.&lt;/p&gt;

&lt;p&gt;That made the changed multimedia libraries worth investigating.&lt;/p&gt;

&lt;p&gt;The troubleshooting path ended up looking like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Media playback failure
        ↓
Browser settings checked
        ↓
VPN/DNS checked
        ↓
Network issue became less likely
        ↓
FFmpeg major update noticed
        ↓
Compatibility tested
        ↓
ffmpeg4.4 restored playback
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The error initially looked like a browser media problem.&lt;/p&gt;

&lt;p&gt;The actual investigation led underneath the browser, into the system multimedia libraries it depended on.&lt;/p&gt;

&lt;h2&gt;
  
  
  If You Hit Something Similar
&lt;/h2&gt;

&lt;p&gt;If LibreWolf or another application suddenly stops playing media after an Arch Linux update, check:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The exact media error&lt;/li&gt;
&lt;li&gt;Recent package upgrades&lt;/li&gt;
&lt;li&gt;FFmpeg version changes&lt;/li&gt;
&lt;li&gt;Browser and multimedia-library dependencies&lt;/li&gt;
&lt;li&gt;Hardware-acceleration settings&lt;/li&gt;
&lt;li&gt;Whether the problem occurs with different media sources&lt;/li&gt;
&lt;li&gt;Whether a newer application build is available&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The package-update history can be especially useful when the failure starts immediately after a system upgrade.&lt;/p&gt;

&lt;p&gt;Don't keep changing browser settings indefinitely if the problem started at exactly the same time as a major dependency update.&lt;/p&gt;

&lt;p&gt;Check what changed underneath the application.&lt;/p&gt;

&lt;h2&gt;
  
  
  Environment
&lt;/h2&gt;

&lt;p&gt;The relevant environment during testing was:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;OS: CachyOS
Browser: LibreWolf
Session: Wayland
Desktop: KDE Plasma 6
FFmpeg: 8 → 9 transition
Compatibility package tested: ffmpeg4.4
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Your exact behavior may differ depending on your LibreWolf version, FFmpeg version, package source, and system configuration.&lt;/p&gt;

&lt;h2&gt;
  
  
  Final Takeaway
&lt;/h2&gt;

&lt;p&gt;The problem initially looked like a LibreWolf media error.&lt;/p&gt;

&lt;p&gt;The timing of the failure pointed toward the FFmpeg 8 → 9 transition.&lt;/p&gt;

&lt;p&gt;Installing &lt;code&gt;ffmpeg4.4&lt;/code&gt; restored video playback on my system, which provided a practical compatibility workaround.&lt;/p&gt;

&lt;p&gt;I would treat that as a temporary measure rather than a permanent fix. Once the affected LibreWolf package supports the newer FFmpeg libraries correctly, the compatibility package should no longer be necessary.&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://archlinux.org/packages/extra/x86_64/ffmpeg/" rel="noopener noreferrer"&gt;Arch Linux: FFmpeg package&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://archlinux.org/packages/extra/x86_64/ffmpeg4.4/" rel="noopener noreferrer"&gt;Arch Linux: ffmpeg4.4 compatibility package&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://librewolf.net/installation/arch/" rel="noopener noreferrer"&gt;LibreWolf: Arch Linux installation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://bbs.archlinux.org/viewtopic.php?pid=2306687" rel="noopener noreferrer"&gt;Arch Linux Forums: LibreWolf video playback issue after FFmpeg 9 update&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>linux</category>
      <category>cachyos</category>
      <category>ffmpeg</category>
      <category>librewolf</category>
    </item>
    <item>
      <title>KWin Crashed Every Time I Fullscreened Edge on My Second Monitor: Direct Scanout Was the Culprit</title>
      <dc:creator>Musa Nayyer</dc:creator>
      <pubDate>Wed, 23 Sep 2026 18:09:41 +0000</pubDate>
      <link>https://dev.to/muzasio/kwin-crashed-every-time-i-fullscreened-edge-on-my-second-monitor-direct-scanout-was-the-culprit-53m1</link>
      <guid>https://dev.to/muzasio/kwin-crashed-every-time-i-fullscreened-edge-on-my-second-monitor-direct-scanout-was-the-culprit-53m1</guid>
      <description>&lt;h1&gt;
  
  
  KWin Crashed Every Time I Fullscreened Edge on My Second Monitor: Direct Scanout Was the Culprit
&lt;/h1&gt;

&lt;p&gt;I ran into a reproducible KWin Wayland crash that happened whenever I fullscreened Microsoft Edge on my secondary monitor.&lt;/p&gt;

&lt;p&gt;The strange part was that the monitor itself wasn't disconnecting. The screen briefly went black, KWin crashed, and the desktop came back as if a new display had been connected.&lt;/p&gt;

&lt;p&gt;After narrowing down the symptoms, I traced the crash to KWin's direct-scanout path.&lt;/p&gt;

&lt;p&gt;The workaround was to disable direct scanout.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Setup
&lt;/h2&gt;

&lt;p&gt;This happened on a Linux desktop running:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;CachyOS&lt;/li&gt;
&lt;li&gt;KDE Plasma 6&lt;/li&gt;
&lt;li&gt;Wayland&lt;/li&gt;
&lt;li&gt;Hybrid AMD/NVIDIA graphics&lt;/li&gt;
&lt;li&gt;Microsoft Edge&lt;/li&gt;
&lt;li&gt;Two monitors&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The secondary display was connected through HDMI and was running at 1920×1080 around 60 Hz.&lt;/p&gt;

&lt;p&gt;The primary display was running at a much higher refresh rate.&lt;/p&gt;

&lt;p&gt;The problem only appeared when I fullscreened Edge on the secondary monitor.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Happened
&lt;/h2&gt;

&lt;p&gt;The first thing I noticed was confusing.&lt;/p&gt;

&lt;p&gt;I would fullscreen an Edge window on the second monitor and the display would suddenly go black for about a second.&lt;/p&gt;

&lt;p&gt;Then KDE recovered and showed a notification that a new display had been detected.&lt;/p&gt;

&lt;p&gt;At first, this looked like a display or GPU issue.&lt;/p&gt;

&lt;p&gt;But the monitor itself hadn't actually disconnected.&lt;/p&gt;

&lt;p&gt;KWin was crashing, restarting, and detecting the displays again.&lt;/p&gt;

&lt;p&gt;That distinction turned out to be important.&lt;/p&gt;

&lt;h2&gt;
  
  
  Reproducing the Crash
&lt;/h2&gt;

&lt;p&gt;The crash was surprisingly consistent.&lt;/p&gt;

&lt;p&gt;The sequence was roughly:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Open Microsoft Edge.&lt;/li&gt;
&lt;li&gt;Move the window to the secondary monitor.&lt;/li&gt;
&lt;li&gt;Enter fullscreen.&lt;/li&gt;
&lt;li&gt;The secondary display goes black.&lt;/li&gt;
&lt;li&gt;KWin crashes.&lt;/li&gt;
&lt;li&gt;The desktop recovers.&lt;/li&gt;
&lt;li&gt;KDE reports that a display was detected again.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;I could repeat the same sequence and get the same result.&lt;/p&gt;

&lt;p&gt;That made it look repeatable rather than like a one-off crash.&lt;/p&gt;

&lt;h2&gt;
  
  
  Looking at KWin
&lt;/h2&gt;

&lt;p&gt;I checked the compositor logs after reproducing the problem.&lt;/p&gt;

&lt;p&gt;The relevant crash was occurring inside KWin rather than in Edge itself.&lt;/p&gt;

&lt;p&gt;The display was also being re-detected after the crash, which explained the misleading "new display detected" behavior.&lt;/p&gt;

&lt;p&gt;That narrowed the problem down considerably.&lt;/p&gt;

&lt;p&gt;The next question was what KWin was doing differently when a window entered fullscreen.&lt;/p&gt;

&lt;h2&gt;
  
  
  Direct Scanout
&lt;/h2&gt;

&lt;p&gt;Wayland compositors can sometimes bypass parts of the normal compositing process when a fullscreen surface can be displayed directly on a monitor.&lt;/p&gt;

&lt;p&gt;This is called &lt;strong&gt;direct scanout&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Normally, the compositor combines the visible surfaces into a final frame and sends that frame to the display.&lt;/p&gt;

&lt;p&gt;With direct scanout, a fullscreen surface can potentially be presented directly instead of going through the normal compositing path.&lt;/p&gt;

&lt;p&gt;This reduces compositor work and can lower latency.&lt;/p&gt;

&lt;p&gt;The problem is that this path interacts with the display and GPU stack differently from normal compositing.&lt;/p&gt;

&lt;p&gt;In my case, fullscreening Edge on the secondary monitor appeared to trigger a failure in that path.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Important Clue
&lt;/h2&gt;

&lt;p&gt;The important clue was the sequence of symptoms:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fullscreen Edge → black screen → KWin crash → display re-detected&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The monitor wasn't physically disappearing.&lt;/p&gt;

&lt;p&gt;KWin was failing during the transition into the fullscreen presentation path.&lt;/p&gt;

&lt;p&gt;That pointed to the direct-scanout path as the likely failure point.&lt;/p&gt;

&lt;h2&gt;
  
  
  Testing the Theory
&lt;/h2&gt;

&lt;p&gt;The simplest test was to disable direct scanout and reproduce the same fullscreen action.&lt;/p&gt;

&lt;p&gt;I disabled it using the relevant KWin environment/configuration workaround and then tested Edge again.&lt;/p&gt;

&lt;p&gt;After that, fullscreening Edge on the secondary monitor stopped crashing the session.&lt;/p&gt;

&lt;p&gt;I repeated the same fullscreen action, and the session stayed up.&lt;/p&gt;

&lt;p&gt;That gave me a practical way to isolate the problem.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Workaround
&lt;/h2&gt;

&lt;p&gt;The workaround was to disable direct scanout.&lt;/p&gt;

&lt;p&gt;This isn't a universal fix for KWin crashes. It was useful here because disabling the feature removed the specific presentation path that was triggering the crash.&lt;/p&gt;

&lt;p&gt;The trade-off is that the compositor has to do more work.&lt;/p&gt;

&lt;p&gt;That can slightly increase compositor and GPU usage.&lt;/p&gt;

&lt;p&gt;For my setup, that was a reasonable trade-off compared with having KWin crash whenever Edge entered fullscreen on the second monitor.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Learned
&lt;/h2&gt;

&lt;p&gt;The confusing part of this bug was that the visible symptom looked like a monitor problem.&lt;/p&gt;

&lt;p&gt;The display went black and KDE reported that a display had been connected again.&lt;/p&gt;

&lt;p&gt;But the actual failure was KWin.&lt;/p&gt;

&lt;p&gt;The useful debugging lesson was to follow the complete sequence instead of assuming the first visible symptom was the cause.&lt;/p&gt;

&lt;p&gt;In this case:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Fullscreen
    ↓
Direct-scanout transition
    ↓
KWin crash
    ↓
Display re-detection
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Once the crash itself was identified, the display behavior made much more sense.&lt;/p&gt;

&lt;h2&gt;
  
  
  If You Hit Something Similar
&lt;/h2&gt;

&lt;p&gt;If KWin crashes specifically when fullscreening an application on Wayland, check:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;KWin crash logs&lt;/li&gt;
&lt;li&gt;GPU driver information&lt;/li&gt;
&lt;li&gt;Whether the problem is limited to one display&lt;/li&gt;
&lt;li&gt;Whether it happens only with fullscreen windows&lt;/li&gt;
&lt;li&gt;Whether disabling direct scanout changes the behavior&lt;/li&gt;
&lt;li&gt;Whether the issue is reproducible with another application&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Also check whether the problem changes when using different GPU/display combinations on hybrid graphics systems.&lt;/p&gt;

&lt;p&gt;Don't immediately blame the monitor just because the screen goes black.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Check the compositor before blaming the monitor.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Environment
&lt;/h2&gt;

&lt;p&gt;My relevant environment at the time of testing included:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;OS: CachyOS
Desktop: KDE Plasma 6
Session: Wayland
GPU: Hybrid AMD/NVIDIA
Browser: Microsoft Edge
Primary display: 1920×1080 @ ~180 Hz
Secondary display: 1920×1080 @ ~60 Hz
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Your exact behavior may differ depending on your GPU, driver version, display configuration, and KWin version.&lt;/p&gt;

&lt;h2&gt;
  
  
  Final Takeaway
&lt;/h2&gt;

&lt;p&gt;The crash initially looked like a monitor disconnect.&lt;/p&gt;

&lt;p&gt;It wasn't.&lt;/p&gt;

&lt;p&gt;The reproducible trigger was fullscreening Edge on the secondary display, and disabling direct scanout prevented the KWin crash on my system.&lt;/p&gt;

&lt;p&gt;For now, I'm keeping the environment/configuration change as a workaround rather than treating it as a permanent fix.&lt;/p&gt;

&lt;p&gt;The proper long-term solution would be for the underlying KWin/graphics-stack issue to be identified and fixed upstream.&lt;/p&gt;




&lt;h3&gt;
  
  
  Sources
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;KDE / KWin documentation&lt;/li&gt;
&lt;li&gt;Relevant KWin crash logs from the affected system&lt;/li&gt;
&lt;li&gt;Linux graphics/display documentation&lt;/li&gt;
&lt;li&gt;Relevant upstream issue and driver information&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>linux</category>
      <category>kde</category>
      <category>wayland</category>
      <category>kwin</category>
    </item>
    <item>
      <title>Samsung A15 Has No App Lock - Samsung Calls It 'Secure' Anyway</title>
      <dc:creator>Musa Nayyer</dc:creator>
      <pubDate>Mon, 29 Jun 2026 11:05:55 +0000</pubDate>
      <link>https://dev.to/muzasio/samsung-a15-has-no-app-lock-samsung-calls-it-secure-anyway-35a2</link>
      <guid>https://dev.to/muzasio/samsung-a15-has-no-app-lock-samsung-calls-it-secure-anyway-35a2</guid>
      <description>&lt;h1&gt;
  
  
  Samsung A15 Has No App Lock - Samsung Calls It 'Secure' Anyway
&lt;/h1&gt;

&lt;p&gt;I wanted to lock one app with a fingerprint. No Secure Folder, no third-party garbage, just a native PIN or fingerprint lock on a single app. Straightforward. Except it isn't - because Samsung has no native App Lock in One UI, and there's no clean way to get it.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Problem
&lt;/h2&gt;

&lt;p&gt;The Samsung Galaxy A15 is running One UI 8, Android 16. Biometric hardware is there. The "secure" branding is all over the box. But when I went to Settings → Biometrics and Security, App Lock was nowhere.&lt;/p&gt;

&lt;p&gt;I searched "App Lock" in the Settings search bar. Nothing. I checked under Lock Screen. Nothing. The only lock-related result I found was &lt;strong&gt;"Hide content while locked"&lt;/strong&gt; - which just hides notification previews on the lock screen. Completely different feature.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Tried
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Settings → Biometrics and Security → no App Lock entry&lt;/li&gt;
&lt;li&gt;Settings search bar → "App Lock" → zero results&lt;/li&gt;
&lt;li&gt;Settings → Lock Screen → scrolled everything → not there&lt;/li&gt;
&lt;li&gt;Galaxy Store → searched "S Secure" (Samsung's dedicated app locker) → not found&lt;/li&gt;
&lt;li&gt;Device: Samsung Galaxy A15, One UI 8.0, Android 16
## What's Actually Going On&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This isn't a budget phone problem. App Lock is missing from Samsung's One UI across the board - A series, S series, all of them. Users on flagship devices are filing the exact same complaint on Samsung Community forums. Samsung never had a dedicated native App Lock built into One UI itself; it was always either Secure Folder or the now-discontinued S Secure app.&lt;/p&gt;

&lt;p&gt;S Secure was Samsung's standalone app locker. It's been pulled from the Galaxy Store entirely. You won't find it.&lt;/p&gt;

&lt;p&gt;And One UI 8.5 doesn't fix this for global users either - the App Lock in 8.5 is part of Samsung's Smart Manager, which is exclusive to China. The rest of the world gets nothing new.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why All the Alternatives Fall Short
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Secure Folder&lt;/strong&gt; - Requires a Samsung account. Creates a separate storage partition. Overkill for locking one app, and it ties you into Samsung's ecosystem just to get basic access control.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;F-Droid open source app lockers&lt;/strong&gt; - No mature, actively maintained option exists. Most are abandoned or launcher-dependent.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Google Play third-party apps&lt;/strong&gt; - Full of ads, background services, and privacy red flags. And they can be bypassed - anyone can go to Settings → Apps → uninstall the locker with zero authentication, and everything unlocks instantly. That completely defeats the purpose.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Should Be Different
&lt;/h2&gt;

&lt;p&gt;App Lock is not a complicated feature. Xiaomi, OPPO, OnePlus ship it natively even on their cheapest devices. Samsung had S Secure, pulled it, never replaced it in the OS, and still sells these phones as "secure."&lt;/p&gt;

&lt;p&gt;The "secure" in Samsung's branding refers to hardware-level encryption and Knox architecture - not feature completeness. That's a real distinction, but it should be on the box, not discovered after the fact by the person who just bought the phone.&lt;/p&gt;

&lt;h2&gt;
  
  
  TIL
&lt;/h2&gt;

&lt;p&gt;App Lock being missing from Samsung isn't a Galaxy A15 problem - it's missing from the entire One UI lineup globally. S Secure is gone. The 8.5 fix is China-only. There is no timeline for global users.&lt;/p&gt;

</description>
      <category>samsung</category>
      <category>security</category>
      <category>android</category>
      <category>oneui</category>
    </item>
    <item>
      <title>Local AI SOC Server on a Laptop - What Actually Broke</title>
      <dc:creator>Musa Nayyer</dc:creator>
      <pubDate>Tue, 09 Jun 2026 10:57:12 +0000</pubDate>
      <link>https://dev.to/muzasio/local-ai-soc-server-on-a-laptop-what-actually-broke-517k</link>
      <guid>https://dev.to/muzasio/local-ai-soc-server-on-a-laptop-what-actually-broke-517k</guid>
      <description>&lt;p&gt;I'm a cybersecurity student. No budget, no enterprise hardware, no client environments.&lt;/p&gt;

&lt;p&gt;I set it up as a local AI server for SOC automation: log triage, CVE analysis, security bots. The internet made it look straightforward. It wasn't. This is the honest version of that story.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;One thing upfront:&lt;/strong&gt; everything here ran on self-generated test data and my own lab logs. No client data, no sensitive information, nothing real. I'm documenting a learning build, not a deployment guide. Don't point this at anyone's actual environment until you've properly hardened it beyond what I did here.&lt;/p&gt;




&lt;h2&gt;
  
  
  The OS Decision - What I Considered and Why It Actually Matters
&lt;/h2&gt;

&lt;p&gt;Most answers online say "just use Ubuntu" with zero justification. Here's the real breakdown.&lt;/p&gt;

&lt;h3&gt;
  
  
  Ubuntu 24.04 LTS - What I Used
&lt;/h3&gt;

&lt;p&gt;This was the right call for my situation. NVIDIA driver support is the most mature here, &lt;code&gt;nvidia-container-toolkit&lt;/code&gt; installs without a fight, and when something breaks at 2am there's a Stack Overflow thread for it.&lt;/p&gt;

&lt;p&gt;The downside: blank slate for security work. You're manually installing &lt;code&gt;nmap&lt;/code&gt;, &lt;code&gt;gobuster&lt;/code&gt;, everything. For a dedicated inference server that's fine, less noise. For a daily driver where you also want to pentest, it gets tedious.&lt;/p&gt;

&lt;h3&gt;
  
  
  Parrot OS / Kali Linux - Worth Considering
&lt;/h3&gt;

&lt;p&gt;Both are Debian-based and ship with the full offensive toolkit pre-installed. Parrot is lighter and more workable alongside inference workloads. Kali is the standard for dedicated pentest infrastructure.&lt;/p&gt;

&lt;p&gt;The appeal: your AI stack and your security toolchain live in the same environment. No dependency wrangling between two systems. The catch: don't run heavy inference and an active &lt;code&gt;nmap&lt;/code&gt; scan simultaneously without capping container resources. They'll fight over RAM and both degrade.&lt;/p&gt;

&lt;h3&gt;
  
  
  Proxmox VE - Good Idea, Painful on Laptops
&lt;/h3&gt;

&lt;p&gt;Turns the machine into a bare-metal hypervisor. Ubuntu for Ollama, Kali for tooling, LXC containers for bots, all isolated, all snapshotted. Great for client work in theory. GPU passthrough on a laptop is genuinely painful though: IOMMU, VFIO config, WiFi driver issues in guests. Skipped it for that reason.&lt;/p&gt;

&lt;h3&gt;
  
  
  NixOS - Long-Term Interesting, Steep Curve
&lt;/h3&gt;

&lt;p&gt;Entire server state lives in one declarative file. Reproducible, version-controlled, no config drift. Compelling on paper but CUDA/GPU support requires specific Nix module knowledge. Worth it eventually if you're comfortable with infrastructure-as-code. Not where I started.&lt;/p&gt;




&lt;h2&gt;
  
  
  Model Stack - What the Internet Says vs. What I Hit
&lt;/h2&gt;

&lt;p&gt;You'll see this configuration recommended everywhere:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Workload&lt;/th&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Approx. RAM (inference only)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Code audit / vuln review&lt;/td&gt;
&lt;td&gt;&lt;code&gt;deepseek-coder-v2&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;~10 GB (16B)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Log analysis / SOC triage&lt;/td&gt;
&lt;td&gt;&lt;code&gt;deepseek-r1&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;6-9 GB (7B-14B)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;General reasoning / recon&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;llama3.1&lt;/code&gt; or &lt;code&gt;mistral&lt;/code&gt;
&lt;/td&gt;
&lt;td&gt;~5 GB (7B)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Automation bots / agents&lt;/td&gt;
&lt;td&gt;&lt;code&gt;qwen2.5-coder&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;~5 GB (7B)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;What the table doesn't tell you:&lt;/strong&gt; these numbers are for inference at idle. In a real pipeline: ingestion scripts, scan tools, and a bot running concurrently, add 4-6 GB on top. On older or low-spec hardware, even 7B models stall after a few runs if the settings aren't tuned.&lt;/p&gt;

&lt;p&gt;My actual experience: the models needed more manual configuration than any tutorial mentioned. Default settings caused the model to get stuck mid-automation after a few cycles. Context window size, temperature, and output format all needed tuning before anything ran reliably.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;Q4_K_M&lt;/code&gt; quantization is a solid starting point, best quality-to-size ratio for most workloads. But treat the whole table as a starting baseline, not a guarantee.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Automation Pipeline
&lt;/h2&gt;

&lt;p&gt;The basic setup I wired together:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fnrdfhyuw57q1gw5hwu4w.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fnrdfhyuw57q1gw5hwu4w.png" alt="Automation pipeline diagram for local AI SOC server" width="800" height="640"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Calling the Ollama API is simple enough:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl http://localhost:11434/api/generate &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'{"model": "llama3.1", "prompt": "Analyze this log entry for anomalies: [LOG]", "stream": false}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The curl call works fine in isolation. Getting it to behave consistently inside an automated pipeline is the different problem, more on that below.&lt;/p&gt;

&lt;p&gt;For visual workflow automation, n8n self-hosted in Docker handles the glue between file watchers, Ollama calls, and notifications well. For CLI piping, Fabric is cleaner: &lt;code&gt;cat auth.log | fabric --pattern analyze_logs&lt;/code&gt; just works with less overhead.&lt;/p&gt;




&lt;h2&gt;
  
  
  One Thing I'd Do Differently
&lt;/h2&gt;

&lt;p&gt;Isolate Ollama in Docker from the start. I didn't, and when an automation script misbehaved it had more filesystem access than it should have. On a personal lab machine that's annoying. On anything with real data it would be a serious problem.&lt;/p&gt;

&lt;p&gt;For anyone building something similar, even just for learning:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Default-deny at the container network level&lt;/li&gt;
&lt;li&gt;Never expose port &lt;code&gt;11434&lt;/code&gt; outside localhost&lt;/li&gt;
&lt;li&gt;Put Nginx with auth in front if you need remote access&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  The Part Nobody Talks About - The Actual Hard Part
&lt;/h2&gt;

&lt;p&gt;The bottleneck isn't inference speed. It's not even which model you pick.&lt;/p&gt;

&lt;p&gt;It's getting &lt;strong&gt;structured, parseable output consistently&lt;/strong&gt; enough to feed into downstream scripts.&lt;/p&gt;

&lt;p&gt;Models are inconsistent by default. Valid JSON for 20 runs, then a plain sentence on run 21, then a half-finished response on run 22. Your pipeline breaks on the edge cases, not the happy path.&lt;/p&gt;

&lt;p&gt;Prompt engineering for reliability matters more than model size here. A smaller model with a well-structured prompt will outperform a larger model with a vague one inside an automation pipeline every time.&lt;/p&gt;

&lt;p&gt;That's the thing I'd tell myself at the start.&lt;/p&gt;




</description>
      <category>linux</category>
      <category>ollama</category>
      <category>cybersecurity</category>
      <category>infosec</category>
    </item>
    <item>
      <title>Claude's Visualize Feature Is Broken — Here's a One-Line Workaround</title>
      <dc:creator>Musa Nayyer</dc:creator>
      <pubDate>Fri, 05 Jun 2026 06:50:19 +0000</pubDate>
      <link>https://dev.to/muzasio/claudes-visualize-feature-is-broken-heres-a-one-line-workaround-ol8</link>
      <guid>https://dev.to/muzasio/claudes-visualize-feature-is-broken-heres-a-one-line-workaround-ol8</guid>
      <description>&lt;p&gt;Since mid-March 2026, a significant chunk of Claude users have been hitting this error whenever Claude tries to render an inline diagram, chart, or interactive widget:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Failed to set up MCP app for "visualize".
Check that claudemcpcontent.com is not blocked by your network or browser.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The instinct is to blame your network. I went through the same cycle — switched DNS to Cloudflare 1.1.1.1, tried Google 8.8.8.8, disabled browser extensions, tested across browsers. Nothing worked.&lt;/p&gt;

&lt;p&gt;Then I ran a direct DNS lookup:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;nslookup claudemcpcontent.com 1.1.1.1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Output:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="go"&gt;Server:   1.1.1.1
&lt;/span&gt;&lt;span class="gp"&gt;Address:  1.1.1.1#&lt;/span&gt;53
&lt;span class="go"&gt;
*** Can't find claudemcpcontent.com: No answer
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Same result with 8.8.8.8. The domain doesn't resolve — at all, from any resolver. Not a user-side issue.&lt;/p&gt;

&lt;h2&gt;
  
  
  What's Actually Happening
&lt;/h2&gt;

&lt;p&gt;Claude's visualize feature depends on an external domain — &lt;code&gt;claudemcpcontent.com&lt;/code&gt; — to serve the MCP app that renders inline SVG/HTML widgets. When that domain goes down, the feature breaks silently with a misleading error that makes it look like a local network problem.&lt;/p&gt;

&lt;p&gt;There's an open GitHub issue tracking this (#34820 on anthropics/claude-code) filed March 16, 2026. It has 50+ affected users, no official fix, and was labeled &lt;code&gt;invalid&lt;/code&gt; because it was filed on the wrong repo. Anthropic hasn't responded substantively. The visualize infrastructure had multiple incidents throughout April 2026.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Workaround
&lt;/h2&gt;

&lt;p&gt;Instead of asking Claude to generate a diagram or chart (which triggers the broken MCP visualizer), ask it to generate a PNG file using Pillow.&lt;/p&gt;

&lt;p&gt;Instead of:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"Create a bar chart showing X"&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Say:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"Create a bar chart showing X as a PNG file using Pillow"&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Claude writes Python, executes it via its bash tool, and drops a downloadable PNG in the outputs directory. No MCP dependency. No &lt;code&gt;claudemcpcontent.com&lt;/code&gt;. Completely different rendering pipeline.&lt;/p&gt;

&lt;p&gt;Works for bar charts, line graphs, flowcharts, architecture diagrams — anything you'd normally visualize inline.&lt;/p&gt;

&lt;h2&gt;
  
  
  TIL
&lt;/h2&gt;

&lt;p&gt;Claude's inline visualizer depends on an external domain serving an MCP app. When that domain goes down, the feature fails with an error that looks like a local network problem. Asking for a PNG via Pillow bypasses the dependency entirely.&lt;/p&gt;

</description>
      <category>claude</category>
      <category>devlog</category>
      <category>todayilearned</category>
      <category>muzasio</category>
    </item>
    <item>
      <title>CachyOS Wouldn't Install and My NVMe Was the Problem (Corrupted GPT + chwd Fix)</title>
      <dc:creator>Musa Nayyer</dc:creator>
      <pubDate>Fri, 29 May 2026 11:47:30 +0000</pubDate>
      <link>https://dev.to/muzasio/cachyos-wouldnt-install-and-my-nvme-was-the-problem-corrupted-gpt-chwd-fix-2heb</link>
      <guid>https://dev.to/muzasio/cachyos-wouldnt-install-and-my-nvme-was-the-problem-corrupted-gpt-chwd-fix-2heb</guid>
      <description>&lt;h1&gt;
  
  
  CachyOS Wouldn't Install and My NVMe Was the Problem (Corrupted GPT + chwd Fix)
&lt;/h1&gt;




&lt;p&gt;I was setting up CachyOS on a secondary laptop — i7 11th Gen, NVIDIA mobile GPU, Micron 2210 NVMe. Should've been a clean install. It wasn't. Two specific things broke and both of them looked like my fault until they weren't.&lt;/p&gt;

&lt;h2&gt;
  
  
  Problem 1: The NVMe with an Identity Crisis
&lt;/h2&gt;

&lt;p&gt;The CachyOS installer (Calamares) failed to create a partition table on the Micron 2210 NVMe. No useful error, just a failure. I dropped into the terminal and ran &lt;code&gt;gdisk&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;sudo &lt;/span&gt;gdisk /dev/nvme0n1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Three warnings. Then this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;MBR: not present
BSD: not present
APM: not present
GPT: damaged
Found invalid MBR and corrupt GPT.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The drive had a corrupted GPT and ghost MBR headers from a previous install. The Calamares installer couldn't write a clean partition table because it was fighting old metadata it couldn't fully read or overwrite.&lt;/p&gt;

&lt;p&gt;The fix is a full zap from inside &lt;code&gt;gdisk&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;sudo &lt;/span&gt;gdisk /dev/nvme0n1
&lt;span class="c"&gt;# at the prompt:&lt;/span&gt;
x        &lt;span class="c"&gt;# enter expert mode&lt;/span&gt;
z        &lt;span class="c"&gt;# zap GPT data — confirm with y&lt;/span&gt;
         &lt;span class="c"&gt;# then zap MBR — confirm with y&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Drive came out completely clean. Re-ran the installer, partition table created without issue.&lt;/p&gt;

&lt;p&gt;If &lt;code&gt;gdisk&lt;/code&gt; can't get there, hit it with &lt;code&gt;wipefs&lt;/code&gt; first:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;sudo &lt;/span&gt;wipefs &lt;span class="nt"&gt;-a&lt;/span&gt; /dev/nvme0n1
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Intel VMD note:&lt;/strong&gt; If you're on an 11th Gen Intel laptop, check your BIOS for Intel VMD (Volume Management Device). If it's enabled, Linux can see the NVMe drive but can't write to it properly. Disable VMD or switch from RAID to AHCI/NVMe mode before trying any of the above.&lt;/p&gt;

&lt;h2&gt;
  
  
  Problem 2: chwd Killed the Install Mid-Way
&lt;/h2&gt;

&lt;p&gt;After getting past the NVMe issue, the installer failed again — this time at &lt;code&gt;chwd&lt;/code&gt;, the CachyOS hardware detection tool. The online installer couldn't pull the driver profiles, likely a mirror timeout.&lt;/p&gt;

&lt;p&gt;The fix: switch to the offline installer.&lt;/p&gt;

&lt;p&gt;It installs a default desktop from the USB without pulling anything from the network. Once booted into the system, run hardware detection manually:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;sudo &lt;/span&gt;chwd &lt;span class="nt"&gt;-a&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That scans your hardware and applies the right driver profiles — in my case, the Intel + NVIDIA hybrid (Optimus) setup for the laptop.&lt;/p&gt;

&lt;h2&gt;
  
  
  TIL
&lt;/h2&gt;

&lt;p&gt;Old partition metadata is silent until it isn't. Before installing anything on a drive that's been used before, zap it manually — don't trust the installer to do it cleanly.&lt;/p&gt;




&lt;h1&gt;
  
  
  muzasio #til #devlog #techexperiment #linux #cachyos #nvme #archlinux
&lt;/h1&gt;

</description>
      <category>archlinux</category>
      <category>devops</category>
      <category>opensource</category>
      <category>cachyos</category>
    </item>
    <item>
      <title>I Tried Building a Complex Security Tool with a 1.5B Local Model — Here's What Broke</title>
      <dc:creator>Musa Nayyer</dc:creator>
      <pubDate>Wed, 27 May 2026 20:06:50 +0000</pubDate>
      <link>https://dev.to/muzasio/i-tried-building-a-complex-security-tool-with-a-15b-local-model-heres-what-broke-2ld2</link>
      <guid>https://dev.to/muzasio/i-tried-building-a-complex-security-tool-with-a-15b-local-model-heres-what-broke-2ld2</guid>
      <description>&lt;p&gt;&lt;strong&gt;Problem:&lt;/strong&gt; I had aider running on Lubuntu, three API keys configured, a detailed architecture diagram, and a clear goal — build a modular forensic data analysis pipeline. What I actually got was token walls, a model that replied "Ok." to everything, and a half-made project directory with nothing inside it.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Setup
&lt;/h2&gt;

&lt;p&gt;Installed aider via the official shell script on Lubuntu. Connected three models:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;OpenAI — failed outright, key didn't authenticate&lt;/li&gt;
&lt;li&gt;Grok — worked partially, got some directory scaffolding done before tokens ran out mid-session&lt;/li&gt;
&lt;li&gt;Ollama local (Qwen 2.5 Coder 1.5B) — connected fine after setting &lt;code&gt;OLLAMA_API_BASE=http://localhost:11434&lt;/code&gt;, but the model kept responding with "Ok." and two tokens flat
The architecture I was trying to build had seven layers: data acquisition, integrity hashing, normalization, schema mapping, anomaly detection, explainability, and 3D visualization with Three.js. Ambitious. Too much for a 1.5B model to hold in its head at once.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What I Tried
&lt;/h2&gt;

&lt;p&gt;First I threw the whole diagram at aider and asked it to scaffold everything. Grok started making directories. Tokens ended. Switched to local Ollama. Local model saw the repo map, said "Ok." — twice. No output, no errors, no explanation.&lt;/p&gt;

&lt;p&gt;I tried prompting it to list the project structure. Still "Ok." Tried asking it to read what happened in the folder. "Ok." The model wasn't broken — it was overwhelmed. A 1.5B model with a full repo map in context has almost no room left to actually generate code.&lt;/p&gt;

&lt;p&gt;Then I realized the real problem: I was treating a small local model like a senior engineer. It's not. It's a fast, cheap code-completer that needs a tight scope and a single task.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Actually Worked
&lt;/h2&gt;

&lt;p&gt;Three things changed the outcome:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;1. One module at a time.&lt;/strong&gt; Instead of "build the pipeline," I asked for one function in one file. The model went from "Ok." to producing real code.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. &lt;code&gt;/read&lt;/code&gt; instead of &lt;code&gt;/add&lt;/code&gt;.&lt;/strong&gt; Using &lt;code&gt;/read&lt;/code&gt; for files the model doesn't need to edit cuts token usage significantly. The model gets context without burning its window on write permissions.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Git as memory across models.&lt;/strong&gt; When Grok's tokens ran out mid-session, I committed whatever it had produced. When I switched to Ollama, I opened with: &lt;em&gt;"The previous model created the acquisition and integrity folders. Look at the current files and continue with the normalization script."&lt;/em&gt; The local model picked up cleanly — because git was the real memory, not the chat history.&lt;/p&gt;

&lt;h2&gt;
  
  
  Result
&lt;/h2&gt;

&lt;p&gt;Got a working &lt;code&gt;acquisition/&lt;/code&gt; module and a &lt;code&gt;schema/event_schema.json&lt;/code&gt; with the Canonical Event Schema defined. The hashing layer and normalization parser came from the local model once I stopped giving it too much context at once. The Three.js visualization is still pending — that's a job for Gemini's free tier, which handles frontend/creative code better than a 1.5B coder model.&lt;/p&gt;

&lt;p&gt;The pipeline isn't done. But the architecture is solid and every module has a defined input/output contract, which means any model can continue any piece without needing the full history.&lt;/p&gt;




&lt;h2&gt;
  
  
  TIL
&lt;/h2&gt;

&lt;p&gt;A 1.5B local model isn't bad at coding — it's bad at holding a complex project in context. Split the work into single-function tasks, use git commits as handoff points between models, and let the repo map do the heavy lifting instead of bloating the chat history.&lt;/p&gt;




&lt;h1&gt;
  
  
  muzasio #til #devlog #techexperiment #aider #ollama #localai #cybersecurity #linux
&lt;/h1&gt;

</description>
      <category>ollama</category>
      <category>aider</category>
      <category>localai</category>
      <category>cybersecurity</category>
    </item>
    <item>
      <title>Signal QR Stuck on Spinner — University Firewall Was the Culprit</title>
      <dc:creator>Musa Nayyer</dc:creator>
      <pubDate>Sun, 24 May 2026 06:43:16 +0000</pubDate>
      <link>https://dev.to/muzasio/signal-qr-stuck-on-spinner-university-firewall-was-the-culprit-1ch7</link>
      <guid>https://dev.to/muzasio/signal-qr-stuck-on-spinner-university-firewall-was-the-culprit-1ch7</guid>
      <description>&lt;p&gt;&lt;strong&gt;Problem:&lt;/strong&gt; I tried linking my iPhone as a secondary Signal device. The QR code just spun forever — a loading spinner inside a white box. No error. Just nothing.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;My setup:&lt;/strong&gt; Primary device — Redmi 13C (PTA), Signal registered and working. Secondary device — iPhone (non-PTA). University Wi-Fi. I wasn't doing a full registration or transfer. Just: &lt;em&gt;Settings → Linked Devices → Link New Device&lt;/em&gt; on iPhone. Dead silence from Signal.&lt;/p&gt;




&lt;h2&gt;
  
  
  What I Tried
&lt;/h2&gt;

&lt;p&gt;First assumption: it's the non-PTA iPhone. Maybe Signal's blocking it? Nope — PTA status is a cellular registration thing. It has zero effect on Wi-Fi app behavior.&lt;/p&gt;

&lt;p&gt;Second assumption: maybe I need to select "transfer from Android"? Nope — I didn't want a transfer. Just a secondary link. The flow is: iPhone shows QR → Redmi scans it.&lt;/p&gt;

&lt;p&gt;I got the flow backwards at first too. Thought Redmi generates QR, iPhone scans. Wrong. &lt;strong&gt;iPhone (secondary) generates the QR. Redmi (primary) scans it.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The QR never loaded on iPhone. That's when it clicked — it's not the phone, it's the network.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Actually Happened
&lt;/h2&gt;

&lt;p&gt;Signal's QR code generation isn't local. The iPhone hits Signal's provisioning API (&lt;code&gt;api.signal.org&lt;/code&gt;, &lt;code&gt;storage.signal.org&lt;/code&gt;) to create a time-sensitive token before displaying the code. My university network was blocking those endpoints.&lt;/p&gt;

&lt;p&gt;Why did Redmi register fine earlier? Probably a different time, different access point, or Android and iOS clients have different TLS fingerprints — and the firewall's application-layer inspection treats them differently. University-grade firewalls (Fortinet, Palo Alto) do this.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Fix
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Proton VPN on both devices, same server location.&lt;/strong&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Installed Proton VPN (free tier) on both iPhone and Redmi&lt;/li&gt;
&lt;li&gt;Connected both to the same server (Netherlands)&lt;/li&gt;
&lt;li&gt;On iPhone → Signal → Linked Devices → Link New Device&lt;/li&gt;
&lt;li&gt;QR code appeared instantly&lt;/li&gt;
&lt;li&gt;Redmi scanned it — linked successfully&lt;/li&gt;
&lt;/ol&gt;

&lt;h2&gt;
  
  
  6. Redmi got logged out immediately after. That's not a bug — Signal logs out existing sessions when a new device links, by design. Re-registered with the phone number and everything was fine. Message history is local anyway; it doesn't sync across linked devices.
&lt;/h2&gt;

&lt;h2&gt;
  
  
  Key Insights
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The QR code requires a live API call.&lt;/strong&gt; If Signal's servers are unreachable, the spinner loops forever with no error message. Signal could honestly show a better error here.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;iOS and Android Signal clients behave differently at the network level.&lt;/strong&gt; A firewall can block one and pass the other.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;University networks block more than you think.&lt;/strong&gt; VoIP, messaging APIs, certain CDN routes — it's not just torrents.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Free VPN is enough for this.&lt;/strong&gt; Proton VPN free tier worked fine. No speed requirements for a one-time provisioning handshake.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  - &lt;strong&gt;Both devices need to be on the same network exit.&lt;/strong&gt; The provisioning token is bound to the originating network context. If iPhone generated it over VPN and Redmi scanned from plain university Wi-Fi, it failed with "QR code not valid."
&lt;/h2&gt;

&lt;h2&gt;
  
  
  TIL
&lt;/h2&gt;

&lt;p&gt;Signal's device linking QR is a remote token, not a local code — and if your network blocks Signal's API endpoints, you'll get an endless spinner with zero explanation.&lt;/p&gt;

</description>
      <category>signal</category>
      <category>vpn</category>
      <category>till</category>
      <category>muzasio</category>
    </item>
    <item>
      <title>Firefox Profiles Frozen and Corrupted on CachyOS — What Broke and How psd Was the Culprit</title>
      <dc:creator>Musa Nayyer</dc:creator>
      <pubDate>Fri, 22 May 2026 12:35:15 +0000</pubDate>
      <link>https://dev.to/muzasio/firefox-profiles-frozen-and-corrupted-on-cachyos-what-broke-and-how-psd-was-the-culprit-39hc</link>
      <guid>https://dev.to/muzasio/firefox-profiles-frozen-and-corrupted-on-cachyos-what-broke-and-how-psd-was-the-culprit-39hc</guid>
      <description>&lt;h2&gt;
  
  
  The Problem
&lt;/h2&gt;

&lt;p&gt;Opened Firefox one morning and the profile picker showed up. Clicked my main profile — nothing happened. Hover worked, tooltips appeared, but clicks registered nowhere. Profile 1 was already dead the day before. Only the third profile (sss) responded. Main profile had six accounts in it.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I Tried
&lt;/h2&gt;

&lt;p&gt;Keyboard navigation — nothing. Tab and Enter didn't work either.&lt;/p&gt;

&lt;p&gt;Checked the Firefox profile directory and found this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight console"&gt;&lt;code&gt;&lt;span class="gp"&gt;lrwxrwxrwx  lbjm2hpz.Profile 2 -&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;/run/user/1000/psd/user-firefox-lbjm2hpz.Profile 2
&lt;span class="gp"&gt;lrwxrwxrwx  xcpk3xfu.default   -&amp;gt;&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;/run/user/1000/psd/user-firefox-xcpk3xfu.default
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Symlinks. Not real directories. psd (profile-sync-daemon) moves Firefox profiles to RAM and replaces them with symlinks. The RAM targets were either empty or broken.&lt;/p&gt;

&lt;p&gt;Ran &lt;code&gt;psd preview&lt;/code&gt; — confirmed psd was managing both profiles. &lt;code&gt;xcpk3xfu.default&lt;/code&gt; showed profile size 0. That profile was already gone.&lt;/p&gt;

&lt;p&gt;Tried &lt;code&gt;psd resync&lt;/code&gt; — ran without errors but didn't fix the picker clicks.&lt;/p&gt;

&lt;p&gt;Tried launching directly from terminal:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;firefox &lt;span class="nt"&gt;--profile&lt;/span&gt; &lt;span class="s2"&gt;"/home/user/.config/mozilla/firefox/lbjm2hpz.Profile 2"&lt;/span&gt;
firefox &lt;span class="nt"&gt;-P&lt;/span&gt; &lt;span class="s2"&gt;"default-release"&lt;/span&gt; &lt;span class="nt"&gt;--no-remote&lt;/span&gt;
&lt;span class="nv"&gt;MOZ_ENABLE_WAYLAND&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;0 firefox &lt;span class="nt"&gt;--profile&lt;/span&gt; &lt;span class="s2"&gt;"/home/user/.config/mozilla/firefox/lbjm2hpz.Profile 2"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;All three kept opening the sss profile instead. The &lt;code&gt;--no-remote&lt;/code&gt; attempts threw repeated channel errors because Firefox was already running.&lt;/p&gt;

&lt;p&gt;Checked &lt;code&gt;about:profiles&lt;/code&gt; — confirmed the current window was actually running &lt;code&gt;default-release&lt;/code&gt; (the main profile), but it was completely empty. No accounts, no history, nothing. The 49M of data was somewhere but Firefox wasn't seeing it.&lt;/p&gt;

&lt;p&gt;Stopped psd, deleted the broken profile directory, tried restoring from the &lt;code&gt;-backup&lt;/code&gt; folder — backup had disappeared when psd stopped (psd overwrites it on shutdown with whatever is in RAM). Only a crashrecovery folder from earlier that morning survived with 49M of actual data.&lt;/p&gt;

&lt;p&gt;Restored from crashrecovery, started psd again — psd immediately created a new symlink pointing to RAM and the profile went missing again. psd was overwriting the restore on every start.&lt;/p&gt;

&lt;p&gt;The GUI click issue turned out to be a separate Wayland input problem with the Firefox profile picker — unrelated to psd but compounding the mess.&lt;/p&gt;

&lt;h2&gt;
  
  
  Root Cause
&lt;/h2&gt;

&lt;p&gt;psd syncs browser profiles to RAM for speed. On unclean shutdowns or crashes, the sync back to disk fails. The &lt;code&gt;-backup&lt;/code&gt; folder gets overwritten on the next psd start with whatever empty state is in RAM. Over time this had been happening repeatedly — the profile directory had 64 crash recovery folders going back months, all showing the same pattern.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;xcpk3xfu.default&lt;/code&gt; was already fully gone. &lt;code&gt;lbjm2hpz.Profile 2&lt;/code&gt; survived only because a crashrecovery snapshot existed from earlier that same day.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Fix
&lt;/h2&gt;

&lt;p&gt;Disabled Firefox in psd by adding this to &lt;code&gt;~/.config/psd/psd.conf&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nv"&gt;BROWSERS&lt;/span&gt;&lt;span class="o"&gt;=(&lt;/span&gt;google-chrome&lt;span class="o"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Restarted psd. Firefox now reads directly from disk. No more symlinks, no more RAM sync, no more silent corruption on shutdown.&lt;/p&gt;

&lt;p&gt;Also ran &lt;code&gt;psd clean&lt;/code&gt; to delete the accumulated crash recovery folders and reclaim disk space.&lt;/p&gt;

&lt;h2&gt;
  
  
  TIL
&lt;/h2&gt;

&lt;p&gt;psd is a silent data risk on systems that crash or suspend frequently — it will eat your browser profiles and leave behind only empty backup folders.&lt;/p&gt;

</description>
      <category>till</category>
      <category>devlog</category>
      <category>tech</category>
      <category>experiment</category>
    </item>
  </channel>
</rss>
