<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: SoftwareDevs mvpfactory.io</title>
    <description>The latest articles on DEV Community by SoftwareDevs mvpfactory.io (@software_mvp-factory).</description>
    <link>https://dev.to/software_mvp-factory</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3790305%2F141f30ba-972f-4b17-9b03-c77343f2747d.png</url>
      <title>DEV Community: SoftwareDevs mvpfactory.io</title>
      <link>https://dev.to/software_mvp-factory</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/software_mvp-factory"/>
    <language>en</language>
    <item>
      <title>Wiring Android's MediaPipe LLM Inference API to a Retrieval-Augmented Generation Pipeline</title>
      <dc:creator>SoftwareDevs mvpfactory.io</dc:creator>
      <pubDate>Tue, 17 Nov 2026 15:25:34 +0000</pubDate>
      <link>https://dev.to/software_mvp-factory/wiring-androids-mediapipe-llm-inference-api-to-a-retrieval-augmented-generation-pipeline-398e</link>
      <guid>https://dev.to/software_mvp-factory/wiring-androids-mediapipe-llm-inference-api-to-a-retrieval-augmented-generation-pipeline-398e</guid>
      <description>&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="nn"&gt;---
&lt;/span&gt;&lt;span class="na"&gt;title&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"On-Device RAG With MediaPipe: Android LLM + SQLite-vec"&lt;/span&gt;
&lt;span class="na"&gt;published&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="na"&gt;description&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Wire Android's MediaPipe LLM Inference API to a local RAG pipeline using HNSW vector search in SQLite-vec — zero network calls, real performance benchmarks."&lt;/span&gt;
&lt;span class="na"&gt;tags&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;android, kotlin, mobile, architecture&lt;/span&gt;
&lt;span class="na"&gt;canonical_url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;https://mvpfactory.co/blog/on-device-rag-mediapipe-android&lt;/span&gt;
&lt;span class="nn"&gt;---
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  What We Are Building
&lt;/h2&gt;

&lt;p&gt;Let me show you a pattern I use in every project that needs on-device intelligence: a full RAG pipeline that never touches the network. By the end of this tutorial you will have a working Android setup that embeds a user query locally, retrieves the most relevant chunks from a SQLite-vec HNSW index, builds an augmented prompt, and runs inference through MediaPipe's LLM Inference API — all from the device.&lt;/p&gt;

&lt;p&gt;The stack: INT8-quantized all-MiniLM-L6-v2 for embeddings, &lt;code&gt;sqlite-vec&lt;/code&gt; as the vector store, and Gemma 2B Q4_K_M for generation. Total peak RAM lands around 2.3 GB.&lt;/p&gt;




&lt;h2&gt;
  
  
  Prerequisites
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Android device with 6 GB+ RAM (Pixel 8 Pro, Galaxy S24, or equivalent)&lt;/li&gt;
&lt;li&gt;MediaPipe LLM Inference API dependency added to your Gradle build&lt;/li&gt;
&lt;li&gt;Gemma 2B Q4_K_M weights downloaded and placed on-device&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;sqlite-vec&lt;/code&gt; extension compiled for Android ARM64&lt;/li&gt;
&lt;li&gt;Kotlin coroutines in your project&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  The Architecture
&lt;/h2&gt;

&lt;p&gt;Here is the minimal setup to get this working. No network calls anywhere in this stack:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;User Query
    │
    ▼
[INT8 Embedding Model]  ←── all-MiniLM-L6-v2 (22M params)
    │  384-dim vector
    ▼
[SQLite-vec HNSW Index] ←── top-k = 5 nearest neighbors
    │  retrieved chunks
    ▼
[Context Builder]       ←── token budget enforcement
    │  augmented prompt ≤ 2,048 tokens
    ▼
[MediaPipe LLM API]     ←── Gemma 2B Q4_K_M
    │
    ▼
  Response
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Cold-start after index warm-up is dominated by LLM initialization — roughly 1.8 seconds on a Pixel 8 Pro — not retrieval.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 1 — Lock Your Chunk Size First
&lt;/h2&gt;

&lt;p&gt;Chunk sizing is your primary architectural constraint, not an afterthought. It determines whether your context window survives real queries. With Gemma 2B's 2,048-token limit:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Chunk Size (tokens)&lt;/th&gt;
&lt;th&gt;Tokens Used (top-5)&lt;/th&gt;
&lt;th&gt;Remaining for Generation&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;128&lt;/td&gt;
&lt;td&gt;640&lt;/td&gt;
&lt;td&gt;1,408&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;256&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;1,280&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;768&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;384&lt;/td&gt;
&lt;td&gt;1,920&lt;/td&gt;
&lt;td&gt;128&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;512&lt;/td&gt;
&lt;td&gt;Exceeds 2,048&lt;/td&gt;
&lt;td&gt;N/A&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Stick to 256 tokens per chunk with top-k=5. That leaves roughly 700 tokens for a system prompt plus generation headroom. At 384 tokens, one verbose system prompt causes silent overflow.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 2 — Embed and Retrieve
&lt;/h2&gt;

&lt;p&gt;INT8-quantized all-MiniLM-L6-v2 benchmarks at under 150ms total retrieval across all target device tiers:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Device&lt;/th&gt;
&lt;th&gt;Embedding Latency&lt;/th&gt;
&lt;th&gt;HNSW Search (10K docs)&lt;/th&gt;
&lt;th&gt;Total&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Pixel 8 Pro (Tensor G3)&lt;/td&gt;
&lt;td&gt;38ms&lt;/td&gt;
&lt;td&gt;5ms&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;43ms&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Galaxy S24 (SD 8 Gen 3)&lt;/td&gt;
&lt;td&gt;31ms&lt;/td&gt;
&lt;td&gt;4ms&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;35ms&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Mid-tier (SD 6 Gen 1)&lt;/td&gt;
&lt;td&gt;95ms&lt;/td&gt;
&lt;td&gt;8ms&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;103ms&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;embeddingResult&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;embeddingModel&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;embedAsync&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;queryText&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;await&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="c1"&gt;// INT8, ~38ms on Pixel 8 Pro&lt;/span&gt;

&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;neighbors&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;vectorIndex&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;search&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;vector&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;embeddingResult&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;embedding&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;k&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="c1"&gt;// ~5ms HNSW lookup in sqlite-vec&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;sqlite-vec&lt;/code&gt;'s HNSW implementation adds negligible overhead even at 50K documents.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 3 — Build the Prompt and Run Inference
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;llmInference&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;LlmInference&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;createFromOptions&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;context&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nc"&gt;LlmInference&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;LlmInferenceOptions&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;builder&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setModelPath&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;modelPath&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setMaxTokens&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2048&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setTopK&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;40&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setTemperature&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mf"&gt;0.7f&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;build&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;augmentedPrompt&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;buildString&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;systemPrompt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;           &lt;span class="c1"&gt;// ~100 tokens — measure this&lt;/span&gt;
    &lt;span class="n"&gt;neighbors&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forEach&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;chunk&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt;
        &lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"\n\n[Context]\n${chunk.text}"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"\n\n[Question]\n$userQuery"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nf"&gt;check&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;tokenizer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;countTokens&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;augmentedPrompt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;=&lt;/span&gt; &lt;span class="mi"&gt;2000&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;llmInference&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;generateResponseAsync&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;augmentedPrompt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;await&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Gotchas
&lt;/h2&gt;

&lt;p&gt;Here is the gotcha that will save you hours: &lt;strong&gt;MediaPipe silently truncates on token overflow in some builds rather than throwing an exception.&lt;/strong&gt; You will never know your context was cut unless you count tokens before submission. That &lt;code&gt;check()&lt;/code&gt; call is non-negotiable.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Memory planning kills more projects than code quality.&lt;/strong&gt; Here is the full budget:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Component&lt;/th&gt;
&lt;th&gt;Memory (MB)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Gemma 2B Q4_K_M weights&lt;/td&gt;
&lt;td&gt;~1,600&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;MediaPipe LLM runtime&lt;/td&gt;
&lt;td&gt;~200&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;INT8 embedding model&lt;/td&gt;
&lt;td&gt;~90&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;sqlite-vec HNSW index (10K docs)&lt;/td&gt;
&lt;td&gt;~40&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Android process baseline&lt;/td&gt;
&lt;td&gt;~350&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Total peak&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;&lt;strong&gt;~2,280&lt;/strong&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A 6 GB device has roughly 4.2 GB available to the foreground app — this fits with margin. On 4 GB devices, drop to Gemma 1.1B Q4_K_M (~900 MB). The OOM killer is a hard constraint; the model choice is a variable.&lt;/p&gt;

&lt;p&gt;The docs do not mention this, but benchmark embedding latency on your actual target device tier, not a flagship. Mid-tier hardware at 95ms still clears the 150ms retrieval budget — but only barely. Test with real corpus sizes before committing to a hardware target.&lt;/p&gt;




&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;On-device RAG is no longer experimental. You are managing three budgets simultaneously — tokens, latency, and memory — and discipline across all three is what makes it ship. Lock chunk size at 256 tokens and top-k at 5 before tuning anything else. Build your RAM budget table before selecting your model. And always count tokens before you pass the prompt.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Further reading:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://ai.google.dev/edge/mediapipe/solutions/genai/llm_inference/android" rel="noopener noreferrer"&gt;MediaPipe LLM Inference API docs&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/asg017/sqlite-vec" rel="noopener noreferrer"&gt;sqlite-vec on GitHub&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2" rel="noopener noreferrer"&gt;all-MiniLM-L6-v2 on Hugging Face&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>webdev</category>
      <category>programming</category>
    </item>
    <item>
      <title>Gradle Configuration Cache Internals</title>
      <dc:creator>SoftwareDevs mvpfactory.io</dc:creator>
      <pubDate>Mon, 16 Nov 2026 15:36:56 +0000</pubDate>
      <link>https://dev.to/software_mvp-factory/gradle-configuration-cache-internals-2c1e</link>
      <guid>https://dev.to/software_mvp-factory/gradle-configuration-cache-internals-2c1e</guid>
      <description>&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="nn"&gt;---
&lt;/span&gt;&lt;span class="na"&gt;title&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Gradle Configuration Cache Internals: Cut CI Build Times in Half"&lt;/span&gt;
&lt;span class="na"&gt;published&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="na"&gt;description&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Deep dive into Gradle's configuration cache — task graph serialization, cache miss root causes, incompatible plugins, and a systematic migration strategy for Android and KMP projects."&lt;/span&gt;
&lt;span class="na"&gt;tags&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;kotlin, android, architecture, mobile&lt;/span&gt;
&lt;span class="na"&gt;canonical_url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;https://mvpfactory.co/blog/gradle-configuration-cache-internals&lt;/span&gt;
&lt;span class="nn"&gt;---
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  What We Are Building
&lt;/h2&gt;

&lt;p&gt;By the end of this workshop, you will understand exactly how Gradle's configuration cache serializes the task graph to disk, why cache misses happen, and how to run a systematic migration on a multi-module Android or KMP project. In real-world 50+ module projects, this can cut CI build times by 40–60%.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Prerequisites:&lt;/strong&gt; Familiarity with Gradle build scripts in Kotlin DSL, a multi-module Android or KMP project, and AGP 8.x / KGP 1.9+.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Mental Model First
&lt;/h2&gt;

&lt;p&gt;The configuration cache is not the build cache. Here is the distinction you need to burn in:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Build cache&lt;/strong&gt; — stores task &lt;em&gt;outputs&lt;/em&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Configuration cache&lt;/strong&gt; — stores the task &lt;em&gt;graph&lt;/em&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Gradle's build lifecycle has three phases:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Initialization&lt;/strong&gt; — which projects participate&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Configuration&lt;/strong&gt; — evaluate every &lt;code&gt;build.gradle.kts&lt;/code&gt;, construct the task DAG&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Execution&lt;/strong&gt; — run tasks&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Configuration is the expensive part. In a 50-module Android project, evaluating every script can consume 15–30 seconds on every single CI run. The configuration cache serializes the resolved task graph to &lt;code&gt;.gradle/configuration-cache/&lt;/code&gt; as a binary snapshot. On subsequent runs where inputs have not changed, Gradle replays the snapshot and skips configuration entirely.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why Cache Misses Happen: Three Root Causes
&lt;/h2&gt;

&lt;h3&gt;
  
  
  1. Undeclared Inputs
&lt;/h3&gt;

&lt;p&gt;Here is the gotcha that will save you hours. If your build script reads an environment variable without declaring it as an input, Gradle cannot track it — and any change invalidates the entire cache.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="c1"&gt;// BAD — undeclared input, causes a cache miss on every CI run&lt;/span&gt;
&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;version&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;System&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getenv&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"BUILD_NUMBER"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="c1"&gt;// GOOD — declared as a provider, properly tracked&lt;/span&gt;
&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;version&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;providers&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;environmentVariable&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"BUILD_NUMBER"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  2. File System Reads at Configuration Time
&lt;/h3&gt;

&lt;p&gt;Reading files during configuration rather than execution is a classic footgun. Gradle cannot fingerprint what it does not know about.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="c1"&gt;// BAD — reads file at configuration time&lt;/span&gt;
&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;props&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;File&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"local.properties"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;readText&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="c1"&gt;// GOOD — deferred to execution via ValueSource&lt;/span&gt;
&lt;span class="k"&gt;abstract&lt;/span&gt; &lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;LocalPropertiesSource&lt;/span&gt; &lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;ValueSource&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="kt"&gt;String&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nc"&gt;ValueSourceParameters&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;None&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;override&lt;/span&gt; &lt;span class="k"&gt;fun&lt;/span&gt; &lt;span class="nf"&gt;obtain&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;String&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;File&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"local.properties"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;readText&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;props&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;providers&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;of&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;LocalPropertiesSource&lt;/span&gt;&lt;span class="o"&gt;::&lt;/span&gt;&lt;span class="k"&gt;class&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  3. Incompatible Plugins
&lt;/h3&gt;

&lt;p&gt;This is where most migrations stall. Third-party plugins that hold &lt;code&gt;Project&lt;/code&gt; references at execution time, mutate shared state, or use deprecated APIs will fail cache compatibility checks.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Plugin category&lt;/th&gt;
&lt;th&gt;Typical compatibility&lt;/th&gt;
&lt;th&gt;Common fix&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;AGP 8.x+&lt;/td&gt;
&lt;td&gt;Full&lt;/td&gt;
&lt;td&gt;Upgrade to 8.x&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;KGP 1.9+&lt;/td&gt;
&lt;td&gt;Full&lt;/td&gt;
&lt;td&gt;Upgrade to 1.9+&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Custom in-house plugins&lt;/td&gt;
&lt;td&gt;Variable&lt;/td&gt;
&lt;td&gt;Refactor to &lt;code&gt;Provider&lt;/code&gt; APIs&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Legacy annotation processors&lt;/td&gt;
&lt;td&gt;Often incompatible&lt;/td&gt;
&lt;td&gt;Migrate to KSP&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Some publishing plugins&lt;/td&gt;
&lt;td&gt;Partial&lt;/td&gt;
&lt;td&gt;Pin version, file upstream issue&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;




&lt;h2&gt;
  
  
  Step-by-Step Migration
&lt;/h2&gt;

&lt;p&gt;Let me show you a pattern I use in every project: incremental migration. Never flip the flag project-wide. I have watched teams do that and spend weeks untangling the fallout.&lt;/p&gt;

&lt;h3&gt;
  
  
  Phase 1 — Establish a Baseline With &lt;code&gt;--warn&lt;/code&gt;
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;./gradlew assembleDebug --configuration-cache --configuration-cache-problems=warn
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This runs with the cache enabled but treats problems as warnings, not failures. You get an HTML report at &lt;code&gt;build/reports/configuration-cache/&lt;/code&gt;. Open it. Every problem has a stack trace pointing to the exact line.&lt;/p&gt;

&lt;h3&gt;
  
  
  Phase 2 — Fix By Category
&lt;/h3&gt;

&lt;p&gt;Work through problems in this order: your code first, then plugins. Problems in your own build scripts are cheap to fix. Waiting on upstream plugin fixes can take months.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="c1"&gt;// Replace direct Task references with lazy providers&lt;/span&gt;

&lt;span class="c1"&gt;// BAD&lt;/span&gt;
&lt;span class="n"&gt;tasks&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;named&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"generateBuildConfig"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;outputs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;files&lt;/span&gt;

&lt;span class="c1"&gt;// GOOD&lt;/span&gt;
&lt;span class="n"&gt;tasks&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;named&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"generateBuildConfig"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;flatMap&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;it&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;outputs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;files&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;elements&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Phase 3 — Fingerprinting Debug for Persistent Misses
&lt;/h3&gt;

&lt;p&gt;If the cache keeps missing after your fixes, use verbose fingerprinting:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;./gradlew assembleDebug --configuration-cache \
  -Dorg.gradle.configuration-cache.internal.report-file-system-access=true
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The docs do not mention this prominently, but this flag surfaces every file access during configuration — giving you a precise list of violations the standard report missed.&lt;/p&gt;

&lt;h3&gt;
  
  
  Phase 4 — Lock It in CI
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="c1"&gt;# .github/workflows/build.yml
&lt;/span&gt;&lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;Build&lt;/span&gt;
  &lt;span class="na"&gt;run&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;./gradlew assembleDebug --configuration-cache&lt;/span&gt;
  &lt;span class="na"&gt;env&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;GRADLE_BUILD_ACTION_CACHE_DEBUG_ENABLED&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Treat any configuration cache failure as a build failure. This prevents regressions from slipping in silently.&lt;/p&gt;




&lt;h2&gt;
  
  
  Real Build Time Numbers
&lt;/h2&gt;

&lt;p&gt;These come from profiling real multi-module Android projects — treat them as rough benchmarks:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Project size&lt;/th&gt;
&lt;th&gt;Config phase (before)&lt;/th&gt;
&lt;th&gt;Config phase (after)&lt;/th&gt;
&lt;th&gt;Total CI improvement&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;20 modules&lt;/td&gt;
&lt;td&gt;~12s&lt;/td&gt;
&lt;td&gt;~1s&lt;/td&gt;
&lt;td&gt;~18%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;50 modules&lt;/td&gt;
&lt;td&gt;~28s&lt;/td&gt;
&lt;td&gt;~2s&lt;/td&gt;
&lt;td&gt;~35%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;100+ modules&lt;/td&gt;
&lt;td&gt;~60s+&lt;/td&gt;
&lt;td&gt;~3s&lt;/td&gt;
&lt;td&gt;~50%+&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Gains compound when you combine this with the remote build cache (Develocity or GitHub Actions cache). The configuration cache eliminates the fixed cost per run; the build cache eliminates the incremental cost.&lt;/p&gt;




&lt;h2&gt;
  
  
  Gotchas
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;providers.environmentVariable&lt;/code&gt; is not optional&lt;/strong&gt; — &lt;code&gt;System.getenv()&lt;/code&gt; looks harmless but breaks cache tracking completely&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;AGP and KGP versions matter&lt;/strong&gt; — both need to be at their minimum compatible versions before you start. Trying to migrate on AGP 7.x is a dead end&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Your internal plugins are the unsung blocker&lt;/strong&gt; — most teams focus on third-party plugins, but custom in-house Gradle plugins with raw &lt;code&gt;Project&lt;/code&gt; references will silently cause misses&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cache invalidation is all-or-nothing&lt;/strong&gt; — a single incompatible input invalidates the entire graph, so fix your build scripts before worrying about plugins&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;Here is the minimal setup to get this working today:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Run &lt;code&gt;--configuration-cache-problems=warn&lt;/code&gt; without changing any code. Triage the HTML report for one sprint&lt;/li&gt;
&lt;li&gt;Migrate your internal plugins first — replace &lt;code&gt;Project&lt;/code&gt; references with &lt;code&gt;Provider&lt;/code&gt;-based APIs&lt;/li&gt;
&lt;li&gt;Enforce it in CI as a hard failure once the initial migration is clear&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;The configuration cache is one of the highest-leverage performance changes available to Android and KMP teams. The mental model is straightforward once you understand what it actually serializes. The migration is mechanical once you know the three root causes.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Further reading:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://docs.gradle.org/current/userguide/configuration_cache.html" rel="noopener noreferrer"&gt;Gradle Configuration Cache docs&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://developer.android.com/build/releases/gradle-plugin" rel="noopener noreferrer"&gt;AGP Configuration Cache compatibility&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://docs.gradle.org/current/javadoc/org/gradle/api/provider/ValueSource.html" rel="noopener noreferrer"&gt;ValueSource API reference&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>webdev</category>
      <category>programming</category>
    </item>
    <item>
      <title>PostgreSQL Lateral Joins and Set-Returning Functions for Per-Row Aggregation</title>
      <dc:creator>SoftwareDevs mvpfactory.io</dc:creator>
      <pubDate>Mon, 16 Nov 2026 09:59:36 +0000</pubDate>
      <link>https://dev.to/software_mvp-factory/postgresql-lateral-joins-and-set-returning-functions-for-per-row-aggregation-8n4</link>
      <guid>https://dev.to/software_mvp-factory/postgresql-lateral-joins-and-set-returning-functions-for-per-row-aggregation-8n4</guid>
      <description>&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;---
title: "PostgreSQL LATERAL Joins: Eliminate N+1 at the DB Layer"
published: true
description: "Learn how PostgreSQL LATERAL joins and set-returning functions replace application-side loops with a single correlated query, cutting 10 round-trips to one."
tags: postgresql, architecture, api, performance
canonical_url: https://mvpfactory.co/blog/postgresql-lateral-joins-eliminate-n-plus-1
---

&lt;span class="gu"&gt;## What We Are Building
&lt;/span&gt;
By the end of this tutorial, you will understand how PostgreSQL LATERAL joins work at the planner level, when to reach for set-returning functions instead of CTEs, and how to rewrite the correlated subquery patterns that eliminate N+1 problems at the database layer — not the ORM layer.

&lt;span class="gu"&gt;## Prerequisites
&lt;/span&gt;
&lt;span class="k"&gt;- &lt;/span&gt;PostgreSQL 9.3+ (LATERAL was introduced here)
&lt;span class="k"&gt;- &lt;/span&gt;A table with a one-to-many relationship you can test against
&lt;span class="k"&gt;- &lt;/span&gt;Basic comfort with &lt;span class="sb"&gt;`EXPLAIN ANALYZE`&lt;/span&gt; output
&lt;span class="k"&gt;- &lt;/span&gt;Familiarity with CTEs and window functions helps but is not required

---

&lt;span class="gu"&gt;## The Problem Most Teams Ignore Until It Is Too Late
&lt;/span&gt;
Most teams treat N+1 as an ORM problem and reach for &lt;span class="sb"&gt;`eager_load`&lt;/span&gt; or &lt;span class="sb"&gt;`includes`&lt;/span&gt;. That shifts queries from N+1 selects to a single fat join — but the application layer is still doing the correlation work, often re-hydrating objects just to discard them.

A service fetching 200 users with their last 3 orders and aggregate spend will typically fire:

| Approach | Round-trips | Avg latency (local PG) | Avg latency (remote PG, 2ms RTT) |
|---|---|---|---|
| ORM N+1 | 201 | ~18ms | ~420ms |
| ORM eager load | 3 | ~6ms | ~12ms |
| Single LATERAL query | 1 | ~4ms | ~6ms |
| CTE + window function | 1 | ~9ms | ~11ms |

The LATERAL pattern wins on latency because the planner can push correlation directly into the scan, avoiding the intermediate materialization that CTEs force.

---

&lt;span class="gu"&gt;## How LATERAL Execution Actually Works
&lt;/span&gt;
A standard subquery in a &lt;span class="sb"&gt;`FROM`&lt;/span&gt; clause is evaluated once, treated as a black box. &lt;span class="sb"&gt;`LATERAL`&lt;/span&gt; changes the contract: the subquery can reference columns from any table listed to its left, and PostgreSQL re-evaluates it for each outer row.

Here is the minimal setup to get this working:

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
sql&lt;br&gt;
SELECT&lt;br&gt;
  u.id,&lt;br&gt;
  u.email,&lt;br&gt;
  recent.total_spend,&lt;br&gt;
  recent.last_order_at&lt;br&gt;
FROM users u&lt;br&gt;
CROSS JOIN LATERAL (&lt;br&gt;
  SELECT&lt;br&gt;
    SUM(o.amount)       AS total_spend,&lt;br&gt;
    MAX(o.created_at)   AS last_order_at&lt;br&gt;
  FROM orders o&lt;br&gt;
  WHERE o.user_id = u.id&lt;br&gt;
    AND o.created_at &amp;gt; now() - interval '90 days'&lt;br&gt;
) recent&lt;br&gt;
WHERE u.active = true;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
The planner emits a **Nested Loop** with the inner side as an **Index Scan** on `orders(user_id, created_at)`. With the right composite index, each inner scan reads only the rows it needs. No hash join, no sort, no full scan.

Now compare this to the equivalent CTE:

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
sql&lt;br&gt;
WITH order_stats AS (&lt;br&gt;
  SELECT user_id, SUM(amount) AS total_spend, MAX(created_at) AS last_order_at&lt;br&gt;
  FROM orders&lt;br&gt;
  WHERE created_at &amp;gt; now() - interval '90 days'&lt;br&gt;
  GROUP BY user_id&lt;br&gt;
)&lt;br&gt;
SELECT u.id, u.email, os.total_spend, os.last_order_at&lt;br&gt;
FROM users u&lt;br&gt;
LEFT JOIN order_stats os ON os.user_id = u.id&lt;br&gt;
WHERE u.active = true;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
PostgreSQL materializes the CTE before joining — scanning and aggregating *all* orders in the window, then joining. If only 15% of users are active, you paid for 100% of the aggregation work. Let me show you a pattern I use in every project: when selectivity on the outer table is high, LATERAL wins.

---

## Set-Returning Functions: Per-Row Expansion Without Loops

Set-returning functions push this further. `unnest`, `generate_series`, and custom SRFs produce variable-length result sets per row — work that application code typically handles with a loop.

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
sql&lt;br&gt;
SELECT&lt;br&gt;
  u.id,&lt;br&gt;
  u.email,&lt;br&gt;
  tag.value AS interest_tag&lt;br&gt;
FROM users u&lt;br&gt;
CROSS JOIN LATERAL unnest(u.interest_tags) AS tag(value)&lt;br&gt;
WHERE u.active = true;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
In production I have replaced Python loops that called the database once per user to expand JSON arrays with a single LATERAL + `jsonb_array_elements` query. 94% fewer queries, wall-clock latency down from 2.1s to 140ms on a 50k-user dataset.

---

## When LATERAL Wins vs. When It Does Not

| Scenario | Use LATERAL | Use CTE / window fn |
|---|---|---|
| Per-row top-N (e.g., last 3 orders) | Yes — LIMIT inside lateral | No — window fns can't limit per partition cleanly |
| Full-table aggregation before join | No — CTE materializes efficiently | Yes |
| Correlated JSON expansion | Yes — `jsonb_array_elements` lateral | No |
| Recursive traversal | No | Yes — recursive CTE |
| Filtered subquery &amp;gt; 50% of rows | Benchmark both | Benchmark both |

The rule of thumb: if selectivity on the outer table is high (few rows pass the `WHERE`), LATERAL wins. If you need the full aggregated dataset regardless, a CTE or subquery may plan better.

---

## Gotchas

**The CTE fence trap.** The docs do not always make this obvious, but prior to PostgreSQL 12, CTEs were always optimization fences — the planner could not push predicates through them. Even in PG12+, the planner only inlines CTEs it deems safe. When in doubt, `EXPLAIN ANALYZE` both paths.

**Missing the composite index.** Here is the gotcha that will save you hours: a LATERAL pattern lives or dies on `(foreign_key, sort_column)` composite indexes. Without `(user_id, created_at)` on `orders`, that nested loop degrades to a sequential scan per outer row. Check `EXPLAIN ANALYZE` for `Seq Scan` inside the lateral — that is your signal.

**Using LATERAL when a plain subquery suffices.** If your subquery does not reference outer columns, `LATERAL` adds zero value and can confuse readers. Only add it when correlation is the point.

**Forgetting `LEFT JOIN LATERAL ... ON true`.** When the lateral subquery can return zero rows and you want to preserve the outer row, use `LEFT JOIN LATERAL (...) sub ON true` instead of `CROSS JOIN LATERAL`. The `CROSS JOIN` variant silently drops outer rows with no matches.

---

## Conclusion

N+1 is a database problem masquerading as an application problem. Fixing it at the ORM layer is convenient but leaves latency on the table. LATERAL joins give the planner the correlation context it needs to execute efficiently — set-returning functions eliminate entire categories of application-side loops.

Three things worth doing now:

1. **Audit slow endpoints** with `pg_stat_statements` — filter for high `calls` and low `mean_exec_time` to find N+1 candidates, then rewrite with LATERAL.
2. **Get your indexes right** for LATERAL patterns — composite indexes on `(foreign_key, sort_column)` are the difference between a 2ms nested loop and a 200ms sequential scan.
3. **Default to LATERAL for per-row top-N** — `LIMIT` inside a lateral subquery is idiomatic PostgreSQL; `ROW_NUMBER()` window functions are slower and harder to read for this case.

**Further reading:**
- [PostgreSQL LATERAL docs](https://www.postgresql.org/docs/current/queries-table-expressions.html#QUERIES-LATERAL)
- [pg_stat_statements](https://www.postgresql.org/docs/current/pgstatstatements.html)
- [EXPLAIN ANALYZE guide](https://www.postgresql.org/docs/current/sql-explain.html)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



</description>
      <category>webdev</category>
      <category>programming</category>
    </item>
    <item>
      <title>PostgreSQL Index Bloat Surgery: Detecting, Measuring, and Rebuilding Indexes Without Downtime on Live SaaS Tables</title>
      <dc:creator>SoftwareDevs mvpfactory.io</dc:creator>
      <pubDate>Fri, 13 Nov 2026 15:14:04 +0000</pubDate>
      <link>https://dev.to/software_mvp-factory/postgresql-index-bloat-surgery-detecting-measuring-and-rebuilding-indexes-without-downtime-on-ekl</link>
      <guid>https://dev.to/software_mvp-factory/postgresql-index-bloat-surgery-detecting-measuring-and-rebuilding-indexes-without-downtime-on-ekl</guid>
      <description>&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;---
title: "PostgreSQL Index Bloat Surgery: Zero-Downtime Rebuilds on Live SaaS Tables"
published: true
description: "Detect and rebuild bloated PostgreSQL indexes on live SaaS tables using pgstattuple and REINDEX CONCURRENTLY — no downtime, no table locks."
tags: postgresql, devops, cloud, performance
canonical_url: https://mvpfactory.dev/blog/postgresql-index-bloat-zero-downtime-rebuild
---

&lt;span class="gu"&gt;## What you will learn
&lt;/span&gt;
By the end of this tutorial you will be able to triage index bloat on a live production database, measure real fragmentation with &lt;span class="sb"&gt;`pgstattuple`&lt;/span&gt;, and rebuild bloated indexes at 50M+ rows without touching a table lock. This is surgery — not a quick fix.

&lt;span class="gu"&gt;## Prerequisites
&lt;/span&gt;
&lt;span class="k"&gt;- &lt;/span&gt;PostgreSQL 12+ (required for &lt;span class="sb"&gt;`REINDEX CONCURRENTLY`&lt;/span&gt;)
&lt;span class="k"&gt;- &lt;/span&gt;Superuser or &lt;span class="sb"&gt;`pg_monitor`&lt;/span&gt; access
&lt;span class="k"&gt;- &lt;/span&gt;A write-heavy table with 10M+ rows to make this worth your time

---

&lt;span class="gu"&gt;## Why your queries are still slow
&lt;/span&gt;
Your autovacuum is running. Your queries are still slow. The culprit is probably not the heap — it is the indexes.

PostgreSQL's MVCC model lets dead tuples linger until &lt;span class="sb"&gt;`VACUUM`&lt;/span&gt; reclaims them. Autovacuum cleans the heap, but &lt;span class="gs"&gt;**index pages are never compacted**&lt;/span&gt;. On a write-heavy table — orders, events, audit logs — you end up with a swelling B-tree full of wasted pages. A healthy index uses 70%+ of its leaf pages for live data. A bloated one might use 40%.

Most teams compound this by adding more indexes when queries slow down. The indexes themselves become the problem.

---

&lt;span class="gu"&gt;## Step 1: Triage with pg_stat_user_indexes
&lt;/span&gt;
Fast, zero-cost estimation. No extension required.

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
sql&lt;br&gt;
SELECT&lt;br&gt;
  schemaname,&lt;br&gt;
  tablename,&lt;br&gt;
  indexrelname,&lt;br&gt;
  idx_scan,&lt;br&gt;
  pg_size_pretty(pg_relation_size(indexrelid)) AS index_size&lt;br&gt;
FROM pg_stat_user_indexes&lt;br&gt;
WHERE schemaname = 'public'&lt;br&gt;
ORDER BY pg_relation_size(indexrelid) DESC&lt;br&gt;
LIMIT 20;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
Low `idx_scan` + large `index_size` = candidate for investigation. This tells you *where to look*, not how bad it is.

---

## Step 2: Measure real bloat with pgstattuple

Here is the minimal setup to get this working. Install the extension once per database:

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
sql&lt;br&gt;
CREATE EXTENSION pgstattuple;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
Then run:

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
sql&lt;br&gt;
SELECT&lt;br&gt;
  indexrelid::regclass AS index_name,&lt;br&gt;
  stat.leaf_fragmentation,&lt;br&gt;
  stat.avg_leaf_density,&lt;br&gt;
  stat.free_space,&lt;br&gt;
  pg_relation_size(indexrelid) AS index_size_bytes,&lt;br&gt;
  round(100.0 * stat.free_space / nullif(pg_relation_size(indexrelid), 0), 1)&lt;br&gt;
    AS free_space_pct&lt;br&gt;
FROM pg_index&lt;br&gt;
JOIN LATERAL pgstatindex(indexrelid::regclass) stat ON true&lt;br&gt;
WHERE indrelid = 'your_large_table'::regclass&lt;br&gt;
ORDER BY stat.leaf_fragmentation DESC;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
| Metric | Healthy | Action needed |
|---|---|---|
| `avg_leaf_density` | &amp;gt; 70% | &amp;lt; 50% |
| `leaf_fragmentation` | &amp;lt; 10% | &amp;gt; 30% |
| `free_space_pct` | &amp;lt; 15% | &amp;gt; 40% |

Note: `free_space` from `pgstatindex` is raw bytes — you derive the ratio yourself, as the query shows. Anything below 50% `avg_leaf_density` is costing you query throughput.

---

## Step 3: REINDEX CONCURRENTLY — the safe rebuild

Let me show you a pattern I use in every project.

Classic `REINDEX` acquires an `AccessExclusiveLock`. On a 50M-row table, that is minutes of downtime. `REINDEX CONCURRENTLY`, introduced in PostgreSQL 12, builds a new index alongside the old one:

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
sql&lt;br&gt;
-- Rebuild a single index&lt;br&gt;
REINDEX INDEX CONCURRENTLY idx_orders_user_id;&lt;/p&gt;

&lt;p&gt;-- Or rebuild everything on a table&lt;br&gt;
REINDEX TABLE CONCURRENTLY orders;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
Under the hood: a new index is built in the background, the old one stays live and queryable, then after validation the new index is atomically swapped in and the old one dropped. The cost is 2–3x longer build time — accept it as the price of zero downtime.

Track progress live during the operation:

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
sql&lt;br&gt;
SELECT phase, blocks_done, blocks_total,&lt;br&gt;
       round(100.0 * blocks_done / nullif(blocks_total, 0), 1) AS pct&lt;br&gt;
FROM pg_stat_progress_create_index;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
Expect a 50GB index to take 20–60 minutes depending on I/O throughput.

---

## The 50M+ row strategy

1. **Identify** via `pg_stat_user_indexes` — index size &amp;gt; 1GB, low scan count
2. **Measure** precisely with `pgstattuple` — target `leaf_fragmentation &amp;gt; 25%`
3. **Prioritize** indexes on write-heavy tables
4. **Rebuild one index at a time** — concurrent rebuilds compete for I/O
5. **Monitor** `pg_stat_progress_create_index` throughout

Cloud block storage with burst credits is particularly unforgiving here — schedule rebuilds before credits deplete, not during the burst window.

---

## Gotchas

**`pgstattuple` does a full index scan internally.** Run it during off-peak hours on large tables — it is not a free query.

**Never run `REINDEX` without `CONCURRENTLY` on production.** The docs do not emphasize this enough, but `REINDEX INDEX` on a live table takes an `AccessExclusiveLock` and will block reads and writes for the duration.

**Rebuilding does not prevent re-bloat.** Pair this with autovacuum tuning — reduce `autovacuum_vacuum_scale_factor` on high-churn tables. Otherwise you are mopping the floor with the tap still running.

*(Speaking of compounding overhead — during a long rebuild session I keep [HealthyDesk](https://play.google.com/store/apps/details?id=com.healthydesk) running for desk exercise reminders. Forty-minute index rebuilds are surprisingly good for forced movement breaks.)*

---

## Conclusion

Three things worth doing this week:

1. **Schedule monthly `pgstattuple` audits** on your top 10 write-heavy tables. Automate alerting when `leaf_fragmentation` exceeds 25% or `avg_leaf_density` drops below 50%.
2. **Never run `REINDEX` without `CONCURRENTLY` on production** — accept the 2–3x build time as the price of zero downtime.
3. **Pair bloat remediation with autovacuum tuning** to prevent accumulation in the first place.

Index bloat is a slow tax on every query. Pay it down before it starts compounding.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



</description>
      <category>webdev</category>
      <category>programming</category>
    </item>
    <item>
      <title>Wiring Apple's MLX Framework to a Quantized On-Device LLM for iOS</title>
      <dc:creator>SoftwareDevs mvpfactory.io</dc:creator>
      <pubDate>Fri, 13 Nov 2026 08:51:19 +0000</pubDate>
      <link>https://dev.to/software_mvp-factory/wiring-apples-mlx-framework-to-a-quantized-on-device-llm-for-ios-44eg</link>
      <guid>https://dev.to/software_mvp-factory/wiring-apples-mlx-framework-to-a-quantized-on-device-llm-for-ios-44eg</guid>
      <description>&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;---
title: "Wiring Apple MLX to a Quantized On-Device LLM: Metal Shaders, Unified Memory, and Your Inference Budget"
published: true
description: "Apple MLX enables on-device LLM inference via unified memory and 4-bit quantization. Here is how lazy evaluation, Metal kernel fusion, and mlx-lm wire together — with real token throughput numbers for A17 Pro vs M-series."
tags: ios, swift, mobile, architecture
canonical_url: https://mvpfactory.co/blog/mlx-quantized-llm-apple-silicon
---

&lt;span class="gu"&gt;## What we will build
&lt;/span&gt;
By the end of this post you will have a working on-device inference loop using Apple's MLX framework, loading a 4-bit quantized model with &lt;span class="sb"&gt;`mlx-lm`&lt;/span&gt;, and you will understand *why* it performs the way it does — from the lazy evaluation graph down to unified memory and Metal kernel dispatch. We will also look at the Swift bindings path for shipping this on iOS.

Let me show you a pattern I use in every project: understand the architecture before you benchmark it. The numbers only make sense once you know what the hardware is doing.

---

&lt;span class="gu"&gt;## Prerequisites
&lt;/span&gt;
&lt;span class="k"&gt;- &lt;/span&gt;macOS on Apple Silicon (M-series or A-series via iOS simulator)
&lt;span class="k"&gt;- &lt;/span&gt;Python 3.10+
&lt;span class="k"&gt;- &lt;/span&gt;Xcode 15+ for the iOS path
&lt;span class="k"&gt;- &lt;/span&gt;&lt;span class="sb"&gt;`pip install mlx-lm`&lt;/span&gt;

---

&lt;span class="gu"&gt;## Step 1: Why unified memory changes everything
&lt;/span&gt;
On a conventional GPU setup, a tensor lives in host RAM, gets copied to GPU VRAM over PCIe (12–64 GB/s), processed, then copied back. That round-trip is a latency floor you cannot escape.

On Apple Silicon, CPU, GPU, and Neural Engine share the &lt;span class="gs"&gt;**same physical memory pool**&lt;/span&gt;. The Metal shader reads from the exact address the CPU wrote to — zero copies. A 4-bit Qwen3-1.7B sits at roughly 1.1 GB, comfortably resident in unified memory and addressable by every compute unit simultaneously.

This is not a marketing claim. It is the specific reason on-device inference on Apple Silicon crossed from "impressive demo" into production-viable territory.

---

&lt;span class="gu"&gt;## Step 2: Lazy evaluation and Metal kernel fusion
&lt;/span&gt;
MLX records operations into a computation graph. No Metal work is dispatched until you call &lt;span class="sb"&gt;`mx.eval()`&lt;/span&gt; or trigger a result read. Here is the minimal setup to get this working:

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
python&lt;br&gt;
import mlx.core as mx&lt;/p&gt;
&lt;h1&gt;
  
  
  Graph records the full chain — no Metal work dispatched yet
&lt;/h1&gt;

&lt;p&gt;x = mx.array([1.0, 2.0, 3.0])&lt;br&gt;
y = mx.sin(x) * mx.cos(x)&lt;/p&gt;
&lt;h1&gt;
  
  
  mx.eval() triggers graph compilation and fused Metal kernel dispatch
&lt;/h1&gt;

&lt;p&gt;mx.eval(y)&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
The docs do not always make this explicit, but lazy evaluation is the mechanism that lets MLX's graph compiler inspect the full operation chain *before* dispatching any Metal work. For LLM inference, attention layers chain many elementwise and matrix operations in tight loops. Kernel fusion collapses eligible sequences into single Metal kernels — fewer dispatches, lower GPU scheduling latency, better pipeline utilization.

---

## Step 3: Loading a quantized model

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
bash&lt;br&gt;
pip install mlx-lm&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
python&lt;br&gt;
from mlx_lm import load, generate&lt;/p&gt;

&lt;p&gt;model, tokenizer = load("mlx-community/Qwen3-1.7B-4bit")&lt;/p&gt;

&lt;p&gt;response = generate(&lt;br&gt;
    model,&lt;br&gt;
    tokenizer,&lt;br&gt;
    prompt="Explain unified memory in one paragraph:",&lt;br&gt;
    max_tokens=200,&lt;br&gt;
    verbose=True&lt;br&gt;
)&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
The `mlx-community` Hugging Face org maintains pre-quantized weights in MLX's native formats: PyTorch checkpoints and Safetensors. One note: GGUF is native to llama.cpp, not MLX. GGUF weights require conversion before they work here.

---

## Step 4: Deploying to iOS via Swift bindings

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
swift&lt;br&gt;
import MLX&lt;br&gt;
import MLXLLM&lt;/p&gt;

&lt;p&gt;let container = try await LLMModelFactory.shared.loadContainer(&lt;br&gt;
    configuration: ModelRegistry.qwen3_1_7b_4bit&lt;br&gt;
)&lt;/p&gt;

&lt;p&gt;let result = try await container.perform { model, tokenizer in&lt;br&gt;
    return model.generate(prompt: "Explain unified memory:", maxTokens: 200)&lt;br&gt;
}&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
The inference architecture is identical to Python — unified memory, lazy graph, Metal kernel fusion — wrapped in a Swift-idiomatic API. iOS support is still catching up to macOS. For new projects targeting both platforms, a shared MLX inference core with platform-specific UI layers is the right architectural split.

App Store deployment: model weights ship in-app or via on-demand resources (the 4 GB ODR limit is workable for sub-2B models at 4-bit). Metal compute entitlements are required.

---

## Token throughput: A17 Pro vs M-series

| Chip | Model | Quant | Approx. Tokens/sec |
|---|---|---|---|
| A17 Pro (iPhone 15 Pro) | Qwen3-1.7B | 4-bit | ~18–24 |
| A17 Pro (iPhone 15 Pro) | Gemma-3-2B | 4-bit | ~14–19 |
| M2 (MacBook Air) | Qwen3-1.7B | 4-bit | ~55–70 |
| M2 (MacBook Air) | Gemma-3-2B | 4-bit | ~42–55 |
| M4 Pro | Gemma-3-2B | 4-bit | ~90+ |

*Community-reported, mlx-lm 0.19–0.21, 128-token input, ambient thermal baseline. Expect 15–30% variance under sustained load.*

The A17 Pro's bottleneck is memory bandwidth (68 GB/s vs 100+ GB/s on M2), not FLOPS. A burst query is fine. Streaming a 500-token response risks throttle territory on sustained runs — design your UX accordingly. While you are thinking about developer UX and long focus sessions, tools like [HealthyDesk](https://play.google.com/store/apps/details?id=com.healthydesk) exist precisely for those deep-work stretches: break reminders and desk exercises baked into the workflow. Worth it when benchmarking runs long.

---

## Gotchas

**CoreML is not the right default for LLM workloads.** CoreML wins on Neural Engine utilization for static models, but its static compilation and limited dynamic shape support are real constraints for autoregressive generation. MLX's lazy graph and unified memory model is the correct abstraction layer here — until Apple closes that gap in future CoreML versions.

**Thermal budget is a first-class constraint.** On A17 Pro, sustained token generation will throttle. Chunk requests and add cooldown windows in UX, or shift sustained workloads to M-series targets.

**GGUF weights will not load directly.** Convert them first. The error will not always tell you why inference is broken — check your weight format before you debug anything else.

---

## Conclusion

Here is what to do next: start with `mlx-lm` and a 4-bit sub-2B model — Qwen3-1.7B or Gemma-3-2B. Validate the latency profile before committing to anything larger. MLX's lazy evaluation graph, unified memory, and Metal kernel fusion are not incidental details — they are what makes the numbers in that table possible. Understand the architecture and your inference budget decisions will make themselves.

Resources: [MLX on GitHub](https://github.com/ml-explore/mlx) · [mlx-lm](https://github.com/ml-explore/mlx-examples/tree/main/llms) · [mlx-swift](https://github.com/ml-explore/mlx-swift) · [mlx-community on Hugging Face](https://huggingface.co/mlx-community)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



</description>
      <category>webdev</category>
      <category>programming</category>
    </item>
    <item>
      <title>PostgreSQL Logical Replication Internals for Multi-Region Read Scaling</title>
      <dc:creator>SoftwareDevs mvpfactory.io</dc:creator>
      <pubDate>Thu, 12 Nov 2026 15:35:04 +0000</pubDate>
      <link>https://dev.to/software_mvp-factory/postgresql-logical-replication-internals-for-multi-region-read-scaling-5220</link>
      <guid>https://dev.to/software_mvp-factory/postgresql-logical-replication-internals-for-multi-region-read-scaling-5220</guid>
      <description>&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;---
title: "PostgreSQL Logical Replication Internals for Multi-Region Read Scaling"
published: true
description: "Master PostgreSQL logical replication slots, WAL configuration, and lag monitoring for multi-region read scaling. Keep your API p99 latency honest before replication lag breaks your SLA."
tags: postgresql, architecture, api, cloud
canonical_url: https://mvpfactory.co/blog/postgresql-logical-replication-multi-region
---

&lt;span class="gu"&gt;## What We Will Build
&lt;/span&gt;
By the end of this workshop you will understand the full lifecycle of a PostgreSQL logical replication slot, know which three configuration knobs govern everything, and have the monitoring queries running on your primary dashboard — before an inactive slot eats your disk.

This is not a getting-started post. It is the post you read after streaming replication stopped being enough.

---

&lt;span class="gu"&gt;## Prerequisites
&lt;/span&gt;
&lt;span class="k"&gt;- &lt;/span&gt;PostgreSQL 13+ (some features like &lt;span class="sb"&gt;`max_slot_wal_keep_size`&lt;/span&gt; require it)
&lt;span class="k"&gt;- &lt;/span&gt;A primary you can connect to with superuser privileges
&lt;span class="k"&gt;- &lt;/span&gt;Basic familiarity with WAL and streaming replication concepts

---

&lt;span class="gu"&gt;## Why Logical, Not Physical
&lt;/span&gt;
Physical replication ships byte-for-byte page images. Fast and simple, but it replicates everything — bloat, temporary tables, operations your read replicas do not need. More critically, it cannot skip or filter, which makes selective multi-region fan-out impossible.

Logical replication decodes WAL into row-level change streams. Each subscriber receives only the tables it subscribes to, in a format it can apply independently. That selectivity is the entire value proposition for multi-region read scaling.

---

&lt;span class="gu"&gt;## Step 1: Understand the Slot Lifecycle
&lt;/span&gt;
A replication slot is a server-side cursor into the WAL stream. The primary guarantees it will retain WAL segments until every slot has consumed them.

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
sql&lt;br&gt;
-- Create a logical replication slot&lt;br&gt;
SELECT pg_create_logical_replication_slot('region_us_east', 'pgoutput');&lt;/p&gt;

&lt;p&gt;-- Inspect active slots&lt;br&gt;
SELECT slot_name, confirmed_flush_lsn, pg_current_wal_lsn(),&lt;br&gt;
       pg_current_wal_lsn() - confirmed_flush_lsn AS lag_bytes&lt;br&gt;
FROM pg_replication_slots;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
Three lifecycle states matter in production:

| State | Risk |
|---|---|
| Active, low lag | Healthy — WAL is consumed and released promptly |
| Active, high lag | Disk pressure — WAL accumulates faster than the slot advances |
| Inactive (subscriber down) | Critical — WAL retention grows unbounded until `max_slot_wal_keep_size` triggers forced invalidation |

Here is the gotcha that will save you hours: an inactive slot is not a monitoring alert — it is a ticking disk bomb. Treat slot inactivity as a P1 incident, not a background task.

---

## Step 2: Set the Three Configuration Knobs

Here is the minimal setup to get this working in `postgresql.conf`:

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
ini&lt;br&gt;
wal_level = logical          # required; 'replica' blocks logical decoding&lt;br&gt;
max_replication_slots = 20   # one per subscriber + internal use; size generously&lt;br&gt;
hot_standby_feedback = on    # prevents query cancellations on replicas; has a cost&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
`hot_standby_feedback` deserves more than a comment. It prevents the primary from vacuuming rows that a long-running replica query still needs — eliminating the dreaded `ERROR: canceling statement due to conflict with recovery`. But it also pins dead row versions on the primary, inflating table bloat under heavy OLAP workloads.

Let me show you a pattern I use in every project: if p99 replica queries are under two seconds, enable it. If you are running analytics joins that run for minutes, use a dedicated replica with a relaxed `max_standby_streaming_delay` instead.

---

## Step 3: Handle Conflicts When Bi-Directional Writes Enter the Picture

Logical replication is single-primary by default. If you allow writes on a subscriber via `pglogical` or Postgres 16's native bi-directional support, conflicts are inevitable.

| Conflict type | Default resolution | Production recommendation |
|---|---|---|
| INSERT on existing key | Error, halt replication | Use `SKIP` with application-layer reconciliation |
| UPDATE on missing row | Skip silently | Log and alert — data drift indicator |
| DELETE on missing row | Skip silently | Generally safe to ignore |

The docs do not mention this, but the safest posture for production bi-directional replication is last-write-wins based on an `updated_at` timestamp column, enforced via a conflict trigger on the subscriber side — not at the replication layer. The replication layer's job is delivery, not business logic.

---

## Step 4: Define Your Replication Lag Budget

Define your lag budget before your SLA is at risk. This query surfaces per-slot lag in bytes and estimated seconds:

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
sql&lt;br&gt;
SELECT&lt;br&gt;
  slot_name,&lt;br&gt;
  pg_size_pretty(pg_current_wal_lsn() - confirmed_flush_lsn) AS lag_size,&lt;br&gt;
  EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp()))::int AS lag_seconds&lt;br&gt;
FROM pg_replication_slots&lt;br&gt;
WHERE slot_type = 'logical';&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
For a p99 API budget of 50ms on a read-replica path, lag exceeding 500ms on OLTP workloads warrants immediate investigation. For async reporting reads, 30 seconds may be acceptable. Encode the number — do not leave it implicit.

---

## Gotchas

**Unlimited WAL retention by default.** `max_slot_wal_keep_size` defaults to unlimited in PostgreSQL 13+. Set it to 20–40% of your WAL partition before you need it, not after a 3am disk-full page.

**You cannot observe what you do not monitor.** Replication lag is not a metric for weekly reviews. Add `confirmed_flush_lsn` lag to your primary dashboard today and alert on it continuously.

**Lag is latency, exactly.** If your read path uses replica data, replication lag is not approximately your users' latency — it is exactly their latency. Treat it as a first-class SLO tied to your API p99 target.

---

## Conclusion

Logical replication slots give you fine-grained, per-consumer WAL delivery across regions. Mismanage them and you will blow your disk, stall your primary, and watch p99 latency climb. The slot lifecycle, the three `postgresql.conf` knobs, and the lag monitoring queries above are the foundation. Inactive slots with unbounded WAL retention have caused more surprise outages than misconfigured `wal_level` — now you have the tools to prevent that.

**Further reading:**
- [PostgreSQL: Logical Replication](https://www.postgresql.org/docs/current/logical-replication.html)
- [pg_replication_slots system view](https://www.postgresql.org/docs/current/view-pg-replication-slots.html)
- [max_slot_wal_keep_size](https://www.postgresql.org/docs/current/runtime-config-replication.html#GUC-MAX-SLOT-WAL-KEEP-SIZE)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



</description>
      <category>webdev</category>
      <category>programming</category>
    </item>
    <item>
      <title>Wiring gRPC-Web to a Kotlin Multiplatform Backend</title>
      <dc:creator>SoftwareDevs mvpfactory.io</dc:creator>
      <pubDate>Thu, 12 Nov 2026 08:41:24 +0000</pubDate>
      <link>https://dev.to/software_mvp-factory/wiring-grpc-web-to-a-kotlin-multiplatform-backend-2619</link>
      <guid>https://dev.to/software_mvp-factory/wiring-grpc-web-to-a-kotlin-multiplatform-backend-2619</guid>
      <description>&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="nn"&gt;---
&lt;/span&gt;&lt;span class="na"&gt;title&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"gRPC-Web + Kotlin Multiplatform: Protobuf at Scale"&lt;/span&gt;
&lt;span class="na"&gt;published&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="na"&gt;description&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Wire gRPC-Web to a KMP shared client layer using buf CLI, expect/actual transport boundaries, coroutine streaming lifecycle, and benchmarks showing protobuf beats REST above 10k req/s."&lt;/span&gt;
&lt;span class="na"&gt;tags&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;kotlin, mobile, architecture, performance&lt;/span&gt;
&lt;span class="na"&gt;canonical_url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;https://mvpfactory.co/blog/grpc-web-kotlin-multiplatform-protobuf-scale&lt;/span&gt;
&lt;span class="nn"&gt;---
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  What You Will Build
&lt;/h2&gt;

&lt;p&gt;By the end of this tutorial you will have a working Kotlin Multiplatform client wired to a gRPC-Web backend — with buf CLI generating your Protobuf descriptors, platform-correct transport behind an expect/actual boundary (OkHttp on Android, URLSession on iOS), and bidirectional streaming tied to your coroutine lifecycle. You will also walk away with the numbers that tell you exactly when this migration is worth the operational overhead.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prerequisites
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;KMP project targeting Android and iOS&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;buf&lt;/code&gt; CLI installed (&lt;code&gt;brew install bufbuild/buf/buf&lt;/code&gt; or equivalent)&lt;/li&gt;
&lt;li&gt;Familiarity with Kotlin coroutines and &lt;code&gt;Flow&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;A gRPC-Web-compatible backend (Envoy proxy or native gRPC-Web server)&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Step 1 — Generate Protobuf Descriptors with buf CLI
&lt;/h2&gt;

&lt;p&gt;Let me show you a pattern I use in every project: skip hand-rolling &lt;code&gt;.proto&lt;/code&gt; files and skip invoking &lt;code&gt;protoc&lt;/code&gt; directly. The buf CLI gives you a reproducible, lint-enforced pipeline.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="c1"&gt;# buf.gen.yaml
&lt;/span&gt;&lt;span class="na"&gt;version&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;v1&lt;/span&gt;
&lt;span class="na"&gt;plugins&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
  &lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="na"&gt;plugin&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;buf.build/grpc/kotlin&lt;/span&gt;
    &lt;span class="na"&gt;out&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;shared/src/commonMain/kotlin&lt;/span&gt;
    &lt;span class="na"&gt;opt&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;lite&lt;/span&gt;
  &lt;span class="p"&gt;-&lt;/span&gt; &lt;span class="na"&gt;plugin&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;buf.build/grpc/java&lt;/span&gt;
    &lt;span class="na"&gt;out&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;shared/src/androidMain/kotlin&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Run &lt;code&gt;buf generate&lt;/code&gt; from your repo root. The &lt;code&gt;lite&lt;/code&gt; option on the Kotlin plugin keeps the generated runtime lean — on mobile the full Protobuf runtime adds overhead you do not need.&lt;/p&gt;

&lt;p&gt;Add &lt;code&gt;buf lint&lt;/code&gt; to your CI gate immediately. Schema drift between client and server is cheap to prevent and genuinely painful to debug at scale.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 2 — The expect/actual Transport Boundary
&lt;/h2&gt;

&lt;p&gt;This is where KMP's platform split matters most. gRPC-Web requires an HTTP/1.1-compatible transport — browsers and mobile HTTP stacks do not support HTTP/2 framing natively — so channel construction must live behind an expect/actual boundary.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="c1"&gt;// commonMain&lt;/span&gt;
&lt;span class="n"&gt;expect&lt;/span&gt; &lt;span class="k"&gt;fun&lt;/span&gt; &lt;span class="nf"&gt;buildGrpcChannel&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;host&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;String&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;Int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;ManagedChannel&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="c1"&gt;// androidMain&lt;/span&gt;
&lt;span class="n"&gt;actual&lt;/span&gt; &lt;span class="k"&gt;fun&lt;/span&gt; &lt;span class="nf"&gt;buildGrpcChannel&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;host&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;String&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;Int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;ManagedChannel&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;
    &lt;span class="nc"&gt;OkHttpChannelBuilder&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;forAddress&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;host&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;usePlaintext&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="c1"&gt;// TLS terminated at proxy in prod&lt;/span&gt;
        &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;build&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="c1"&gt;// iosMain&lt;/span&gt;
&lt;span class="n"&gt;actual&lt;/span&gt; &lt;span class="k"&gt;fun&lt;/span&gt; &lt;span class="nf"&gt;buildGrpcChannel&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;host&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;String&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;Int&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;ManagedChannel&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;
    &lt;span class="nc"&gt;GRPCChannelPool&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sharedPool&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;channel&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;withHost&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;host&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;port&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;port&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="c1"&gt;// URLSession-backed transport&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Keep these &lt;code&gt;actual&lt;/code&gt; implementations thin — channel construction only, no business logic.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 3 — Bidirectional Streaming Lifecycle in Coroutines
&lt;/h2&gt;

&lt;p&gt;The gRPC-Kotlin stub exposes &lt;code&gt;Flow&lt;/code&gt;-based APIs natively, mapping cleanly onto KMP's coroutine layer.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="c1"&gt;// commonMain shared business logic&lt;/span&gt;
&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;ProductStreamRepository&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;private&lt;/span&gt; &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;stub&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;CatalogServiceCoroutineStub&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;

    &lt;span class="k"&gt;fun&lt;/span&gt; &lt;span class="nf"&gt;observeInventory&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;skus&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Flow&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nc"&gt;SkuRequest&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Flow&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="nc"&gt;InventoryUpdate&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt;
        &lt;span class="n"&gt;stub&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;streamInventory&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;skus&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
            &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="k"&gt;catch&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nf"&gt;emit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;InventoryUpdate&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;error&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
            &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;flowOn&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;Dispatchers&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;IO&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Tie stream cancellation to your UI scope:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="n"&gt;viewModelScope&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;launch&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;repository&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;observeInventory&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;requestFlow&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;collect&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;update&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;_state&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;value&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;update&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="c1"&gt;// Cancelled automatically on ViewModel clear&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;viewModelScope&lt;/code&gt; handles cleanup automatically when the ViewModel is cleared. Do not manage stream lifecycle manually.&lt;/p&gt;




&lt;h2&gt;
  
  
  Gotchas
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Leaked gRPC streams are silent killers.&lt;/strong&gt; I have seen mobile apps leak open streams across navigation events and quietly exhaust server connection limits under load. Always bind stream lifecycle to a structured scope — &lt;code&gt;viewModelScope&lt;/code&gt; on Android, the equivalent lifecycle-aware scope on iOS.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Business logic in platform actuals defeats KMP.&lt;/strong&gt; The moment you put logic into &lt;code&gt;androidMain&lt;/code&gt; or &lt;code&gt;iosMain&lt;/code&gt; transport implementations, you create platform divergence that compounds over time. Keep actuals to channel construction only.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The buf CLI crossover point matters.&lt;/strong&gt; At low request rates the operational overhead of Protobuf — schema management, generated code, buf pipeline — outweighs the gains. The crossover in production systems sits around 5–10k req/s per service.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Numbers
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Metric&lt;/th&gt;
&lt;th&gt;JSON/REST&lt;/th&gt;
&lt;th&gt;Protobuf/gRPC-Web&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Avg payload size&lt;/td&gt;
&lt;td&gt;1.0x&lt;/td&gt;
&lt;td&gt;0.35x&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Parse time per message (mobile)&lt;/td&gt;
&lt;td&gt;1.0x&lt;/td&gt;
&lt;td&gt;~0.45x&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Schema enforcement&lt;/td&gt;
&lt;td&gt;Runtime&lt;/td&gt;
&lt;td&gt;Compile-time&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Streaming support&lt;/td&gt;
&lt;td&gt;Polling / SSE&lt;/td&gt;
&lt;td&gt;Native bidirectional&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Above 10k req/s, the wire size reduction hits egress costs directly. The parse CPU reduction means fewer compute instances. Compile-time schema enforcement shrinks the runtime error surface at high call volumes.&lt;/p&gt;




&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;Three things to take away: gate on &lt;code&gt;buf lint&lt;/code&gt; in CI before any &lt;code&gt;.proto&lt;/code&gt; merge, keep expect/actual implementations to channel construction only, and tie stream lifecycle to your UI scope — not to a manual flag. The docs do not emphasize that last one loudly enough, but it is the gotcha that will save you hours in production.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Further reading:&lt;/strong&gt; &lt;a href="https://buf.build/docs/" rel="noopener noreferrer"&gt;buf CLI docs&lt;/a&gt;, &lt;a href="https://grpc.io/docs/languages/kotlin/" rel="noopener noreferrer"&gt;grpc-kotlin coroutine stubs&lt;/a&gt;, &lt;a href="https://kotlinlang.org/docs/multiplatform-connect-to-apis.html" rel="noopener noreferrer"&gt;KMP expect/actual reference&lt;/a&gt;&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>programming</category>
    </item>
    <item>
      <title>Wiring Android's TensorFlow Lite Delegate API to a Quantized On-Device Vision Transformer</title>
      <dc:creator>SoftwareDevs mvpfactory.io</dc:creator>
      <pubDate>Wed, 11 Nov 2026 14:58:57 +0000</pubDate>
      <link>https://dev.to/software_mvp-factory/wiring-androids-tensorflow-lite-delegate-api-to-a-quantized-on-device-vision-transformer-1ljg</link>
      <guid>https://dev.to/software_mvp-factory/wiring-androids-tensorflow-lite-delegate-api-to-a-quantized-on-device-vision-transformer-1ljg</guid>
      <description>&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;---
title: "On-Device Vision Transformers on Android: Wiring TFLite Delegates the Right Way"
published: true
description: "How to wire TFLite delegate hierarchy, calibrate INT8 quantization, pre-allocate tensor arena, and tune XNNPACK threading for Vision Transformers on Android."
tags: android, kotlin, mobile, architecture
canonical_url: https://blog.mvp-factory.dev/on-device-vision-transformers-tflite-android
---

&lt;span class="gu"&gt;## What We Are Building
&lt;/span&gt;
By the end of this tutorial you will know how to deploy a quantized Vision Transformer on mid-range Android hardware without crashing on NNAPI, bleeding latency on GPU transitions, or throttling a Snapdragon 7-series into uselessness within 30 seconds. We will cover the four production mistakes I see in almost every TFLite deployment — and the fixes that take 10 minutes each.

&lt;span class="gu"&gt;## Prerequisites
&lt;/span&gt;
&lt;span class="k"&gt;- &lt;/span&gt;Android project targeting API 26+
&lt;span class="k"&gt;- &lt;/span&gt;TensorFlow Lite AAR (&lt;span class="sb"&gt;`org.tensorflow:tensorflow-lite`&lt;/span&gt;) added to your Gradle dependencies
&lt;span class="k"&gt;- &lt;/span&gt;A SavedModel you want to convert to INT8
&lt;span class="k"&gt;- &lt;/span&gt;Basic familiarity with Kotlin coroutines and Android threading

---

&lt;span class="gu"&gt;## Step 1 — Understand the Delegate Selection Hierarchy
&lt;/span&gt;
Here is the gotcha that will save you hours: most teams treat TFLite delegate selection as a binary NNAPI-on/NNAPI-off switch. In production, the runtime attempts delegation in a priority chain, and each fallback has a measurable cost.

| Delegate | Best For | Failure Mode | Fallback |
|---|---|---|---|
| NNAPI | DSP/NPU offload, Qualcomm Hexagon | Op coverage gaps, driver bugs | GPU |
| GPU Delegate | Parallelizable FP16 ops | Thermal, memory bandwidth | XNNPACK |
| XNNPACK | ARM NEON optimized CPU ops | None — always available | Interpreter |

Vision Transformer attention layers frequently contain ops that NNAPI delegates *partially* — triggering a split-graph execution that costs more than a full CPU path. Always validate op coverage before committing to NNAPI.

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
kotlin&lt;br&gt;
val options = Interpreter.Options().apply {&lt;br&gt;
    val nnapi = NnApiDelegate.Options().apply {&lt;br&gt;
        executionPreference = NnApiDelegate.Options.EXECUTION_PREFERENCE_SUSTAINED_SPEED&lt;br&gt;
        allowFp16 = true&lt;br&gt;
    }&lt;br&gt;
    addDelegate(NnApiDelegate(nnapi))&lt;br&gt;
    // GPU fallback if NNAPI coverage is incomplete&lt;br&gt;
    addDelegate(GpuDelegate(GpuDelegate.Options().apply {&lt;br&gt;
        inferencePreference = GpuDelegate.Options.INFERENCE_PREFERENCE_SUSTAINED_SPEED&lt;br&gt;
    }))&lt;br&gt;
}&lt;br&gt;
val interpreter = Interpreter(modelBuffer, options)&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
After initialization, log the number of resolved input tensors and cross-reference with your expected graph shape. For GPU delegates, wrap initialization in a `try-catch` — driver-level failures on some Android 12 OEM builds surface as silent no-ops, not exceptions. **Silence is not success.**

---

## Step 2 — Calibrate INT8 Quantization With a Representative Dataset

Let me show you a pattern I use in every project: post-training INT8 quantization without a representative calibration dataset ships accuracy regressions disguised as a latency optimization.

The docs do not mention this clearly enough, but the calibration set must cover the realistic distribution of inference inputs your production model will actually see — not ImageNet defaults.

Teams that use 100–500 representative samples from their actual domain see accuracy degradation under 1% versus FP32. Teams that skip calibration entirely and use synthetic or generic data regularly see 5–8% drops on tail distributions.

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
python&lt;br&gt;
def representative_dataset():&lt;br&gt;
    for image in calibration_samples:  # domain-specific images&lt;br&gt;
        yield [image.astype(np.float32)]&lt;/p&gt;

&lt;p&gt;converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)&lt;br&gt;
converter.optimizations = [tf.lite.Optimize.DEFAULT]&lt;br&gt;
converter.representative_dataset = representative_dataset&lt;br&gt;
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]&lt;br&gt;
converter.inference_input_type = tf.int8&lt;br&gt;
converter.inference_output_type = tf.int8&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
---

## Step 3 — Pre-Allocate the Tensor Arena

The tensor arena is TFLite's scratch memory for intermediate activations. If you let the runtime allocate it lazily, you trigger garbage collection during inference — invisible in benchmarks, catastrophic in production when the GC pause lands mid-frame.

Here is the minimal setup to get this working: call `allocateTensors()` immediately after interpreter construction, not on the first inference call.

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
kotlin&lt;br&gt;
val interpreter = Interpreter(modelBuffer, options)&lt;br&gt;
interpreter.allocateTensors() // do this NOW, not at first inference&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
For Vision Transformers with sequence lengths typical of mobile-sized patches, arena sizes in the 20–60 MB range are common. Profile with `Interpreter.getInputTensor(0).numBytes()` and walk the graph to size it correctly.

---

## Step 4 — Tune XNNPACK Threading Against Your Thermal Budget

On Snapdragon 7-series silicon, XNNPACK scales well up to 4 threads for throughput — beyond that, you gain diminishing returns while accelerating thermal throttling. For sustained inference workloads like camera-feed analysis or real-time classification, 2–3 threads is the production-validated sweet spot.

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
kotlin&lt;br&gt;
val xnnpackOptions = XNNPackDelegate.Options().apply {&lt;br&gt;
    numThreads = 3 // thermal-safe for sustained inference on 7-series&lt;br&gt;
}&lt;br&gt;
options.addDelegate(XNNPackDelegate(xnnpackOptions))&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
Default thread counts optimize for peak benchmark numbers, not sustained real-world performance. Here is what that looks like across a sample of mid-range devices:

| Device | SoC | Android | Driver | Safe thread count |
|---|---|---|---|---|
| Pixel 7a | Tensor G2 | 14 | Adreno 660 | 3 |
| Galaxy A54 | Exynos 1380 | 14 | Mali-G68 | 2 |
| Moto G84 | Snapdragon 695 | 13 | Adreno 619 | 3 |
| Redmi Note 12 | Snapdragon 685 | 13 | Adreno 610 | 2 |

Treat these as starting points. Instrument thermal state via `PowerManager.getThermalHeadroom()` to detect throttle onset dynamically — the difference between a smooth experience and a device that starts dropping frames after 30 seconds is usually just a thread or two. Speaking of sustained work sessions without throttling your own hardware: [HealthyDesk](https://play.google.com/store/apps/details?id=com.healthydesk) reminds you to step away from the desk during those long profiling sessions — worth having running in the background while you wait for benchmark results.

---

## Gotchas

**NNAPI silent partial delegation.** When NNAPI cannot cover your full graph, it splits execution rather than failing loudly. You will not see an exception. You will see slower inference than a full CPU path and wonder why. Log resolved tensor counts.

**GPU delegate silent no-ops on Android 12 OEMs.** Some OEM builds fail to initialize the GPU delegate without throwing. Always validate after construction, not just during.

**Lazy arena allocation in benchmarks.** Your benchmark numbers will look fine because GC pauses do not show up in isolated timing. Your production metrics will not. Call `allocateTensors()` at load time, full stop.

**Calibration dataset domain shift.** Using ImageNet samples to calibrate a model that will see medical imaging or industrial inspection inputs is not calibration — it is wishful thinking. Your 100-sample set needs to come from your actual domain.

**Default XNNPACK thread count is not for sustained loads.** The default is tuned for single-inference benchmarks. For any real-time or continuous-inference use case, start at 2–3 threads and instrument thermal headroom before shipping.

---

## Conclusion

The same four mistakes keep appearing in production TFLite deployments: shipping NNAPI without validating op coverage, running INT8 conversion without a domain-specific calibration set, allocating tensor arena lazily, and leaving XNNPACK thread count at whatever the default is.

Each one individually is a 10-minute fix. Together they are the difference between an on-device model that works and one you end up blaming on the hardware. Get these right and mid-range Android hardware is more capable than most teams expect.

**Further reading:**
- [TFLite Delegates overview](https://www.tensorflow.org/lite/performance/delegates)
- [TFLite post-training quantization](https://www.tensorflow.org/lite/performance/post_training_quantization)
- [PowerManager thermal API](https://developer.android.com/reference/android/os/PowerManager#getThermalHeadroom(int))
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



</description>
      <category>webdev</category>
      <category>programming</category>
    </item>
    <item>
      <title>Wiring Android's CameraX ML Kit Pipeline to a On-Device Vision-Language Model</title>
      <dc:creator>SoftwareDevs mvpfactory.io</dc:creator>
      <pubDate>Wed, 11 Nov 2026 09:30:02 +0000</pubDate>
      <link>https://dev.to/software_mvp-factory/wiring-androids-camerax-ml-kit-pipeline-to-a-on-device-vision-language-model-4ghf</link>
      <guid>https://dev.to/software_mvp-factory/wiring-androids-camerax-ml-kit-pipeline-to-a-on-device-vision-language-model-4ghf</guid>
      <description>&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight markdown"&gt;&lt;code&gt;---
title: "CameraX + On-Device VLM: Frame Sampling and the Inference Budget That Fits on a Pixel 9"
published: true
description: "Wire CameraX ImageAnalysis to a quantized vision-language model on Android. Frame drop strategy, YUV tensor conversion, NNAPI vs GPU delegate, and memory budgeting on Snapdragon 8 Gen 3."
tags: android, kotlin, mobile, architecture
canonical_url: https://mvpfactory.co/blog/camerax-on-device-vlm-frame-sampling-pixel9
---

&lt;span class="gu"&gt;## What We Are Building
&lt;/span&gt;
A live-camera inference pipeline that wires CameraX &lt;span class="sb"&gt;`ImageAnalysis`&lt;/span&gt; to a quantized vision-language model (MobileVLM-class, int4) on a Pixel 9. By the end you will have a coherent pipeline contract — frame drop policy, zero-copy YUV-to-RGB preprocessing, delegate routing, and a memory budget that holds under sustained load.

This is not a getting-started guide. It is the guide for after you benchmarked the model in isolation, declared victory at 300ms per frame, wired it to a live feed, and watched the system collapse within 90 seconds.

---

&lt;span class="gu"&gt;## Prerequisites
&lt;/span&gt;
&lt;span class="k"&gt;- &lt;/span&gt;Android project targeting API 26+
&lt;span class="k"&gt;- &lt;/span&gt;CameraX &lt;span class="sb"&gt;`1.3.x`&lt;/span&gt; added to your Gradle dependencies
&lt;span class="k"&gt;- &lt;/span&gt;A quantized VLM loaded via TFLite (MobileVLM or equivalent int4/int8 model)
&lt;span class="k"&gt;- &lt;/span&gt;Basic familiarity with &lt;span class="sb"&gt;`ImageAnalysis`&lt;/span&gt; use case setup

---

&lt;span class="gu"&gt;## Step 1 — Understand the Real Problem
&lt;/span&gt;
The camera does not wait for your model. &lt;span class="sb"&gt;`ImageAnalysis`&lt;/span&gt; delivers frames at the cadence the sensor dictates — 30fps by default. At 300ms per inference, you have a 9× overproduction ratio before you account for KV-cache memory pressure. The fix is not faster inference. It is a coherent pipeline contract.

Let me show you a pattern I use in every project: treat the inference budget as a hard constraint *before* you write a single line of pipeline code.

---

&lt;span class="gu"&gt;## Step 2 — Frame Drop Policy Under Backpressure
&lt;/span&gt;
&lt;span class="sb"&gt;`ImageAnalysis`&lt;/span&gt; gives you two backpressure strategies. Never use &lt;span class="sb"&gt;`STRATEGY_BLOCK_PRODUCER`&lt;/span&gt; with a slow model — you will starve the camera HAL. Always use &lt;span class="sb"&gt;`STRATEGY_KEEP_ONLY_LATEST`&lt;/span&gt;, then gate your sample rate explicitly with a token-bucket in the analyzer.

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
kotlin&lt;br&gt;
val imageAnalysis = ImageAnalysis.Builder()&lt;br&gt;
    .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)&lt;br&gt;
    .setTargetResolution(Size(640, 480))&lt;br&gt;
    .build()&lt;br&gt;
    .also { it.setAnalyzer(inferenceExecutor, ::analyzeFrame) }&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
Target 2–4fps for most VLM tasks. Scene understanding does not require 30fps temporal resolution.

---

## Step 3 — Zero-Copy YUV-to-RGB Conversion

Here is the gotcha that will save you hours. CameraX delivers `ImageProxy` frames in `YUV_420_888`. Your VLM expects a normalized RGB tensor. The naive CPU path — convert, write to `ByteBuffer`, pass to TFLite — involves two full-frame copies and a format-conversion pass that burns ~8ms on a Snapdragon 8 Gen 3 for a 640×480 input.

The correct path uses a RenderScript or Vulkan compute shader to perform the conversion directly in GPU memory, then hands the texture pointer to the GPU delegate without a CPU round-trip:

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
kotlin&lt;br&gt;
// Acquire the HardwareBuffer from ImageProxy&lt;br&gt;
val hardwareBuffer = imageProxy.image?.let {&lt;br&gt;
    ImageReader.acquireLatestImage()?.hardwareBitmap?.asHardwareBuffer()&lt;br&gt;
}&lt;/p&gt;

&lt;p&gt;// Bind to GPU delegate input tensor via TFLite GPU buffer API&lt;br&gt;
val gpuBufferOptions = GpuDelegateFactory.Options().apply {&lt;br&gt;
    setInferencePreference(INFERENCE_PREFERENCE_SUSTAINED_SPEED)&lt;br&gt;
}&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
This alone reduces preprocessing latency by roughly 60% on sustained throughput workloads. Keep tensors on GPU from acquisition through inference — the CPU round-trip is the hidden cost.

---

## Step 4 — NNAPI vs GPU Delegate Routing

The docs do not make this decision obvious, but here is the breakdown for Snapdragon 8 Gen 3 at 640×480 int8:

| Delegate | Latency | Power | KV-cache | Cold Start |
|---|---|---|---|---|
| CPU (4 big cores) | ~420ms | High | Full | ~0ms |
| NNAPI (DSP path) | ~180ms | Low | Partial | ~220ms |
| GPU delegate | ~140ms | Medium | Full | ~350ms |
| NNAPI (GPU path) | ~155ms | Medium | Partial | ~300ms |

For sustained live-camera inference, the GPU delegate wins on latency with acceptable thermal behavior. NNAPI DSP is better for battery-sensitive background tasks, but its partial KV-cache support means you will manage context windowing manually for multi-turn queries.

---

## Step 5 — Budget Memory Before You Benchmark Latency

Here is the minimal setup constraint to get this working on a Pixel 9. A quantized 3B-parameter VLM (int4, MobileVLM-class) requires approximately 1.6GB of model weights plus 300–500MB of KV-cache at a 512-token context window. Add ~200MB peak for the GPU delegate's internal working memory. Budget 2.2GB total for the pipeline.

CameraX camera buffers at 640×480 `YUV_420_888` are ~460KB each — negligible. Upsample to 1080p with 4 buffers in-flight and you are at ~12MB camera-side, which still fits. The Pixel 9's 12GB helps, but on 8GB devices this hard-caps you at int4 quantization with a reduced context window.

Profile under load, not idle:

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;&lt;br&gt;
shell&lt;br&gt;
adb shell dumpsys meminfo &lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
Run that under sustained 30-second camera inference before committing to a model size.

---

## Gotchas

**`STRATEGY_BLOCK_PRODUCER` will silently stall your pipeline.** There is no exception — the camera HAL just falls behind and frames queue. Use `KEEP_ONLY_LATEST` unconditionally with a slow model.

**Benchmarking in isolation is a lie.** A 300ms single-shot inference becomes a 600ms+ sustained-load inference once KV-cache pressure and thermal throttling kick in after 30–60 seconds of live use.

**Cold start latency on the GPU delegate is 350ms.** Initialize it once at startup, not per-frame. Lazy initialization on the first analyzed frame will drop that frame and spike your first-inference latency.

**YUV conversion is not free.** Even at 640×480, skipping the hardware buffer path costs you 8ms per frame and blocks a CPU core. At 4fps sampling that is tolerable. At anything higher, you will see it in your profiler traces.

---

## Conclusion

The pipeline contract is: `KEEP_ONLY_LATEST` + token-bucket sampler at 2–4fps + hardware buffer YUV path + GPU delegate initialized at startup + 2.2GB memory budget verified under load. Miss any one of these and you will hit a wall that looks like a model performance problem but is actually a systems integration problem.

**Further reading:**
- [CameraX ImageAnalysis reference](https://developer.android.com/training/camerax/analyze)
- [TFLite GPU delegate guide](https://www.tensorflow.org/lite/performance/gpu)
- [MobileVLM model card](https://huggingface.co/mtgv/MobileVLM-3B)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



</description>
      <category>webdev</category>
      <category>programming</category>
    </item>
    <item>
      <title>Wiring CoreML to a Quantized On-Device LLM for Streaming Text Generation</title>
      <dc:creator>SoftwareDevs mvpfactory.io</dc:creator>
      <pubDate>Tue, 10 Nov 2026 16:02:16 +0000</pubDate>
      <link>https://dev.to/software_mvp-factory/wiring-coreml-to-a-quantized-on-device-llm-for-streaming-text-generation-22g2</link>
      <guid>https://dev.to/software_mvp-factory/wiring-coreml-to-a-quantized-on-device-llm-for-streaming-text-generation-22g2</guid>
      <description>&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="nn"&gt;---
&lt;/span&gt;&lt;span class="na"&gt;title&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Wiring CoreML to a Quantized On-Device LLM: KV-Cache, ANE Scheduling, and the Token Budget for iPhone 16"&lt;/span&gt;
&lt;span class="na"&gt;published&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="na"&gt;description&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"CoreML's ML Program format for on-device transformer inference — ANE scheduling, 4-bit quantization tradeoffs, and the KV-cache formula that keeps you under iOS memory limits."&lt;/span&gt;
&lt;span class="na"&gt;tags&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ios, mobile, architecture, api&lt;/span&gt;
&lt;span class="na"&gt;canonical_url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;https://mvpfactory.co/blog/coreml-on-device-llm-kv-cache-ane-iphone16&lt;/span&gt;
&lt;span class="nn"&gt;---
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  What We Are Building
&lt;/h2&gt;

&lt;p&gt;By the end of this tutorial you will understand — and be able to implement — a streaming token generation pipeline on iPhone using CoreML. Specifically: how to schedule transformer layers across the ANE and GPU correctly, which quantization format keeps your model weights under the 2GB threshold, and how to calculate the KV-cache ceiling before jetsam kills your process in production.&lt;/p&gt;

&lt;p&gt;Get any one of these three wrong and you are debugging OOM crashes on a user's device. Let me show you the pattern I use in every on-device inference project.&lt;/p&gt;




&lt;h2&gt;
  
  
  Prerequisites
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Xcode 15+ with a physical device running iOS 17 (simulator will not reflect ANE behaviour)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;coremltools&lt;/code&gt; 7.x installed in your Python environment&lt;/li&gt;
&lt;li&gt;A base transformer model checkpoint (3B parameters or smaller for iPhone 16)&lt;/li&gt;
&lt;li&gt;Familiarity with CoreML's &lt;code&gt;.mlpackage&lt;/code&gt; format&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Step 1 — Understand the Hardware Scheduling Problem
&lt;/h2&gt;

&lt;p&gt;Apple's unified memory architecture means the CPU, GPU, and Neural Engine share the same physical memory pool. No PCIe bottleneck — but everything competes for the same budget, and iOS reclaims it without mercy.&lt;/p&gt;

&lt;p&gt;CoreML's &lt;strong&gt;ML Program format&lt;/strong&gt; (&lt;code&gt;.mlpackage&lt;/code&gt; with a &lt;code&gt;model.mlmodel&lt;/code&gt; spec) is your abstraction layer between Swift and that hardware. The compiler partitions your compute graph across available units. Here is where most teams go wrong: they assume the ANE and GPU handle attention and feed-forward layers identically. They do not.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Layer Type&lt;/th&gt;
&lt;th&gt;Preferred Backend&lt;/th&gt;
&lt;th&gt;Reason&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Feed-forward (FFN)&lt;/td&gt;
&lt;td&gt;ANE&lt;/td&gt;
&lt;td&gt;Fixed shapes, 4-bit weight load, high throughput&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Multi-head attention (MHA)&lt;/td&gt;
&lt;td&gt;GPU&lt;/td&gt;
&lt;td&gt;Dynamic sequence length, flexible memory access&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Embedding lookup&lt;/td&gt;
&lt;td&gt;CPU&lt;/td&gt;
&lt;td&gt;Small, irregular access pattern&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Layer norm / softmax&lt;/td&gt;
&lt;td&gt;ANE or GPU&lt;/td&gt;
&lt;td&gt;Compiler decides based on input rank&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The ANE is optimized for fixed-shape, low-precision matrix multiply — that maps cleanly to FFN layers. Attention has dynamic sequence length and flexible memory access patterns, which the GPU handles better.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 2 — Export With the Right Compute Unit Setting
&lt;/h2&gt;

&lt;p&gt;Here is the minimal setup to get this working. When exporting with &lt;code&gt;coremltools&lt;/code&gt;, use &lt;code&gt;ComputeUnit.ALL&lt;/code&gt; and let the compiler make scheduling decisions:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;coremltools&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;ct&lt;/span&gt;

&lt;span class="n"&gt;model_spec&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;ct&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;convert&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;traced_model&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;compute_units&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;ct&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;ComputeUnit&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;ALL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;  &lt;span class="c1"&gt;# Let the compiler decide&lt;/span&gt;
    &lt;span class="n"&gt;minimum_deployment_target&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;ct&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;target&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;iOS17&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;model_spec&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;save&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"MyLLM.mlpackage"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Do not force &lt;code&gt;ANE_ONLY&lt;/code&gt; on attention blocks. The docs do not make this obvious, but doing so triggers a silent fallback to CPU and destroys your latency budget. Profile first with Instruments' Core ML template before you override anything.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 3 — Choose Your Quantization Strategy
&lt;/h2&gt;

&lt;p&gt;To fit under the 2GB weight threshold, you have two practical options.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4-bit palettization&lt;/strong&gt; maps each weight to an index in a learned codebook — roughly 4x compression over float16. The ANE loads palettized weights natively on A17 Pro and M-series chips:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;coremltools&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;optimize&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;coreml&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;cto&lt;/span&gt;

&lt;span class="n"&gt;op_config&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;cto&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;OpPalettizerConfig&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;mode&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"kmeans"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;nbits&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;config&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;cto&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;OptimizationConfig&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;global_config&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;op_config&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;palettized_model&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;cto&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;palettize_weights&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;model_spec&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;config&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Block-wise float16&lt;/strong&gt; (per-channel or per-group) retains float16 precision per block of N weights. Less aggressive compression, but better for layers sensitive to weight distribution — specifically the first and last projection layers.&lt;/p&gt;

&lt;p&gt;The combination that works: 4-bit palettization across the bulk of the model, float16 for the first and last projection layers. A 3B parameter model lands around 1.7GB — under threshold, with headroom left for activations and the KV-cache.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 4 — Implement the Streaming Token Loop
&lt;/h2&gt;

&lt;p&gt;Here is the working Swift pattern for token-by-token generation with KV-cache threading:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight swift"&gt;&lt;code&gt;&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="n"&gt;config&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;MLModelConfiguration&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;config&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;computeUnits&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;all&lt;/span&gt;

&lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="nc"&gt;MLModel&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;contentsOf&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;modelURL&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;configuration&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;config&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="kd"&gt;var&lt;/span&gt; &lt;span class="n"&gt;inputTokens&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;Int32&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;tokenize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="o"&gt;!&lt;/span&gt;&lt;span class="n"&gt;stopConditionMet&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="n"&gt;inputFeatures&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="nc"&gt;MLDictionaryFeatureProvider&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;dictionary&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;
        &lt;span class="s2"&gt;"input_ids"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;MLMultiArray&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;inputTokens&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"attention_mask"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;currentMask&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="s2"&gt;"past_key_values"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;kvCache&lt;/span&gt;
    &lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="n"&gt;output&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;try&lt;/span&gt; &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;prediction&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;from&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;inputFeatures&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="kd"&gt;let&lt;/span&gt; &lt;span class="n"&gt;nextToken&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;argmax&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;output&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;featureValue&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;for&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"logits"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;kvCache&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;output&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;featureValue&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;for&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"present_key_values"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;inputTokens&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;nextToken&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="nf"&gt;emit&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;nextToken&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;h2&gt;
  
  
  Step 5 — Calculate Your KV-Cache Ceiling Before Shipping
&lt;/h2&gt;

&lt;p&gt;Here is the gotcha that will save you hours. KV-cache memory scales linearly with sequence length:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;kv_bytes = 2 × n_layers × n_heads × head_dim × seq_len × dtype_bytes
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For a 32-layer, 32-head model with &lt;code&gt;head_dim=128&lt;/code&gt; in float16 at 2048 tokens:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;2 × 32 × 32 × 128 × 2048 × 2 = ~1.07 GB
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Add that to 1.7GB of model weights and you are at ~2.8GB total — well past the jetsam threshold on most iPhone configurations. On iPhone 16 with 6GB LPDDR5, the practical usable context ceiling is roughly 1024–1500 tokens before the OS starts issuing memory pressure notifications.&lt;/p&gt;

&lt;p&gt;The fix is static KV-cache allocation: pre-allocate the cache tensor at max sequence length and use a position index to avoid dynamic reallocation, which trips memory fragmentation warnings. Set a hard &lt;code&gt;max_sequence_length&lt;/code&gt; in your ML Program spec.&lt;/p&gt;




&lt;h2&gt;
  
  
  Gotchas
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Forcing ANE_ONLY on attention blocks.&lt;/strong&gt; This is the most common mistake I see. It looks like it should be faster. It is not — the compiler falls back silently to CPU. Always start with &lt;code&gt;ComputeUnit.ALL&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Skipping the KV-cache math.&lt;/strong&gt; Crashing with jetsam in the background is a one-star review waiting to happen. Run the formula with your model's actual architecture constants before you set a context window length in your UI.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Uniform quantization across all layers.&lt;/strong&gt; The first and last projection layers are sensitive to weight distribution. Palettizing them at 4-bit degrades output quality noticeably. Keep those in float16.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Testing only on simulator.&lt;/strong&gt; The ANE does not run in the iOS simulator. All your latency and memory measurements need to come from a physical A17 Pro or later device.&lt;/p&gt;




&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;Three constraints, and you have to get all three right: ANE scheduling, quantization strategy, and KV-cache sizing. Use &lt;code&gt;ComputeUnit.ALL&lt;/code&gt; and profile before overriding. Default to 4-bit palettization with float16 boundaries. Calculate your cache ceiling with the formula above and enforce it with a hard limit in your ML Program spec.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Further reading:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://coremltools.readme.io" rel="noopener noreferrer"&gt;Apple CoreML Tools documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://developer.apple.com/documentation/coreml" rel="noopener noreferrer"&gt;Apple Instruments Core ML template guide&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://apple.github.io/coremltools/docs-guides/source/mlprogram.html" rel="noopener noreferrer"&gt;ML Program format specification&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>webdev</category>
      <category>programming</category>
    </item>
    <item>
      <title>Wiring Android's NNAPI to a Quantized On-Device ASR Model</title>
      <dc:creator>SoftwareDevs mvpfactory.io</dc:creator>
      <pubDate>Tue, 10 Nov 2026 09:11:39 +0000</pubDate>
      <link>https://dev.to/software_mvp-factory/wiring-androids-nnapi-to-a-quantized-on-device-asr-model-ahh</link>
      <guid>https://dev.to/software_mvp-factory/wiring-androids-nnapi-to-a-quantized-on-device-asr-model-ahh</guid>
      <description>&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="nn"&gt;---
&lt;/span&gt;&lt;span class="na"&gt;title&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Android NNAPI + Quantized ASR: Hit 150ms On-Device"&lt;/span&gt;
&lt;span class="na"&gt;published&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="na"&gt;description&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"Wire Android's NNAPI to a quantized Whisper-tiny model for real-time speech recognition — covering buffer sizing, INT8 vs INT4 tradeoffs on Adreno/Mali, and surviving 3GB RAM constraints."&lt;/span&gt;
&lt;span class="na"&gt;tags&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;android&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;kotlin&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;mobile&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;architecture&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="na"&gt;canonical_url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;https://blog.mvpfactory.co/android-nnapi-quantized-asr-150ms&lt;/span&gt;
&lt;span class="nn"&gt;---
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  What We Are Building
&lt;/h2&gt;

&lt;p&gt;By the end of this tutorial you will have a working on-device ASR pipeline on mid-range Android hardware that lands under 150ms end-to-end. We are wiring Android's Neural Networks API to a quantized Whisper-tiny model (39M parameters), and we are doing it in a way that survives Adreno/Mali GPU differences and 3GB RAM constraints without blowing up in production.&lt;/p&gt;

&lt;p&gt;Network-round-trip ASR adds 300–800ms depending on signal quality. A properly tuned on-device pipeline on a Snapdragon 6-series chip cuts that to under 150ms. That is the difference between a feature that feels native and one that feels broken.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prerequisites
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Android project targeting API 29+ (NNAPI stable delegate support)&lt;/li&gt;
&lt;li&gt;TensorFlow Lite with the NNAPI delegate dependency&lt;/li&gt;
&lt;li&gt;A quantized Whisper-tiny &lt;code&gt;.tflite&lt;/code&gt; model (INT8 or INT4)&lt;/li&gt;
&lt;li&gt;Basic familiarity with &lt;code&gt;AudioRecord&lt;/code&gt; and &lt;code&gt;ByteBuffer&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Step 1: Size Your AudioRecord Buffer Correctly
&lt;/h2&gt;

&lt;p&gt;Here is the gotcha that will save you hours: most teams let Android choose the minimum buffer size and wonder why they get choppy, context-broken transcriptions.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;sampleRate&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;16000&lt;/span&gt;
&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;frameSize&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;1280&lt;/span&gt; &lt;span class="c1"&gt;// 80ms window at 16kHz&lt;/span&gt;
&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;bufferSize&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;maxOf&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="nc"&gt;AudioRecord&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getMinBufferSize&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;sampleRate&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="no"&gt;CHANNEL_IN_MONO&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="no"&gt;ENCODING_PCM_16BIT&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;frameSize&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt; &lt;span class="c1"&gt;// 4 buffers of safety&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;recorder&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;AudioRecord&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="nc"&gt;MediaRecorder&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;AudioSource&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;VOICE_RECOGNITION&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;sampleRate&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="no"&gt;CHANNEL_IN_MONO&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="no"&gt;ENCODING_PCM_16BIT&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;bufferSize&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The 1280-frame chunk (80ms at 16kHz) maps cleanly to Whisper's mel-filterbank input stride. Going smaller increases callback overhead and risks underruns on loaded CPUs. Going larger introduces unacceptable latency before the first inference fires. Use &lt;code&gt;VOICE_RECOGNITION&lt;/code&gt; as the audio source — it activates hardware noise suppression on most Qualcomm and MediaTek SoCs at zero CPU cost.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix buffer sizing before touching model parameters.&lt;/strong&gt; It is the highest-leverage, lowest-risk change in this whole stack.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 2: Probe Your NNAPI Delegate at Startup — Not at Inference Time
&lt;/h2&gt;

&lt;p&gt;Let me show you a pattern I use in every project: fail fast on delegate configuration.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;options&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Interpreter&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Options&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;apply&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nf"&gt;addDelegate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;NnApiDelegate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;NnApiDelegate&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Options&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;apply&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nf"&gt;setAcceleratorName&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"google-edgetpu"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="c1"&gt;// or null for auto&lt;/span&gt;
        &lt;span class="nf"&gt;setUseNnapiCpu&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kc"&gt;false&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="c1"&gt;// fail fast rather than silent CPU fallback&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The docs do not mention this, but INT8 vs INT4 tradeoffs are not uniform across Android GPUs. Adreno and Mali have meaningfully different delegate behaviors:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Configuration&lt;/th&gt;
&lt;th&gt;Adreno 610&lt;/th&gt;
&lt;th&gt;Mali-G57&lt;/th&gt;
&lt;th&gt;WER Delta vs FP32&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;FP32 baseline&lt;/td&gt;
&lt;td&gt;480ms&lt;/td&gt;
&lt;td&gt;510ms&lt;/td&gt;
&lt;td&gt;0%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;INT8 (symmetric)&lt;/td&gt;
&lt;td&gt;148ms&lt;/td&gt;
&lt;td&gt;162ms&lt;/td&gt;
&lt;td&gt;+1.8%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;INT4 (GPU delegate)&lt;/td&gt;
&lt;td&gt;141ms&lt;/td&gt;
&lt;td&gt;103ms&lt;/td&gt;
&lt;td&gt;+4.1%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;INT4 (CPU fallback)&lt;/td&gt;
&lt;td&gt;310ms&lt;/td&gt;
&lt;td&gt;298ms&lt;/td&gt;
&lt;td&gt;+4.1%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;INT4 on Mali with the GPU delegate is fast. INT4 on Adreno without confirmed delegate acceleration silently falls back to CPU and blows your latency budget. Log whether GPU acceleration is active on first run. Silent delegate fallback is the single most common cause of "works on my Pixel, broken on everything else" bugs.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 3: Cap Beam Width and Profile Before Going Wider
&lt;/h2&gt;

&lt;p&gt;Beam search is the hidden latency multiplier nobody talks about. Each beam width increment roughly doubles compute and memory bandwidth pressure:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Beam Width&lt;/th&gt;
&lt;th&gt;Relative Latency&lt;/th&gt;
&lt;th&gt;WER (LibriSpeech clean)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;1 (greedy)&lt;/td&gt;
&lt;td&gt;1.0x&lt;/td&gt;
&lt;td&gt;8.4%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;2&lt;/td&gt;
&lt;td&gt;1.6x&lt;/td&gt;
&lt;td&gt;7.1%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;3&lt;/td&gt;
&lt;td&gt;2.3x&lt;/td&gt;
&lt;td&gt;6.8%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;5&lt;/td&gt;
&lt;td&gt;3.8x&lt;/td&gt;
&lt;td&gt;6.6%&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Past width 3, the WER improvement stops justifying itself for conversational ASR. Under a 150ms budget on a Snapdragon 695, width 2 is the practical ceiling. Use greedy decoding for real-time caption overlays; use width 2–3 when transcription accuracy matters.&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 4: Pre-Allocate Tensor Buffers Once
&lt;/h2&gt;

&lt;p&gt;Here is the minimal setup to get this working without OOM on 3GB devices. The issue is almost never the model weights — it is intermediate activation buffers being re-allocated per inference call.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight kotlin"&gt;&lt;code&gt;&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;AsrSession&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="k"&gt;private&lt;/span&gt; &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;interpreter&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;Interpreter&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;private&lt;/span&gt; &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;inputBuffer&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;ByteBuffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;allocateDirect&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;80&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;3000&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;order&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;ByteOrder&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;nativeOrder&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;private&lt;/span&gt; &lt;span class="kd"&gt;val&lt;/span&gt; &lt;span class="n"&gt;outputBuffer&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Array&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="nc"&gt;FloatArray&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;51865&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt; &lt;span class="c1"&gt;// vocab size&lt;/span&gt;

    &lt;span class="k"&gt;fun&lt;/span&gt; &lt;span class="nf"&gt;transcribe&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;melFrames&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;FloatArray&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;IntArray&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="n"&gt;inputBuffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;rewind&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;melFrames&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;forEach&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;inputBuffer&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;putFloat&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;it&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
        &lt;span class="n"&gt;interpreter&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;run&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;inputBuffer&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;outputBuffer&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;outputBuffer&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;indices&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;sortedByDescending&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;outputBuffer&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;it&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;take&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;toIntArray&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Keep model weights in &lt;code&gt;MappedByteBuffer&lt;/code&gt; via &lt;code&gt;FileInputStream.channel.map()&lt;/code&gt;. The OS can page them out under memory pressure and reload without your process dying. Never load weights into a heap &lt;code&gt;ByteArray&lt;/code&gt;.&lt;/p&gt;




&lt;h2&gt;
  
  
  Gotchas
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Silent CPU fallback is invisible by default.&lt;/strong&gt; Set &lt;code&gt;setUseNnapiCpu(false)&lt;/code&gt; and log delegate status at startup or you will not know your latency budget is broken until a user reports it.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;CHANNEL_IN_MONO&lt;/code&gt; is non-negotiable for Whisper.&lt;/strong&gt; Stereo input doubles your buffer and misaligns the mel-filterbank stride.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;INT4 on Adreno is slower than INT8&lt;/strong&gt; without a confirmed GPU delegate. Profile on the actual device family you are targeting, not your flagship test device.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Beam width 5 is not "safer" than width 2.&lt;/strong&gt; It is 3.8x the compute cost for less than 2% WER gain. Measure before you widen.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Wrapping Up
&lt;/h2&gt;

&lt;p&gt;The 150ms ceiling is achievable on mid-range hardware, but each of these four steps has to be right. Buffer sizing gives you clean audio. Delegate probing gives you real GPU acceleration. Beam width capping keeps you inside the latency budget. Pre-allocated buffers keep you alive on constrained RAM.&lt;/p&gt;

&lt;p&gt;Miss any one of them and you are either blowing latency or OOMing on a 3GB device — and the failure will not be obvious until it is in production.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Further reading:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://developer.android.com/ndk/guides/neuralnetworks" rel="noopener noreferrer"&gt;Android NNAPI guide&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.tensorflow.org/lite/android/delegates/nnapi" rel="noopener noreferrer"&gt;TFLite NNAPI delegate docs&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://huggingface.co/openai/whisper-tiny" rel="noopener noreferrer"&gt;Whisper model card&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>webdev</category>
      <category>programming</category>
    </item>
    <item>
      <title>Wiring eBPF-Based Profiling to Your Kubernetes Backend: Continuous CPU and Latency Flame Graphs Without Instrumentation Overhead</title>
      <dc:creator>SoftwareDevs mvpfactory.io</dc:creator>
      <pubDate>Mon, 09 Nov 2026 14:26:45 +0000</pubDate>
      <link>https://dev.to/software_mvp-factory/wiring-ebpf-based-profiling-to-your-kubernetes-backend-continuous-cpu-and-latency-flame-graphs-5coc</link>
      <guid>https://dev.to/software_mvp-factory/wiring-ebpf-based-profiling-to-your-kubernetes-backend-continuous-cpu-and-latency-flame-graphs-5coc</guid>
      <description>&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;span class="na"&gt;title&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;eBPF-Based&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;Continuous&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;Profiling&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;in&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;Kubernetes:&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;Flame&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;Graphs&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;Without&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;the&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;Overhead"&lt;/span&gt;
&lt;span class="na"&gt;published&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="na"&gt;description&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Deploy&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;Parca&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;or&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;Pyroscope&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;as&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;a&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;Kubernetes&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;DaemonSet&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;for&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;always-on&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;CPU&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;flame&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;graphs.&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;Correlate&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;hot&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;paths&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;with&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;HPA&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;events&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;—&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;zero&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;code&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;changes,&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;zero&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;APM&lt;/span&gt;&lt;span class="nv"&gt; &lt;/span&gt;&lt;span class="s"&gt;tax."&lt;/span&gt;
&lt;span class="na"&gt;tags&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;kubernetes, devops, architecture, performance&lt;/span&gt;
&lt;span class="na"&gt;canonical_url&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;https://blog.mvpfactory.co/ebpf-continuous-profiling-kubernetes&lt;/span&gt;
&lt;span class="nn"&gt;---&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  What You Will Build
&lt;/h2&gt;

&lt;p&gt;By the end of this tutorial you will have Parca running as a Kubernetes DaemonSet, sampling CPU stack traces across every node at kernel level, with flame graphs you can timestamp-align to HPA scaling events. No SDK. No code changes. No 15–20% APM overhead.&lt;/p&gt;

&lt;p&gt;Let me show you a pattern I use in every production Kubernetes cluster.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prerequisites
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Kubernetes cluster running kernel ≥ 4.9 (≥ 5.8 recommended for DWARF unwinding)&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;kubectl&lt;/code&gt; access with ability to create privileged workloads&lt;/li&gt;
&lt;li&gt;An observability namespace (we will use &lt;code&gt;observability&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;Go or JVM microservices under real traffic load — this tool earns its keep there&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Why Not Just Use an APM Agent?
&lt;/h2&gt;

&lt;p&gt;Traditional APM agents extract a 15–20% CPU and memory tax just to tell you something is slow. eBPF programs run inside the Linux kernel's sandboxed VM, using &lt;code&gt;perf_event_open&lt;/code&gt; syscalls to sample stack traces at a fixed interval (typically 19–99 Hz). Here is what that looks like in practice:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Approach&lt;/th&gt;
&lt;th&gt;Overhead&lt;/th&gt;
&lt;th&gt;Code Changes&lt;/th&gt;
&lt;th&gt;Always-On&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Agent-based APM (Datadog, New Relic)&lt;/td&gt;
&lt;td&gt;15–20%&lt;/td&gt;
&lt;td&gt;Yes (SDK)&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;async-profiler / pprof manual&lt;/td&gt;
&lt;td&gt;Negligible&lt;/td&gt;
&lt;td&gt;Yes (endpoint)&lt;/td&gt;
&lt;td&gt;No&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;eBPF (Parca / Pyroscope)&lt;/td&gt;
&lt;td&gt;1–3%&lt;/td&gt;
&lt;td&gt;None&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;eBPF with DWARF unwinding&lt;/td&gt;
&lt;td&gt;2–5%&lt;/td&gt;
&lt;td&gt;None&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The docs do not mention this, but for Go services this works out of the box — Go has preserved frame pointers by default since 1.12. JVM services need one extra flag, which we cover in the Gotchas section.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 1 — Deploy Parca Agent as a DaemonSet
&lt;/h2&gt;

&lt;p&gt;One pod per node. It needs &lt;code&gt;hostPID&lt;/code&gt;, &lt;code&gt;hostNetwork&lt;/code&gt;, and a privileged security context to load its eBPF programs. Here is the minimal setup to get this working:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;apiVersion&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;apps/v1&lt;/span&gt;
&lt;span class="na"&gt;kind&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;DaemonSet&lt;/span&gt;
&lt;span class="na"&gt;metadata&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;parca-agent&lt;/span&gt;
  &lt;span class="na"&gt;namespace&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;observability&lt;/span&gt;
&lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;selector&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;matchLabels&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;app&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;parca-agent&lt;/span&gt;
  &lt;span class="na"&gt;template&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;spec&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;hostPID&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
      &lt;span class="na"&gt;hostNetwork&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
      &lt;span class="na"&gt;containers&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;parca-agent&lt;/span&gt;
        &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;ghcr.io/parca-dev/parca-agent:latest&lt;/span&gt;
        &lt;span class="na"&gt;securityContext&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
          &lt;span class="na"&gt;privileged&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
        &lt;span class="na"&gt;args&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;--node=$(NODE_NAME)&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;--remote-store-address=parca.observability.svc:7070&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;--remote-store-insecure&lt;/span&gt;
        &lt;span class="na"&gt;volumeMounts&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;mountPath&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;/sys/fs/cgroup&lt;/span&gt;
          &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;cgroup&lt;/span&gt;
        &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;mountPath&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;/sys/kernel/debug&lt;/span&gt;
          &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;kerneldebug&lt;/span&gt;
      &lt;span class="na"&gt;volumes&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;cgroup&lt;/span&gt;
        &lt;span class="na"&gt;hostPath&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;{&lt;/span&gt; &lt;span class="nv"&gt;path&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="nv"&gt;/sys/fs/cgroup&lt;/span&gt; &lt;span class="pi"&gt;}&lt;/span&gt;
      &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;kerneldebug&lt;/span&gt;
        &lt;span class="na"&gt;hostPath&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="pi"&gt;{&lt;/span&gt; &lt;span class="nv"&gt;path&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="nv"&gt;/sys/kernel/debug&lt;/span&gt; &lt;span class="pi"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The Parca server stores profiles in a time-series-aware format backed by object storage. Its query UI is Prometheus-compatible.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 2 — Correlate Flame Graphs With HPA Scaling Events
&lt;/h2&gt;

&lt;p&gt;This is the part that actually earns its keep. When your HPA scales from 4 to 12 replicas under a mobile traffic surge, you want to know &lt;em&gt;why&lt;/em&gt; CPU spiked, not just &lt;em&gt;that&lt;/em&gt; it did.&lt;/p&gt;

&lt;p&gt;Parca exposes a gRPC Query API. Annotate your HPA events into your observability pipeline, then timestamp-align them against flame graph deltas:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;parcactl query &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--start&lt;/span&gt; 2026-11-09T14:30:00Z &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--end&lt;/span&gt;   2026-11-09T14:35:00Z &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--label-selector&lt;/span&gt; &lt;span class="s1"&gt;'app=payment-service'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--profile-type&lt;/span&gt; process_cpu:cpu:nanoseconds:cpu:nanoseconds
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;In my experience, the most expensive surprises are rarely in application logic. They show up in Jackson serialization, Protobuf reflection, connection pool contention, and G1GC humongous allocations — things that only appear past a certain concurrency threshold. Flame graphs timed to HPA events expose exactly this.&lt;/p&gt;

&lt;h2&gt;
  
  
  Gotchas
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;JVMs omit frame pointers by default.&lt;/strong&gt; Without them, eBPF unwinding falls back to DWARF — slower and less accurate. Add this flag to every JVM service:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;-XX:+PreserveFramePointer
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This carries a measured 1–3% throughput cost. That is well below what you recover by dropping APM agents entirely, so it is not really a trade-off.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Hot paths under real mobile traffic do not appear locally.&lt;/strong&gt; A JVM service under sustained load at 2,000 RPS will exhibit lock contention and GC pressure that simply does not appear at 50 RPS on a developer laptop. Always-on production profiling is the only way to see this. Do not profile in development and ship the fix without validating under production traffic patterns.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Privileged DaemonSet is a one-time ops conversation.&lt;/strong&gt; Front-load it. What you get back is permanent visibility into production behavior without touching application code.&lt;/p&gt;

&lt;h2&gt;
  
  
  Conclusion
&lt;/h2&gt;

&lt;p&gt;Three changes get you there: deploy Parca or Pyroscope as a DaemonSet, annotate HPA events into your observability pipeline, and add &lt;code&gt;-XX:+PreserveFramePointer&lt;/code&gt; to all JVM services. That combination turns autoscaling from a reactive measure into a diagnostic tool — you see the root cause of load-induced CPU growth, not just the symptom.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Resources:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://www.parca.dev/docs/overview" rel="noopener noreferrer"&gt;Parca documentation&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://grafana.com/docs/pyroscope/latest/deploy-kubernetes/" rel="noopener noreferrer"&gt;Pyroscope Kubernetes deployment&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://www.kernel.org/doc/html/latest/admin-guide/perf-security.html" rel="noopener noreferrer"&gt;eBPF and &lt;code&gt;perf_event_open&lt;/code&gt; — Linux kernel docs&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>webdev</category>
      <category>programming</category>
    </item>
  </channel>
</rss>
