<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Tabish Hassan</title>
    <description>The latest articles on DEV Community by Tabish Hassan (@tabishhassan5121472).</description>
    <link>https://dev.to/tabishhassan5121472</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4248844%2Fed0ce505-0723-46e1-92f0-9ed83afe9169.jpg</url>
      <title>DEV Community: Tabish Hassan</title>
      <link>https://dev.to/tabishhassan5121472</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/tabishhassan5121472"/>
    <language>en</language>
    <item>
      <title>Why AI coding agents slow down in long sessions, and 5 prompt-caching mistakes to avoid</title>
      <dc:creator>Tabish Hassan</dc:creator>
      <pubDate>Tue, 17 Nov 2026 10:27:23 +0000</pubDate>
      <link>https://dev.to/tabishhassan5121472/why-ai-coding-agents-slow-down-in-long-sessions-and-5-prompt-caching-mistakes-to-avoid-2hk4</link>
      <guid>https://dev.to/tabishhassan5121472/why-ai-coding-agents-slow-down-in-long-sessions-and-5-prompt-caching-mistakes-to-avoid-2hk4</guid>
      <description>&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://tabishapps.com/blog/prompt-caching-coding-agents/" rel="noopener noreferrer"&gt;tabishapps.com/blog/prompt-caching-coding-agents&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;If you use Claude Code, Codex or an open model through Ollama, you've probably seen an agent that was quick at the start get slower and more expensive as the session went on. Often the reason isn't the model. It's the prompt cache being thrown away, sometimes by something as small as a timestamp.&lt;/p&gt;

&lt;p&gt;This week &lt;a href="https://x.com/ollama/status/2122531964154712541" rel="noopener noreferrer"&gt;Ollama shared three lessons&lt;/a&gt; on this, from a longer &lt;a href="https://www.together.ai/blog/making-open-models-work-for-long-horizon-agentic-workloads" rel="noopener noreferrer"&gt;write-up by Together AI and Ollama&lt;/a&gt;. I read it, then tested the main claim on my own Mac. Here's what's going on, in plain words, and what to do about it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Every turn, the agent sends everything again
&lt;/h2&gt;

&lt;p&gt;A coding agent doesn't "remember" your session the way a person does. On every step, the tool you use (the &lt;em&gt;harness&lt;/em&gt;: Claude Code, Codex, Cursor, Pi and so on) sends the model the whole conversation again: the system prompt, every tool definition, your request, every file it has read and every command it has run. The model usually answers with a short tool call, like "read this file". Then the next turn sends all of that again, plus a bit more.&lt;/p&gt;

&lt;p&gt;So for agents, the expensive part is reading the input, not writing the answer. Without help, turn 100 would mean re-reading 100 turns of history.&lt;/p&gt;

&lt;h2&gt;
  
  
  Prefix caching: only read what's new
&lt;/h2&gt;

&lt;p&gt;Inference engines (Ollama, vLLM, MLX, and hosted APIs like Anthropic's and OpenAI's) keep the work they did on a prompt for a while. If the next request &lt;strong&gt;starts with exactly the same tokens&lt;/strong&gt;, they reuse that work and only process the new part at the end.&lt;/p&gt;

&lt;p&gt;The Together write-up gives the simple maths: if each turn adds about the same amount, ten turns without a cache process 1 + 2 + … + 10 = 55 units of prompt, and with a cache only 10. At a hundred turns it's 5,050 against 100.&lt;/p&gt;

&lt;p&gt;But there's one rule that everything else follows from:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The cache matches from the very first token up to the first token that's different, and stops there.&lt;/strong&gt; Adding text at the end is cheap. Changing anything earlier, even one character, means everything after that point is processed again.&lt;/p&gt;

&lt;h2&gt;
  
  
  I tested it on a Mac
&lt;/h2&gt;

&lt;p&gt;I wanted to see this rule in real numbers, so I ran a small test with Apple's MLX and Google's Gemma 3 12B (4-bit) on a MacBook Pro with an M3 Pro chip and 36 GB of memory. The script builds a fake agent session: a system prompt with tool definitions, then a few turns where the agent reads files. It then times how long the model takes to read the prompt for the next turn in three cases:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Case&lt;/th&gt;
&lt;th&gt;Tokens the model had to read&lt;/th&gt;
&lt;th&gt;Time before the first word&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;No cache (fresh start)&lt;/td&gt;
&lt;td&gt;10,275 (the whole prompt)&lt;/td&gt;
&lt;td&gt;about 2–4 minutes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cache kept, new turn added at the end&lt;/td&gt;
&lt;td&gt;2,513 (only the new turn)&lt;/td&gt;
&lt;td&gt;about 1 minute (66 s)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Cache kept, but the system prompt has a new timestamp&lt;/td&gt;
&lt;td&gt;10,244 (only the first 32 tokens matched)&lt;/td&gt;
&lt;td&gt;about 2–4 minutes&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The token counts are the important part, because they don't depend on what else the laptop was doing. With the cache kept, the model read 2,513 tokens instead of 10,275, about four times less, and that gap keeps growing as the session gets longer. With one changed timestamp near the top, only the first 32 tokens matched, so the cache saved almost nothing: the model read 10,244 tokens, the same as starting fresh. (The times are rough. My two full re-reads took 134 and 234 seconds because the Mac was busy with other apps, which is why I show a range.)&lt;/p&gt;

&lt;p&gt;A cloud model on big GPUs is much faster than a laptop, but the shape is the same: a broken cache means paying again, in time and money, for the whole session.&lt;/p&gt;

&lt;p&gt;The test script is short and at the end of this post if you want to try it.&lt;/p&gt;

&lt;h2&gt;
  
  
  5 things that break the cache
&lt;/h2&gt;

&lt;h3&gt;
  
  
  1. Anything in the system prompt that changes each turn
&lt;/h3&gt;

&lt;p&gt;A current time, the &lt;code&gt;git status&lt;/code&gt;, the current folder, "turn 14 of 40". It feels helpful to keep the model up to date, but the system prompt sits at the very top, so a new value there means the whole session is read again on every turn. That's the "timestamp" row in my test.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix:&lt;/strong&gt; keep the system prompt identical for the whole session (and ideally across sessions). Put changing information in a new message at the end instead. If you write your own agent or prompt templates, check for &lt;code&gt;datetime.now()&lt;/code&gt; or similar in the system prompt first.&lt;/p&gt;

&lt;h3&gt;
  
  
  2. Changing reasoning effort in the middle of a session
&lt;/h3&gt;

&lt;p&gt;For some open models, the chat template writes the reasoning-effort setting into the first system block. Ollama found this with GLM. So switching from "low" to "high" halfway through a long task changes the prompt almost at the first token, and the whole session is processed again.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix:&lt;/strong&gt; choose the effort level at the start and leave it. If you really need more thinking, it's often cheaper to start a fresh session (or compact the old one first) than to flip the setting on a long one.&lt;/p&gt;

&lt;h3&gt;
  
  
  3. Adding tools near the top
&lt;/h3&gt;

&lt;p&gt;Newer harnesses load tools only when the agent searches for them, which saves context. But if the new tool definition is added to the tool list at the top, everything after it is re-read. The better place is the end of the conversation. Not every model's chat template accepts that, so this depends on the harness and model.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix (if you build tools or MCP servers):&lt;/strong&gt; avoid changing the tool list during a session, and prefer harnesses that add late-loaded tools at the end.&lt;/p&gt;

&lt;h3&gt;
  
  
  4. Huge tool results
&lt;/h3&gt;

&lt;p&gt;A file the agent reads becomes part of every request after it. If it reads 3,000 lines when it needed 40, you carry those 3,000 lines for the rest of the session. Even cached, they fill the context window and memory.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix:&lt;/strong&gt; ask the agent to search before it reads, and to read line ranges. If you build tools, return less by default.&lt;/p&gt;

&lt;h3&gt;
  
  
  5. Cutting output without saying so
&lt;/h3&gt;

&lt;p&gt;This one isn't about speed, but it's the third point in Ollama's post and it matters. Many tools trim long output. If a tool returns the first 100 lines of a 640-line file without saying it stopped, the model doesn't know the rest exists and will happily edit code it never saw.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Fix:&lt;/strong&gt; when output is cut, add one line like &lt;code&gt;[showing lines 1–100 of 640; search or ask for another range]&lt;/code&gt;. With that hint, the model goes and reads the part it needs.&lt;/p&gt;

&lt;h2&gt;
  
  
  A quick checklist
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;System prompt: no dates, times, git status or folder paths that change. Put those in the latest message.&lt;/li&gt;
&lt;li&gt;Pick model, reasoning effort and tools at the start of a session and don't change them midway.&lt;/li&gt;
&lt;li&gt;Long, messy session? Start fresh or compact, rather than changing settings.&lt;/li&gt;
&lt;li&gt;Read files by searching and line ranges, not whole files.&lt;/li&gt;
&lt;li&gt;Truncated output should always say it was truncated.&lt;/li&gt;
&lt;li&gt;For subagents: give them a short brief instead of the whole history when they only need a few files. A cached prefix is cheap, but not free.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The test script
&lt;/h2&gt;

&lt;p&gt;It needs a Mac with Apple silicon and Python. Install &lt;code&gt;mlx-lm&lt;/code&gt; with &lt;code&gt;pip install mlx-lm&lt;/code&gt;; the first run downloads the model (about 8 GB for the 12B, under 1 GB for &lt;code&gt;mlx-community/gemma-3-1b-it-4bit&lt;/code&gt;).&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="s2"&gt;"""Measure how prefix caching behaves for an agent-style prompt with mlx_lm.

Scenarios (all time only prefill + 1 token):
  cold      : whole prompt for turn N+1, empty cache
  append    : cache holds turn N, only the new turn is processed
  timestamp : same as append, but the system prompt has a new timestamp -&amp;gt; first
              differing token is near the top, so nothing after it can be reused
"""&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;sys&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;random&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;mlx_lm&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;load&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;stream_generate&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;mlx_lm&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;models&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;cache&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;make_prompt_cache&lt;/span&gt;

&lt;span class="no"&gt;MODEL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;sys&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;argv&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;sys&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;argv&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="s2"&gt;"mlx-community/gemma-3-1b-it-4bit"&lt;/span&gt;
&lt;span class="no"&gt;TURNS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kt"&gt;int&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;sys&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;argv&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;sys&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;argv&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt; &lt;span class="k"&gt;else&lt;/span&gt; &lt;span class="mi"&gt;12&lt;/span&gt;
&lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tok&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;load&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;MODEL&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="no"&gt;TOOLS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"\n"&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"- {n}(path: str, start: int = 1, end: int = 200): {d}"&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;d&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"read_file"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Read part of a file and return numbered lines."&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                 &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"search"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Search the project for a string and return matching lines."&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                 &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"run"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Run a shell command and return stdout and stderr."&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                 &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"edit_file"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"Replace an exact string in a file with new text."&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                 &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"list_dir"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"List the files in a directory."&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;random&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;seed&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;fake_file&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;rnd&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;random&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Random&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;i&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;lines&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;j&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;60&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;lines&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"{j+1:4d}  def handler_{i}_{j}(event, tz='UTC'):  # export step {j}"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;lines&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"      return convert(event['start'], tz) + {rnd.randint(1, 999)}"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="s2"&gt;"\n"&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;join&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;lines&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;messages&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;turns&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;stamp&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;sys_prompt&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"You are a coding agent.\nCurrent time: {stamp}\nTools:\n{TOOLS}"&lt;/span&gt;
    &lt;span class="n"&gt;msgs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"role"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"user"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"content"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;sys_prompt&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="s2"&gt;"\n\nTask: fix the timezone bug in the calendar export."&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;t&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;turns&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;msgs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"role"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"assistant"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"content"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s1"&gt;'read_file(path="src/module_{t}.py")'&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;msgs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"role"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"user"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"content"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"Tool result:\n{fake_file(t)}"&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;msgs&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;ids&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;turns&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;stamp&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;tok&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;apply_chat_template&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;messages&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;turns&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;stamp&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;add_generation_prompt&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;prefill&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;tokens&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;cache&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;t0&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;perf_counter&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;stream_generate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;tok&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;prompt&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;tokens&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;max_tokens&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;prompt_cache&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;cache&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;pass&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;perf_counter&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="n"&gt;t0&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;lcp&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;y&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;zip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;a&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;b&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="n"&gt;y&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;break&lt;/span&gt;
        &lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt;

&lt;span class="c1"&gt;# warm up the GPU kernels so the first timing is fair&lt;/span&gt;
&lt;span class="nf"&gt;prefill&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;ids&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"warmup"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nf"&gt;make_prompt_cache&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="no"&gt;S1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="no"&gt;S2&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"2026-11-17 10:04:11"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"2026-11-17 10:04:52"&lt;/span&gt;
&lt;span class="n"&gt;prev&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;ids&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;TURNS&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="no"&gt;S1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;nxt_same&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;ids&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;TURNS&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="no"&gt;S1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;nxt_new_stamp&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;ids&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="no"&gt;TURNS&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="no"&gt;S2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"model {MODEL}"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"turn {TURNS} prompt: {len(prev)} tokens; turn {TURNS+1}: {len(nxt_same)} tokens"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"shared prefix, same system prompt: {lcp(prev, nxt_same)} tokens"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"shared prefix, new timestamp:      {lcp(prev, nxt_new_stamp)} tokens"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="c1"&gt;# chat templates may re-render the end of the previous prompt (generation prompt),&lt;/span&gt;
&lt;span class="c1"&gt;# so reuse exactly the shared prefix: prime the cache with it, then send the rest.&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;warm&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prev_ids&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;new_ids&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;lcp&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prev_ids&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;new_ids&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;c&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;make_prompt_cache&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="nf"&gt;prefill&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;new_ids&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;       &lt;span class="c1"&gt;# what the server already has from turn N&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;prefill&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;new_ids&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;new_ids&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="o"&gt;-&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;prefill&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;new_ids&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;c&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;new_ids&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;cold&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;prefill&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;nxt_same&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nf"&gt;make_prompt_cache&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;app&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;app_n&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;warm&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prev&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;nxt_same&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;ts&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;ts_n&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;warm&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;prev&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;nxt_new_stamp&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"cold      : {cold:6.2f}s  ({len(nxt_same)} tokens processed)"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"append    : {app:6.2f}s  ({app_n} tokens processed)"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"timestamp : {ts:6.2f}s  ({ts_n} tokens processed)"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Run it with &lt;code&gt;python cache_test.py mlx-community/gemma-3-12b-it-4bit 3&lt;/code&gt;. The last number is how many file-reading turns to put in the session.&lt;/p&gt;

&lt;h2&gt;
  
  
  Sources
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Ollama on X, 17 November 2026: &lt;a href="https://x.com/ollama/status/2122531964154712541" rel="noopener noreferrer"&gt;lessons on keeping coding agents fast&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;Together AI and Ollama: &lt;a href="https://www.together.ai/blog/making-open-models-work-for-long-horizon-agentic-workloads" rel="noopener noreferrer"&gt;Making open models work for long-horizon agentic workloads&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://github.com/ml-explore/mlx-lm" rel="noopener noreferrer"&gt;mlx-lm&lt;/a&gt;, used for the test.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;em&gt;Written by Tabish Hassan, a software developer. Found a mistake or got different numbers on your machine? Let me know in the comments.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>programming</category>
      <category>productivity</category>
    </item>
    <item>
      <title>How to add subtitles to a video on your Mac, offline, in 100 languages</title>
      <dc:creator>Tabish Hassan</dc:creator>
      <pubDate>Tue, 17 Nov 2026 04:30:34 +0000</pubDate>
      <link>https://dev.to/tabishhassan5121472/how-to-add-subtitles-to-a-video-on-your-mac-offline-in-100-languages-1ecj</link>
      <guid>https://dev.to/tabishhassan5121472/how-to-add-subtitles-to-a-video-on-your-mac-offline-in-100-languages-1ecj</guid>
      <description>&lt;p&gt;&lt;em&gt;Originally published at &lt;a href="https://tabishapps.com/blog/add-subtitles-to-video-offline-mac/" rel="noopener noreferrer"&gt;tabishapps.com&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;Most subtitle tools upload your video to a server and charge by the minute. If you have an Apple silicon Mac, you can do the whole job on your own machine instead: transcribe the speech, split it into readable subtitles, translate them, and export files that YouTube and every major editor accept.&lt;/p&gt;

&lt;p&gt;This guide walks through that process with &lt;a href="https://tabishapps.com/mlx-caption/" rel="noopener noreferrer"&gt;MLX Caption&lt;/a&gt;, a Mac app I built for exactly this job. The first five videos are free, so you can follow along without paying. Near the end I also explain what happens under the hood, for anyone curious about running AI models locally.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why do it offline?
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Privacy.&lt;/strong&gt; Client footage, unreleased videos and internal recordings never leave your Mac.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cost.&lt;/strong&gt; Online services usually charge per minute of video, and translation into extra languages costs more again. A local tool costs the same whether you caption one video or a thousand.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No waiting on uploads.&lt;/strong&gt; Large video files don't need to travel anywhere.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What you need
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Mac&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Apple silicon: M1, M2, M3, M4 or newer. Intel Macs can't run it.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;macOS&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;14 Sonoma or newer&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Memory&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;16 GB recommended. 8 GB works with smaller models, chosen automatically.&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Disk&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;8 to 15 GB free for the AI models, downloaded once&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Video&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;MP4, MOV, MKV, WebM, AVI, or an audio file&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;h2&gt;
  
  
  Step 1: Download and open the app
&lt;/h2&gt;

&lt;p&gt;Download the free trial from the &lt;a href="https://tabishapps.com/mlx-caption/" rel="noopener noreferrer"&gt;MLX Caption page&lt;/a&gt; and unzip it. No account, email address or card is needed.&lt;/p&gt;

&lt;p&gt;Because the app is sold outside the Mac App Store, macOS asks you to confirm the first time you open it. Open &lt;strong&gt;System Settings → Privacy &amp;amp; Security&lt;/strong&gt;, scroll down and click &lt;strong&gt;Open Anyway&lt;/strong&gt;. You only do this once, and the download includes a short guide with the exact steps.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 2: Let it download the models
&lt;/h2&gt;

&lt;p&gt;On first launch the app installs its components and downloads the AI models: about 6.5 GB on Macs with 8 to 16 GB of memory, and about 13 GB on Macs with 24 GB or more. This is the only step that needs the internet. After that, everything works offline.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 3: Drop in a video and choose languages
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fxkfokuzfjuibeg0stuvb.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fxkfokuzfjuibeg0stuvb.jpg" alt="The start screen: drop a file, pick languages, press start" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Drag your video onto the window. The spoken language is detected automatically. Then pick the subtitle languages you want. Fifteen common languages are selected by default, and there are buttons for "All core" (54 languages) and "All 100".&lt;/p&gt;

&lt;p&gt;If your video has names or jargon the speech model might mishear, add them as a vocabulary hint. Then press start.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 4: Check the result
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftzyp30apmjfosdq6j5x5.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Ftzyp30apmjfosdq6j5x5.jpg" alt="A video playing with Hindi subtitles next to the transcript" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;When it finishes you can play the video with any subtitle language, read the transcript, and compare a translation side by side with the original. It is worth a quick read before you publish, especially for names and for the less common languages.&lt;/p&gt;

&lt;p&gt;For a sense of speed: on an M3 Pro, a 28-second clip was transcribed in 5.2 seconds. Translating that same clip into all 100 languages, plus scene descriptions and a summary, took 6 minutes 35 seconds. Older Macs and 8 GB machines are slower.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 5: Export and use the subtitles
&lt;/h2&gt;

&lt;p&gt;Each job produces a folder with:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;SRT and VTT files&lt;/strong&gt; for every language, the formats every platform and editor understands&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;An MP4 with every language embedded&lt;/strong&gt; as a selectable subtitle track, without re-encoding the video&lt;/li&gt;
&lt;li&gt;Optionally, &lt;strong&gt;a copy with one language burned in&lt;/strong&gt;, for social media where viewers don't turn captions on&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;A plain-text transcript&lt;/strong&gt;, plus a &lt;strong&gt;summary with chapter timestamps&lt;/strong&gt; you can paste into a video description&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  YouTube
&lt;/h3&gt;

&lt;p&gt;In YouTube Studio, open &lt;strong&gt;Subtitles&lt;/strong&gt;, choose your video, click &lt;strong&gt;Add language&lt;/strong&gt;, then &lt;strong&gt;Add&lt;/strong&gt; under Subtitles and choose &lt;strong&gt;Upload file → With timing&lt;/strong&gt;. Select the matching &lt;code&gt;.srt&lt;/code&gt; file. Repeat for each language. Viewers in other countries can then pick their own language.&lt;/p&gt;

&lt;h3&gt;
  
  
  Premiere Pro
&lt;/h3&gt;

&lt;p&gt;Import the &lt;code&gt;.srt&lt;/code&gt; file into your project like any other media, then drag it onto the timeline. It becomes a caption track you can restyle.&lt;/p&gt;

&lt;h3&gt;
  
  
  Final Cut Pro
&lt;/h3&gt;

&lt;p&gt;Choose &lt;strong&gt;File → Import → Captions&lt;/strong&gt; and pick the &lt;code&gt;.srt&lt;/code&gt; file. Recent versions of Final Cut Pro import SRT directly.&lt;/p&gt;

&lt;h3&gt;
  
  
  DaVinci Resolve
&lt;/h3&gt;

&lt;p&gt;Import the &lt;code&gt;.srt&lt;/code&gt; file into the Media Pool, then drag it onto the timeline to create a subtitle track.&lt;/p&gt;

&lt;h2&gt;
  
  
  What happens under the hood
&lt;/h2&gt;

&lt;p&gt;Everything runs on the Mac's GPU through &lt;a href="https://github.com/ml-explore/mlx" rel="noopener noreferrer"&gt;MLX&lt;/a&gt;, Apple's machine learning framework for Apple silicon. Four models do the work, one at a time so memory stays manageable:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Job&lt;/th&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Speech to text&lt;/td&gt;
&lt;td&gt;Whisper large-v3 turbo, with timing for every word&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Translation and summary&lt;/td&gt;
&lt;td&gt;Gemma 3 12B, 4-bit&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Scene descriptions&lt;/td&gt;
&lt;td&gt;Qwen3-VL, one keyframe per shot&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Silence and shot detection&lt;/td&gt;
&lt;td&gt;Small custom routines written directly in MLX&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The interesting trick is in translation. Generating text on a GPU is limited by memory bandwidth, not by raw compute, so translating into 16 languages in one batch costs little more than translating into one. The app reads the shared part of each prompt once, then decodes all the target languages together. That is why adding languages is cheap.&lt;/p&gt;

&lt;p&gt;Two smaller details make the output usable. Silent stretches are skipped, because speech models tend to invent text in silence. And every translated line is checked for letters from the wrong writing system, for example Hindi script inside Urdu, and translated again if needed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Limits worth knowing
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;54 "core" languages translate at high quality. The other 46 are lower-resource languages where quality varies, so review them before publishing.&lt;/li&gt;
&lt;li&gt;It needs an Apple silicon Mac. There is no Windows or Intel version.&lt;/li&gt;
&lt;li&gt;Machine translation is a strong first draft, not a certified translation. For legal or medical content, have a native speaker check it.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Disclosure:&lt;/strong&gt; I am the developer of MLX Caption. After the five free videos it costs $49 once, with a 14-day refund. There is no subscription.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;a href="https://tabishapps.com/mlx-caption/" rel="noopener noreferrer"&gt;Download the free trial&lt;/a&gt;&lt;/strong&gt;: your first 5 videos are free, with every feature, and nothing is uploaded.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;I'm Tabish Hassan, a software developer in Pakistan. I build websites, apps and business systems: &lt;a href="https://tabishapps.com/" rel="noopener noreferrer"&gt;tabishapps.com&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>macos</category>
      <category>ai</category>
      <category>machinelearning</category>
      <category>tutorial</category>
    </item>
  </channel>
</rss>
