<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Sakshi Lunawat</title>
    <description>The latest articles on DEV Community by Sakshi Lunawat (@sakshi916).</description>
    <link>https://dev.to/sakshi916</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4257569%2F7be474e5-8572-4092-8c67-9df5780fa590.png</url>
      <title>DEV Community: Sakshi Lunawat</title>
      <link>https://dev.to/sakshi916</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/sakshi916"/>
    <language>en</language>
    <item>
      <title>Why I removed the dedup from my coding agent</title>
      <dc:creator>Sakshi Lunawat</dc:creator>
      <pubDate>Sat, 21 Nov 2026 15:44:50 +0000</pubDate>
      <link>https://dev.to/sakshi916/why-i-removed-the-dedup-from-my-coding-agent-28k</link>
      <guid>https://dev.to/sakshi916/why-i-removed-the-dedup-from-my-coding-agent-28k</guid>
      <description>&lt;h2&gt;
  
  
  The setup
&lt;/h2&gt;

&lt;p&gt;I'm building a local coding agent on a 32GB VRAM machine, running a ~27B model via Ollama. The model has a 128k-token context window, but Ollama is not context-aware — every call requires re-sending the full conversation. I cap that at the last 20 messages.&lt;/p&gt;

&lt;p&gt;For a coding agent, that cap is reasonable. But &lt;code&gt;read_file&lt;/code&gt; returns the entire file as a single tool result. A handful of 100–150-line reads plus the conversation history fills the window fast. And with no cap on tool-call iterations, the agent has room to waste.&lt;/p&gt;

&lt;h2&gt;
  
  
  The incident
&lt;/h2&gt;

&lt;p&gt;I asked the agent to make a small method-parameter change and stepped away. Ten minutes later it was still running — on its 6th read of the same file, out of 15 tool calls. The agent was stuck in a re-read loop: every &lt;code&gt;search_folder&lt;/code&gt; call pointed it back to a file it had already read, and it read it again.&lt;/p&gt;

&lt;h2&gt;
  
  
  First fix: dedup
&lt;/h2&gt;

&lt;p&gt;The obvious move was a dedup layer. In the agent loop I maintained a &lt;code&gt;set&lt;/code&gt; of paths already read in the session. On a subsequent &lt;code&gt;read_file&lt;/code&gt; for a path in that set, I short-circuited the call and returned a message to the model instead of the file contents:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;"This file was already read. Its contents are in your conversation history — refer to them instead of re-reading."&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The re-reads stopped. But the agent still didn't converge. Here's the trace from that session:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Can you check the readNote and listNotes functions, edit them if needed and add the tools?
  ⚙  read_file({'path': 'notes.py'})
  ⚙  read_file({'path': 'main.py'})
  ⚙  read_file({'path': 'notes.py'})  [duplicate — skipped]
  ⚙  search_folder({'path': '.', 'query': 'TOOLS'})
  ⚙  search_folder({'path': '.', 'query': 'dispatch'})
  ⚙  read_file({'path': 'notes.py'})  [duplicate — skipped]
  ⚙  search_folder({'path': '.', 'query': 'show_note'})
  ⚙  search_folder({'path': '.', 'query': 'HANDLERS'})
  ⚙  search_folder({'path': '.', 'query': '"name": "create_note"'})
  ⚙  search_folder({'path': '.', 'query': '"name": "edit_note"'})
  ⚙  search_folder({'path': '.', 'query': '"name": "delete_note"'})
  ⚙  read_file({'path': 'notes.py'})  [duplicate — skipped]
  ⚙  search_folder({'path': '.', 'query': '"parameters"'})
  ⚙  search_folder({'path': '.', 'query': '"required"'})
  ...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The re-reads were skipped, but the model kept searching for symbols that were already in &lt;code&gt;notes.py&lt;/code&gt; — a file it had read three calls earlier. The dedup was working as designed. The problem was elsewhere.&lt;/p&gt;

&lt;h2&gt;
  
  
  Second fix: prompt discipline
&lt;/h2&gt;

&lt;p&gt;I moved the constraint into the system prompt and the tool descriptions:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;read_file&lt;/strong&gt; — Read the full contents of a file. Read each file ONCE. After reading, assess its content and use it from your conversation history. Do NOT re-read a file you have already read; the content is already in your context above.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;search_folder&lt;/strong&gt; — Search all text files in a folder for a keyword. Use this ONLY to find information you have NOT already seen in a file you read. If you already read a file that likely contains the answer, do not search for it — refer to the content in your conversation history instead.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The agent stopped over-searching. The loop converged in a reasonable number of steps.&lt;/p&gt;

&lt;h2&gt;
  
  
  The realization
&lt;/h2&gt;

&lt;p&gt;The dedup trace made me ask a question I should have asked before building it: what if the last &lt;code&gt;read_file('notes.py')&lt;/code&gt; was 10 calls ago? With a 20-message cap, that read — and its full file contents — had already been evicted from the context window.&lt;/p&gt;

&lt;p&gt;The dedup layer was telling the model: "don't re-read, it's in your history." But the history no longer contained it. The model was being told to rely on content it could no longer see.&lt;/p&gt;

&lt;p&gt;This is the &lt;strong&gt;window-blind dedup&lt;/strong&gt; bug: a dedup layer that assumes "already read" implies "currently in context," when the two are only equivalent while the read is inside the window. The moment the history cap evicts it, the optimization becomes a silent correctness bug — the model reasons about a file it has no knowledge of.&lt;/p&gt;

&lt;p&gt;Fixing it properly would have meant tracking which reads were still inside the window and only deduping those. That's a non-trivial piece of state to maintain in the agent loop, and it couples the dedup layer to the eviction policy.&lt;/p&gt;

&lt;p&gt;So I ran the simpler test first: remove the dedup, keep the prompt changes. The prompt discipline was already doing the job — it told the model not to re-read &lt;em&gt;and&lt;/em&gt; not to over-search, without assuming anything about what's in the window. The agent performed fine without the dedup.&lt;/p&gt;

&lt;p&gt;I also capped tool calls at 15 and told the model to wrap up two iterations before the limit — a backstop against unbounded grep loops.&lt;/p&gt;

&lt;h2&gt;
  
  
  Takeaway
&lt;/h2&gt;

&lt;p&gt;For a local, context-unaware model with a bounded history window, "already read" does not mean "the model currently knows it." Any optimization that assumes the former implies the latter is window-blind. The safe fix is a constraint that doesn't depend on window state — in this case, prompt-level discipline. It's a soft constraint, not a hard guarantee, but it's the &lt;em&gt;safe&lt;/em&gt; soft constraint.&lt;/p&gt;

&lt;p&gt;That said, dedup isn't wrong in general — it's just the wrong tool at this stage. With the 15-call cap in place, a stray re-read is cheap: the loop is bounded, and a stricter instruction on the next call ("you've already read this — use it") handles the occasional slip. A context-aware dedup layer — one that tracks which reads are still inside the window — is the right fix, but it's overkill until the window pressure actually forces it. I'll add it when the cost of a stray re-read stops being negligible.&lt;/p&gt;

</description>
      <category>llm</category>
      <category>ai</category>
      <category>firstpost</category>
    </item>
  </channel>
</rss>
