<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Samyak Vaghela</title>
    <description>The latest articles on DEV Community by Samyak Vaghela (@samyak_vaghela_ff0e1e6dd8).</description>
    <link>https://dev.to/samyak_vaghela_ff0e1e6dd8</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4243408%2Fc1356f6c-b7ba-4cf0-859e-cfe11653becc.jpg</url>
      <title>DEV Community: Samyak Vaghela</title>
      <link>https://dev.to/samyak_vaghela_ff0e1e6dd8</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/samyak_vaghela_ff0e1e6dd8"/>
    <language>en</language>
    <item>
      <title>I scanned 6 popular repos the way a coding agent reads them. LangChain's lockfiles outweigh its code.</title>
      <dc:creator>Samyak Vaghela</dc:creator>
      <pubDate>Fri, 13 Nov 2026 19:01:13 +0000</pubDate>
      <link>https://dev.to/samyak_vaghela_ff0e1e6dd8/i-scanned-6-popular-repos-the-way-a-coding-agent-reads-them-langchains-lockfiles-outweigh-its-1i38</link>
      <guid>https://dev.to/samyak_vaghela_ff0e1e6dd8/i-scanned-6-popular-repos-the-way-a-coding-agent-reads-them-langchains-lockfiles-outweigh-its-1i38</guid>
      <description>&lt;p&gt;Coding agents don't read a repo the way you do. You skim past &lt;code&gt;uv.lock&lt;/code&gt; without thinking; a &lt;code&gt;grep&lt;/code&gt;, a glob or a careless &lt;code&gt;cat&lt;/code&gt; from an agent doesn't. I wanted to know how much of a typical repo is text a model pays for but can't learn anything from, so I built a small CLI to count it and ran it on six well-known projects.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbdmcjq4604mu9b996kl6.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbdmcjq4604mu9b996kl6.png" alt="Six repos scanned with ctxray" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The numbers
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Repo&lt;/th&gt;
&lt;th&gt;Total tokens&lt;/th&gt;
&lt;th&gt;Flagged&lt;/th&gt;
&lt;th&gt;Biggest single source&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;langchain&lt;/td&gt;
&lt;td&gt;9.82M&lt;/td&gt;
&lt;td&gt;55%&lt;/td&gt;
&lt;td&gt;22 &lt;code&gt;uv.lock&lt;/code&gt; files, 5.11M tokens&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ollama&lt;/td&gt;
&lt;td&gt;9.26M&lt;/td&gt;
&lt;td&gt;51%&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;integration/testdata/shakespeare.txt&lt;/code&gt;, 1.33M tokens&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;excalidraw&lt;/td&gt;
&lt;td&gt;4.50M&lt;/td&gt;
&lt;td&gt;54%&lt;/td&gt;
&lt;td&gt;a wasm wrapper and a font source file, ~590k&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;fastapi&lt;/td&gt;
&lt;td&gt;8.47M&lt;/td&gt;
&lt;td&gt;43%&lt;/td&gt;
&lt;td&gt;one drawio SVG, 631k tokens&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;shadcn/ui&lt;/td&gt;
&lt;td&gt;7.79M&lt;/td&gt;
&lt;td&gt;38%&lt;/td&gt;
&lt;td&gt;22 lockfiles, 813k tokens&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;express&lt;/td&gt;
&lt;td&gt;196k&lt;/td&gt;
&lt;td&gt;~0%&lt;/td&gt;
&lt;td&gt;nothing worth flagging&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Shallow clones, scanned 12 and 13 Nov 2026.&lt;/p&gt;

&lt;p&gt;These are whole-tree totals, so read them as an upper bound. No agent loads an entire repo in one go. But agents do search, glob and open files they don't need, and every file above is a place where that goes badly.&lt;/p&gt;

&lt;h2&gt;
  
  
  What stood out
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;LangChain's lockfiles outweigh its code.&lt;/strong&gt; It's a monorepo and every package carries its own &lt;code&gt;uv.lock&lt;/code&gt;. Together they come to 5.11M tokens. Everything an agent could actually use comes to 4.45M.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;ollama keeps the complete works of Shakespeare as test data.&lt;/strong&gt; 1.33M tokens, more than a whole 1M-token context window. A perfectly reasonable test fixture, and a terrible thing for an agent to open.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;fastapi's biggest file is a diagram.&lt;/strong&gt; A single drawio SVG is 631k tokens of XML path data.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;express is clean.&lt;/strong&gt; Small, hand-written repos are fine. The problem is rarely your code. It's three or four files nobody thinks about.&lt;/p&gt;

&lt;h2&gt;
  
  
  How it decides what counts as waste
&lt;/h2&gt;

&lt;p&gt;There are 14 categories: lockfiles, build output, minified bundles, generated code, snapshots, bulk data, duplicates, vendored dependencies, binaries and so on.&lt;/p&gt;

&lt;p&gt;The rules are deliberately conservative. Folders like &lt;code&gt;packages/&lt;/code&gt;, &lt;code&gt;lib/&lt;/code&gt; and &lt;code&gt;bin/&lt;/code&gt; are not flagged, because monorepos and CLIs keep real source there. &lt;code&gt;.d.ts&lt;/code&gt; files are kept too, since type declarations are good context. Telling you to hide your own code from your agent would be worse than missing a lockfile.&lt;/p&gt;

&lt;p&gt;Token counts come from a rule-based BPE approximation: no vocabulary file, no network. Fitted against &lt;code&gt;gpt-tokenizer&lt;/code&gt; (&lt;code&gt;o200k_base&lt;/code&gt;) over 435 files, the median per-file error is 6.1% and the corpus-wide bias is 0.0%. The calibration script is in the repo if you want to rerun it. Claude's tokenizer is different, so treat Claude numbers as estimates of an estimate.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it on your repo
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx github:geraltbhai/ctxray
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;No install, no dependencies, no network calls, MIT. It prints the breakdown and prices a full read on current Claude, GPT and Gemini models. &lt;code&gt;ctxray init&lt;/code&gt; then writes an &lt;code&gt;.agentignore&lt;/code&gt; built from what it actually found, plus an &lt;code&gt;AGENTS.md&lt;/code&gt; scaffold with your repo's real shape filled in.&lt;/p&gt;

&lt;p&gt;There's a paid tier for the HTML report, PR diffs and CI checks. Everything in this post is the free part.&lt;/p&gt;

&lt;p&gt;If it flags something on your repo that it shouldn't, drop the path in the comments. I'm treating false positives as bugs.&lt;/p&gt;

&lt;p&gt;Repo: &lt;a href="https://github.com/geraltbhai/ctxray" rel="noopener noreferrer"&gt;https://github.com/geraltbhai/ctxray&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>opensource</category>
      <category>productivity</category>
      <category>llm</category>
    </item>
    <item>
      <title>I counted the tokens in 6 popular repos the way a coding agent sees them</title>
      <dc:creator>Samyak Vaghela</dc:creator>
      <pubDate>Fri, 13 Nov 2026 18:41:53 +0000</pubDate>
      <link>https://dev.to/samyak_vaghela_ff0e1e6dd8/i-counted-the-tokens-in-6-popular-repos-the-way-a-coding-agent-sees-them-oag</link>
      <guid>https://dev.to/samyak_vaghela_ff0e1e6dd8/i-counted-the-tokens-in-6-popular-repos-the-way-a-coding-agent-sees-them-oag</guid>
      <description>&lt;p&gt;I kept wondering how much of what a coding agent reads in a repo is actually useful to it. So I wrote a small CLI that walks a repo, estimates tokens per file the way a model sees them, and flags files that cost tokens without teaching the model anything.&lt;/p&gt;

&lt;p&gt;Then I ran it on six well-known open-source repos (fresh shallow clones, 12–13 Nov 2026).&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbdmcjq4604mu9b996kl6.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbdmcjq4604mu9b996kl6.png" alt="Six popular repos scanned with ctxray: share of tokens flagged as waste" width="800" height="450"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The numbers
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Repo&lt;/th&gt;
&lt;th&gt;Total tokens&lt;/th&gt;
&lt;th&gt;Flagged as waste&lt;/th&gt;
&lt;th&gt;Biggest offender&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;langchain&lt;/td&gt;
&lt;td&gt;9.82M&lt;/td&gt;
&lt;td&gt;55%&lt;/td&gt;
&lt;td&gt;22 &lt;code&gt;uv.lock&lt;/code&gt; files, 5.11M tokens together&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;excalidraw&lt;/td&gt;
&lt;td&gt;4.50M&lt;/td&gt;
&lt;td&gt;54%&lt;/td&gt;
&lt;td&gt;a 296k-token &lt;code&gt;harfbuzz-wasm.ts&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;ollama&lt;/td&gt;
&lt;td&gt;9.26M&lt;/td&gt;
&lt;td&gt;51%&lt;/td&gt;
&lt;td&gt;
&lt;code&gt;integration/testdata/shakespeare.txt&lt;/code&gt;, 1.33M tokens&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;fastapi&lt;/td&gt;
&lt;td&gt;8.47M&lt;/td&gt;
&lt;td&gt;43%&lt;/td&gt;
&lt;td&gt;one drawio SVG, 631k tokens&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;shadcn/ui&lt;/td&gt;
&lt;td&gt;7.79M&lt;/td&gt;
&lt;td&gt;38%&lt;/td&gt;
&lt;td&gt;22 lockfiles, 813k tokens together&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;express&lt;/td&gt;
&lt;td&gt;196k&lt;/td&gt;
&lt;td&gt;~0%&lt;/td&gt;
&lt;td&gt;nothing worth flagging&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;These are whole-tree totals, so treat them as an upper bound. No agent reads a 9M-token repo in one pass. But agents grep, glob and open files all day, and every time one of those lands in a lockfile or a test fixture, that turn's context goes to integrity hashes instead of code.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three things that surprised me
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. It's rarely the code.&lt;/strong&gt; In langchain, the lockfiles alone (5.11M tokens) outweigh everything an agent could actually learn from (4.45M). In ollama, a copy of Shakespeare used as test data is bigger than most entire repos.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Small, hand-written repos are fine.&lt;/strong&gt; express came back clean. The problem shows up in big monorepos with many packages, generated assets and test data.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. The fix is boring.&lt;/strong&gt; Almost all of the waste comes from five kinds of file: lockfiles, snapshots, minified or generated bundles, bulk data, and large SVGs. A dozen ignore rules removes most of it.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why care, if context windows are 1M now?
&lt;/h2&gt;

&lt;p&gt;Input tokens are still the bill, and a bigger window just makes it easier to fill with junk without noticing. It also costs you quality: Chroma's "context rot" research found that models get worse as the input fills up, however large the window is. So you pay more for worse answers.&lt;/p&gt;

&lt;h2&gt;
  
  
  The tool
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx github:geraltbhai/ctxray
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Nothing to install, no dependencies, no network calls, Node 18+, MIT. It prints:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;total tokens if an agent ingested the repo&lt;/li&gt;
&lt;li&gt;the cost of one full-context pass on Claude, GPT and Gemini models&lt;/li&gt;
&lt;li&gt;a grade from A to F&lt;/li&gt;
&lt;li&gt;the worst offenders, file by file, with the reason each one counts as waste&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Then:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;npx github:geraltbhai/ctxray init --dry-run   &lt;span class="c"&gt;# preview; drop --dry-run to write the files&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That writes an &lt;code&gt;.agentignore&lt;/code&gt; built from what the scan actually found. It uses &lt;code&gt;.gitignore&lt;/code&gt; syntax, so it works as a &lt;code&gt;.cursorignore&lt;/code&gt; or &lt;code&gt;.aiexclude&lt;/code&gt; too. It also writes a starter &lt;code&gt;AGENTS.md&lt;/code&gt; pre-filled with the repo's real shape: language mix, directory weights and entry points.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;How the count works:&lt;/strong&gt; loading a full tokenizer vocabulary would mean a multi-megabyte dependency, so ctxray uses a rule-based BPE approximation instead. I fitted its constants against &lt;code&gt;gpt-tokenizer&lt;/code&gt; over 435 files, and it lands at about 6% median error per file. The calibration script is in the repo, so you can check that claim yourself.&lt;/p&gt;

&lt;p&gt;The scanner and &lt;code&gt;init&lt;/code&gt; are free and staying free. A paid tier for HTML reports, PR comments that show what a change did to your context budget, CI thresholds and a context packer is on its way.&lt;/p&gt;

&lt;h2&gt;
  
  
  Try it on your repo
&lt;/h2&gt;

&lt;p&gt;Run it and tell me your grade in the comments. If it flags something it shouldn't, post the path. I'm treating false positives as bugs.&lt;/p&gt;

&lt;p&gt;Repo: &lt;a href="https://github.com/geraltbhai/ctxray" rel="noopener noreferrer"&gt;github.com/geraltbhai/ctxray&lt;/a&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>opensource</category>
      <category>productivity</category>
      <category>javascript</category>
    </item>
  </channel>
</rss>
