DEV Community: Shenao Yu

Is 'Chunkless RAG' Actually Solving the Right Problem?

Shenao Yu — Thu, 17 Sep 2026 03:17:51 +0000

This came up in Chinese AI developer circles recently: IBM has been promoting something called Chunkless RAG, where instead of the standard document-to-chunk-to-embedding pipeline, an AI agent navigates document structure the way a human reader would. Tools like Docling parse PDFs into structured representations, preserving headings, tables, and section hierarchies, and then the agent walks that structure to find relevant content rather than doing cosine similarity on chopped-up text.

The pitch is appealing. Anyone who has watched a RAG pipeline confidently return three fragments from the middle of a dense technical spec, all of which are semantically close to the query but structurally meaningless without their surrounding sections, knows that fixed-size chunking is not a solved problem. So on the surface, this sounds like progress.

But I think the core claim, that document structure is the real problem chunking destroys, is significantly oversold.

The structure argument assumes your documents have structure

Real production corpora are messier than IBM's demos suggest. Legal contracts with inconsistent heading hierarchies. Scanned PDFs where Docling's layout parser confidently returns garbage. Internal wikis where someone decided to put the entire product spec in one enormous table. Support ticket exports. Slack export dumps. Email threads.

For these, the Chunkless approach does not obviously win. If the parser cannot reliably extract a meaningful document tree, the agent navigating that tree is just traversing noise with extra latency. You have replaced one failure mode (lost context from arbitrary splits) with another (hallucinated structure from a bad parser). The latter can be harder to detect because it looks like the system understood the document.

Fixed-size chunking with overlap is dumb, yes. But its failure modes are predictable. You can measure them. You can tune chunk size and overlap against a retrieval benchmark and watch the numbers move. The agent navigation approach has failure modes that are much harder to surface in offline evaluation.

The actual bottleneck is usually not where the cuts happen

In most pipelines I have seen discussed or documented, the retrieval precision problem is not primarily that chunks lose structural context. It is that the query and the relevant passage do not share enough lexical or semantic overlap for retrieval to work at all, regardless of how you chunked. This shows up especially for multi-hop questions where the answer requires synthesizing information from sections that are semantically distant from the query.

Chunkless RAG with agent navigation does not obviously fix this. If anything, it may make it worse: the agent still has to decide which section to navigate to first, and that decision is still a retrieval or classification problem under the hood. You have just moved the hard part one layer up and added agent call overhead on top of it.

The approaches that actually seem to move the needle on this are things like HyDE (hypothetical document embeddings), query expansion through prompt mutation, or hybrid retrieval that blends BM25 with dense vectors. One commenter on the original post mentioned a project called Knowhere that combines MinerU parsing with BM25 and prompt mutation and gets good results. That combination is not about preserving document structure as a graph; it is about improving the match between query intent and retrieved content.

What Chunkless RAG is probably good for

None of this means the idea is wrong everywhere. There is a real use case where it probably wins: long, well-structured technical documents where the user's question maps clearly to a section of the document. API reference docs. Regulatory filings with consistent section numbering. Academic papers with standard IMRaD structure. In these cases, knowing that the answer to "what is the data preparation methodology" lives in section 2.2 and navigating there directly is genuinely better than hoping your embedding space put the question and that section close enough together.

The legaltech commenter on the original post noted their team is building exactly this kind of system, and for structured legal documents, the structured navigation argument is more plausible.

But that is a narrower claim than "chunking is the wrong primitive for RAG." It is closer to "for well-structured documents, structure-aware retrieval outperforms flat chunking," which is much less surprising and much less revolutionary.

The framing bothers me more than the technique

The bigger issue is that "Chunkless RAG" as a brand encourages people to think of their chunking strategy as the core failure in their pipeline, when for most production systems it is not. Developers who are struggling with recall on heterogeneous corpora will read this, spend two weeks integrating Docling and building an agent navigation layer, and find that their benchmark numbers barely moved because the problem was never the chunk boundaries.

The useful question is not "should I chunk or not chunk?" It is "where exactly is my pipeline losing information, and what is the cheapest fix for that specific failure mode?" Sometimes that is better chunking. Sometimes it is a reranker. Sometimes it is query rewriting. Sometimes it is accepting that your source documents are too poorly structured for any retrieval strategy to work well without a preprocessing investment.

Chunkless RAG is a real direction worth watching, especially as document parsers improve. But treating document structure navigation as the successor to chunking, rather than as one useful technique among several, seems like it is going to send a lot of engineers down the wrong rabbit hole.

If you have run a direct comparison between structure-aware agent retrieval and well-tuned chunked retrieval on a messy real-world corpus, what did you actually see in the numbers?

The resume AI prompt that sounds useful but quietly teaches you to lie

Shenao Yu — Thu, 17 Sep 2026 02:24:08 +0000

This came up in Chinese AI dev circles this week: a set of seven Claude prompts for job hunting, shared with genuine enthusiasm. Most of them are fine. One of them is a trap.

The prompt in question is called the "Positioning Rewriter." You give Claude a target job title, and it rewrites your career summary and achievements to highlight relevant accomplishments and industry language. The post even includes a safety note: "Do not fabricate anything I haven't done."

That note is doing a lot of work. Maybe too much.

Here is what actually happens when you run a prompt like that. Claude does not know which parts of your resume are embellished already. It does not know whether the "led a cross-functional initiative" on line three was you coordinating two Slack threads or you genuinely owning a six-month project. It optimizes the language toward whatever sounds best for the target role, and it is very good at making ordinary things sound significant without technically lying. The result reads as confident, polished, and calibrated for keyword matching.

One comment in the thread put this more bluntly. Someone wrote that they had interviewed a candidate recently whose resume had clearly been AI-rewritten to the point of fraud. Halfway through the interview it became obvious the person could not back up what was on the page. The candidate disappeared from the call. That commenter suspected the network cable got pulled.

Now, that is one data point. But it points at something real: there is a gap between "the AI did not fabricate anything" and "the resume accurately represents what you can do on day one." Aggressive reframing fills that gap without crossing the technical line, and most people do not notice they are doing it.

The other six prompts in the set are genuinely useful in a way that sidesteps this problem. The "Hidden Role Finder" asks Claude to list job titles your experience already qualifies you for, including outside your current industry. The "Transferable Skills Translator" extracts what you are actually good at and maps it to sectors that value those things. The "Qualification Gap Analyzer" compares your resume to a specific job description and tells you honestly what you are missing. None of these require rewriting anything. They work with what exists.

Those prompts are doing something different from the Positioning Rewriter. They are helping you see your own experience more clearly, not helping you present it more persuasively. That distinction matters more than it might seem.

Job hunting advice has always contained a tension between honest self-presentation and strategic self-promotion. The usual guidance is to lead with your strongest true framing, use the vocabulary of the target field, and not undersell yourself. That is reasonable. The problem is that AI makes it much easier to slide from "better framing" to "framing that requires a version of you that does not exist yet" without realizing it happened.

The safety disclaimer in the original prompt does not actually prevent this. It tells Claude not to invent experiences. It does not tell Claude to flag when the framing is getting ahead of what the experience can support. Those are different requests, and only one of them is being made.

If you want to use AI for job search prep, the more honest version of the Positioning Rewriter looks like this: ask Claude to rewrite your summary for a target role, then ask it to identify every claim in that rewrite and cite the specific resume line that supports it. If it cannot find the line, the claim comes out. That extra step is what the original prompt is quietly skipping.

The set of prompts is not bad overall. But the framing around the Positioning Rewriter, that it will help you "stand out" for roles you might not have considered, glosses over the risk that standing out and being able to follow through are two separate problems.

So here is what I actually want to know: when you have used AI to rewrite resume language, did you find it made you more honest about your experience or less?

Claude's invisible text watermarks: what practitioners need to know right now

Shenao Yu — Wed, 26 Aug 2026 07:27:31 +0000

This blew up in Chinese AI developer communities this week, and it's worth understanding the actual mechanism before the takes calcify.

On August 2nd, Anthropic updated their support documentation to announce that all Claude models released on or after that date embed an imperceptible watermark directly into generated text. Not appended metadata. Not a visible tag. A statistical signal woven into the word choices and sentence structures themselves.

Here is what the docs say it survives: copy-paste, light editing, synonym substitution. Here is what Anthropic says degrades it: heavy rewriting, large-scale rewrites, back-and-forth translation, mixing in substantial human-written content. The detection tool is not publicly available yet.

The failure mode that matters for practitioners

If you have a pipeline where Claude drafts and a human reviews before publishing, you need to ask a hard question: is your review step heavy rewriting, or is it proofreading?

For most content pipelines I have seen described, the human step involves catching errors, adjusting tone, and fixing the occasional awkward sentence. That is not heavy rewriting. Under Anthropic's own description of how the watermark persists, that workflow probably does not clear the bar.

The practical cost: every piece you ship from that pipeline now carries a detectable signal that Claude was involved, even if a human spent significant time on it. When the detection tool ships, anyone running a check on your content gets that signal.

What coverage actually looks like

This is global, not just the EU, though the EU AI Act's Article 50(2) on AI-generated content transparency is cited as the legal basis. It covers:

Claude.ai (the consumer product)
The API (Claude Platform)
Claude Code
Cloud partners including AWS and Google Cloud

C2PA provenance metadata is attached to generated files like SVG, PNG, and JPG. That is a separate mechanism from the text watermark and follows an open industry standard, so third-party tools can already read it.

The workarounds people are trying

The Chinese dev community comments were instructive. Three categories emerged:

Translation chaining: write in English, translate to Chinese with a general-purpose translator, rewrite in Chinese naturally, translate back to English, check coherence. This adds meaningful time to every piece.

Manual retyping: if you type Claude's output by hand, no watermark transfers. This is the most expensive possible solution and scales to nothing.

Actual rewriting: use Claude for structure and ideas, write the sentences yourself. This was already the best practice recommendation before the watermark existed.

One comment in the thread raised a concern I have not seen addressed elsewhere: Claude's training data came substantially from the public web, so the statistical patterns the watermark encodes already exist in human writing. A detection system built on those patterns will generate false positives for humans who naturally write in styles that overlap with Claude's outputs. For academic integrity applications specifically, that is not a small problem.

What is still unknown

Anthropic says the detection tool is in development and technical documentation is coming. Right now you cannot run a test to see whether your specific editing workflow degrades the watermark sufficiently. That is a significant gap for anyone trying to make compliance decisions today.

The question I keep coming back to: if the detection threshold is never published, how does a human writer who genuinely uses Claude as an outlining tool but writes every sentence themselves demonstrate that their final text is primarily theirs? The burden-of-proof problem is real, and I do not see Anthropic addressing it in the current documentation.

Claude as a trading agent: what does the LLM actually add over a plain script?

Shenao Yu — Fri, 14 Aug 2026 01:36:19 +0000

This came up in Chinese AI developer communities this week, and the numbers are almost beside the point.

A physics researcher (no CS background, no finance training) connected Claude to a US brokerage account via API, wrote a simple automated trading strategy, and ran a live experiment for one week with $500. The rules were deliberately conservative: long equities only, no margin, no futures, no shorting, no HFT. "You can't earn beyond your own understanding," the author wrote, "so I only do things I can understand."

After seven trading days: $510.65, up 2.20%. VOO returned 1.88% that week. QQQ returned 2.05%.

The author is careful about this: one week is not a result, it's a data point. But 374 comments later, the conversation has moved somewhere more interesting than return attribution.

The actual problem being solved

The author runs two accounts in parallel. The Claude-managed account follows the strategy. The manually managed account is, by the author's own description, a textbook retail investor failure mode: constant overconfidence, overtrading, getting whipsawed. The AI account simply executed the plan and did not deviate.

This is a real and documented problem. Systematic strategies outperforming discretionary ones is well-established in quantitative finance, and the reason is usually behavioral rather than informational. Retail investors underperform their own funds because they trade at the wrong moments.

But here is the thing the thread mostly glossed over: retail algo trading infrastructure has been accessible for years. Alpaca offers a free brokerage API with paper and live trading. Interactive Brokers has had algorithmic access for over a decade. QuantConnect lets you backtest and deploy strategies without writing broker integration code from scratch. A fixed rules-based Python script connected to any of these would also not panic, not overtrade, and not second-guess the plan.

So the question worth asking is not "can Claude beat the market" but "what does the LLM layer add that a deterministic script would not?"

What the architecture actually looks like

The author published a simplified, anonymized version of the code. From the trade log visible in the post, the bot executes fractional share market orders across a small set of equities: LCID, KO, SMH, AAPL, among others. The bot runs daily analysis and execution automatically; the author checks logs each evening to confirm no bugs.

This is not a chat-based workflow. It requires agentic tool use with brokerage API access, where Claude is calling functions rather than answering questions. One commenter noted their own Claude just responds with "I am not a financial adviser" to any market question, which points to the gap between a raw chat interface and a purpose-built agentic pipeline with appropriate system prompting.

The cost structure: Claude Pro at $200/year, already paid for research work, making the marginal cost of the trading bot effectively zero. But again, the brokerage API and Python are free regardless of whether you use Claude or a fixed script. The $200 is not the infrastructure cost; it's the LLM cost on top of infrastructure that would exist either way.

What the LLM might actually be doing

The author describes the strategy as "very simple" but the full logic isn't public. There are two meaningfully different versions of this experiment:

Version A: Claude runs daily natural-language market analysis, synthesizes signals, and adjusts position sizing or stock selection based on that analysis. In this version, the LLM is doing something a fixed script cannot, because the inputs are unstructured and the reasoning is adaptive.

Version B: Claude executes a fixed set of rules that could be expressed as deterministic code, and the LLM wrapper is mostly convenience, handling the API calls and logging in a more readable way than raw Python.

A second commenter in the thread reported a parallel result using Claude for chart analysis and GPT for fundamental analysis on AI-related tech stocks during a rough July, coming out slightly positive. That framing suggests Version A, where the LLMs are doing genuine analytical work rather than just executing rules.

But one week of positive returns in a strong tech week does not distinguish between the two. Both versions would have produced similar outputs in a trending market.

The comments worth reading

The skeptical voices focused on two practical issues.

Taxes: short-term capital gains on frequent trades in a taxable account can get complicated fast. One commenter pointed out that accounting costs alone could erase small gains. The author appears to be trading in what looks like a Roth IRA based on the account labels visible in the post, which would sidestep this, but the thread did not confirm it.

Commission structure: the author appears to be on a zero-commission platform. Another commenter mentioned paying a minimum of $3 per trade on their broker, which would materially change the math on small fractional orders.

Geographic limitation: someone noted that A-share markets in mainland China are almost certainly incompatible with this approach, both technically and regulatorily.

What this is and is not

Seven trading days in a strong week for tech stocks is not a backtest, not a stress test, and not a risk-adjusted performance analysis. The author knows this.

What it might be evidence of is something narrower: that for a retail investor whose primary failure mode is behavioral rather than informational, any automated system that removes them from the execution decision could matter more than the strategy itself. Whether Claude specifically is the right tool for that, versus a simpler and more auditable fixed script, is a question this experiment cannot answer yet.

The author plans to continue. The more useful follow-up would be a week where the strategy signal and market direction diverge sharply. That is when you find out whether the discipline holds, and also when you find out whether the LLM's daily analysis is actually doing anything a moving average crossover could not.

What does Claude add in the execution loop that a fixed script with identical rules would not provide?

Does Agentic RAG Actually Help When Your Retrieval Is Bad?

Shenao Yu — Wed, 12 Aug 2026 09:52:30 +0000

This question surfaced in Chinese AI developer communities recently, buried in a thread framed as a one-year post-mortem on RAG. Most of the post argued that naive RAG is dead and agentic retrieval is the future. One comment near the bottom asked the question nobody had a clean answer to: if your underlying recall is poor, does giving the model control over the search loop actually recover quality, or does it just run more iterations of the same broken search?

That's the question I want to think through here, because I don't think the community has answered it honestly.

The case for agentic retrieval

Naive RAG has a specific failure pattern. The system retrieves before the model understands the task. You get top-K chunks ranked by surface similarity to the query, not by what the model actually needs to reason well. The model then has to generate through noise it didn't ask for.

Agentic retrieval changes the control flow. The model first interprets the task, then decides what to search, which tool to use, whether the results are sufficient, and whether to reformulate and search again. In theory, this lets the model escape failure modes that a fixed pipeline can't: try grep instead of vector search, narrow the query scope, call a SQL lookup, recognize when the corpus doesn't contain what's needed and say so.

The thread's author uses a good analogy: naive RAG is like a student who hasn't studied flipping through a textbook based on keyword similarity to the exam question. Agentic retrieval is like a prepared student who knows which chapter and formula to look up. The difference isn't whether you consult external material. It's whether the consultation is driven by understanding.

Where the argument gets uncomfortable

But here's the part the post-mortem glossed over. The prepared student analogy assumes the textbook is well-organized and the student can find what they need. What if the index is incomplete? What if the chunks are noisy and the embeddings are mediocre?

Agentic retrieval is still bounded by your worst tool. If all your retrieval mechanisms have weak recall on a particular corpus, more planning steps don't fix that. The model can reformulate queries, try different tools, recognize insufficient results. But if nothing in the toolbox can surface the right information, the loop terminates with the wrong answer after burning three times the compute.

The optimistic counter is that a model with genuine tool flexibility can sometimes escape this. Maybe vector search fails but grep on a known field name succeeds. Maybe a SQL query over structured metadata retrieves what semantic search missed. The model as planner can try paths a fixed pipeline never would.

The honest answer is: it depends on whether the right information is actually retrievable by any of the available tools, and most production systems haven't been tested with that distinction in mind.

The grep debate is the wrong frame

The thread spent time on grep versus vector search, and the top comment from the Chinese dev community put it well: this is a design tradeoff, not a competition. Claude Code uses grep and works well for codebase navigation because you know the symbol names. Vector search earns its place on unstructured corpora where you don't know the exact terms used to write the document you need.

The more interesting question is who picks which tool. In a fixed pipeline, the system designer decides at build time. In an agentic setup, the model decides at runtime based on the task. That's a real change in flexibility, but it's only valuable if the model's choices are better than the designer's defaults. For well-scoped tasks on clean corpora, the designer's defaults are usually fine and cheaper.

When each approach still makes sense

For simple FAQ systems, product documentation, and enterprise knowledge bases with predictable query patterns, traditional or hybrid RAG is still the right call. The cost and latency profile is better, the behavior is more predictable, and the tasks don't require dynamic tool selection.

For multi-file code understanding, complex debugging across logs and configs, or research tasks where the next query depends on what the previous one returned, agentic retrieval is worth the overhead. The task structure isn't known in advance, so fixed pipelines can't be designed to handle it.

The practical split the thread landed on is reasonable. What it didn't address is the degraded-recall case that sits between these two scenarios.

The empirical gap

What I haven't found: controlled comparisons where retrieval quality is intentionally varied and both approaches are measured against the same degraded conditions. Does agentic retrieval close the gap when the underlying search is weak, or does it require solid retrieval infrastructure to show its advantages?

If the answer is the latter, then the advice to switch from naive RAG to agentic retrieval for complex tasks is incomplete. The real prerequisite might be fixing your index first. More reasoning around bad retrieval is not the same thing as better retrieval.

That's the question the post-mortem didn't finish.

MCP Tool Bloat Hits Local Models Harder: A Constraint Worth Talking About

Shenao Yu — Thu, 06 Aug 2026 07:49:58 +0000

The MCP conversation in developer communities right now focuses heavily on enterprise governance and permission models. Those are real concerns. But there's a more immediate constraint that gets less attention: for anyone running smaller local models, MCP's token consumption pattern is a structural problem, not just an inconvenience.

Here's the shape of the issue.

Context windows are not equal

A cloud-hosted model with 128k context can absorb verbose tool descriptions from three or four MCP servers and still have room for a real conversation. A locally-run 7B model with 8k context cannot. When a single MCP server pushes 30 tool descriptions into your context, you've consumed a meaningful fraction of your budget before the first user message.

This isn't a hypothetical. Tool descriptions in practice are verbose because they need to be — the model uses them to decide when and how to invoke each tool. A terse description saves tokens but degrades routing accuracy. A thorough description costs tokens but works better. The tradeoff is real and there's no free solution.

Teams working on this have landed on a few approaches: stripping descriptions to a minimum and accepting some mis-routing, implementing dynamic tool loading that only injects relevant tools based on detected task context, or hard-capping the number of active servers per session. None of these is clean.

The permission model problem is different for local deployments

Much of the permission discussion frames MCP's all-or-nothing trust model as an enterprise security concern. For local deployments, the concern is more immediate and personal: if you're running an MCP server against your local filesystem or a local database, there's no granular scope between "the agent can read this directory" and "the agent can do anything the server exposes."

Some practitioners have built lightweight gateway layers that intercept MCP calls and apply scope rules before they reach the server. This works, but it adds a component that needs maintenance and introduces its own failure modes. The protocol not solving this at the spec level means every team that cares about it solves it differently.

Server quality varies in ways that matter for smaller models

Most published MCP servers are REST API wrappers. The tool surface reflects the original API's design, which was built for human developers, not for language model consumption. Good tool design for LLMs is different: tools should be narrow, names should be unambiguous, descriptions should front-load the most discriminating information.

For a large model with strong instruction following, a mediocre tool description is recoverable. For a smaller local model, a poorly described tool that looks similar to another tool produces consistent mis-routing. The quality gap in the ecosystem hits smaller models harder.

What the protocol actually gets right

The case for MCP is the glue-code argument, and it holds. Before a common standard, connecting an agent to multiple heterogeneous data sources meant writing custom integration logic for each one. Different auth patterns, different error handling, different tool surface conventions. MCP collapses that into a single interface pattern. Even with the current rough edges, the reduction in integration overhead is real and measurable once you're past the initial setup.

Where this leaves practitioners

The protocol is doing something genuinely useful. The implementation has concrete problems that hit local model users harder than cloud users. The ecosystem is early enough that server quality and tool design norms haven't stabilized.

The practical question for anyone building with local models right now: what's your actual strategy for keeping MCP tool descriptions from consuming your context budget, and does it hold up when you add a third or fourth server to the mix?