I Fed Two Versions of a Policy to My Knowledge Base. The Expired One Won.

Building a Knowledge Base from Scratch, EP03

The setup

EP02 built a knowledge base with three clean, hand-written FAQ files and retrieval worked great. Real companies don't hand you clean files. So this time I ingested five documents of the kind that actually cross your desk: a 2023 expense policy nobody deleted, the 2026 rewrite that contradicts it in six places, the same freight table twice (once as sectioned Markdown, once as a merged-cell Excel with a note rotting in a corner of sheet two), and a 5,000-word support manual.

All five went into the EP02 library (zj0knmrbye), no fresh index, because a real library grows one file at a time. Eight documents, 27 slices, smart chunking. Then ten retrieval queries, three failures.

One detour first

The CLI has a bl file upload command. It is not ingestion. It uploads to DashScope temporary storage, expires in 48 hours, and its mandatory --model flag gives away the purpose: binding a file to a model for multimodal calls. A parcel locker, not a warehouse. Knowledge-base ingestion still lives in the web console, drag and drop.

Failure 1: version blindness

bl knowledge retrieve --index-id zj0knmrbye --query "报销发票怎么提交"

(How do I submit invoices for reimbursement?) Top hit at 0.7226: the 2023 version, demanding paper invoices stapled and delivered to the accountant within 15 working days. The current 2026 version came second at 0.7055 with e-invoices uploaded within 30 days. The gap is 0.017.

Three queries total, both versions neck and neck every time, largest gap 0.05. The winner is decided by whose phrasing matches the question, nothing else. I named the files "2023旧版" and "2026现行版"; filenames don't participate in retrieval at all. There is no version awareness in the retrieval layer. The old version doesn't just coexist, it occasionally wins.

Failure 2: the carrier matters more than the content

Same freight information, two carriers. Ask "新疆买沙发能发货吗" (can a sofa ship to Xinjiang?) and a fragment from the raw Excel takes first at 0.7982:

时效运费:6-8天不发大件（沙发床垫餐桌）差价15

The word "sofa" is literally in there, so the score is fair. But the fragment can't answer anything: 15 RMB is the difference from what? The clean Markdown version came second at 0.6894 with a complete, forwardable rule.

Worse numbers hide in the structure. The 6KB Excel expanded into 11 of the library's 27 slices (the console's slice view counts exactly that), 40%. Its stale corner note, "以上如有变动以客服最新答复为准（2024.6 更新）", crashed the top five in six of my ten queries on high-frequency words alone. One offhand spreadsheet became the library's permanent noise floor.

And one accident I didn't plant: asking the free-shipping threshold returned both 满99包邮 from the new table and 满59包邮 from the EP02 FAQ. Two true-at-the-time documents, zero reconciliation. Retrieval serves both. It doesn't arbitrate.

Failure 3: what smart chunking actually does

The 5,000-word manual finally triggers chunking. Three findings. Titles ride along: each slice carries the headings of its sections, and they participate in matching, so heading hierarchy is load-bearing infrastructure, not typography. Cross-chapter slices exist (a slice starts with the tail of 4.5, then chapter five) but semantics survived. Tables keep their content and lose their syntax: a two-column Markdown table came back as pipe-joined rows, 场景|建议话术|, information intact, structure gone.

Also: the manual's version line, "版本：v3.2 ｜ 更新日期：2026 年 6 月", landed in slice one. Retrieval ranking ignores it. The generation-stage model reads it. In a version conflict, that line is the only thing downstream that can tell old from new.

The checklist

1. Delete expired versions before ingestion (the 2023 file outranked the 2026 one).
2. One fact, one number, reconciled across files (59 and 99 both came back).
3. Defuse Excel: unmerge, re-home corner notes, consolidate sheets, or convert to Markdown (6KB became 40% of all slices).
4. Structure long documents with headings; they become the chunker's road signs.
5. Write version and date in the body's first line, not the filename.

Zero code. The right people to execute it are the document owners, finance and ops and support leads, not the person holding the CLI.

Honest limits

The messy Excel frequently outscores the clean document; if you audit data quality by score, it will fool you. Audit by whether the returned content can answer the question. Smart chunking boundaries aren't controllable from your side, though the console does offer manual slice editing. And my library is now polluted on purpose; production libraries shouldn't be.

Try it

npm install -g bailian-cli
bl auth login --api-key sk-xxxxx

Collect your real documents, run the five-item checklist, ingest in the console, then verify: pick the rule you trust least, ask about its procedure, and check whether the top two hits are two versions of the same document. If they are, the checklist wasn't finished.

---

All ten retrievals ran for real on the Bailian CLI against a Model Studio knowledge base; request IDs are kept in the project repo. CLI install: Bailian CLI docs. API key: get one free, new accounts include free quota for 90 days.