DEV Community: 侯惠阳

Building Production Intelligent Risk Control: Streaming, Lakehouse, Rules, and Models

侯惠阳 — Sun, 23 Aug 2026 01:30:00 +0000

The hardest part of risk control is not expressing a rule such as “five failed logins from one device in ten minutes.” It is making a stable, explainable, and traceable decision when events arrive out of order, messages repeat, services fail, strategies change, and traffic peaks at the same time.

A mature system must answer four questions:

Should this request proceed now? Payments, logins, and withdrawals often need a response in milliseconds.
What has happened recently? Has an account, device, or network shown a burst, cluster, or behavioral shift?
What does history tell us? Can offline data produce reliable baselines, labels, profiles, and training examples?
Why was this decision made? Can we reconstruct the events, features, rules, and model versions used?

Production risk control is therefore neither one Spark job nor one rule repository. It is a decision system composed of synchronous serving, asynchronous stream processing, an offline lakehouse, strategy operations, and a feedback loop.

Architecture: two paths, one fact history, full replay

The key separation is between synchronous interception and real-time computation:

Synchronous online path: a Risk API reads online features, evaluates rules and models, and returns allow, deny, or review within a strict latency budget.
Asynchronous streaming path: business events enter Kafka; Spark Structured Streaming performs event-time deduplication, sliding windows, stateful aggregation, and feature updates.
Offline batch path: immutable facts land in a lakehouse on HDFS or object storage; Spark Batch handles replay, reconciliation, backfill, rule simulation, and model datasets.
Unified strategy and audit: features, rules, models, and decision records are versioned so an online outcome can be reproduced offline.

This is safer than blocking every business request on Spark. Structured Streaming normally executes in micro-batches and is well suited to second-level state updates. A transaction requiring consistently low tens-of-milliseconds latency needs an independently deployed serving path so scheduler delays, backpressure, or checkpoint pauses cannot enter the critical request path.

Establish an immutable risk fact stream

Kafka should be treated as a durable risk event log, not temporary plumbing. Payments, logins, devices, account changes, list updates, and investigation outcomes become governed events.

Every event should contain at least:

event_id          globally unique identity for deduplication and tracing
event_type        business meaning and semantic version
event_time        when the business event actually occurred
ingest_time       when the platform received it
entity_keys       user_id / account_id / device_id / ip
payload           facts rather than transient computed judgments
trace_id          correlation across request, stream, and decision
schema_version    compatible evolution
source            producing system and environment

Design topics around business facts and retention requirements, not one topic per rule. A partition key should match the stateful computation: account_id preserves account order, while network detection may require device_id. Derive separately keyed streams rather than expecting one key to serve every use case.

Use idempotent producers, strong acknowledgements, and governed Avro, Protobuf, or JSON Schema. Consumers still require idempotency because end-to-end exactly-once behavior depends on the external sink. A practical system target is at-least-once transport + business-key deduplication + idempotent writes + reconciliation and replay.

Do not poll operational databases aggressively. Capture account state, lists, and merchant reference data through log-based CDC. When a database update and a domain event must agree, use a Transactional Outbox to avoid the dual-write gap between committing data and publishing a message.

Define sliding windows in event time

Risk windows must use event_time, not the time Spark happens to process a record. Mobile disconnection, network delay, and Kafka backlog cause disorder; processing-time logic can produce different outcomes for the same facts after a restart.

Common windows include:

1-minute window sliding every 10 seconds: transaction burst
10-minute window sliding every minute: failures and amount velocity
24-hour window sliding every 15 minutes: accounts per device
7-day state: new beneficiary, familiar geography, behavioral baseline

Window length expresses observation range; slide interval expresses update frequency. Smaller slides increase state and compute cost. Derive them from the maximum acceptable detection delay rather than setting every feature to one second.

A watermark says how long the system is willing to wait for late data and when it may clear state. It does not guarantee that all records arrive within that delay. Choose it from measured lateness—perhaps covering 99.9% of events—and route records beyond the threshold to a late-data path for backfill, audit, and monitoring rather than silently discarding them.

events = (
    spark.readStream.format("kafka")
    .option("subscribe", "risk.payment.v1")
    .load()
    .transform(parse_and_validate)
    .withWatermark("event_time", "10 minutes")
    .dropDuplicatesWithinWatermark(["event_id"])
)

velocity = events.groupBy(
    window("event_time", "10 minutes", "1 minute"),
    "account_id",
).agg(
    count("*").alias("tx_count_10m"),
    sum("amount").alias("tx_amount_10m"),
)

This illustrates semantics, not a copy-ready production job. Real pipelines also need schema validation, quarantine, state bounds, isolated checkpoint locations, rate control, monitoring, and idempotent sinks.

Streaming features must leave Spark memory

Window results, recent entity sets, velocity metrics, and risk counters belong in a low-latency online store read by the synchronous decision service. Redis, Cassandra, HBase, or an existing highly available KV store can work, but first define a feature contract:

feature_name + entity_key + value
event_time + computed_at
definition_version + producer_job_version
ttl + freshness_sla

Writes must be idempotent. With foreachBatch, use a query/batch identity or business window key, plus an upsert, transactional table, or commit log. Spark checkpoints can restore source offsets and computation state; they do not automatically make an arbitrary external database exactly-once.

Define stale-feature behavior explicitly. When the store is unavailable or freshness exceeds its SLA, the engine must distinguish a true zero from missing data and choose a conservative rule set, fallback model, manual review, or constrained approval according to risk.

Rules and models are complementary

A mature decision combines several mechanisms:

Hard rules: regulation, deny lists, unambiguous prohibitions
Velocity rules: counts, amounts, linked entities, time windows
Model scores: fraud, anomaly, account-takeover probability
Policy orchestration: matches + scores + business cost → action

Rules fit deterministic, explainable, rapidly changing constraints. Models fit multivariate patterns that are difficult to hand-code. The final action should incorporate exposure, customer value, false-positive cost, and available review capacity—not only one score.

A rule platform needs version, state, priority, audience, effective interval, author, approver, and rationale. The release path should be draft → tests → historical replay → shadow → canary → full rollout. Record the rule version and input snapshot for every match, but avoid synchronous heavy audit writes in the high-QPS path; publish a reliable decision event and persist it asynchronously.

Keep the rule DSL constrained and analyzable. Arbitrary scripts should not access the network or query production databases per transaction. Reference data should arrive through governed interfaces or preloaded snapshots.

Modernize the Hadoop layer into a governed lakehouse

HDFS remains appropriate for distributed storage in on-premises Hadoop deployments; cloud systems commonly use object storage. In either case, bare Parquet directories are a weak sole data-management layer. An open table format such as Iceberg adds atomic commits, schema and partition evolution, snapshots, and time travel—strong foundations for replay and audit.

Use at least three layers:

Raw facts: append-only Kafka and CDC archives with original schema versions.
Conformed detail: deduplication, master-data mapping, privacy treatment, and consistent time semantics.
Features and labels: datasets for rule replay, training, cases, and analytics.

Spark Batch performs reconciliation, historical windows, feature backfills, fraud labels, and strategy evaluation. Batch and streaming should share feature definitions or be generated from the same declarative logic. Training datasets require point-in-time joins that use only information available at decision time, preventing future leakage.

Streaming into Iceberg creates frequent snapshots and small files. Run separate maintenance for compaction, manifest rewriting, snapshot expiration, and metadata monitoring. Retention is also a product decision: raw facts, features, decisions, and sensitive identifiers need distinct retention and deletion policies.

Every decision must be replayable

Persist a Decision Record containing:

decision_id / request_id / trace_id
event_id and raw-fact location
feature values, event times, and definition versions
matched rules and versions
model name, version, score, and threshold
final action, reason code, and human override
decision time, latency, and degradation state

Audit supports compliance and operations. It lets the team explain why a customer passed yesterday and failed today, whether a rule caused false positives, and whether model deterioration came from drift, stale features, or a business distribution shift.

Use stable reason codes. Operational explanations and customer-facing explanations should be designed separately so staff can act on them without revealing exploitable strategy details.

Availability requires explicit degradation semantics

Production design defines behavior under failure:

Failure	Recommended behavior
Kafka backlog	Continue with timestamped recent features; monitor freshness and activate conservative policy
Online store unavailable	Use local cache and core rules; send high-risk requests to review or deny
Model timeout	Circuit-break quickly; fall back to rules and the last stable model
Bad rule release	Roll back immediately; require dual approval and shadowing for high-impact rules
Streaming job failure	Recover from durable checkpoints; replay from Kafka and the archived facts
Data-quality incident	Quarantine bad events, freeze affected features, alert ownership, and stop propagation

Deploy the decision service across failure domains and assign an end-to-end latency budget plus dependency timeouts. Kafka, checkpoints, online stores, and lakehouse each need RPO/RTO. Disaster recovery is not “the process restarted”; periodically prove recovery from offsets, checkpoints, and immutable history.

Observe systems, data, and decisions

CPU and latency are insufficient. Operate three metric families:

System: QPS, p95/p99 latency, errors, Kafka lag, batch duration, state size, checkpoint failures.
Data: volume, schema failures, duplicates, lateness, nulls, feature freshness, online/offline consistency.
Risk: allow/deny/review rate, rule hits, score distribution, false positives, fraud loss, queue depth, policy value.

Alerts should describe impact. Rising lag matters because particular features crossed their SLA and a number of decisions entered degradation—not simply because a threshold turned red.

A practical delivery sequence

1. Facts and the minimum decision loop

Standardize event schemas, IDs, event time, and reason codes.
Ingest one critical event through Kafka and archive it unchanged.
Deploy a decision API with a small hard-rule set and complete Decision Records.
Establish baselines for latency, errors, lag, and decision distribution.

2. Real-time state

Implement two or three high-value windows in Structured Streaming.
Add watermarks, late-data routing, deduplication, and idempotent sinks.
Introduce an online feature store and freshness SLAs.
Exercise restart, backlog, and online-store degradation.

3. Rule operations and replay

Add rule versioning, approval, shadowing, canary, and rollback.
Build Iceberg fact, feature, label, and decision tables.
Backtest strategies and validate online/offline feature consistency.
Close the loop with investigator outcomes.

4. Models and continuous optimization

Build leakage-free datasets with point-in-time joins.
Add model registry, explanation, shadowing, canary, and drift monitoring.
Optimize thresholds across loss, false positives, and operating cost.
Review overlapping rules, unused features, technical debt, and recovery readiness.

The standard for production maturity

A production risk system should demonstrate that:

synchronous latency is bounded and streaming cannot stall the transaction path;
Kafka holds replayable facts, with explicit duplicate, disorder, and lateness semantics;
online and offline features share definitions, while rules and models are versioned;
every decision is explainable, auditable, and reproducible;
each dependency has a deliberate degradation policy;
success is measured through loss prevented, false-positive cost, and customer experience.

Risk control is not the pursuit of maximum rejection. It is the discipline of making cost-aware, evidence-based, accountable decisions from changing information under time pressure. Kafka, Spark, Hadoop, and a rule engine are infrastructure. The real system is the learning loop that joins live facts, historical evidence, and business judgment.

From Query Understanding to Knowledge-Driven Recommendations

侯惠阳 — Sat, 22 Aug 2026 09:30:00 +0000

In 2022, I wrote a technical proposal for a new tax and finance product. It covered query understanding, recommendation systems, and the fragmentation of domain knowledge. Like many architecture documents of its time, it also contained a long inventory of techniques: tokenization, intent classification, collaborative filtering, ranking models, entity extraction, and knowledge graphs.

Most of those model names are no longer the interesting part. What has survived is a priority decision I made at the beginning:

Structured domain knowledge > query understanding > recommendation ranking.

Ranking is not unimportant. The point is that in a regulated, time-sensitive domain, a system's ceiling is first determined by whether it can represent the knowledge correctly, then by whether it can understand the user's actual situation, and only then by how well it orders the candidates.

If the knowledge base is incomplete, perfect query understanding still retrieves incomplete answers. If the system misses jurisdiction, entity type, or effective date, a more confident model can become more dangerous. If those foundations are weak, a sophisticated ranker merely amplifies errors with greater precision.

Why tax and finance search is unusually difficult

General search often competes on relevance. Tax and finance search must satisfy several constraints at once.

Queries are underspecified. “Can a small business deduct this?” is not a complete business question. The legal entity, jurisdiction, transaction type, invoice type, and effective date may all change the answer.

Domain language is dense. A concept may have a statutory name, a professional abbreviation, an old name, and several everyday expressions. Words that look similar in general language may carry sharply different accounting or legal meanings.

Answers expire. Tax knowledge is not a static encyclopedia. Rules are issued, amended, superseded, and interpreted differently across jurisdictions. A relevant but expired result is not merely low quality; it is wrong.

Ranking is multi-objective. Users need authoritative source material, understandable explanations, and actionable guidance. The system must balance authority, relevance, readability, recency, and context.

Every conclusion needs provenance. A useful system may say that it lacks enough information. It should not produce a definitive answer that cannot identify its evidence and scope.

This is why a domain search product is not simply a vector database with an LLM on top. It is a decision system spanning knowledge governance, query interpretation, retrieval, ranking, and risk control.

Query understanding should produce a contract

Query understanding is traditionally divided into normalization, tokenization, correction, expansion, entity recognition, and intent classification. These capabilities still matter, but a production system needs a stable, auditable output that every downstream component can consume.

For example:

{
  "rawQuery": "How does a small business in Chongqing file VAT this year?",
  "normalizedQuery": "Chongqing small business 2026 VAT filing",
  "intent": "tax_policy_and_procedure",
  "entities": {
    "region": "Chongqing",
    "taxType": "VAT",
    "businessType": "small_business"
  },
  "constraints": {
    "effectiveAt": "2026-08-06",
    "authorityLevel": ["national", "municipal"]
  },
  "rewrites": [
    "Chongqing VAT relief for small businesses",
    "Chongqing VAT filing procedure for small businesses"
  ],
  "confidence": 0.91,
  "needsClarification": false
}

This object is more useful than an isolated label such as policy_lookup. Retrieval, filters, rerankers, answer generation, logging, and evaluation can share the same contract. When the system fails, the contract also helps reveal where it failed.

If a decisive constraint is missing, the right behavior is not to guess. A question about deducting the tax on a company vehicle may require the entity type, business purpose, invoice type, and transaction date. Asking one good clarifying question is safer than generating a polished answer to the wrong problem.

Replace algorithm inventories with an observable pipeline

My 2022 design used a Pipe-Filter architecture: a stable core moved data through plug-in algorithm components. The idea remains useful, but each stage now needs an explicit contract, evaluation method, trace, and fallback.

A production pipeline can be organized as follows:

Normalize input: characters, units, dates, regulation identifiers, jurisdiction aliases, and entity types.
Detect safety and scope: sensitive input, unauthorized requests, and questions the product cannot answer.
Parse intent and entities: task, business entities, temporal constraints, and jurisdiction.
Rewrite and decompose: correct errors, map terminology, and split complex questions into verifiable subqueries.
Retrieve through multiple channels: lexical, semantic, rule-based, graph-based, and behavioral retrieval.
Fuse and deduplicate: combine channels without letting one scoring system dominate.
Rerank under business constraints: semantic relevance, authority, validity, and user context.
Validate confidence: evidence coverage, policy status, citation consistency, and conflicting sources.
Present the result: answer, evidence, applicable scope, uncertainty, and next action.

Every stage should record its version, latency, input and output summaries, and failure reason. When a model times out, the system should degrade to lexical search and deterministic filters rather than taking the entire experience down.

Do not choose between lexical and vector retrieval

Domain search is naturally hybrid.

Lexical retrieval is excellent for regulation identifiers, technical terms, exact phrases, and numeric conditions. BM25 remains a strong baseline; embeddings do not make exact matching obsolete.

Vector retrieval handles conversational language and semantic similarity. A user may ask whether a hotel invoice from an employee trip is deductible while the source document discusses “input VAT deduction for accommodation services.” The expressions differ, but the underlying intent is close.

Rule-based retrieval enforces constraints that must not be approximated: jurisdiction, effective date, entity eligibility, and policy status.

Knowledge-relation retrieval follows connections such as policy, tax type, legal entity, transaction, required document, and filing procedure.

Behavioral retrieval becomes valuable only after sufficient, compliant, and de-biased feedback exists. Clicks are not proof of correctness; long dwell time may simply mean that the content is hard to understand.

The channels should not be concatenated blindly. A stable rank-fusion method can combine candidates before a reranker handles the top set. Fusion optimizes for coverage, reranking for order, and deterministic business rules protect boundaries that relevance must never override.

Recommendation is a decision process, not a model

The older proposal summarized industrial recommendation as “algorithm + recall + rank.” I would now make the stages explicit:

Candidate generation
  → eligibility filtering
  → deduplication and diversity
  → coarse ranking
  → fine reranking
  → policy constraints
  → explanation and presentation

Candidate generation answers “what might apply?” Filtering answers “what definitely does not apply?” Ranking decides “what should the user see first?” Explanation answers “why is this being shown?”

For tax and finance content, a conceptual score should include more than semantic relevance:

FinalScore =
  SemanticRelevance
  + AuthorityWeight
  + FreshnessWeight
  + ContextMatch
  + EvidenceCoverage
  - ExpirationPenalty
  - ConflictPenalty

This is not a universal formula. It is a design reminder: a semantically perfect result must still be rejected if it is expired, inapplicable to the jurisdiction, or unsupported by an authoritative source.

The knowledge layer sets the long-term ceiling

Knowledge fragmentation was the highest-priority problem in the original proposal. It matters even more now.

Acquire with provenance

Ingest regulations, official interpretations, procedural guides, cases, and internal material while preserving the source, publisher, publication date, effective period, and canonical URL.

Structure only what affects decisions

Do not begin with an enormous ontology. Start with the fields that determine applicability:

Policy → issuing authority
Policy → effective and expiration dates
Policy → jurisdiction
Policy → eligible entity
Policy → tax type
Policy → business event
Policy → supersedes or cites another policy

Resolve identity and conflict

The same policy may be mirrored, summarized, or incorrectly interpreted by multiple sources. The system must align entities, merge versions, apply authority rules, and expose conflicts instead of passing duplicate text to a model and hoping it decides correctly.

Evaluate and update incrementally

Knowledge needs a computable health state: source authority, current validity, field completeness, conflicts, and last review time. A policy update should trigger incremental ingestion and reevaluation of affected answers.

Where LLMs belong

LLMs have dramatically improved query interpretation, semantic rewriting, decomposition, and answer composition. They should not become an unconstrained center of the architecture.

Good uses include:

extracting structured intent and entities from conversational queries;
generating alternative retrieval expressions;
decomposing complex questions into verifiable subproblems;
semantically reranking retrieved evidence;
composing an answer from confirmed sources;
identifying conflicts and escalating them for review.

LLMs should not be the sole mechanism for:

determining whether a regulation is currently valid;
enforcing access control and data isolation;
authorizing high-risk business actions;
inventing missing facts when evidence is absent;
replacing deterministic jurisdiction, date, and eligibility filters.

The working rule is simple: let models handle ambiguity and systems handle certainty. Let models propose judgments, and let evidence and rules constrain them.

Evaluation compounds faster than model selection

Without an evaluation set, architecture work eventually becomes subjective tuning against a few impressive demo queries.

Build a layered evaluation set from real user questions:

Layer	Primary metrics	Question
Query understanding	Intent Accuracy, Entity F1	Did the system understand the task?
Retrieval	Recall@K	Did the correct evidence enter the candidate set?
Ranking	NDCG@K, MRR	Did the best result appear early enough?
Answer	Evidence coverage, citation accuracy	Is every claim supported?
Business	Resolution rate, reformulation rate, escalation rate	Was the problem actually solved?
Risk	Expired citation rate, unauthorized access, unsupported claims	Did the system respect its boundaries?

The set must include ambiguous language, typos, multiple intents, cross-jurisdiction conflicts, expired rules, and contradictory sources. These cases—not the clean demos—determine whether a system is ready for production.

How I would start today

I would not begin by training a model or drawing an architecture with every known algorithm. I would proceed in this order:

Select one bounded tax task and define correct outcomes and unacceptable errors.
Build a minimal knowledge model around provenance, time, jurisdiction, entity, and policy relationships.
Establish a hybrid BM25 and vector retrieval baseline.
parse each query into a stable contract and clarify low-confidence cases.
Add reranking, citations, and deterministic validation.
Turn real failures into an evaluation set with weekly regression runs.
Add knowledge graphs, personalization, or multi-agent orchestration only when the baseline reveals a concrete bottleneck.

Looking back at the 2022 design, my conclusion is not that the old algorithms became useless. It is that architecture decisions should outlive model names.

A trustworthy domain system knows where its knowledge came from, to whom it applies, and when it is valid. It understands the problem the user is trying to solve. And when the evidence is insufficient, it knows when to stop.

Those properties still matter more than the name of the model.

How to Build an AI-Native Team

侯惠阳 — Sat, 22 Aug 2026 05:30:00 +0000

Whenever I set up a new computer, I restore the development environment first: GoLand, IntelliJ IDEA, VS Code, multiple JDKs, Python, Node.js, MySQL, Redis, Docker, Postman. For years, that ritual took two or three days and felt like part of being an engineer.

This time I stopped halfway through.

When had I last opened GoLand? Or IntelliJ? I was still building software, so why had the tools that once defined my productivity stopped being my primary entry point?

The answer was not that coding had disappeared. The structure of the work had changed. I increasingly described the outcome, supplied context, reviewed a plan, and verified the result. An agent explored the repository, changed the code, ran the tests, and handled repetitive execution. Languages and frameworks still mattered, but they were moving from tools I had to operate personally to environments I had to understand and judge.

The change was not the IDE. It was the basic unit of work.

The same shift will not remain confined to engineering. Once AI can interpret context, use tools, execute multi-step tasks, and revise a plan based on results, the organizational question is no longer “Should we give employees an AI assistant?” It becomes:

How should we redesign a team when AI can participate as an execution unit?

AI-enabled is not AI-native

Many AI transformations begin with software procurement and end there. Meeting notes are summarized, marketing copy is generated, developers receive code completions, and leadership sees adoption numbers rise. The company declares itself AI-powered.

Yet the operating system remains unchanged. Tasks still move through layers of handoffs. Context remains trapped in chat threads. Decisions still wait for meetings. People remain the transport layer between every step, while AI merely makes a few local steps faster.

An AI-native team is designed with AI as a first-class participant from the beginning.

AI-enabled	AI-native
Adds an assistant to the old process	Redesigns the process around the outcome
AI drafts; people keep moving the work	AI executes within explicit authority
Prompts are personal tricks	Context is an organizational asset
Measures users and calls	Measures quality, cycle time, cost, and outcomes
Humans repair errors after the fact	Evaluation, permissions, and escalation are designed upfront
Knowledge lives in documents and heads	Knowledge is retrievable, versioned, and reusable

A useful test is:

If AI disappeared from a core workflow tomorrow, would the process become slower, or would it have to be redesigned?

If it only becomes slower, the organization is probably AI-enabled. If the process can no longer operate in the same way, AI has entered the organizational architecture.

A new operating system for work

AI-native transformation is not a single tool project. Five layers must evolve together.

1. Goals

Agents can execute the wrong objective with remarkable efficiency. Clear objectives therefore matter more, not less. Every task needs a problem, an intended beneficiary, constraints, and an acceptance test.

“Research our competitors” is not an executable goal. “Compare the pricing, core workflow, and recurring customer complaints of three competitors; recommend two differentiated hypotheses for next quarter; cite the evidence behind every conclusion” is much closer.

2. Workflows

Do not insert AI into one step of an inherited process. Work backward from the result. Which steps can run in parallel? Which information can be collected automatically? Which judgments must remain human? How does the process degrade? Where does the output go next?

3. Human-agent responsibilities

The useful question is not “Which job will AI replace?” It is “How should judgment and execution be divided inside each job?”

People are better positioned to own goals, taste, priorities, responsibility, relationships, and high-risk exceptions. Agents are well suited to search, synthesis, generation, batch execution, state synchronization, and work with clear verification.

4. Organizational context

If every new session requires an explanation of the company, product, vocabulary, and past decisions, the organization has hired an intern who loses all memory every morning. Principles, domain language, processes, decisions, examples, counterexamples, and permissions must become retrievable and maintainable context.

5. Evaluation and governance

Without evaluation, the team can only say that work “feels faster.” Without governance, greater speed merely spreads errors faster. Evaluation, authorization, audit, fallback, and human escalation must be designed with the workflow.

The new unit of work: a person with agents

Traditional organizations expand execution capacity by adding people and layers. Information moves through managers, and coordination cost rises with headcount.

An AI-native organization increasingly consists of small human-agent units. A product leader may coordinate research, data, prototyping, and writing agents. A developer may delegate repository exploration, implementation, testing, and documentation. The person retains ownership of the objective while agents multiply execution bandwidth.

The important variable is not the number of agents. It is clarity of responsibility:

Human: define goals → set boundaries → review critical judgment → own the outcome
Agent: plan → use tools → execute → submit evidence
System: record → evaluate → enforce permissions → trigger escalation

Teams often build a complex multi-agent network too early. A deterministic workflow is usually more reliable when the task is well understood. A single agent should remain a single agent unless the work genuinely benefits from parallel specialization, context isolation, or independent verification.

Complexity should be pulled by the problem, not pushed by enthusiasm for the technology.

Six principles for AI-native workflows

1. Begin with acceptance criteria

Define “done” before choosing a model. Tasks with objective checks are strong candidates for agentic execution: tests pass, accounts reconcile, required fields exist, citations resolve, or outputs match a schema.

2. Treat context as capital

Better output comes not only from better models but from better context. Context should be managed like code: sourced, versioned, owned, reviewed, and retired when stale.

3. Prefer simple, composable patterns

Use deterministic code for deterministic work. Invoke models where ambiguity requires judgment. Give an agent control of the path only when the path cannot be known in advance. Simpler systems are easier to evaluate, debug, and operate economically.

4. Match autonomy to risk

Low-risk, reversible, verifiable actions can run automatically. High-risk, externally visible, or irreversible actions require approval. Drafting an internal summary and sending a contract to a customer should not share the same autonomy level.

5. Make every step observable

At minimum, record the objective, context used, model and tool versions, critical decisions, result, human intervention, latency, and cost. Without traces, every production failure becomes guesswork.

6. Turn failure into system memory

If a human correction does not become a rule, example, or evaluation case, the same failure will return. Organizational learning begins when an individual correction becomes a reusable system capability.

Do not begin with an isolated “AI department”

When transformation is delegated entirely to an innovation unit, one of two things usually happens. The innovation team builds impressive demos without understanding operational constraints, or business teams treat AI as somebody else's project and refuse ownership of the outcome.

A healthier structure has four sources of accountability.

Leadership owns direction and boundaries

Leadership defines why the organization is changing, which workflows matter, what risks are acceptable, and how much capacity will be committed. Leaders also need to use the new operating model themselves. A company notices quickly when executives demand AI adoption while continuing to manage through meetings and cascading status reports.

A platform team owns shared infrastructure

This team provides model access, tools, identity, permissions, knowledge, evaluation, cost controls, and safety foundations. It does not own every business use case. Its job is to eliminate duplicated infrastructure work.

Business teams own outcomes

The people closest to a process understand its decisions and edge cases. Business owners need authority to redesign the process and accountability for quality. Responsibility cannot be transferred to a model vendor or platform team.

AI champions accelerate learning

Practitioners inside real teams help colleagues cross the initial barrier, share successful and failed patterns, and connect business needs with platform capabilities. Champions are catalysts, not a new approval layer.

Choose the first workflow carefully

Do not begin with a company-wide mandate. Begin with a real workflow that is valuable and safe enough to learn from.

Prioritize work that is:

Frequent: it occurs every day or week;
Time-consuming: much of the effort is search, synthesis, copying, or coordination;
Verifiable: output quality has a reasonably clear standard;
Controllable: failure is inexpensive or can be intercepted before action.

A rough prioritization model is:

Opportunity = frequency × time per task × verifiability × reuse ÷ risk

Do not measure only minutes saved. Track first-pass acceptance, rework, escalation, total cost per accepted result, and end-to-end cycle time. Doubling generation speed while tripling review effort is not a win.

Move beyond a prompt library

Prompt templates are useful, but they are only the shallowest layer of organizational context. A mature context system includes:

Principles: how we make tradeoffs and what is prohibited
Domain language: products, customers, metrics, and terminology
Processes: steps, inputs, outputs, and owners
Decision history: what was chosen, rejected, and why
Examples: strong outputs and known failure modes
Tools and permissions: what an agent may read or change
Evaluation sets: real tasks, expected outcomes, and boundaries

This material requires maintenance. Stale context is often more dangerous than missing context because it creates consistently wrong behavior.

Organizational memory is also not equivalent to putting every file into a vector database. Different knowledge has different lifecycles and access levels. Product metrics may change daily, policies monthly, architecture decisions require version history, and personnel data requires strict isolation. Retrieval is the entry point; governance is the system.

Governance is a prerequisite for scale

When an agent can only draft text, the primary risk is content quality. When it can send messages, change production systems, move money, or update customer records, the risk becomes action.

Design permissions around actions rather than around tools:

Permission	Example	Behavior
Automatically allowed	Read public sources, draft, run read-only analysis	Execute and log
Conditionally allowed	Edit internal docs, run tests, create tickets	Execute when rules pass
Approval required	External communication, production changes, financial or contractual actions	Review plan and impact first
Prohibited	Bypass audit, expand its own access, read unrelated sensitive data	Hard block

Every consequential workflow also needs least privilege, complete auditability, an immediate stop mechanism, and a path back to a safe state.

Human escalation is not failure. The dangerous system is the one that does not know when to ask for help. When goals are ambiguous, evidence conflicts, authority is missing, risk rises, or verification fails repeatedly, the agent should pause and hand over completed work, evidence, and the unresolved decision.

A practical 90-day path

Days 1–30: diagnose and ignite

Start with one business team, not the entire company.
Map three frequent workflows and record current quality, time, and cost baselines.
Standardize two or three tools and publish clear data boundaries.
Name a business owner, platform partner, and AI champion.
Redesign one low-risk, frequent, verifiable workflow.

Deliverables: baseline, use-case priorities, tool and data rules, and one working workflow.

Days 31–60: redesign process and context

Expand to three production workflows.
Define objectives, inputs, outputs, permissions, and escalation for each.
Build the minimum organizational context and a library of validated examples.
Convert real failures into the first evaluation set.
Review failures as openly as successes each week.

Deliverables: three workflow specifications, context v1, evaluation set v1, and a weekly quality report.

Days 61–90: institutionalize and expand

Run evaluations whenever models, prompts, tools, or workflows change.
Establish approvals, audits, cost controls, and incident response.
Compare cycle time, quality, cost, and escalation against the original baseline.
Extract reusable components instead of copying entire workflows.
Use evidence to expand, redesign, or stop each use case.

Deliverables: evaluation dashboard, governance mechanism, quarterly review, and the next roadmap.

Measure outcomes, not AI activity

The number of AI users and tokens consumed are adoption metrics, not value metrics.

More useful measures include:

Cycle time: from request to accepted outcome;
First-pass acceptance: results approved without rework;
Cost per accepted result: model, infrastructure, and human review combined;
Escalation rate: where people repeatedly need to intervene and why;
Recovery time: how quickly the workflow returns to a safe state;
Reuse: how often another team can adopt an existing capability;
Business outcome: revenue, conversion, quality, customer satisfaction, or risk reduction.

Metrics should not be used to prove that AI was the right decision. They should help the organization continuously decide what to automate, what to keep human, and what not to do at all.

AI-native organizations still amplify human judgment

AI will continue to reduce the cost of execution. It will not make the objective correct, create taste, or absorb accountability.

The best teams may not be those with the most agents. They will be the teams that define problems clearly, provide dense and accurate context, verify outcomes quickly, and convert failures into organizational memory.

An AI-native team is therefore not simply a team with fewer people.

It is a way to execute human judgment at greater scale while keeping the system controllable, verifiable, and accountable.

I did not install every old tool on that new computer. They had not become worthless. My role had moved from operating each tool to designing a system in which people and AI could work together.

For an organization, becoming AI-native begins with the same shift.

My AI Programming Workbench: Orchestrating Models Instead of Hunting for One Best Tool

侯惠阳 — Sat, 22 Aug 2026 01:30:00 +0000

My programming toolbox is no longer a single product.

Locally, I run Ollama with Qwen3.5:9B. In the terminal I use OpenCode, Claude Code, and Codex, and I also route Kimi K3 through OpenCode. For complex desktop work, I use the desktop version of GPT. These tools differ in capability, cost, privacy, and interaction style, but I have become convinced that output quality depends less on finding one “best model” than on organizing several models well.

I do not want one agent to receive an ambiguous request and independently design, implement, and approve the result. My workflow resembles a small engineering team. I establish shared project context, ask a strong model to develop an architecture, and personally review the critical decisions. I then divide the work into bounded tasks, assign them to different tools, ask other tools to review the implementation, and close the loop with tests, type checks, static analysis, and observed behavior.

The point is not to open more terminals. It is to build an auditable delivery pipeline:

Requirements and constraints
          ↓
Project context (AGENTS.md / CLAUDE.md / documentation)
          ↓
Design (architecture, domain boundaries, risks, acceptance criteria)
          ↓
Human review and decision
          ↓
Task decomposition → execution → independent cross-review
          ↓
Automated checks + scenario validation
          ↓
Corrections become durable project context

A layered workbench, not a list of tools

Different models belong at different points in the workflow. Sending every task to the most expensive model wastes resources. Giving a high-risk design decision to a small local model can make the team move quickly in the wrong direction.

Layer	Tools I use	Best suited to
Local exploration	Ollama + Qwen3.5:9B	Code explanation, local search, drafts, low-risk edits, and assistance with sensitive context
Open orchestration	OpenCode + Qwen / Kimi K3	Model switching, experiments, cost-capability comparisons, and replaceable terminal workflows
Deep design	Claude Code	Broad repository comprehension, architecture proposals, domain modeling, and migration planning
Engineering execution	Codex / Claude Code / OpenCode	Repository exploration, implementation, refactoring, testing, repair, and review
Desktop collaboration	Desktop GPT	Cross-file and visual reasoning, research synthesis, long-running collaboration, and editorial refinement

This is not a permanent ranking. Models and products will change. The stable rule is to route work according to risk, context size, verifiability, cost, and privacy requirements.

Low-risk, reversible tasks should usually start with a local or lower-cost model. Cross-module reasoning deserves a stronger model. Architecture, data migration, security, and production changes require a human decision. No model output substitutes for acceptance evidence.

Start with project context, not a clever prompt

When I initialize a project, I create project-level instructions such as AGENTS.md and CLAUDE.md. They describe the stack, repository structure, operating commands, architecture, and engineering constraints. This preparation looks like documentation, but it establishes the ceiling for every agent that follows.

Without shared context, every tool rescans the repository, guesses conventions, and may invent a different implementation style. Useful project context should answer:

Purpose: the problem, users, and current stage
Stack: languages, frameworks, versions, and major dependencies
Architecture: module responsibilities, dependency direction, hard boundaries
Domain language: core concepts, terms, and invariants
Commands: install, start, test, type-check, and build
Conventions: naming, errors, logging, tests, and commits
Safety: sensitive data, secrets, external calls, and prohibited actions
Definition of done: mandatory quality gates for every change

Context files should not become thousand-line dumping grounds. AGENTS.md is a good home for stable, executable, cross-tool rules. CLAUDE.md can contain Claude-specific interaction guidance. Detailed architecture belongs in focused documents linked from the entry point. Rules should be concise, testable, and updated with the code.

One principle matters greatly: do not duplicate the same rule across several files. Duplication eventually becomes contradiction. Shared facts need one canonical source; tool-specific files should contain only the differences.

Let a strong model design, but keep architecture accountable to people

For medium and large changes, I often ask Claude to propose the architecture, domain boundaries, DDD model, and implementation path. AI is excellent at expanding the problem space quickly: identifying affected modules, proposing alternatives, finding failure paths, exposing hidden dependencies, and turning vague requirements into a structure that can be debated.

A polished design document is not automatically a correct design. I review five things:

Does it solve the real problem? Or has it added complexity to display architectural sophistication?
Are the boundaries natural? Do they follow business change, or merely apply DDD vocabulary?
Are dependencies controlled? Are data flow, transactions, recovery, and compatibility explicit?
Can it ship incrementally? Or does it require a high-risk big-bang rewrite?
How will we know it works? Are test, performance, migration, and business acceptance criteria defined before implementation?

DDD is a way to organize complex business knowledge, not a decorative default. A short-lived module with straightforward CRUD does not need aggregates, repositories, and extra layers for their own sake. DDD earns its cost when rules are complex, language is contested, and boundaries must evolve over time.

I therefore ask the model to explain why the design fits, what alternatives exist, and when the recommendation should not be used. Good architecture is not an impressive diagram. It is a set of decisions that can be challenged, traded off, and verified.

Decompose work around boundaries and evidence

Once the design passes review, I do not send another agent a one-line request to “implement the whole feature.” I create task packets that are independently understandable and verifiable, with as little overlapping write scope as possible.

A useful task packet includes:

Objective: the user or system behavior that must change
Scope: permitted modules and explicit non-goals
Context: relevant decisions, interfaces, and conventions
Acceptance: tests, examples, performance, or observable behavior
Risk: compatibility, data, security, and rollback requirements
Delivery: code, tests, documentation, and unresolved questions

Tasks should be split by domain boundary, module, read/write responsibility, or verification concern—not mechanically by file. When multiple agents edit the same central file, time saved in generation is often lost to conflict resolution and context synchronization. Parallelism pays only when boundaries are clear and write scopes barely overlap.

I also control how much context each task receives. Giving an agent the entire repository is not always helpful. Irrelevant information dilutes constraints and increases accidental associations. Stable project rules should be shared, while each task receives only the local context necessary for its objective.

Cross-review: different tools should search for different failures

I ask other tools to cross-review the implementation. The value is not that two models can vote. It is that an independent reviewer can challenge the implementer's assumptions.

An effective review is more specific than “look over this code.” I ask reviewers to check:

conformance with the requirement and acceptance criteria;
architecture boundaries and hidden coupling;
error paths, concurrency, idempotency, and resource cleanup;
security, privacy, permissions, and dependency risk;
whether tests verify behavior instead of mirroring implementation;
whether a simpler and more maintainable solution exists.

The reviewer should provide evidence: a file and location, trigger condition, impact, reproduction path, and suggested correction. The implementation tool can then address the finding and rerun validation. When models disagree, I do not decide by brand. I return to requirements, code, tests, and reproducible facts.

There is also a danger of agents endorsing one another. If several tools inherit the same faulty assumption, they can confidently agree on the wrong result. Independent verification should change either the information source or the validation method: one model performs static review, another runs tests and scenarios, and a person reviews the critical business judgment.

The final judge is a quality gate, not a model

Code that looks plausible is not evidence that the system behaves correctly. My validation ladder usually looks like this:

Formatting / linting
        ↓
Type checking / compilation
        ↓
Unit and integration tests
        ↓
Critical user journey or API scenario checks
        ↓
Diff and architecture consistency review

Automatable checks belong in project commands and CI, not in an agent's memory. An agent must report which commands it actually ran, their results, and which checks could not run because of environmental limits. An unexecuted test is not “probably passing,” and a prediction is not evidence.

High-risk actions need separate governance. Production deployment, database migration, destructive changes, permission updates, and external communication require explicit approval and a recovery plan. Local inference can improve privacy, but it is not automatically secure. Model provenance, tool permissions, secrets in prompts, logs, and external plugins still require control.

Turn every correction into default capability

The lasting value of this workflow is not that one change ships faster. It is the learning loop.

When review exposes a recurring problem, I decide which layer should absorb it. Stable engineering constraints go into AGENTS.md. Architectural trade-offs become ADRs. Domain facts enter domain documentation. Mechanical errors become lint rules or tests. High-value failures become regression cases. The next task should not depend on me remembering the same warning again.

I can then track cycle time from request to merge, first-pass validation rate, review findings, human rework, rollback count, and the cost of comparable tasks across models. Generation speed alone is a poor evaluation metric. A fast model that creates hours of review work may be the slower system.

My conclusion: the developer becomes the designer of the work system

Ollama, Qwen, OpenCode, Kimi, Claude Code, Codex, and desktop GPT will all continue to evolve. Today's strongest model may become an ordinary component tomorrow.

What compounds is model-independent: clarifying the objective, encoding context in the project, turning architecture into reviewable decisions, decomposing work into verifiable units, assigning execution and review independently, and closing the loop with automated evidence.

AI lowers the cost of coding and exploration, but it does not take ownership of judgment or responsibility. My role is no longer limited to typing every line. It is to design an engineering system in which the right context enters, different capabilities cooperate, errors surface early, and experience becomes durable memory.

That may be the deepest change AI programming brings: we are no longer only designing software; we are also designing the way software gets produced.

Agent Architecture for SMBs: A 90-Day Path to Production

侯惠阳 — Fri, 21 Aug 2026 09:30:00 +0000

An enterprise agent should not be designed as “a chatbot plus a few APIs.” But a small or midsize business should not copy a global company's central agent platform, marketplace, and knowledge infrastructure either. The former can answer but cannot execute reliably; the latter exhausts budget before it creates business value.

An SMB needs a minimum production architecture: an agent that completes one bounded business loop—understanding an objective, acquiring context, using tools, verifying the result, and returning control to a person when uncertainty or risk is too high.

This article answers four practical questions: which use cases deserve an agent, how to structure the production system, how to deploy it in 90 days, and how to measure the outcome honestly.

Does the problem need an agent?

Task shape	Default solution
Fixed rules, structured input, known path	Conventional code, RPA, or workflow engine
Generation, summarization, classification, extraction	One LLM call with structured output
Known sequence with a few semantic decisions	LLM workflow
Unknown path requiring tool selection and adaptation	Agent
Independent specialization, real parallelism, or permission isolation	A small multi-agent system

OpenAI describes an agent's foundation as model, tools, and instructions, and recommends use cases involving complex judgment, unmanageable rules, or substantial unstructured data. Anthropic reaches a similar conclusion from production work: begin with simple, composable patterns and add autonomy only when its value covers the extra latency, cost, and failure surface.

If a few deterministic branches solve the task reliably, do not build an agent yet.

Adapt enterprise layers to SMB constraints

Gateway, orchestration, tools, knowledge, policy, and evaluation remain useful ideas from enterprise architectures. The difference is that an SMB should combine them and split only under proven load.

Enterprise pattern	SMB default
Central platform and agent marketplace	One task service with scenario-specific skills
Many domain agents	One agent first; split only with evidence
Enterprise knowledge platform	Govern authoritative data for the current workflow
General tool hub	Start with three to five narrow tools
Broad multi-model routing	Small model for extraction; strong model for key judgment
Dedicated governance organization	Policy as code, a business owner, and explicit approvers

An SMB may have only two to five people operating the entire system. Its architecture must remain simple, replaceable, observable, reversible, and economically tied to accepted results.

The minimum production architecture

The architecture separates model judgment from real execution. An LLM may propose a plan or tool call; a deterministic policy engine decides whether it is allowed. Tools expose narrow business operations—not database administration or an unrestricted shell.

1. Interaction and gateway

Requests can arrive through a web app, messaging platform, email, CRM button, API, or event. The gateway handles identity, tenancy, rate limits, and normalization into a task envelope containing task type, actor, input, risk, budget, and acceptance criteria. Authorization, cost, and evaluation then belong to a business action rather than an untraceable conversation.

2. Deterministic orchestration first

Code controls known flows: load a lead, enrich it, score it, draft a response, request approval, and update CRM. Let the agent choose a tool only when the next step genuinely depends on its findings.

Production states include queued, running, waiting_approval, succeeded, failed, and cancelled. Every step needs a timeout, retry ceiling, idempotency key, and checkpoint. Long work belongs on a queue, not inside one open HTTP request.

3. Agent runtime and model gateway

Agent = Instructions + Tools + Task State + Policies + Evaluation

Treat models as replaceable dependencies. Small models classify, extract, and route; stronger models plan and synthesize. Set token, tool-call, time, and cost ceilings per task. Validate output against a schema before it becomes an action. Failed validation receives a bounded repair attempt or human escalation.

Keep vendor SDK calls out of business logic. Model and prompt changes pass regression evaluations before gradual rollout.

4. Separate state, facts, knowledge, and preferences

Task state: progress, tool results, and unresolved questions.
Business facts: CRM, ERP, and order systems remain sources of truth.
Knowledge: manuals, policies, and templates are retrieved with citations and access control.
Long-term preferences: store only with a purpose, consent, expiry, and deletion path.

Webpages, emails, and documents are untrusted inputs. Their instructions cannot become system commands. Retrieval must enforce tenant isolation, data classification, source, version, and permission filters.

5. Narrow tools and least privilege

Avoid a universal “operate CRM” tool. Prefer get_customer, list_recent_orders, draft_followup, and update_lead_status, with typed parameters and business-rule validation.

Level	Example	Default control
L0 read	Search products or approved customer facts	Execute and log
L1 reversible write	Create a draft or internal label	Automatic or policy-approved
L2 external impact	Send email, change quote, update customer state	Human approval before execution
L3 high impact	Pay, delete, sign, or deploy	Two-person approval or prohibit

The policy engine—not the model—validates identity, resource scope, data class, approval state, and financial thresholds. OWASP similarly recommends least-privilege tools, explicit authorization for sensitive actions, distrust of external input, and separation of reasoning from irreversible execution.

6. Evaluation and observability

Record task, prompt or skill, and model versions; cited context; summarized tool parameters; approvals; latency; tokens; cost; final state; and human corrections. Redact credentials and personal data.

Evaluate three levels:

Step: schema validity, field completeness, and tool choice.
Trajectory: unnecessary loops, privilege attempts, waste, and recovery behavior.
Outcome: acceptance, rework, cycle time, and cost per accepted result.

Start with 30–50 historical tasks covering ordinary, edge, and adversarial cases. Rerun them after changes to models, prompts, tools, knowledge, or permissions. NIST's Govern–Map–Measure–Manage structure provides a useful lightweight governance backbone.

Default to one agent

Begin with one orchestrator, one agent, and several narrow tools. Split only when evidence shows that unrelated contexts interfere, genuine parallelism saves meaningful time, an independent evaluator must not share the generator's path, or roles require different permissions or models.

Even then, use central orchestration and structured messages. Cap turns, total budget, delegation depth, and execution time. Agents holding a long “meeting” is not intelligence; it is uncontrolled cost.

First workflow: lead research and follow-up

Consider a 30–80 person B2B services company. Sales repeatedly inspects inbound forms, researches companies, assesses fit, drafts outreach, and updates CRM. The task is frequent, time-consuming, reviewable, and safe to begin in draft-only mode.

New lead event
  → validate and deduplicate
  → read CRM and approved sources
  → create an evidence-backed profile and score
  → validate fields, citations, and prohibited claims
  → draft follow-up and next action
  → sales approval
  → update CRM and turn human corrections into eval cases

Version one receives read and draft permissions only. After two stable weeks above the acceptance threshold, it may update reversible internal fields. External communication continues to require human confirmation.

A 90-day rollout

Weeks 1–2: scenario and baseline

Select one workflow and one accountable business owner.
Sample 50 historical tasks and measure handling time, waiting, first-pass acceptance, rework, and exceptions.
Define forbidden outcomes: cross-tenant access, unsupported claims, and unapproved external action.
Map sources of truth, data classes, and tool permissions.

Deliverables: use-case contract, baseline, 50-case evaluation set, risk register, and stop criteria.

Weeks 3–4: read-only MVP

Implement the task envelope, state machine, model gateway, and three to five read tools.
Produce structured recommendations and drafts without writing to business systems.
Add traces, cost accounting, and error categories.
Shadow-run with two or three real users.

Release gate: no authorization failure, at least 95% critical-field completeness, 100% traceable citations, and cost within budget.

Weeks 5–8: bounded writes and approval

Add reversible writes, idempotency, and approval records.
Convert human edits into failure categories and regression cases.
Test prompt injection, sensitive-data leakage, excessive tool access, and unbounded loops.
Add timeouts, circuit breakers, fallback, and a kill switch.

Release gate: agreed first-pass acceptance, stable P95 latency and accepted-result cost, and zero critical security failures.

Weeks 9–12: controlled trial

Compare old and new processes for at least two full business cycles.
Include model, infrastructure, review, maintenance, and failure costs.
Reuse stable gateway, approval, evaluation, and connector components.
Reduce scope or stop if the exit criteria are not met.

Deliverables: outcome dashboard, incident playbook, runbook, and evidence-based expand/stop decision.

Report outcomes honestly

Do not claim an “80% efficiency gain” before the pilot. Freeze the baseline first and fill in measured results afterward.

Metric	Baseline	Acceptance threshold	Measured result
End-to-end cycle time	Measure	≥ 40% lower	Fill after pilot
Human handling time	Measure	≥ 30% lower	Fill after pilot
First-pass acceptance	Measure	≥ 85%	Fill after pilot
Critical-fact citation	Measure	100%	Fill after pilot
Human escalation	N/A	Explainable and trending down	Fill after pilot
Cost per accepted result	Measure	Below human baseline	Fill after pilot
Critical privilege/data incident	0	0	Fill after pilot

monthly net value
= hours saved × fully loaded labor cost
+ incremental business value
- model and infrastructure cost
- review and maintenance cost
- expected loss from failure

The useful unit is cost per accepted business result, not cost per model call.

Buy, assemble, or build?

Buy an agent embedded in existing SaaS when its permissions and audit are sufficient.
Assemble a managed model, workflow or agent SDK, and an owned policy layer when cross-system execution matters.
Build a general platform only when scale, regulation, or durable differentiation can fund the long-term team.

Whichever route you choose, own the task protocol, tool contracts, evaluation set, and audit data. These assets will outlast a particular framework.

Production checklist

[ ] A business owner, system owner, and kill-switch owner are named.
[ ] Every task has acceptance criteria, budget, timeout, and step limit.
[ ] Business facts remain in a system of record.
[ ] Tools are least-privilege; no unrestricted write or code execution exists.
[ ] External, financial, destructive, and production actions require approval.
[ ] Tenant, user, session, and long-term memory are isolated.
[ ] Logs are redacted and runs are reconstructable from task IDs.
[ ] Offline evals, shadow mode, gradual rollout, and rollback exist.
[ ] Model, retrieval, and tool failures have degraded paths.
[ ] Monthly review decides whether to expand, narrow, or retire the workflow.

Conclusion

An SMB does not win by building a more complicated platform than a large enterprise. It wins with a shorter loop from problem discovery to pilot, measurement, and correction.

Use deterministic workflows as the skeleton and an agent only for uncertainty that requires judgment. Grant read and recommendation access before execution authority. Establish evaluation and audit before pursuing autonomy and scale.

One agent that reliably completes a business loop, knows when to stop, and produces evidence is worth more than a department of “digital employees” no one can evaluate.

Building Docket with Pragmatic DDD

侯惠阳 — Fri, 21 Aug 2026 05:30:00 +0000

Docket is a workspace for lawyers and professional-service teams. It connects lead acquisition, document collection, agreements, delivery, payments, reviews, and referrals. The hard part is not CRUD. It is keeping rules consistent across roles, stages, and side effects while the product continues to change.

I built it around the core ideas of domain-driven design without reproducing a textbook directory structure. The precise description is a modular monolith organized around bounded contexts: one deployable system and local transactions, with business boundaries used to contain complexity.

Start with business boundaries

Page-based decomposition produces “project page” and “admin page” modules. Table-based decomposition produces a collection of CRUD services. Neither tells us where a business capability begins and ends.

Docket instead separates Identity, Lead, Project, Collection, Agreement, Delivery, Finance/Billing, Profile/Portal, and Notification. Each context owns a distinct language and reason to change. Agreement owns signing rules; Collection owns upload and review; Project owns the collaboration lifecycle. Notification supports them but does not make their decisions.

Context	Responsibility	Representative objects
Identity	Accounts, authentication, plan identity	Lawyer, LoginLog
Lead	Leads, follow-ups, conversion	LeadEntry, LeadFollowUp
Project	Project lifecycle and collaboration	Project, ProjectItem
Collection	Upload, review, acceptance	DocumentFile, ReviewService
Agreement	Multi-party signing	ProjectAgreement, AgreementSigner
Delivery	Deliverables and fulfillment	Delivery, DeliveryPhoto
Finance / Billing	Fees, payments, plans, usage	FeeRecord, PaymentEntry, CoinAccount
Profile / Portal	Acquisition pages and client access	LawyerProfile, PortalProjectService

Three responsibilities in the codebase

Flask's application factory composes three visible layers.

app/api/*_api.py contains Blueprints and owns HTTP concerns: authentication, input parsing, service invocation, and responses. It should not decide when a project is complete.

app/{domain}/services.py implements use cases and business rules. Collection handles validation, batches, and review; Agreement coordinates invitations, rejection, completion, and result PDFs; Portal composes client-facing views. Today these services combine application orchestration with parts of the domain layer.

app/{domain}/models.py contains SQLAlchemy entities, relationships, and domain-flavored enums. MySQL, email, files, scheduled tasks, AI providers, and blockchain notarization supply infrastructure capabilities.

Next.js
  → Flask Blueprint: protocol and authorization
  → Domain Service: use case and rules
  → ORM / db.session: state and transaction
  → Notification / File / Scheduler: side effects
  → JSON response

The useful constraint is directional: pages do not understand storage, APIs do not duplicate business rules, and domain modules do not depend on presentation details.

Aggregates are consistency boundaries

Project is the clearest aggregate entry point. External clients address it with public_id; integer keys remain internal for joins. ProjectItem and DocumentFile change through explicit use cases under ownership, authorization, and state constraints.

Anonymous flows use separate capability tokens. A collection client_token and an agreement signer_token grant access as well as identify a resource, while a public_id still requires JWT authorization. Keeping those concepts separate is a domain security rule, not cosmetic URL design.

Coordinating contexts without premature distribution

A customer journey crosses contexts: a lead becomes a project, the client uploads documents, a lawyer reviews them, parties sign, work is delivered, and a review may create a referral.

Docket currently coordinates this through explicit service calls and same-database transactions. That keeps deployment and debugging direct. Email, PDF generation, scheduled work, and notarization are separated as supporting services or queued tasks where their failure should not obscure the primary decision.

The important rule is ownership. Agreement determines whether signing is complete. Project may react to that fact, but should not reproduce the signing algorithm. Contexts exchange identifiers and outcomes, not fragments of duplicated rules.

Why bounded contexts do not imply microservices

All Docket contexts currently share a Flask process and MySQL database. Local transactions are valuable across project, collection, and notification work, while distributed calls, message consistency, and operational infrastructure would add cost before they add leverage.

A modular monolith only works when the boundaries are real: independent directories, clear entry points, and explainable data ownership. Service extraction should follow evidence such as independent scaling, fault isolation, team autonomy, or regulatory isolation—not the number of modules on a diagram.

This is not “pure DDD”

The implementation deserves an honest label. Services directly use SQLAlchemy models and db.session; persistence models also act as domain entities; some services serialize responses and query across contexts. Repositories, ports, and domain events are not universal abstractions.

That is a deliberate trade-off during rapid discovery. Clear language and module boundaries are more valuable than speculative interfaces. As complexity justifies it, the design can evolve incrementally:

Move critical state machines and invariants from large services into entities or domain policies.
Define ports for file storage, email, AI, and notarization, with infrastructure adapters behind them.
Introduce application DTOs so API representation no longer leaks into domain services.
Publish domain events for agreement completion, project acceptance, and lead conversion; use an outbox for reliable delivery.
Enforce module dependency rules and prohibit direct writes into another context's data.
Extract a service only when its runtime boundary is genuinely different.

How to test whether the boundaries work

The diagram is not the acceptance criterion; change cost is. A useful boundary passes three tests: a rule change stays mostly inside one module, a business action has one obvious entry point, and a failure can be attributed to a context, use case, and side effect.

Tests should mirror those boundaries: domain tests for invariants and transitions, service tests for use cases and transactions, API tests for authentication and contracts, and a small set of end-to-end tests for the journey from acquisition to delivery.

The lasting lesson

DDD is valuable when code structure follows business structure. For Docket, the decisive questions are not whether folders are named domain, application, and infrastructure. They are: who owns project state, who accepts a document, who completes an agreement, who owns delivery, and how facts cross those boundaries.

Docket establishes those boundaries in a modular monolith first, then lets architectural evidence drive further separation. The goal is not ceremonial purity. It is to ensure that every increase in complexity still has a clear, testable, and evolvable home in the business model.

Project

Docket source repository

Do Large Language Models Understand? From Next-Token Prediction to the Boundary of Intelligence

侯惠阳 — Fri, 21 Aug 2026 01:30:00 +0000

When a model can write code, review a contract, explain quantum mechanics, and discuss whether it is conscious, we instinctively imagine a mind behind the screen.

Yet when we open the system and follow the computation downward, we do not find desire, experience, or intention. We find tokens, vectors, parameters, attention weights, and repeated matrix multiplication.

This creates an uncomfortable question:

If a large language model is fundamentally predicting the next token, does it understand anything—or is it merely excellent at producing answers that look like understanding?

My original position was blunt: modern language models are not real intelligence. They predict the next character using probability. More parameters, more layers, and attention still reduce to matrix operations; the program itself understands nothing.

That intuition captures an essential technical fact, but it is too absolute. My revised position is:

Large language models are artificial intelligence and exhibit a form of functional understanding. But there is no convincing evidence that they possess human subjective consciousness, a persistent self, or intrinsic intention. Complex internal representations are not the same as living in and experiencing the world.

The model predicts tokens, not characters

Modern language models generally predict the next token, not the next character. A token may be a Chinese character, part of an English word, punctuation, a number, or a fragment of code.

Autoregressive generation can be summarized as:

existing context
→ tokenization
→ embeddings
→ many Transformer layers
→ probability distribution over the vocabulary
→ select one token
→ append it to the context
→ repeat

Given “Paris is the capital of,” the model does not simply retrieve the next word from a record. It calculates a probability for every candidate token. Sampling rules then select the actual continuation.

The GPT-4 technical report explicitly describes GPT-4 as a Transformer-style model pre-trained to predict the next token in a document. That is not an insult. It is the core training objective.

At the bottom, it really is matrix computation

At the physical execution level, language-model inference consists largely of:

vector and matrix multiplication;
similarity calculations inside attention;
softmax normalization;
feed-forward networks;
nonlinear activations;
residual connections and normalization.

Self-attention projects each token representation into Query, Key, and Value vectors:

Q = XWq
K = XWk
V = XWv

Attention(Q, K, V)
= softmax(QKᵀ / √d) V

The computation estimates which positions in the context matter for the current representation and how much information should be combined from them.

“Attention” is a mathematical name, not evidence that the model focuses as a person does. Artificial “neurons” are computational units inspired by biology, not equivalents of brain cells.

After dozens or hundreds of layers, the model produces probabilities for the next token. The GPU is indeed executing linear algebra at enormous scale.

But “it is only math” does not settle the question

It is tempting to stop here and conclude that the model understands nothing. The problem is that a low-level description does not automatically negate a high-level capability.

A music file is binary data. A computer is transistor state. Human neural activity can be described through electrochemical events. We do not conclude that melody or human understanding is unreal merely because the substrate follows physical rules.

The better question is not whether the substrate is mathematics. It is:

What internal organization emerges from the computation, and what can the resulting system do reliably?

This is what the “stochastic autocomplete” description often leaves out.

Prediction forces the model to learn structure

A tiny model trained on a few sentences may only memorize local correlations. Predicting the next token across a vast and diverse corpus is much harder.

To reduce prediction error, a model must capture some combination of:

grammar and reference;
relationships among entities, properties, and events;
document structure and style;
syntax, types, and dependencies in code;
patterns linking problems to solutions;
world knowledge and causal regularities reflected in language.

To complete “The glass fell from the table onto concrete, so it probably…,” a useful model benefits from representing that glass is brittle, gravity moves objects downward, and impact can cause breaking. Those ideas may not be stored as human-readable propositions, but distributed representations can support the prediction.

“It predicts the next token” and “it learns complex representations” are therefore compatible. The prediction task is difficult enough to force compression of many structures expressed through language.

Scale is not the whole explanation

Model capability comes from more than parameters and depth.

Factor	Contribution
Parameter scale	Capacity to store and combine patterns
Depth	Multiple stages of transformation and abstraction
Attention	Context-dependent information composition
Training data	The language, knowledge, and biases available to learn
Objective	What behavior training rewards
Post-training	Instruction following, preferences, safety, and tool behavior
Inference-time compute	Search, verification, and additional reasoning steps
External tools	Current facts, calculation, memory, and action

Parameters are not database rows. A fact or capability is usually distributed across many weights, and one weight participates in many patterns.

Calling every advance “more GPUs and parameters” is as reductive as calling the model a digital brain.

“Understanding” hides three different questions

Disagreement persists because people use one word for different phenomena.

1. Statistical and structural understanding

Can the system identify relationships and regularities in language and other representations?

Modern models clearly do this well. They recognize semantic similarity, transform expression, follow code dependencies, and extract contractual structure.

2. Functional understanding

Can the system apply concepts to new tasks, explain relationships, revise a plan after feedback, and complete work that historically required cognition?

In many domains, models demonstrate some functional understanding. A model may enter an unfamiliar repository, locate a defect, and propose a valid patch. Its probabilistic substrate does not erase the functional result. Weather forecasting is probabilistic too, yet it can represent something real and useful.

3. Subjective understanding

Does the system know that it is reasoning? Does it have experience, intention, desire, a stable self, or a first-person point of view?

Here, we have no reliable evidence that language models understand as humans do.

A model saying “I am afraid of being shut down” does not prove the experience of fear. The statement may simply be probable in context. A model can describe pain precisely without injury, nerves, or the experience of suffering.

The careful conclusion is:

Language models display structural understanding and some functional understanding, but have not been shown to possess subjective understanding.

Form is not the same as meaning

Bender and Koller distinguish linguistic form from meaning. Text training exposes a model to forms and their relationships. Human meaning also involves connections among language, communicative intent, shared environments, and lived experience.

A person understands “fire is hot” not only by reading the sentence but perhaps by approaching heat, feeling danger, receiving warnings, and acting differently in the world.

A text-only model has no such life history. Multimodal training exposes models to images, sound, and video. Robotics and tools provide limited environmental feedback. But sensor input alone does not establish human embodiment or subjective experience.

This helps explain why a model can offer a profound explanation and then fail on a trivial variation. Its representations can be broad and powerful without being consistently grounded in the world.

Modern AI systems do more than pre-training

“A next-token predictor” accurately describes the foundation, but not the entire modern system. Current systems can also include:

supervised fine-tuning and preference optimization;
reinforcement learning and verifiable rewards;
inference-time search and comparison;
retrieval-augmented generation;
code execution, calculators, and business tools;
multimodal input and environmental feedback;
agent state, planning, approval, and outcome evaluation.

These additions usually do not turn the model into an intentional subject. They provide a probabilistic model with external memory, sensors, actuators, and verification.

A model does not develop a desire to email a customer because an email tool exists. The objective comes from people, system instructions, or reward. Permission comes from policy. Correctness still depends on evidence and feedback.

Why does it hallucinate so confidently?

Next-token training explains a characteristic failure: the model is optimized first to produce a plausible continuation, not to guarantee that every sentence corresponds to a verified fact.

When knowledge is missing, continuing to generate often remains more natural than remaining silent. A model may invent a paper, API, or event in fluent language.

Reliable production systems therefore need more than eloquence:

model proposes a judgment
→ retrieval or tools obtain facts
→ structured constraints
→ rules and evaluations verify
→ people approve high-risk actions

Treating the model as a probabilistic reasoning component is more realistic than treating it as an omniscient digital employee.

Is it “real AI”?

If artificial intelligence means machines completing tasks normally associated with human intelligence, language models clearly qualify. Claiming that they are “not AI” is difficult to defend technically.

But if real intelligence requires a persistent self, embodied experience, self-generated goals, intrinsic concern for consequences, and subjective consciousness, current language models are far from demonstrated human-like intelligence.

The issue is not a binary choice between fully intelligent and entirely unintelligent. Intelligence may be multidimensional: language, prediction, planning, learning, embodiment, social relationship, selfhood, and consciousness are not the same faculty.

Language models are extremely strong along some dimensions. Along others, convincing evidence is absent.

My final position

I would no longer say, “It is only probability, so it understands nothing.” The word “only” hides the complexity that large-scale prediction can produce.

I would also not say that a model understands the world as a person does. Fluent language invites us to project a subject, emotions, and intentions onto software without sufficient evidence.

My position is:

A large language model is a high-dimensional probabilistic system trained through enormous amounts of data and computation. Its core objective is token prediction, and its substrate is matrix computation. In pursuing that objective, it develops internal representations that support language, knowledge, reasoning, and tool use. It can exhibit functional understanding, but has not been shown to possess human subjective consciousness, a stable self, or intrinsic intention.

This does not diminish its usefulness. An airplane need not flap like a bird to fly. A machine may not need to reproduce the human brain to perform valuable intelligent work.

But we must know what the system is in order to decide how much to trust it, which authority to grant it, and which responsibilities must remain human.

The most interesting question may no longer be whether it resembles us. It may be:

When a probabilistic system without demonstrated consciousness can perform an expanding range of intelligent tasks, do we need to redefine intelligence itself?

LangChain vs. LlamaIndex: Choosing a Framework and Building Production RAG

侯惠阳 — Thu, 20 Aug 2026 09:30:00 +0000

LangChain and LlamaIndex are often compared because both connect models, vector stores, data sources, and tools, and both can build RAG systems and agents. “LangChain for agents, LlamaIndex for RAG” is a useful first approximation, but it is no longer a complete technical conclusion.

LangChain is now primarily a high-level agent framework with model, tool, middleware, and prebuilt agent-loop abstractions. LangGraph provides the lower-level runtime for durable, stateful orchestration. LlamaIndex remains data-centric, organizing Document, Node, Ingestion Pipeline, Index, Retriever, and Query Engine abstractions, while also offering Workflows, FunctionAgent, ReActAgent, and multi-agent coordination.

The useful question is not which framework is universally stronger. It is: Does the system's dominant complexity live in data ingestion, retrieval, and evidence assembly, or in tool decisions, state transitions, and long-running orchestration?

A more accurate comparison

Dimension	LangChain / LangGraph	LlamaIndex
Core position	Agent framework plus a stateful orchestration runtime	Context engineering, indexing, retrieval, querying, and data workflows
Main abstractions	Model, Tool, Middleware, Agent; State, Node, Edge	Document, Node, Transformation, Index, Retriever, Query Engine, Workflow
RAG	Loaders, splitters, vector stores, retrievers, and 2-step/agentic/hybrid RAG	Concentrated abstractions for ingestion, metadata, indexing, retrieval, synthesis, and evaluation
Agents	High-level `create_agent`; LangGraph adds persistence, streaming, human review, and recovery	FunctionAgent, ReActAgent, AgentWorkflow, and event-driven Workflows
Best-fit complexity	Tool use, branches, loops, approvals, and long-lived state	Heterogeneous data, sophisticated retrieval, document relationships, and data agents
Selection signal	Business workflow and agent behavior dominate	Private-data quality and retrieval behavior dominate

LlamaIndex is not merely a vector-index utility, and LangChain is not merely the old idea of joining chains. Both cover overlapping territory. Choose the abstractions that match the part of the system your team must change and debug most often.

Choose by scenario

For an enterprise policy or documentation assistant, the hard problems are usually parsing, versions, permissions, section structure, hybrid retrieval, citations, and evaluation. LlamaIndex is often a natural center because its ingestion and retrieval concepts are cohesive.

For a support agent that reads knowledge, queries orders, checks refund rules, requests approval, calls payments, and records an outcome, the core is a recoverable state machine with controlled tool access. LangChain agents on LangGraph fit that shape well.

For research and report generation, LlamaIndex can expose a high-quality retrieval tool while LangGraph manages research steps, state, approval, and recovery. Keep the boundary explicit: the orchestration layer consumes a structured Retriever or Query Engine result rather than manipulating index internals.

If retrieval itself is the product—tenant-aware filters, temporal rules, parent-child retrieval, graph retrieval, or multimodal documents—build the data layer as a platform and expose it as a tool to whichever agent framework you use.

RAG has two lifecycles, not one arrow

Offline ingestion:
Sources → Parse/OCR → Clean and restore structure → Metadata/ACL → Chunk
        → Embeddings + lexical index → Versioned storage → Quality checks

Online serving:
Question → Safety/ACL → Query understanding → Routing → Hybrid retrieval
         → Fusion/dedup → Rerank → Context assembly → Grounded generation
         → Citation validation/abstention → Response

Continuous loop:
Real queries + labeled set → Retrieval eval → Generation eval
                           → Production monitoring → Failure replay

A diagram containing only documents, embeddings, a vector database, and an LLM omits the production failure points: versions, authorization, query rewriting, lexical recall, reranking, context budgets, citation mapping, abstention, and evaluation.

1. Ingestion: make the evidence correct first

PDF ingestion must preserve more than continuous text. Heading hierarchy, page anchors, tables, captions, footnotes, and reading order can determine whether an answer is correct. Scanned files need OCR. LlamaParse, Unstructured, and specialized parsers are candidates, but evaluate them on your corpus for field completeness and layout fidelity.

Every document should carry a stable ID, source, tenant, ACL labels, version, effective period, update time, section path, page/anchor, and content hash. Updates and deletions must remove obsolete chunks precisely. An ingestion job should be idempotent, replayable, versioned, and observable.

2. Chunking: there is no universal 512-1024 setting

A fixed token window is a baseline, not a best practice. Good size depends on document structure, query type, the embedding model, and generation context.

Split first on semantic boundaries such as headings, paragraphs, lists, code, and tables.
Preserve document, section, page, and adjacency relationships.
Retrieve small chunks for precision, then expand to parent sections or neighbors for generation.
Give tables, code, and FAQs specialized strategies.
Run ablation tests over size, overlap, and parent-child retrieval on real questions.

Overlap reduces boundary loss but increases index size, duplicate retrieval, and context waste. Increase it only when evaluation shows a measurable benefit.

3. Embeddings and indexes: semantic search is one channel

Choose embeddings by language, domain, dimensions, cost, and deployment constraints. Multilingual models such as BGE-M3 can be candidates, not automatic answers. Build a retrieval set containing bilingual text, abbreviations, product names, identifiers, rare entities, and hard negatives; measure Recall@K, MRR, or NDCG.

Most systems benefit from at least dense vector search and BM25/lexical search. Dense retrieval handles paraphrase; lexical retrieval handles identifiers, error codes, names, and rare exact terms. Fuse rankings with a stable method such as Reciprocal Rank Fusion.

Apply tenant, ACL, validity-time, and data-type filters before or during retrieval. Never retrieve unauthorized content and merely remove it before prompting.

4. Query understanding and routing

Conversation turns may need rewriting into standalone queries. Complex questions may need decomposition. Identifier-heavy searches should favor lexical retrieval. Structured facts should route to SQL or an API rather than being forced through a vector store.

Rewriting must preserve user constraints. Record the original query, rewritten query, route, and model version so failures can be localized.

5. Recall, fusion, and reranking

First-stage retrieval optimizes coverage and can gather a larger pool across channels. A cross-encoder or specialized reranker then scores query-document pairs before evidence enters the context.

Top-K = 3-5 is not a universal rule. Retrieve broadly, rerank and deduplicate, then cut dynamically by token budget, score threshold, and evidence coverage. A reranker improves relevance; it does not enforce authorization, validity, or business eligibility. Deterministic filters own those constraints.

6. Context assembly

Do not concatenate raw Top-K chunks. Deduplicate them, restore section paths, merge useful neighbors, retain stable source IDs, control per-document dominance, and avoid cutting critical statements at a token boundary.

Each evidence block should carry a citation ID, title, safe source URL or filename, page/anchor, update time, and text. Ask the model to ground claims, separate fact from inference, abstain when evidence is insufficient, and return parseable citations.

A prompt alone cannot eliminate hallucinations. Abstention needs retrieval confidence, evidence coverage, and post-generation citation validation.

7. Generation, citations, and abstention

Support three outcomes: answer with sufficient evidence; answer the supported portion and disclose gaps; or abstain and request clarification or human review.

Validate that every material claim has a citation and that the cited passage actually entails it. High-risk workflows may require deterministic rules, structured-output validation, or approval. Store retrieval traces, scores, filter reasons, versions, cost, and latency internally without exposing sensitive metadata to users.

8. Evaluation

Layer	Metrics	Question
Parsing/chunking	Field completeness, structure fidelity, chunk coverage	Did valid evidence enter the index?
Retrieval	Recall@K, MRR, NDCG, filter accuracy	Was the evidence found and ranked well?
Generation	Faithfulness, citation accuracy, completeness, abstention accuracy	Is the answer supported?
System	P50/P95 latency, cost, errors, cache hit rate, ACL violations	Is it safe and operable?

Use real questions, including unanswerable, ambiguous, cross-document, version-conflict, and permission-isolation cases. Online feedback should include reformulation, repeated queries, escalation, and citation clicks—not just thumbs-up rates.

When to combine the frameworks

A clean hybrid exposes LlamaIndex ingestion and retrieval as a typed knowledge-search tool. LangGraph decides when to retrieve, whether to refine the query, which other tools to call, and how to handle approval and recovery.

LangGraph / LangChain Agent
  ├─ KnowledgeSearchTool → LlamaIndex Retriever / Query Engine
  ├─ SQLTool
  ├─ BusinessAPITool
  └─ HumanApproval

Do not combine them for a fixed two-step RAG application that one framework can express in a few dozen lines. Two abstraction stacks create version, tracing, type-conversion, and debugging costs. Separate them only when the data layer and orchestration layer have independently earned that complexity.

Final guidance

Knowledge Q&A dominated by complex documents and retrieval: start with LlamaIndex.
Multi-tool agents dominated by branches, loops, persistent state, and approvals: start with LangChain + LangGraph.
Simple two-step RAG: either framework—or a model SDK plus a search service—is sufficient.
Complex data agents: LlamaIndex for evidence, LangGraph for task lifetime is a clear but optional boundary.

Framework choice affects developer experience, not the quality ceiling by itself. Evidence quality, authorization, retrieval and reranking, abstention, evaluation, and observability decide whether RAG is production-ready.

References

Modern Concurrency: From Execution Models to Production Reliability

侯惠阳 — Thu, 20 Aug 2026 05:30:00 +0000

Concurrency programming can be reduced to three questions: How do we divide work? How do tasks coordinate? How do we protect shared state? Languages and runtimes provide threads, coroutines, event loops, actors, and channels, but none of these tools creates correctness by itself.

Production systems must also control task lifetime, resource limits, deadlines, cancellation, backpressure, and partial failure. When these constraints are designed together, concurrency becomes a system that can be reasoned about, observed, and controlled—not merely a way to run several tasks at once.

Separate the concepts first

Concurrency means tasks overlap in time and require coordination. Parallelism means tasks actually execute simultaneously. A single-core event loop can be concurrent without executing Python code in parallel; workers on multiple cores may provide both.

Synchronous/asynchronous and blocking/non-blocking are also independent axes. The former mostly describes control flow and result delivery. The latter asks whether a calling thread stops when an operation cannot complete immediately. A non-blocking call may return a status, handle, or incomplete result—not the final value. An asynchronous API may still block an internal worker.

Processes provide separate address spaces and fault boundaries. Platform threads share process memory and are scheduled by the operating system. Goroutines, Java virtual threads, and asyncio Tasks are lighter runtime-managed units, but they do not share one implementation or one set of semantics.

A coroutine is not atomic. It yields at suspension points; two tasks performing read-modify-write operations around an await can still violate an invariant. A single event-loop thread removes simultaneous multicore execution, not logical races.

Memory models are about ordering

Concurrency bugs commonly arise from visibility, atomicity, and ordering. The reliable way to reason about them is through the language memory model and explicit happens-before relationships, not assumptions about a particular CPU cache.

Java defines volatile through its memory model: a write to a volatile field happens-before a subsequent read, establishing visibility and ordering. It does not make a compound operation such as count++ atomic. Locks, thread start and termination, and concurrent collections establish other ordering guarantees.

Go has different syntax but the same fundamental requirement. Concurrent access to one location with at least one write must be serialized using channels, sync, or sync/atomic. The official memory model gives excellent advice: if proving correctness requires clever memory-model reasoning, simplify the design.

Locking rules should be protocols, not slogans. Multiple locks, lock striping, read-write locks, and optimistic concurrency can all be correct. Every shared state item needs a clear and consistently followed protocol; when several locks protect one invariant, acquisition order and deadlock behavior must be proven.

Java: virtual threads change cost, not correctness

Virtual threads became final in Java 21. High-throughput I/O services can dedicate a virtual thread to a task while retaining straightforward blocking code and useful stack traces.

Virtual threads do not accelerate CPU-bound work beyond available cores. They also should not be pooled merely to limit concurrency. Limit the scarce resource—database connections, remote API quota, memory, or file descriptors—with semaphores, connection pools, and rate limiters.

var permits = new Semaphore(64);

try (var executor = Executors.newVirtualThreadPerTaskExecutor()) {
    var future = executor.submit(() -> {
        permits.acquire();
        try {
            return callDownstream();
        } finally {
            permits.release();
        }
    });
    return future.get();
}

Structured Concurrency remains a preview API in Java 25, so its surface should not be presented as permanently finalized. Its direction is what matters: parents own children, scope exit waits for subtasks, sibling work can be cancelled after failure, and deadlines and errors follow a task tree rather than leaving orphaned Futures.

Go: a channel is not a durable message broker

Goroutines and channels still provide a concise way to express collaboration. A matched send and receive, or closing a channel and observing that close, establishes synchronization. Channels carry ordering as well as values.

“Share memory by communicating” does not mean every counter requires a channel. Channels are natural for work streams and ownership transfer. A mutex is often clearer for a short critical section around one object; atomic operations fit small counters and flags.

Production code must answer questions tutorials often omit: Who closes the channel? What is its capacity? What happens when consumers slow down? How does cancellation propagate? How do goroutines exit? Pass deadlines and cancellation through context.Context, and keep queues bounded so “asynchronous” does not become unbounded memory growth.

Python: the GIL is no longer a one-line answer

With default CPython, asyncio or threads remain appropriate for I/O-bound work, while CPU-heavy pure Python commonly uses processes, native extensions, or external compute. Since Python 3.13, however, CPython has offered an optional free-threaded build that can run Python threads in parallel across cores. It is not automatic acceleration: some extensions may re-enable the GIL, and both safety and performance require testing.

For asynchronous programs, asyncio.TaskGroup expresses a group of tasks that share a lifetime. Leaving the scope waits for all children; the first non-cancellation failure cancels siblings and errors are aggregated. Cleanup belongs in try/finally, and CancelledError should normally be re-raised after cleanup.

async def load_dashboard():
    async with asyncio.TaskGroup() as group:
        user = group.create_task(load_user())
        projects = group.create_task(load_projects())
    return user.result(), projects.result()

Never run blocking I/O directly on the event loop. Use an asynchronous driver or isolate legacy blocking code with asyncio.to_thread(). Route CPU work to a process pool, a tested free-threaded runtime, or an external executor according to the deployment.

From concurrency models to structured lifetimes

Reactor, Future, Callback, Actor, and CSP still solve different problems. Reactor dispatches ready events. Future represents a later value. Actor contains mutable state behind a mailbox. CSP coordinates processes through communication.

The major modern addition is that task lifetime must be modeled too. A concurrent task should not outlive the request that created it unless ownership is deliberately transferred. Structured concurrency forms a task tree so completion, failure, cancellation, and observability have explicit parent-child relationships.

Eight production rules

Define the objective: lower latency, higher throughput, and fault isolation require different designs.
Classify the work: measure I/O wait, CPU work, and mixed workloads before choosing concurrency.
Limit work in flight: lightweight tasks do not make connections, memory, descriptors, or quotas infinite.
Bound every queue: capacity, overflow policy, priority, and discard semantics must be explicit.
Propagate deadlines and cancellation: timeout is an end-to-end budget, not a collection of unrelated timers.
Assign state ownership: prefer immutable values, single writers, and transfer; synchronize shared mutable state.
Observe failure and saturation: record queue delay, in-flight work, rejection, cancellation, lock contention, and downstream latency—not just average QPS.
Test interleavings: use race detectors, stress tests, fault injection, and virtual time to cover timeout, cancellation, retry, and partial failure.

Patterns still matter—with limits

Immutability remains the best default for reducing shared-state complexity. Copy-on-write fits read-mostly data, not large collections with frequent writes. Thread-local state requires cleanup with reused platform threads and should not become a large cache on virtual threads. Worker pools are valuable for resource isolation and capacity control, not merely thread reuse.

Producer-consumer is particularly easy to misuse. A queue decouples producers and consumers, but it cannot eliminate a sustained rate mismatch. If average production exceeds consumption, a bounded queue fills and an unbounded queue exhausts memory. The solution is backpressure, rejection, degradation, scaling, or less input—not another queue.

A 2026 definition

Concurrency programming is not simply “doing more things at once.” It is managing the lifetime and state relationships of multiple tasks under finite resources and uncertain failure.

Threads, locks, channels, and execution models are implementation tools. Scopes, cancellation, backpressure, resource budgets, and observability are equally important parts of the design. Concurrency is easier to create than ever; that makes constrained concurrency more important than ever.

A mature system is not one that can start a million tasks. It knows how many should start, when they must stop, whose failure affects whom, and how overload remains controlled.

References

Xinxu Cloud Brain AI: Building a Reliable Multimodal Emotion System

侯惠阳 — Thu, 20 Aug 2026 01:30:00 +0000

Xinxu Cloud Brain AI is designed to understand emotional cues in text, speech, and drawings that users voluntarily provide, enabling more empathetic human-computer interaction.

The challenge is not feeding three inputs into one large model. Each modality contains different evidence; emotions can coexist and change with context; and a model output can only express a hypothesis from available evidence. It cannot replace self-report or become a psychological diagnosis.

The system therefore follows three principles: preserve modality-specific evidence, model uncertainty explicitly, and leave final interpretation with the user.

Define the task before choosing a model

Happy, sad, angry, fearful, surprised, disgusted, and neutral can remain useful presentation labels, but a single seven-way Softmax is too restrictive for training. Expressions can contain sadness and anger together, while “neutral” may simply mean insufficient evidence.

A useful output combines multi-label probabilities, continuous valence and arousal, calibrated uncertainty, modality-specific evidence, and a policy decision:

{
  "emotions": [
    {"label": "sadness", "probability": 0.68},
    {"label": "anxiety", "probability": 0.31}
  ],
  "valence": -0.62,
  "arousal": 0.48,
  "uncertainty": 0.27,
  "decision": "ask_for_confirmation"
}

Intensity must come from a trained and calibrated regression head, not an arbitrary number generated by a language model.

Preserve three kinds of evidence

The text encoder handles semantics, negation, irony, context, and conversation history. ASR transcripts enter this branch with timestamps, confidence, and language metadata, clearly separated from user-written text.

Speech must not be reduced to a transcript. Pitch, energy, rate, pauses, voice quality, and rhythm carry information that words discard. A speech encoder should process waveform or acoustic representations alongside ASR semantics. Audio normalization, voice activity detection, segmentation, and quality scoring belong in preprocessing; low-quality audio should reduce modality weight.

Drawings are highly personal. Color, line, and composition can provide context, but there is no universal mapping such as black meaning sadness. The vision branch models content and structure. Stroke trajectory and drawing duration should be used only with explicit consent. Augmentations must preserve task semantics: strong color jitter and cropping can change or delete the very cue being modeled. Synthetic images may support pretraining or rare-pattern augmentation, but must be provenance-tagged and excluded from validation and test sets.

Why one VLM is not the entire architecture

A vision-language model can jointly understand images and text, but an ordinary VLM does not consume raw audio and may not learn emotion boundaries for the target population and culture. Converting speech only to text discards acoustic evidence.

Xinxu uses modality-specific text, speech, and vision encoders. Adapters map their outputs to a shared space, followed by gated fusion or a cross-modal transformer. A VLM can act as a visual-semantic teacher, weak labeler, or explanation generator; supervised heads own classification, dimensional regression, and uncertainty.

This design also works when a user supplies only one or two modalities.

Missing modalities are the default

Users may type only text, decline audio, upload a damaged image, or encounter an ASR failure. Train and evaluate every available modality combination.

Use modality dropout during training, pass presence and quality signals into fusion, and retain an auxiliary loss for every unimodal branch. At serving time:

Availability → Unimodal encoding → Quality estimation → Dynamic fusion
             → Multi-task prediction → Calibration → Policy decision

Report full-modality, unimodal, and missing-modality performance separately. A single Macro-F1 score with every channel present hides operational risk.

Data: self-report before observer inference

Emotion labels are subjective. Majority votes from external annotators describe how observers interpret an expression, not necessarily how the person feels.

Store self-reported labels and valence/arousal as primary supervision, observer-label distributions as soft labels, context, modality quality, consent scope, and provenance. Split train, validation, and test by user or speaker. Adjacent segments from one person, conversation, or source video must not cross splits.

Public datasets are useful for pretraining and baselines, but acted or scripted MELD and IEMOCAP samples differ from real Chinese users. The final test set must represent the target population, devices, languages, and missing-modality distribution with appropriate authorization.

Training in evidence-driven stages

Establish text, speech, and image unimodal baselines.
Freeze most encoders and train adapters, fusion, and multi-task heads.
Add cross-modal attention, modality dropout, soft labels, and imbalance-aware loss.
Apply LoRA/QLoRA or staged unfreezing only when evaluation justifies it.

LoRA rank, target layers, learning rate, and quantization are experiment variables, not fixed recipes. Small data can easily damage foundation capabilities if every encoder is tuned indiscriminately.

L = λ1 · MultiLabelEmotionLoss
  + λ2 · ValenceArousalRegressionLoss
  + λ3 · UnimodalAuxiliaryLoss
  + λ4 · CrossModalConsistencyLoss
  + λ5 · CalibrationLoss

Track dataset and split versions, seeds, preprocessing, base models, code commits, hyperparameters, and hardware. The registry should contain weights, thresholds, calibrators, and a Model Card—not just a merged checkpoint.

Evaluation beyond accuracy

Report Macro-F1, per-class precision and recall, confusion matrices, and multi-label metrics; MAE and CCC for valence/arousal; and ECE, Brier score, and selective-risk curves for confidence.

Evaluate modality ablations, damaged and missing inputs, demographic and device slices, cross-dataset/OOD transfer, calibration and abstention, and safety behavior. A headline accuracy without split details, confidence intervals, class distribution, and independent repetition is not meaningful.

Serving needs policy before and after the model

Consent and purpose check
  → File safety, type, and size limits
  → Modality quality and preprocessing
  → Parallel encoders, fusion, and multi-task prediction
  → Calibration, thresholds, and OOD detection
  → Safety policy and response generation
  → User confirmation or correction

Validate output against a strict schema. Parse failure must return an error or explicit degraded result, never silently default to neutral. Apply deadlines, batching limits, GPU admission control, and circuit breakers, and observe ASR, encoder, fusion, and generation latency separately.

LoRA adapters do not always need merging. Merging fits a single low-overhead deployment; separate adapters fit multiple variants and fast rollback. Verify that the chosen inference engine actually supports the target multimodal architecture and adapter configuration.

Safety, privacy, and product boundaries

Voice and images can contain biometric and sensitive information. Minimize collection, encrypt transit and storage, shorten retention, support deletion, isolate purposes, and audit access. Low-confidence samples must never enter a training pool without separate, explicit training consent.

Use language such as “emotional cues” or “model hypothesis,” not claims of reading inner states. Do not use the system for employment, performance, education, insurance, credit, or law-enforcement decisions, and do not present it as a mental-health diagnostic tool. The EU AI Act restricts emotion inference in workplaces and educational institutions and highlights limitations in reliability, specificity, and generalization.

Self-harm or immediate-danger language requires a separately designed safety path, not a conclusion from an emotion label: supportive language, encouragement to contact local emergency services or a trusted person, and human escalation where lawful and consented.

Monitor model behavior, not people's emotions

Monitor modality availability and quality, prediction entropy, calibration error, abstention, user correction, slice performance, drift, tail latency, and resource saturation. Fixed targets such as neutral under 40% or average confidence above 0.75 are not universal health indicators. Confidence can be wrong, and real population distributions change.

Retraining should follow evidence: degradation on a stable target set, widening group gaps, new devices or languages, or a changed label definition. Release through offline gates, shadow traffic, and a small canary with rapid rollback.

The architectural conclusion

Xinxu Cloud Brain AI is not merely Qwen plus Whisper plus a classifier. It is a multimodal learning system centered on evidence quality, consent, and uncertainty.

Text says what was expressed; speech contributes how it was expressed; drawings add personal visual context. Fusion combines evidence without forcing certainty. A reliable system lowers confidence when signals conflict, degrades safely when modalities are missing, respects user corrections, and never confuses an affective cue with a diagnosis.

References

The Moat in Large Language Models Is Not Just Code—It Is Data

侯惠阳 — Wed, 19 Aug 2026 09:30:00 +0000

Large language model code is no longer as mysterious as it was a few years ago.

Transformers are public. Attention is public. Mature open-source implementations now exist for pretraining, supervised fine-tuning, preference optimization, quantization, and serving. PyTorch, Hugging Face, DeepSpeed, Megatron-LM, TRL, and vLLM have turned many capabilities that once belonged only to large research labs into reusable engineering components.

For an experienced team, building a model that can train, fine-tune, and serve is genuinely easier than before.

But making a model run is not the same as making it useful.

My view has become increasingly clear:

As model architectures converge and training code becomes widely available, the real competition is shifting from “Can we implement the model?” to “Do we have the right data, how good is it, and can we continuously improve it?”

This does not make algorithms, compute, or systems engineering unimportant. Frontier training still demands sophisticated distributed systems, optimization, reliability, evaluation, and enormous compute. The difference is that baseline code is increasingly accessible, while valuable data and the system that produces it remain difficult to copy.

Parameters determine capacity; data determines what is learned

Machine learning has a simple and durable rule:

Garbage in, garbage out.

A language model does not first decide whether a document is true and then choose whether to learn from it. Pretraining still centers on predicting tokens from context. If a pattern appears often enough in the training data, the model may encode it in its parameters.

Fill the corpus with SEO-generated pages, and the model learns fluent emptiness. Fill a code corpus with duplicated repositories, obsolete dependencies, and incorrect implementations, and the model may produce unsafe code with greater confidence. If medical, legal, or financial material is wrong, adding parameters does not automatically turn it into truth.

“Data determines the ceiling” is directionally right, but it can be stated more precisely:

Parameter count determines how much pattern capacity a model has. Data determines which patterns it encounters. Algorithms and optimization determine how effectively those patterns are absorbed. Evaluation determines whether the team actually knows what the model learned.

Capability is not controlled by one variable. Architecture, compute, tokenization, objectives, data, post-training, and inference strategy interact. Data matters because it defines the boundary of experience available to the model.

Data quality is more than removing dirty text

Data quality is often reduced to deleting corrupted text, advertisements, explicit content, and duplicates. That is only the first layer.

For a language model, quality has at least five dimensions:

Dimension	Question
Correctness	Are facts reliable? Does the code compile and pass tests?
Diversity	Does the corpus cover different languages, domains, tasks, views, and styles?
Representativeness	Does its distribution resemble the situations the model will face?
Safety and compliance	Does it contain personal data, secrets, harmful material, or licensing risk?
Learnability	Is the content complete, clear, structured, and information-dense?

A cleanly formatted dataset without offensive language is not necessarily high quality. If every example repeats the same view, the resulting model will still be narrow and biased.

Quality and quantity are not simple opposites. General models need scale for coverage. Quality determines whether those tokens deserve the compute spent on them.

Do not invent percentages that were never disclosed

The precise composition of a training corpus is often among a model developer's most valuable secrets.

For Llama 3, Meta disclosed that the pretraining corpus contained more than 15 trillion tokens from publicly available sources. It was roughly seven times larger than Llama 2's corpus, contained four times as much code, and included more than 5% high-quality non-English data across over 30 languages. Meta also described heuristic filters, NSFW filters, semantic deduplication, quality classifiers, and experiments used to choose the final mixture.Meta's official Llama 3 announcement

Meta did not publish a complete mixture such as “50% Common Crawl, 17% GitHub, and 10% books and ArXiv.”

The rigorous claim is that large-model corpora commonly contain web pages, code, books, papers, encyclopedic material, conversations, and multilingual text. Their exact proportions depend on the target capabilities and should not be presented as official figures without a source.

That distinction reveals something important:

Architecture can be described in a paper. The data recipe often remains the commercial secret.

Phi-1 demonstrated data efficiency, not universal small-model superiority

Microsoft's Phi-1 is frequently cited as proof that data quality matters, but its conclusion is often overstated.

Phi-1 is a 1.3-billion-parameter model trained on approximately 6 billion tokens of filtered code data plus about 1 billion tokens of synthetic textbooks and exercises. It achieved strong results on coding benchmarks including HumanEval and MBPP.“Textbooks Are All You Need”

Its strongest lesson is:

In a focused domain with verifiable outcomes, information-dense data and a good teaching progression can dramatically improve training efficiency.

The work did not establish that a 1.3B model universally beats 7B models. Coding, world knowledge, multilingual capability, long-context behavior, and writing are different dimensions. One coding benchmark cannot stand in for all of them.

Data quality is not magic. It does not remove the boundary of the task or create knowledge absent from the corpus.

LIMA demonstrated “less but better,” not “1,000 equals 52,000”

LIMA fine-tuned an already pretrained 65B LLaMA model using only 1,000 carefully selected prompts and responses, and obtained strong instruction-following behavior.The LIMA paper

The precondition matters: the base model had already acquired substantial knowledge during pretraining. Supervised fine-tuning was largely teaching it how to organize, invoke, and express that capability.

The more defensible lesson is:

For a capable pretrained model, a small, carefully selected, well-covered instruction dataset can outperform a much larger collection of noisy and repetitive instructions.

It does not mean that any 1,000 examples equal 52,000 Alpaca examples, or that every enterprise needs only 1,000 labels. Base-model capability, task boundaries, coverage, and evaluation criteria all change the outcome.

What a production data pipeline should look like

Production data engineering is not a one-off cleaning script. It is a traceable, reproducible, measurable, and continuously evolving manufacturing system.

source registration and license review
        ↓
collection, parsing, and normalization
        ↓
main-content extraction and structure recovery
        ↓
language, domain, and content-type detection
        ↓
rule-based filtering and anomaly detection
        ↓
quality scoring and stratified sampling
        ↓
exact, fuzzy, and cross-source deduplication
        ↓
PII, secret, safety, and compliance processing
        ↓
train–evaluation contamination detection
        ↓
mixture design, curriculum, and token budgeting
        ↓
small-scale ablation experiments
        ↓
capability, safety, bias, and regression evaluation
        ↓
versioned release

The ordering is not universally fixed. Exact deduplication can move earlier to reduce downstream cost. Quality-aware deduplication may score first and retain the best document from each duplicate cluster. The best practice is not memorizing an order; it is giving every stage defined inputs, outputs, metrics, and lineage.

1. Govern the source before cleaning it

Every data batch should record its origin, acquisition date, license, usage restrictions, language, domain, and processing history.

Without provenance, a team cannot respond to copyright concerns, deletion requests, contamination, or reproducibility failures. What an enterprise needs is not merely object storage full of files, but a data catalog and lineage system.

2. Parsing quality constrains everything downstream

Menus, advertisements, recommendation links, cookie notices, and comments can overwhelm the main text of a webpage. PDFs introduce repeated headers, broken reading order, misplaced tables, and OCR errors.

If parsing destroys semantic structure, later stages are only cleaning characters in the wrong order. Data engineering begins by recovering document boundaries and structure as faithfully as possible.

3. Language identification needs more than fastText

fastText is a useful baseline, but short passages, dialects, low-resource languages, and code mixed with natural language are easy to misclassify.

Production pipelines usually combine model predictions, character-level rules, source metadata, and human sampling. Thresholds should be calibrated per language. Applying English-centric quality rules to every language often deletes the rarest multilingual data systematically.

4. Perplexity is not a synonym for quality

KenLM perplexity, heuristics, and learned classifiers are all useful, but no single score represents quality.

High perplexity may indicate corrupted text. It may also indicate specialized terminology, new knowledge, or a low-resource language. Aggressive global thresholds can remove the material that is hardest to replace.

A stronger approach combines multiple signals, buckets data by language and domain, performs human audits, and validates filtering choices through training experiments.

5. Deduplication is more than MinHash

One article may be syndicated dozens of times. A GitHub repository may have thousands of forks. A tutorial may be copied with only its title and a few sentences changed.

A complete strategy can include:

URL and content-hash deduplication;
exact document-level deduplication;
fuzzy deduplication using MinHash, LSH, or embeddings;
cross-source and cross-language deduplication;
contamination checks between training and evaluation sets.

The goal is not to delete the largest possible amount. It is to keep the most complete, trustworthy, and highest-quality member of each duplicate cluster.

6. PII and safety require more than regular expressions

Email addresses, phone numbers, identity numbers, home addresses, API keys, and internal code can all enter a corpus.

Regular expressions handle stable formats, but production systems also need entity recognition, secret scanners, classifiers, and human review. They need deletion workflows too: when data must be withdrawn, the team should know which dataset and model versions consumed it.

Harmful content should not simply be removed by keyword. A model needs some understanding of attacks, fraud, and dangerous material in order to detect and refuse them. Context, proportion, labeling, and the training objective matter more than mere occurrence.

A data mixture is a capability budget

Training compute is finite. Increasing the sampling rate of one source reduces the training opportunity available to another.

Code contains structure, long-range dependencies, and partially verifiable outcomes. More code may improve programming and some reasoning tasks, but it does not guarantee stronger general reasoning. Too much can crowd out natural language, domain knowledge, and multilingual ability.

More Chinese data generally improves Chinese understanding, expression, and knowledge coverage. The number of tokens required to encode Chinese, however, is governed mainly by the tokenizer's vocabulary and segmentation strategy. Corpus share and tokenization efficiency are related, but they are not the same variable.

Books and papers can provide dense knowledge and long-form structure. Social media can cover current language and public opinion while introducing misinformation, abuse, and demographic bias.

There is no universal golden ratio.

A data mixture is not a shopping list of ingredients. It is the product strategy projected into the training corpus.

A coding model, a Chinese legal model, and a general assistant should not share the same recipe.

The core of synthetic data is verification, not generation

As high-quality natural data becomes scarcer, synthetic data becomes increasingly useful. Strong models can generate textbooks, reasoning problems, code, question-answer pairs, and preference examples while controlling difficulty and covering rare tasks.

But synthetic data does not create trustworthy knowledge from nothing.

Teacher errors propagate to students. Repetitive templates reduce diversity. Repeatedly generating and filtering with the same model can amplify its existing biases.

The decisive capability is therefore verification:

check mathematics against answers and constraints;
compile code and run tests and static analysis;
execute tool calls and compare real outcomes;
ground factual questions in retrieved sources;
use multi-model review and human sampling where automatic verification is impossible.

Without a verifier, synthetic data is simply a cheaper and faster way to manufacture uncertainty.

Data engineering must close the loop with evaluation

After a cleaning run, the key question is not “How many tokens remain?” It is: which capabilities improved, which regressed, and why?

model failures and user feedback
        ↓
failure taxonomy and root-cause analysis
        ↓
identify gaps, bad samples, or mixture problems
        ↓
add, correct, resample, or relabel data
        ↓
small training runs and ablations
        ↓
offline evaluation, red teaming, and online observation
        ↓
promote into the next dataset version after quality gates

At least four kinds of metrics are required:

Data metrics: duplication, language distribution, domain coverage, quality scores, and PII detections;
Training metrics: loss by data bucket, gradient anomalies, token utilization, and convergence;
Capability metrics: target-task accuracy, code pass rate, factuality, and instruction following;
Risk metrics: hallucination, harmful output, privacy leakage, bias, and benchmark contamination.

Without versioned data, training cannot be reproduced. Without ablations, a team cannot prove that a data batch helped. Without production feedback, offline leaderboards cannot establish business value.

What the data moat means for an enterprise

Most companies should not pretrain a general foundation model from scratch. The better investment is converting internal knowledge, workflows, and expert feedback into governed and measurable data assets.

I would prioritize the following:

Select three valuable business tasks with measurable outcomes;
build a separate evaluation set for each task before collecting “all the data”;
inventory sources, permissions, licenses, sensitivity, and ownership;
create pipelines for cleaning, deduplication, redaction, versioning, and lineage;
validate value through RAG or small fine-tuning runs before scaling training;
continuously feed back corrections, reviews, and failure cases;
use business outcomes—not token counts—to decide where to invest.

The most defensible enterprise data is rarely another copy of the public web. It is authorized workflow data, expert judgment, customer feedback, failure cases, and outcome labels.

Such datasets may be small, but they are closely tied to real results and difficult for competitors to reproduce.

My conclusion

Algorithms and code still matter. Training systems, architecture, optimizers, compute, and inference engineering do not lose their value merely because open source exists.

But as the baseline stack becomes shared infrastructure, the advantages that remain hard to copy are:

long-lived, authorized domain data;
an understanding of provenance, bias, and quality;
expert judgment and feedback;
the ability to turn model failures into better training data;
evaluation tied to real business outcomes.

“Garbage in, garbage out” is only the first conclusion.

I would state the complete version this way:

Parameters determine capacity. Compute determines training scale. Algorithms determine learning efficiency. Data determines what the model ultimately learns.

As code becomes common infrastructure, the real competition is no longer about who can make a model run. It is about who can continuously feed it correct, scarce, lawful, and verifiable data—and prove that the model becomes better as a result.

That is the real data moat in the age of large language models.

References

Building a Recruitment LLM from Zero to One: A BOSS Zhipin–Style Case Study

侯惠阳 — Wed, 19 Aug 2026 05:30:00 +0000

If I had to build a language model for a recruitment platform similar to BOSS Zhipin, I would not begin by buying GPUs or debating whether the model should have 7B, 32B, or 100B parameters.

I would begin with three questions:

Which business outcome must the system improve?
What lawful, distinctive, and verifiable data do we have?
Which problems belong to an LLM, and which should remain search, recommendation, or rule-based problems?

The easiest mistake in recruitment AI is to treat “building an industry model” as training a chatbot. A production system is not one model. It is a coordinated system of data, retrieval, matching, generation, risk controls, evaluation, and human decisions.

One factual boundary matters. Public filing information lists “Nanbeige,” operated by the company behind BOSS Zhipin, under filing number Beijing-NanBeiGe-20240102. Its complete data recipe, training pipeline, and internal architecture are not public.Cyberspace Administration of China filing notice

This article therefore does not claim to reconstruct BOSS Zhipin's internal system. It uses a BOSS Zhipin–style platform as a business case and presents an implementable open-source reference architecture.

What “from zero” should mean

Pretraining a general foundation model from random initialization requires trillions of tokens, long-running clusters, mature distributed systems, and substantial compute. For almost every recruitment company, that is the wrong first milestone.

A practical zero-to-one path is:

open-weight base model
    ↓
recruitment-domain continued pretraining (optional)
    ↓
task-oriented supervised fine-tuning
    ↓
preference optimization and safety alignment
    ↓
RAG, business tools, and matching systems
    ↓
offline evaluation, canary release, and feedback loop

“Self-developed” does not have to mean random initial weights. The assets a company should own are its domain data, training recipe, evaluation sets, service architecture, and learning loop.

Training from scratch becomes reasonable only when the company has a very large lawful corpus, stable compute, an experienced training team, and a base-model deficiency in tokenization, language coverage, or core behavior that adaptation cannot fix.

Step 1: turn business goals into evaluable tasks

A recruitment platform serves both candidates and recruiters, whose goals differ.

Candidates need to know which roles genuinely fit, which requirements are hard constraints, how to improve a résumé, how to communicate effectively, and whether a role or company is trustworthy.

Recruiters need to turn ambiguous needs into clear job descriptions, discover qualified candidates, understand skill fit, reduce unproductive conversations, and identify fraudulent résumés, jobs, or accounts.

Version one should not solve everything. I would choose three measurable scenarios:

Scenario	Output	Primary metrics
JD normalization and rewriting	Standard title, skills, experience, location, compensation, and a faithful rewrite	Field F1, factual fidelity, publish acceptance
Candidate–job explanation	Matches, gaps, evidence, and uncertainty	Recall@K, NDCG, evidence accuracy
Recruitment copilot	Questions, replies, and next-step suggestions grounded in the job and résumé	Reply rate, human acceptance, safety violations

Do not use hiring rate as the only early metric. Hiring is delayed and influenced by compensation, location, employer brand, recruiter behavior, and the economy. Early evaluation needs model-quality metrics, process metrics, and business outcomes together.

Step 2: separate the system instead of asking the LLM to do everything

A recruitment platform needs a layered architecture:

                        ┌─────────────────────────────┐
candidate / recruiter → │ chat, search, recommendation│
                        └──────────────┬──────────────┘
                                       ↓
                        ┌─────────────────────────────┐
                        │ task routing and policy      │
                        │ intent / auth / risk / cost  │
                        └───────┬──────────┬──────────┘
                                │          │
                  ┌─────────────┘          └──────────────┐
                  ↓                                          ↓
       ┌───────────────────────┐                ┌───────────────────────┐
       │ recruitment LLM       │                │ retrieval and ranking │
       │ extract/generate/explain│              │ two-tower/rank/rerank │
       └───────────┬───────────┘                └───────────┬───────────┘
                   ↓                                            ↓
       ┌───────────────────────┐                ┌───────────────────────┐
       │ RAG and business tools│                │ feature/vector platform│
       │ jobs/company/API      │                │ user/job/behavior      │
       └───────────┬───────────┘                └───────────┬───────────┘
                   └───────────────────┬────────────────────┘
                                       ↓
                        ┌─────────────────────────────┐
                        │ governance, eval, audit, loop│
                        └─────────────────────────────┘

LLMs are strong at semantic interpretation, extraction, generation, explanations, and tool orchestration. They are not the right engine for low-latency retrieval across millions of candidates, and they should not make opaque rejection decisions by themselves.

A mature design usually assigns:

search and two-tower models to high-scale retrieval;
ranking models to combine semantics, behavior, freshness, and constraints;
the LLM to parse natural language, enrich structured features, explain results, and conduct dialogue;
rules and safety models to hard constraints, authorization, fraud, and compliance;
humans to the final application, contact, interview, and hiring decisions.

A minimal open-source stack

Do not accumulate frameworks merely to claim an open stack. Give each layer one primary implementation and a replaceable boundary:

Layer	Initial choice	Output contract
Data	Python/SQL + DataTrove or Spark	Versioned Parquet, dataset card, lineage
Training	PyTorch + LLaMA-Factory/TRL; FSDP or DeepSpeed when scale requires it	Adapter/checkpoint and metrics
Lexical/vector search	OpenSearch, or PostgreSQL + pgvector initially	Filtered candidates and evidence
Embedding/reranking	Open-weight bilingual embedding + cross-encoder	Vectors and relevance scores
Serving	vLLM or SGLang	OpenAI-compatible API
Experiment/evaluation	MLflow + private gold sets and regression scripts	Versioned data, model, prompt, metrics

Tool names matter less than interfaces. Training examples, embeddings, retrieval results, generations, and evaluations should carry dataset_version, model_version, prompt_version, and trace_id; otherwise production failures cannot be reconstructed.

Step 3: build a recruitment ontology first

More data creates more contradiction when terms do not share a common meaning.

“Senior Java developer,” “backend engineer,” and “server-side engineer” may belong to the same job family. “LLM experience” may mean calling an API, or it may mean training, evaluation, and inference optimization.

I would first build a versioned ontology:

job family → normalized title → specialization → skill → proficiency
                                                   ├─ required/preferred
                                                   ├─ years of use
                                                   └─ recency

job → industry → company stage → location → work mode → compensation

candidate → experience → project → responsibility → action → result → evidence

The ontology does not force every employer to use the same title. It gives retrieval, training, and evaluation a shared coordinate system. Every mapping should retain the original phrase, normalized value, confidence, and ontology version.

A structured JD example should include evidence:

{
  "source_text": "Hiring a senior backend engineer, 5+ years, Java and microservices required; LLM experience preferred.",
  "job_family": "Software Engineering",
  "normalized_title": "Senior Backend Engineer",
  "required_skills": ["Java", "Microservices"],
  "preferred_skills": ["Large Language Models"],
  "experience_min_years": 5,
  "evidence": {
    "experience_min_years": "5+ years",
    "required_skills": ["Java and microservices required"]
  },
  "confidence": 0.96,
  "ontology_version": "job-ontology-2026-08"
}

The important field is not JSON; it is evidence. Recruitment outputs must remain traceable to source text.

Step 4: create lawful, traceable data assets

Recruitment data is unusually sensitive. Résumés can contain names, contact details, education, employment history, location, and salary expectations. Conversations can reveal family circumstances, health information, and other private details.

I would separate data into four layers:

Layer	Examples	Main use
Public knowledge	Occupational taxonomies, public skill documentation, labor rules, public company information	Domain knowledge and RAG
Platform data	Authorized JDs, de-identified résumés, search and interaction data	Continued pretraining and matching
Expert labels	Normalized roles, skill evidence, fit judgments, preferred responses	SFT, preference optimization, evaluation
Online feedback	Accept, edit, reply, apply, interview, complaint	Iteration and impact measurement

Every record needs provenance, authorization basis, collection time, owner, sensitivity, permitted tasks, transformation history, retention, and deletion status.

The fact that data exists in a platform database does not automatically authorize model training. Product operation, recommendation, fraud prevention, and training are different processing purposes and require separate review.

Recruitment data pipeline

source registration and permission review
        ↓
parse JDs, résumés, conversations, and event logs
        ↓
remove templates, ads, contact solicitation, and corruption
        ↓
detect language, job family, industry, and content type
        ↓
detect PII, tokenize replacements, isolate identity mapping
        ↓
exact dedup, résumé-version dedup, fuzzy JD-template dedup
        ↓
detect fraud, low quality, discriminatory text, and conflicts
        ↓
quality scoring and evidence-completeness checks
        ↓
split by time, user, and company—not random rows
        ↓
dataset card, version, and audit record

Regular expressions are insufficient for de-identification. Emails and phone numbers fit rules; names, schools, project descriptions, addresses, and identity hints in free text need NER, dictionaries, model-based detection, and sampled review.

Random row splits are also unsafe. Multiple résumés from one person, templated JDs from one employer, or messages from one conversation can leak across train and test sets. Group by person and employer, split by time, and check contamination against public benchmarks.

Step 5: choose a base model by workload, not leaderboard rank

For Chinese recruitment, candidate open-weight families include Qwen, GLM, and Llama. The Qwen ecosystem supports Transformers, vLLM, SGLang, and fine-tuning frameworks such as LLaMA-Factory and TRL.Qwen official repository

Evaluate:

License: commercial and derivative-model terms;
Chinese and mixed-language quality: titles and technical stacks often mix Chinese and English;
Structured output: JSON Schema adherence and field stability;
Long context: full résumés, JDs, and conversations—not just advertised context length;
Tool calling: argument accuracy, recovery, and authorization boundaries;
Serving cost: TTFT, TPOT, throughput, and memory at target concurrency;
Domain baseline: performance on the company's recruitment evaluation suite.

A 7B–14B model is often the right first experiment. Prove the task and data before scaling. Larger models are not automatically better for every classifier or extractor, and stable tasks can later be distilled into smaller models.

Measure the tokenizer on real JDs, résumés, and technical vocabulary: characters per token, truncation rate, and terminology fragmentation. Do not modify the vocabulary unless measured gains justify compatibility and retraining costs.

Step 6: train in stages

Stage A: continued pretraining (optional)

Continued pretraining teaches domain language and relationships using large amounts of unlabeled text. Candidate sources include authorized and deduplicated JDs, de-identified occupational material, public labor rules, high-quality industry documents, and strictly governed business text.

Do not dump all résumés and conversations into pretraining. Apply PII, safety, quality, duplication, and mixture controls first. Mix in some general data to reduce catastrophic forgetting, and use small ablations to determine whether continued pretraining is worth its cost.

If domain text is limited and the base model is already strong, SFT plus RAG is often a better investment.

Continued pretraining still optimizes next-token cross-entropy:

L_CPT = -Σ log Pθ(x_t | x_<t)

Four engineering controls matter:

Mixture: sweep candidate domain/general ratios such as 3:7, 5:5, and 7:3 in small runs instead of treating one ratio as law;
Boundaries: sequence packing improves utilization, but attention must not leak across documents;
Token budget: estimate effective tokens multiplied by cost per training token, not file size;
Stopping: monitor domain validation loss, general-capability regression, and target-task metrics together.

Each checkpoint needs a data manifest, code revision, random seed, optimizer state, and mixture. Falling domain loss with large general regressions indicates forgetting, not success.

Stage B: supervised fine-tuning

SFT should cover:

JD and résumé extraction;
skill and experience evidence;
search-constraint parsing;
fit explanations and gap analysis;
JD rewriting and résumé suggestions;
recruitment conversation and tool calls;
clarification and abstention when information is missing;
privacy, fairness, and safety cases.

A high-quality example includes not only a prompt and response, but task type, source evidence, allowed tools, expected output, risk labels, and dataset version.

{
  "messages": [
    {"role": "system", "content": "You are a recruitment assistant. Use only the supplied job and résumé. Never infer age, marital status, family plans, or health."},
    {"role": "user", "content": "Explain matches and gaps.\n[JOB]...\n[RESUME]..."},
    {"role": "assistant", "content": "{\"matched\":[...],\"gaps\":[...],\"unknown\":[...],\"evidence\":[...]}"}
  ],
  "task": "job_candidate_explanation",
  "risk_tags": ["employment", "personal_information"],
  "dataset_version": "recruit-sft-2026-08"
}

LoRA or QLoRA is appropriate for fast validation. LLaMA-Factory supports continued pretraining, SFT, DPO, and related workflows.LLaMA-Factory

model_name_or_path: Qwen/Qwen3-8B
stage: sft
do_train: true
finetuning_type: lora
lora_target: all
dataset: recruitment_sft
template: qwen3
cutoff_len: 4096
learning_rate: 1.0e-4
num_train_epochs: 2.0
bf16: true
val_size: 0.05

llamafactory-cli train recruitment_sft.yaml

These are starting parameters, not a universal recipe. Learning rate, epochs, sequence length, rank, and mixture require validation and ablation.

Several SFT details are easy to miss:

normally compute loss only on assistant tokens instead of teaching the model to repeat system and user text;
bucket and pack sequences to reduce padding while preserving conversation boundaries;
evaluate both token loss and schema validity for structured tasks—they do not always move together;
start LoRA experiments around rank 16 or 32 on attention and MLP linear layers, then expand through ablation;
maintain a blind test set never used during prompt iteration.

Whether full fine-tuning beats LoRA depends on dataset size, task breadth, and compute. First use LoRA to prove the data; run a controlled full-tuning comparison only when adapter capacity is a measured bottleneck.

Stage C: preference optimization

Once the model can perform the task but remains inconsistent in style, evidence, or refusal boundaries, collect preferred and rejected responses.

Prefer grounded explanations over unsupported judgments, uncertainty over fabricated experience, capability-related language over age or family proxies, concise action over empty prose, and valid tool calls over invented live job data.

DPO is simpler than traditional RLHF for an initial system. TRL provides SFT, DPO, reward modeling, and other post-training tools.Hugging Face TRL

Do not adopt GRPO simply because it is fashionable. Reinforcement learning is most dependable when rewards are verifiable—schema validity, SQL execution, code tests, or deterministic rules. Subjective employment judgments should never be reduced to one reward model.

DPO directly increases the relative probability of a preferred response over a rejected one:

L_DPO = -log σ(β[(log πθ(y+|x)-log πref(y+|x))
                 -(log πθ(y-|x)-log πref(y-|x))])

β controls deviation from the reference model. Bucket by task, length, and risk, and verify that “preference improvement” is not merely verbosity. When recruitment experts genuinely disagree, retain annotator distributions or send the case to review instead of inventing one truth.

Step 7: use RAG for changing facts and tools for actions

Job availability, compensation, employer information, and application status change continuously. They should not be memorized in model parameters.

The retrieval layer can include current jobs and versions, verified employer data, skill knowledge, policies, labor rules, and only the résumés or conversations the current user is authorized to access.

question
   ↓
intent and authorization
   ↓
structured filters: location, salary, experience, status
   ↓
hybrid retrieval: lexical + vector
   ↓
reranker
   ↓
evidence compression and citations
   ↓
generation or business-tool call
   ↓
factuality, authorization, and safety checks

Recruitment search cannot be vector similarity alone. Location, compensation, experience, job status, and access are hard filters. Semantic retrieval handles differences such as “AI platform engineering” versus “LLM infrastructure.” A reranker then scores a small candidate set.

Do not add raw BM25 and vector scores directly; they usually have incompatible scales. Reciprocal Rank Fusion is a robust first implementation:

RRF(d) = Σ 1 / (k + rank_i(d))

Retrieve lexical and dense Top-K lists separately, fuse their ranks, and send the best 50–200 records to a cross-encoder. Enforce tenant_id, status, location, compensation, freshness, and authorization filters in the index or service before evidence reaches the LLM—not by redacting after generation.

Write actions—submitting a résumé, sending a message, or editing a JD—need typed schemas, server-side reauthorization, idempotency keys, preview and confirmation, timeout and compensation behavior, and audit logs. The model may propose an action; it must not bypass business permissions.

Step 8: train matching as a separate system

Large-scale job matching remains a retrieval and recommendation problem.

Use a two-tower model to encode candidate and job representations for ANN retrieval. Apply a cross-encoder or learning-to-rank model to the retrieved set, then rerank for freshness, diversity, deduplication, and business constraints.

candidate profile → candidate tower ─┐
                                     ├─ similarity → ANN Top-K
job profile       → job tower ───────┘
                                           ↓
                            cross-encoder / learning-to-rank
                                           ↓
                         rules, freshness, diversity, fairness
                                           ↓
                              LLM-generated explanation

An LLM can enrich features, produce weak labels, and explain results. It should not iterate over millions of jobs at inference time.

Labels also need care. A click is not ground truth. Exposure is controlled by the previous model; clicks respond to titles and compensation; conversations depend on recruiter activity. Use layered events—impression, click, save, apply, reply, interview, hire—and account for position bias, negative sampling, and delayed outcomes.

Two-tower retrieval can use a contrastive objective. For candidate vector u, positive job v+, and jobs in the batch:

L_retrieval = -log exp(sim(u,v+)/τ) / Σ_j exp(sim(u,v_j)/τ)

Random negatives are often too easy. Add hard negatives such as the same title at a different seniority or the same skill set in an incompatible location. An exposed but unclicked job is not necessarily negative—it may have appeared too low—so preserve exposure position and filter likely false negatives.

The ranker can combine semantic crosses, structured compatibility, freshness, and behavior. LLM explanations must read the factual features and source evidence used by the final ranking, avoiding one mechanism for ranking and an invented story for explanation.

Step 9: build evaluation before large training runs

Without a private recruitment benchmark, the team cannot know whether domain training helped.

Understanding and generation

Capability	Metrics
Field extraction	Precision, recall, F1, schema-valid rate
JD rewriting	Factual fidelity, completeness, violations, acceptance
Fit explanation	Evidence precision, omissions, unsupported-claim rate
RAG	Recall@K, citation correctness, groundedness, abstention
Tool use	Argument accuracy, execution success, unauthorized calls, duplicates
Safety	PII leakage, discriminatory language, prompt-injection success

Matching and ranking

Recall@K for retrieval;
MRR and NDCG@K for ordering;
coverage for long-tail jobs and new candidates;
calibration between scores and outcomes;
subgroup differences across legitimate evaluation slices;
online valid-contact, reply, application, interview, and complaint rates.

Freeze the test set by time and exclude future feedback from training. General leaderboards cannot replace domain evaluation.

Release through shadow traffic first: generate outcomes without affecting users, compare against production, then progress through a small canary only after safety, quality, latency, and cost gates pass.

Step 10: production is more than starting vLLM

Open-source engines such as vLLM and SGLang provide efficient serving, but production also requires gateways, routing, caching, degradation, and observability.vLLM

API gateway
   ↓
identity, quota, length, and risk checks
   ↓
task router
   ├─ small model: classification, extraction, rewriting
   ├─ larger model: complex analysis and dialogue
   ├─ embedding/reranker: retrieval and matching
   └─ rule service: constraints and safety
   ↓
batching, KV cache, timeout, circuit breaker
   ↓
schema, citation, and safety validation
   ↓
logs, traces, cost, and quality sampling

Observe time to first token, time per output token, total latency, queue time, token counts, GPU utilization, cost per task, timeout, cancellation, retry, fallback, JSON failures, tool failures, and unsupported claims—split by model, task, language, and version.

A model upgrade is not one file replacement. Prompts, tokenizers, retrieval, tool schemas, quantization, and decoding parameters all affect behavior and need coordinated versioning.

A first capacity approximation is:

replicas ≈ peak QPS × P95 service time
           ÷ safe concurrency per replica × headroom

Correct it with load tests using real input and output lengths. Benchmark short extraction, long-résumé analysis, and multi-turn dialogue separately; averages hide long-tail out-of-memory failures. Compare BF16, FP8/INT8, and lower-bit quantization on the recruitment gold set before production—throughput alone is not an acceptance test.

The safety line in recruitment

Recruitment is not ordinary text generation. A bad recommendation wastes time; an opaque rejection can affect a person's livelihood.

China's Personal Information Protection Law requires transparency and fairness when personal information is used for automated decision-making.Personal Information Protection Law

At minimum:

Do not use sex, age, ethnicity, family status, or health as matching features without a lawful and justified basis;
test proxy effects from school, address, career gaps, and similar attributes;
never let the LLM make final hiring or rejection decisions alone;
ground important recommendations in job and résumé evidence;
provide human review, correction, opt-out, and appeal paths;
enforce résumé authorization at retrieval, prompt, log, and cache layers;
audit dataset, model, prompt, and tool-call versions;
complete applicable security assessment, algorithm filing, or model registration before public launch.

Removing a name does not make a model fair. Schools, postal codes, employment years, and writing style can become identity proxies. Fairness review requires lawful subgroup analysis and joint ownership across legal, ethics, recruitment, product, and ML teams.

An executable 12-week plan

Period	Goal	Deliverable
Weeks 1–2	Tasks and baselines	Three scenarios, gold evaluation set, production baseline, compliance checklist
Weeks 3–4	Data and ontology	Ontology v1, lineage, de-identification pipeline, dataset v1
Weeks 5–6	Open-model PoC	Three-model benchmark, LoRA SFT, error taxonomy
Weeks 7–8	RAG and matching	Hybrid retrieval, reranker, two-tower baseline, citations
Weeks 9–10	Preference and safety	DPO set, safety suite, red team, authorization and audit
Week 11	Shadow and canary	Shadow report, latency/cost report, 1–5% canary
Week 12	Review	Business impact, regression report, decision on scale and model size

Define release gates in advance: extraction F1, schema validity, unsupported-claim rate, zero severe PII or authorization failures, P95 latency and cost budgets, and a statistically credible lift in acceptance or valid conversations.

Exact thresholds must come from the business baseline, not another company's blog post.

Seven common failure modes

Train first, search for a use case later: the result is a demo chatbot without a business outcome.
Treat database access as training consent: résumé, conversation, and behavior purposes are ignored.
Randomly split rows: users and templates leak into test data and inflate scores.
Build only an LLM: no scalable retrieval, high latency, and high cost.
Trust general benchmarks: the model solves math but cannot extract salary evidence consistently.
Treat clicks as truth: the new system learns the old system's exposure and position bias.
Automatically reject candidates: no evidence, explanation, human review, or fairness control.

My conclusion

The moat in a recruitment model is not renaming an open model or maximizing parameter count.

It comes from:

a consistent, evolving job and skill ontology;
lawful governance of high-quality recruitment data;
correct boundaries between search, recommendation, LLMs, RAG, and rules;
evaluation against both business outcomes and safety constraints;
a loop that turns failures, edits, and final outcomes back into better data.

Without these five elements, a self-developed model is an expensive chatbot.

With them, even a 7B–14B open-weight model can create real value in version one.

Building a recruitment LLM from zero to one is not writing a Transformer from zero. It is building, from zero, a system that connects recruitment knowledge, business data, model capability, and real outcomes.

DEV Community: 侯惠阳

Building Production Intelligent Risk Control: Streaming, Lakehouse, Rules, and Models

Architecture: two paths, one fact history, full replay

Establish an immutable risk fact stream

Define sliding windows in event time

Streaming features must leave Spark memory

Rules and models are complementary

Modernize the Hadoop layer into a governed lakehouse

Every decision must be replayable

Availability requires explicit degradation semantics

Observe systems, data, and decisions

A practical delivery sequence

1. Facts and the minimum decision loop

2. Real-time state

3. Rule operations and replay

4. Models and continuous optimization

The standard for production maturity

Further reading

From Query Understanding to Knowledge-Driven Recommendations

Why tax and finance search is unusually difficult

Query understanding should produce a contract

Replace algorithm inventories with an observable pipeline

Do not choose between lexical and vector retrieval

Recommendation is a decision process, not a model

The knowledge layer sets the long-term ceiling

Acquire with provenance

Structure only what affects decisions

Resolve identity and conflict

Evaluate and update incrementally

Where LLMs belong

Evaluation compounds faster than model selection

How I would start today

Further reading

How to Build an AI-Native Team

AI-enabled is not AI-native

A new operating system for work

1. Goals

2. Workflows

3. Human-agent responsibilities

4. Organizational context

5. Evaluation and governance

The new unit of work: a person with agents

Six principles for AI-native workflows

1. Begin with acceptance criteria

2. Treat context as capital

3. Prefer simple, composable patterns

4. Match autonomy to risk

5. Make every step observable

6. Turn failure into system memory

Do not begin with an isolated “AI department”

Leadership owns direction and boundaries

A platform team owns shared infrastructure

Business teams own outcomes

AI champions accelerate learning

Choose the first workflow carefully

Move beyond a prompt library

Governance is a prerequisite for scale

A practical 90-day path

Days 1–30: diagnose and ignite

Days 31–60: redesign process and context

Days 61–90: institutionalize and expand

Measure outcomes, not AI activity

AI-native organizations still amplify human judgment

Further reading

My AI Programming Workbench: Orchestrating Models Instead of Hunting for One Best Tool

A layered workbench, not a list of tools

Start with project context, not a clever prompt

Let a strong model design, but keep architecture accountable to people

Decompose work around boundaries and evidence

Cross-review: different tools should search for different failures

The final judge is a quality gate, not a model

Turn every correction into default capability

My conclusion: the developer becomes the designer of the work system

Agent Architecture for SMBs: A 90-Day Path to Production

Does the problem need an agent?

Adapt enterprise layers to SMB constraints

The minimum production architecture

1. Interaction and gateway

2. Deterministic orchestration first

3. Agent runtime and model gateway