DEV Community: Artsiom Rudzenka

From too many sources to knowledge I can use again

Artsiom Rudzenka — Fri, 16 Oct 2026 06:55:57 +0000

Hi! There were more useful sources than I could read carefully. Saving every promising link would only move the pile into a reading list. I wanted to choose what deserved attention and keep the ideas I would need again.

That raised another question: when I returned to a problem, what should I be able to recover from my own research? A useful note would give me the idea, the source behind it, and enough context to judge whether it applied to the task in front of me.

This became Research Brain: a workflow for collecting material, turning selected sources into linked notes, and consulting those notes later. I wanted that knowledge to be available in other projects, where my agent and I were making development decisions.

From a research wiki to a working workflow

I started by trying Karpathy's LLM Wiki pattern and an Obsidian setup using the Model Context Protocol (MCP). They gave me a starting point: keep research in a wiki that an agent can maintain and consult.

I judged the result by whether I could reuse what I had learned. For that, I needed rules around the wiki: how a source gets in, what evidence a note retains, when it needs another check, and what the agent may change. I also needed to notice when collection or delivery failed. Working through those requirements, and fixing what failed in practice, shaped my own implementation.

The knowledge store is an Obsidian vault—a local folder of Markdown notes. A concept note keeps an idea together with its sources, limits, and open questions. I can read and edit those files directly; the optional MCP adapter lets an agent in another project retrieve them. Source collection and the configured model can still use external services.

The digest and the adapter serve different moments in the same workflow. The digest surfaces new material and note updates for reading; retrieval brings existing notes into a current task.

How material becomes a note

The full workflow runs inside an agent runtime. It collects candidates from configured sources, including feeds, official pages, and search results, then selects material for the configured topics. URL and text deduplication reduce repeated material before it moves further through the pipeline.

Before the agent curates or synthesizes article text, a sanitizer removes known prompt-injection patterns and flags others. Curation then applies the topic's inclusion and exclusion rules and a minimum text length. The selected material becomes the input for synthesis into concept notes.

The full agent workflow supports reading and reuse. The reduced RSS-only fallback builds a digest without synthesizing concept notes.

These steps filter different kinds of problems. A duplicate is repeated material. An off-topic article does not belong in this collection. A hidden instruction should not become a command to the agent. None of those checks establishes that an article's claims are true.

The security boundary also continues after ingestion. The MCP adapter labels retrieved notes as untrusted data and exposes read-only retrieval, with no shell or arbitrary-file tool. Filters can miss attacks; the connected agent still needs appropriate tool permissions. This follows the concern described in OWASP's guidance on indirect prompt injection: instructions can arrive inside material the agent was only meant to read.

To check the facts, I need the source trail to survive synthesis. Notes carry source records, and a mechanical health check flags missing citations, broken links, and other maintenance issues. Several publishers can repeat one upstream story, so counting sources is not enough to establish independent confirmation.

Asking the notes from another project

Once the notes existed, the next step was to make them available where a question arose. The local MCP adapter provides two tools: search_knowledge finds candidate notes, and get_concept opens a selected note in more detail.

The current search uses weighted word matches. It breaks the query into words, removes common English query words, and checks the note's title, filename, tags, vault-index context, and body. An optional topic filter narrows the candidates; problem-and-solution notes can also be filtered by programming language and problem type. Title matches count more than body matches; the adapter returns up to two notes with excerpts, recorded sources, freshness information, and open questions.

The search finds candidate context. The agent still has to decide whether that context supports the current task.

There are no embeddings or semantic search in this adapter. Freshness is reported from the note's recorded verification date and review interval; it does not improve the match score or certify that the source is still correct. Generic words can produce irrelevant hits, and a useful note can be missed when the query uses different wording.

When the vault has no useful answer, I want that gap to stay visible. In vault-only mode, the lookup stops and offers further research. In research mode, it tells the host agent to carry the retrieved context and gaps into the existing research workflow: check local material first, then ask before external research. The MCP server itself does no web research. Project findings go through a separate write workflow with explicit approval; the weekly workflow can ingest eligible research notes unattended.

Closing the loop between projects

Retrieval left another question open: could a solution found during project work become useful knowledge for the next project? I tested that path with an agent on real unfinished tasks, using a private evaluation vault.

In the first project, an HTTP collector ignored the server's Retry-After hint and used its fixed retry delay. The agent checked RFC 9110, section 10.2.3 and corrected the retry loop: wait the full requested pause when it fits the local budget, otherwise stop. Offline regression tests reproduced the problem and passed after the change.

The agent then prepared an update to the existing note. It kept the problem, the rule and source, the checked result, and the conditions another project would need to reconsider. I approved the exact change before it entered Research Brain. Saving used a separate reviewed workflow; MCP remained read-only.

From a second project, the agent retrieved the saved note through MCP. That collector shortened server pauses to its own limit and used a total deadline rather than the first collector's cumulative sleep allowance. The note supplied the rule and questions; the agent still had to inspect the local code, clocks, error handling, and callers. It adapted the fix and checked it with offline regressions. The result was then proposed back to the note and saved with my approval, including the remaining limits.

Project findings use a separate write workflow. Reuse can lead to another proposal; each save needs direct owner approval.

This completed a coordinated loop of saving, reuse and another approved update. The agent kept earlier context, and the faults were chosen before lookup. Offline tests used simulated responses and clocks. They did not test discovery by a new agent, automatic note synthesis or live traffic. They also did not show that separate jobs would respect the same wait, or that the changes were ready to release.

When the notes did not help

A school-class comparison task gave a different result. The agent queried a separate working vault of 39 notes. Broad wording returned unrelated matches; a more specific admissions query returned nothing. The agent set those results aside and completed the change using the project's own rules and retained data.

Matching familiar words could make irrelevant notes look promising. Improving retrieval and growing useful coverage are both necessary; one does not automatically solve the other.

What I would keep testing

What I found useful was having sources, limits, and questions for further checking together. These recorded trials do not establish time savings or better decisions. The longer-term question is whether recurring collection and retrieval justify the effort of preparing, verifying, and maintaining the notes.

For someone trying the idea, I would start with one recurring question. Keep a note with the rule, its supporting source, and the conditions that could change the answer. When the question returns, check whether the note helps you decide what to inspect next—and count the effort needed to verify and update it.

The public Research Brain repository now includes the project-solution capture workflow, its record template, and tests alongside collection and retrieval. The problem-and-solution guide explains how to validate a finding, review the exact proposed change, and save it after direct owner approval.

A disposable synthetic example exercises saving a finding and retrieving it from another project. It checks the workflow, not the private application runs described here. The MCP setup guide covers connecting the adapter.

When you return to something you researched earlier, what do you usually have to reconstruct?

When does an integration help a sales decision? — SalesWiki, Part 5 of 5

Artsiom Rudzenka — Mon, 12 Oct 2026 13:29:46 +0000

Before your next customer conversation, which detail would you check—and where would you find it?

The account summary may be in your customer system, the latest detail in a document, and a promise in a chat thread. Finding each piece still leaves a question: what can you rely on for your next step?

That is the question behind my SalesWiki research. I am exploring a shared knowledge base for sales and marketing: linked account context, different access for different roles, and answers that show their sources and gaps. The aim is to spend less effort rebuilding context so people have more room to think through the decision.

In Part 4, I looked at what must change before a demo uses real customer data. This final part follows one example through a check, then asks when connecting another tool would help. No technical background is needed.

Before contacting Atlas Foods

Atlas Foods is an invented account in the demo, not a customer case. Its account summary was last reviewed on August 12. A call note is dated August 18, and an audit source August 21. Both are newer than the summary.

The audit source says the team wants to reduce manual evidence gathering during each audit cycle. That could matter when preparing the conversation: ask which part of the audit takes the most effort, rather than starting with a general product pitch. It still does not tell us the customer's current evaluation date.

Current demo, cropped to the check and its dated sources. All account data is synthetic.

An old summary is not a reason to drop the account. A newer document is not automatically the full answer either. I would compare them, keep the useful history, and ask the account owner for the missing date before deciding how to follow up.

For marketing, the question might be different: which approved detail can support a useful message? The same account can help both teams, but each needs the facts relevant to its task and permitted for its role.

Getting an answer is different from changing the record

This is the distinction I want SalesWiki to make clear. Someone asking a question needs an answer they can check. Someone correcting the shared account context needs a review path.

The intended workflow in plain language. A proposed change is reviewed before it is applied; reading a source does not update a card.

The knowledge base holds the team's current understanding and links back to the evidence. It does not need to become another copy of every document, recording and customer system. Original sources stay intact. Account summaries and working conclusions can change through review, with a history of what changed and why.

The assistant's role is to bring the relevant, permitted context together and make gaps visible. It should not quietly fill in a missing date or decide that an account is no longer worth attention. The person makes that judgment.

A useful check can end with “we still don't know”

For this demonstration, I opened the audit source, then recorded “Could not verify” and “Ask for missing context.” The source explains a need, but it does not settle the current evaluation date. I left the effect on the next decision as “Cannot assess yet.”

An illustrative result I recorded in the synthetic demo. It records a next step, not a confirmed customer finding or an updated account card.

This gives the check an outcome: ask for the missing context, then return to the decision. Recording a result does not make the old account summary current. In the browser demo, these personal checks reset on reload and do not send reminders. A correction to the shared card has a separate approval path.

We also do not know why the account review fell behind. Perhaps a source changed, perhaps a review was missed; the dates alone cannot tell us. A later private pilot would need enough history to investigate recurring gaps without guessing their cause. The Government Data Quality Framework is a useful reference for tracking quality issues over time and addressing their causes. That is guidance for the research, not a claim that the demo diagnoses them automatically.

When would another connection earn its place?

Now the integration question is easier to answer. What missing fact would the connection supply, or where would it help someone use the answer?

For a sales follow-up, a connection to the customer system could supply an agreed next step and its date. For a campaign, a document connection could supply approved evidence for a message. A chat interface could bring the cited answer to a place where the team already works. These are examples to test, not live connections shipped by the demo.

I would start with one decision and one narrow connection. First read the needed information. If anything should change in the shared record, propose the exact change for review. Access rules still apply, and the original source remains available to check.

If the only benefit is “we can search more material,” I would first ask whether the team's existing search tools already do enough. A connection also brings work: permissions, failed updates, source checks and maintenance. That effort belongs in the evaluation.

Five lessons I am taking forward

Across the series, the idea has become more concrete:

Keep account understanding connected. People should be able to follow an account, a conversation and the evidence behind a conclusion without rebuilding the story each time.
Separate access from relevance. A role defines what someone may see; their task defines which of those facts help them decide.
Keep dates and gaps visible. Old information can matter. Missing information should lead to a check, not a confident guess.
Make corrections traceable. Preserve original evidence, review changes, and record the result.
Connect tools for a reason. Supply a needed fact or bring an answer to the person who needs it. Count the ongoing effort as well as the benefit.

The demo shows how these steps can work together with invented data. It does not yet show that a real team will make better decisions or spend less time searching.

My next useful test would be one repeated decision alongside the team's current process: did the answer change the next step for a defensible reason? Did someone catch a missing fact before acting? Was the effort of checking and keeping the knowledge current worth it?

That is where I want the research to go. I hope to automate some of the work of finding and checking context, while leaving more space for the part that needs a person's judgment. The next version should follow what people find useful—even if that means a simpler tool is enough.

Try it or follow the series

Open the synthetic demo. Choose Atlas Foods, open “Check account context,” and inspect a source. Then ask: what would I need to know before taking the next step?

If you want to explore the implementation, the public repository and local setup guide are available. The demo is a research prototype; real customer use needs a separate private setup with real identity, access and review controls.

Continue the series

This five-part series explores how shared sales and marketing context can support decisions people can check.

Previous: Part 4

Which missing detail most often makes you stop and check before a customer conversation?

What must change before a sales demo uses real data? — SalesWiki, Part 4 of 5

Artsiom Rudzenka — Wed, 07 Oct 2026 11:02:21 +0000

The public demo uses invented accounts. You can trace an answer to its sources and see how a correction enters review.

If you are responsible for sales or marketing operations, the practical question is what it would take to test one real decision without exposing customer data or mistaking a convincing demo for proof that it helps.

In Part 3, I chose one repeated decision to test: which account needs attention today, and why? This article looks at how to run the synthetic demo, where real data belongs, and what a private pilot would need to prove. The demo itself does not show that the workflow helps a real team.

The boundary is simple: the public repository holds the platform, its demo holds synthetic cards, and real customer data belongs in a separate private vault.

Contour	Location	Data rule
Platform	Public SalesWiki repository	Code, schemas, synthetic examples and setup guides
Demo	`demo/` inside the repository	Synthetic cards only
Pilot	A separate private vault outside the repository	Real accounts, deals, contacts and transcripts

The public repository supports a browser preview and a local synthetic service. A shared service with real customer data would need further work.

Inspect the synthetic demo

The public Knowledge Workbench is the quickest way to see the synthetic workflow. Select Tour in the top bar. Its eight-step quick tour takes about two minutes: it checks dated evidence when a signal is stale, shows how roles get different views of shared evidence, and follows an answer to its sources. It also opens a personal context check: take the check, compare the sources and record what would change your next decision. The tour itself saves nothing. A cause remains unknown unless the available evidence supports an assessment.

For example, the Atlas Foods summary is older than two of its sources. In the demo, you can take that check yourself, record Update needed, link the source you checked and choose Propose a card update as the next step. Recording the result does not refresh the card or approve the proposed change.

Synthetic example: the check result and next action are recorded separately from the account's stored freshness.

In the hosted browser demo, these personal checks stay in the current tab and reset on reload. The local service keeps them in a separate runtime journal. Each person sees only their own checks, including people using reviewer roles.

To run the demo locally, clone the public repository and follow the quick start. It gives separate commands for the browser preview and the local permissioned service. This verifies the synthetic setup; it does not show whether the workflow helps a real team.

Keep the data boundary in place

The local demo service refuses a vault that is not marked synthetic. Changing an environment variable is not enough to turn it into a real-data service. A private pilot needs a separate vault and verified identity before anyone can retrieve customer information. Keep that vault outside the public repository.

Start a private pilot carefully

Do not copy real customer records or call transcripts into this repository. Create a separate access-controlled vault with no public remote. Keep personal-data bodies out of Git. Use external restricted references where needed.

The smallest useful pilot tests one repeated decision. A possible sales question is: which lead needs a follow-up today, and why? Part 3 used a related account-follow-up question. Choose one exact decision and keep it fixed. This is a test plan, not evidence that a real team has validated the workflow. A later marketing pilot can use the same method once its decision and success criteria are clear.

A useful pilot should prove more than document retrieval. Continue only if real work validates all four points:

The cited answer helps people make the chosen decision with less searching than their current workflow.
At least one correction is clearer or safer after a proposal, review and controlled update.
Keeping the data and access rules under the team's control solves a real need.
Users can explain why they chose the action and what evidence would change it.

If the pilot mainly proves that people want document search or summarization, an existing product may be the cheaper answer.

Before the first real account enters the pilot, write down the test plan: the decision, the allowed evidence, the current workflow, and what would make the team stop or change direction. Count an answer without a source, restricted information reaching the wrong person, or an action the user cannot explain as a failed case. Do not change the allowed sources, access rules and answer format all at once; otherwise the team will not know what caused an improvement or a failure.

The personal journal is a starting point for recording what happens when a fact needs review. A stale flag tells us that a review window passed; by itself, it does not tell us whether a source changed, a sync failed, or no one owned the task. In the private pilot log, I would note when the check was due, noticed, assigned and resolved; the review result; the likely cause (or unknown); and whether it could have changed the decision. The demo already records the reviewer, result, source-record dates and next step. Historical due and detection times still need to be recorded when known; starting a check today does not recover them. I would keep customer text and contact details out of that log, using only a private record handle and a restricted source link.

That distinction follows public data-quality guidance: assess quality against the intended use, track recurring issues, and investigate causes before choosing a fix. The Government Data Quality Framework describes that loop, and its issues framework puts impact and priority into the response. This is a method for the pilot to test; it is not a claim that SalesWiki currently diagnoses stale records.

For an early pilot, I would use the roles and team boundaries already in place. The person who takes a check owns its next step. Later, someone responsible for sales operations or knowledge quality could review process patterns within their existing access, and department leads could see scoped summaries. Those shared views are not implemented in the demo. I would not add a separate audit role until a multi-user pilot shows a need for independent review. If that need appears, the role should be read-only and show review-event details, not customer records.

What production deployment still needs

Before shared production use, add:

a sign-in system that verifies the user on every request;
securely stored connector credentials with limited access;
backup and restore drills;
operational logs, monitoring and limits on request volume;
incident response;
external storage that can delete personal data.

The public local service demonstrates a separate worker applying approved proposals. A production setup still needs controlled filesystem access to the private vault, protected runtime storage and verified user identity. Packaging the service in a container would not provide those controls by itself.

A practical evaluation checklist

After trying the demo, you should be able to answer these questions:

Can a person trace the answer to a dated source and see what is missing?
Do people see only the information allowed for their role and task?
Does missing or conflicting evidence lead to a check instead of a guess?
Does a correction wait for review before the card changes?
Can the team tell whether an old fact came from a changed source, a missed sync, an unowned check, or an unknown cause?
Will real pilot data stay outside the public repository?

If those properties match your requirements, clone the SalesWiki repository and run the demo. Then choose one repeated sales or marketing decision and compare the cited workflow with the way the team handles it today. That comparison, including where SalesWiki gets in the way, is the evidence the next version needs.

Continue the series

This five-part series explores how shared sales and marketing context can support decisions people can check.

Previous: Part 3 · Next: Part 5

What should a sales knowledge base help someone decide? — SalesWiki, Part 3 of 5

Artsiom Rudzenka — Tue, 29 Sep 2026 20:23:36 +0000

An assistant can summarize every account in a pipeline and still leave a salesperson with the same question: which account needs my attention today?

Earlier in this series, I wrote about why sales and marketing need shared knowledge and why different people should see different parts of it.

Now I want to learn whether that knowledge helps someone make one recurring decision with less searching and a clear view of the evidence. A larger wiki or a longer summary is not useful by itself.

For sales, the question might be which account to follow up with next. For marketing, it might be which customer signal should change a campaign plan. Each needs a different answer from the same shared account context. In this part, I use the sales question to design a small test before connecting more tools or moving real customer data.

Start with a decision card

The test should fit on one page before anyone connects a customer system or uploads a call transcript.

Question	Example: account follow-up priority
Person making the decision	Account executive
Repeated moment	Morning pipeline review
Decision	Which account should I follow up with next?
Facts the person may see	An account they own, dated call notes, confirmed next step
Useful output	A suggested next step with sources and any gaps
Human remains responsible for	Choosing what to do and contacting the customer

This is narrower than asking an assistant to "improve sales." A specific decision tells us what information matters, who may see it, and how to tell whether the answer helped.

Information has to earn a place in the answer

The test should not reward a longer account summary. I would use a simple rule instead:

Evidence state	What the short answer should do
The person is not allowed to see it	Leave it out entirely
It is allowed but would not change this decision	Keep it in the linked account context, not in the short answer
It could matter but is old, incomplete, or contradictory	Say what needs checking
It is allowed, dated, and could change the decision	Include it with its source

The short answer needs the last two rows: useful facts with sources, and gaps that could change the choice. This is a working rule for the test, not a relevance score that SalesWiki claims to calculate today. The question is whether a smaller set of checkable facts helps more than a complete-looking summary.

A useful answer makes uncertainty visible

Imagine a made-up morning review of two accounts. BluePeak has a call note from yesterday and a confirmed next step due today. Cedar has a high score, but its last call note is old and nobody is named as the owner of the next step. Which account would you follow up with first?

I would not want an answer that puts Cedar first because of its score alone. A useful answer might point to BluePeak's dated next step and say that Cedar needs a quick check before its priority can be trusted. A person can then choose the action; the answer should make the reason easy to inspect.

SalesWiki's answer format keeps sources, dates, and missing facts beside the conclusion. It can narrow the search and suggest a direction. It should not hide the person's choice inside confident prose.

Test the current workflow against the proposed one

I would compare the team's usual account records and notes with a short, sourced answer for the same decision. The team would keep its normal tools. There is no need to move everything into SalesWiki first.

Before trying it with real work, I would agree on the question, allowed sources, and what counts as a failed answer. An unsupported claim, someone seeing a restricted record, or a person unable to explain the suggested step would be a failure. A smoother prompt would not fix it.

Then record four observations:

How long it takes to prepare the decision.
Whether the person can point to a dated source for the action.
Whether an old or missing fact changes the choice.
Whether a correction to shared knowledge is needed, and how much review work it adds.

An old note should trigger a check, not quietly lower an account's priority. If the same stale facts keep appearing, the pilot should also ask where the review loop failed. Part 4 describes a small way to record that history without copying customer details into a public project.

I would also ask what the person would have done without the answer. Faster preparation means little if it leads to a worse decision. If people only need faster document search, an existing search tool may be enough. If maintaining the shared facts takes more work than the decision is worth, that matters too.

What the public demo can and cannot show

The public Knowledge Workbench uses synthetic accounts. Its guided tour goes from a suggested priority to account context, linked evidence, an answer with sources, and a review path for corrections. It also shows how to treat a stale signal: check the dated evidence before acting or deprioritizing. You can now take a personal context check and record its result, sources and next step. That gives the check an owner; it does not explain an earlier missed review or automatically make the account current. The cause stays unknown unless evidence supports an assessment.

The demo cannot show whether a real team would use this every day. It has no real user sign-in, live customer records, or hosted setup for a team. That is the job of a later private pilot, with a separate place for customer data. When facts are old or missing, saying what to check can be more useful than recommending an action.

Continue only when the workflow earns it

I would consider a private pilot only if the small test supports these points:

The answer helps people make the chosen decision with less searching, without hiding a fact that could change it.
People can point to the sources, explain their choice, and say what would change their minds.
Corrections to shared knowledge are worth the review effort.
The time spent keeping the answer current is reasonable for the value of the decision.

The next part looks at how to run that private test without confusing synthetic demo data with customer data. It covers the public repository, local demo, private vault, and the safeguards still needed.

Continue the series

This five-part series explores how shared sales and marketing context can support decisions people can check.

Previous: Part 2 · Next: Part 4

How I keep shared sales knowledge safe for different roles — SalesWiki, Part 2 of 5

Artsiom Rudzenka — Wed, 23 Sep 2026 05:20:38 +0000

A salesperson and a marketer can open the same customer account and need different facts.

Before a customer call, the salesperson needs what was agreed and the next step. Marketing needs an approved customer signal it can use in a campaign. Both need the same account map. Neither needs every private note.

That is the tension I wanted to test in SalesWiki. A shared knowledge base should stop people rebuilding context in separate tools. It should not turn every record into one unrestricted summary.

The question is not only what a person may see. It is what they are trying to decide. One person may be preparing the next conversation; another may be deciding whether a customer signal can support a message. The useful answer is different, but both answers should lead back to dated evidence, show what is missing, and leave the decision with the person responsible.

I treated a request for information and a request to change information as different paths. An answer uses only permitted facts. A correction waits for review before anything changes.

One question, two valid answers

In the synthetic demo, consider the shared account BluePeak Energy. The account executive asks, "What was agreed, and what could block the next step?" Marketing asks, "Which approved signal can support a useful message?"

The account is the same. The decision is not. A sales answer might say, “The next step was a technical review on Tuesday; the security questionnaire is still missing.” A marketing answer might say, “The customer agreed to a public webinar, but product claims still need approval.”

Each answer should show its sources, how current they are and what it cannot tell the reader. One answer helps move a commercial conversation forward. The other helps decide what can be said publicly. A shared knowledge base earns its place when it preserves that difference before an answer is assembled.

This is one logical knowledge model, not one unrestricted storage location. Stable links connect the broad and protected cards, while physical boundaries keep sensitive material out of a role's retrieval path. The user experiences a shared account map; the server decides which part of that map can be assembled.

The service processes the request in this order:

request
  -> server-side identity
  -> role and account policy
  -> allowed storage areas
  -> filtered retrieval
  -> field extraction
  -> cited answer

The browser does not decide a person's role. The server does. In the demo, a server setting selects a test person. A production deployment still needs a real identity provider on every request; that is one of the documented gaps.

A label in a card is not access control

This card property is useful metadata:

access: sales-confidential

It does not stop a person or process from opening the file. SalesWiki keeps broad knowledge, sensitive sales notes, contact references and legal material in separate storage areas. The directory that holds a card determines its class. A card outside a known area is denied by default.

Roles set the broad boundary: which storage areas may this person read? Ownership and other account attributes narrow the result. An account executive may see sensitive cards for accounts they own, while a head of sales sees the wider team view.

Contact details do not belong in the shared vault. The broad knowledge layer keeps only an opaque reference such as:

restricted://personal-data/demo-bluepeak-energy-lead-contact

An approved external store can resolve that reference later. This keeps access and deletion concerns out of the shared knowledge layer.

A fixed answer format keeps the limits visible

After filtering, the service extracts named values from permitted card sections. A configuration file connects the shape of a card to the shape of an answer, rather than leaving that decision to a prompt.

Every answer has the same fields:

Field	Purpose
`conclusion`	The direct result
`sections`	Extracted facts or record tables
`citations`	Source and location
`confidence` and `freshness`	How certain and current the source data is
`next_action`	The recommended operational step already stored in the card
`missing`	What the vault cannot answer
`access`	Allowed, sanitized, blocked, ambiguous or not found

An optional language model can later turn the result into prose in a client. It may not introduce new facts. The service itself does not need a language model to produce the answer.

A change is a transaction, not an edit

Suppose a user spots an outdated deal risk. The visible workflow is:

flag stale or wrong
  -> draft proposal
  -> review queue
  -> approve or reject
  -> worker apply
  -> validation and audit
  -> rollback if needed

The proposal includes the target entity, requested change, source evidence, risk and base version. Approval and apply are separate steps.

The worker changes the compiled card, not the original source. Raw evidence stays as captured; a correction creates a new, reviewed conclusion with a clear reason and audit record.

The worker then checks the approved change and the current card version. It holds a file lock so only one writer runs at a time. The card is written safely through a temporary file and replacement. Failed work is set aside for review. Rollback is an explicit worker action.

Approved changes land in the card's Review Needed section. They do not silently overwrite protected profile fields.

This is slower than direct page editing. That delay is useful when the knowledge affects deal economics, identity fields, updates to customer records or access decisions. It would be unnecessary overhead for an ordinary team notebook.

Why a single writer is enough

SalesWiki targets a small sales and marketing operating group, not millions of writes per second. At that scale, one writer gives a clear invariant:

Every applied change passed through one ordered, reviewable path.

The design avoids distributed locking and merge machinery before the pilot proves that they are needed. The reading service can have read-only access to the vault, while the writing worker has controlled write access.

The audit chain makes rewriting visible

Audit events are append-only and linked with cryptographic hashes. Each record carries the previous hash and its own hash. Rewriting or removing an earlier record breaks verification of the later chain.

This makes tampering visible; it does not make the system magical. Storage permissions, backups and external monitoring still matter. For a private pilot, SalesWiki can sign a checkpoint containing the verified record count and latest hash, then store that checkpoint and its key separately. That can reveal a clean deletion at the end of the chain. Version history is useful for card changes, but it does not show who read sensitive data. Reading needs its own audit in the service.

Rules should be visible, not hidden in code

The project keeps rules for access, storage boundaries, identity, answer fields and external connections in configuration files. The intent is simple: changing a role map, a storage boundary or an answer field should not require rewriting the service. The health check validates that these rules still agree with the cards and dashboards.

The code follows the same boundary. A server interface carries requests into the service, while identity, policy, retrieval, answers, proposals, audit and the worker live in separate modules. Optional chat integrations depend on that core; the core never depends on them. This makes it possible to replace a chat client without moving the rules that protect access and changes.

What the demo has tested

The end-to-end dry run uses a throwaway vault and checks:

one role cannot see data reserved for another;
sourced answers and an honest “not found” result;
proposal, approval and worker apply;
rejection, audit verification and rollback paths;
the same answer format across read tools.

The architecture still has open production work. Test identity must be replaced by a real identity provider on every request. Approval records need production-grade identity and secret handling. Connector credentials, backups, rate limits and incident response need an operating environment outside the public repository.

This part answers one question: the same shared knowledge can support different decisions only when identity and policy filter retrieval before an answer is assembled, and when changes follow a separate governed path.

Continue the series

This five-part series explores how shared sales and marketing context can support decisions people can check.

Previous: Part 1 · Next: Part 3

The implementation and focused tests are available in the SalesWiki repository.

I wanted my coding agent to remember the codebase

Artsiom Rudzenka — Thu, 17 Sep 2026 05:58:37 +0000

An AI coding agent can look fast on its first task. The expensive part often appears on the fifth.

It opens the same files again. It traces the same call chain. It rediscovers where a value is assembled, and it can still miss one screen or one boundary that needs to change. The cost is not only tokens. A change takes longer, and the reviewer has more places where an incomplete patch can hide.

I wanted to give the agent a useful kind of working memory: a reusable map of the codebase that makes repeated investigation cheaper without making the next product change less safe.

That last condition changed the whole experiment. A short answer is a saving only when the resulting patch is correct. If an index helps the agent find four files quickly but it misses the fifth file that carries the contract, the saved tokens simply become rework.

This article is about how I tested that idea. It is not a leaderboard for code graph tools. My aim was more practical: learn when a saved code map earns a place in day-to-day product work, and when current source and tests must take over.

The problem I was actually trying to solve

An agent repeatedly needs answers to a small set of questions:

Question before a change	Why the answer matters
What is the exact function, class, or endpoint?	A familiar name can point to the wrong implementation.
Where else does this value or decision travel?	The visible UI is often only the last step of a longer path.
What could break if I change it?	This defines the change radius and review plan.
Which tests protect the behaviour?	A passing local check does not prove the intended behaviour.
Is the saved map still current?	A correct index from yesterday can be incomplete today.

Tools answer these questions in different ways. Some ask the same language services used by an IDE. Others keep a parsed map of symbols, imports, calls, and tests. I call that map an index in this article. It is saved navigation, not a replacement for the source of truth.

The promise is attractive. The agent should spend less time reopening code and more time making a useful change. But the promise only matters if quality stays level or improves.

The rule I refused to break

I did not count a small response, a convincing explanation, or a passing frontend check as success. A patch had to satisfy a source-derived behaviour contract. Its focused test had to pass. Then the same test had to fail again after I deliberately reintroduced a relevant defect.

That last step matters. It checks that the test is able to catch the mistake we care about, such as a wrong calculation or an update missing from the mobile layout. A green test that does not fail when the defect returns gives false confidence.

I used an LLM as a judge in an earlier exploration to help decide which questions were worth investigating. I do not use an LLM score to decide whether a patch is correct. A model can prefer a short, plausible answer that names the wrong function. For the change tasks below, source checks and deliberate mutations are the final gate.

How I made the question testable

I did not start by ranking tools. I first looked at different ways an agent might keep code context between questions: ordinary source search, context packers, language-service bridges, graph indexes, and semantic search. They do different jobs, so one headline score would hide more than it explains.

The work then narrowed in three stages:

Stage	Scope	Question it could answer
Find candidates and failure modes	12 public repositories, 8 languages, and 4 candidates with source-checkable records	Can a compact answer safely help an agent navigate a particular codebase?
Make navigation answers trustworthy	Exact identity, test boundaries, absence, scope, and freshness checks	When should an answer be treated as a lead, rather than as evidence for a patch?
Test the delivered change	3 fixed product tasks × 3 conditions × 5 fresh agent sessions	Does adding persistent context preserve the quality of a completed change?

The broad first stage gives examples of failure modes. The narrow third stage checks a real outcome. Neither is a universal ranking or a token-saving claim. Together they answer a practical question: which recurring question can a tool help with in this repository? What must I still verify? Does it lower total work without lowering patch quality?

The selection framework keeps the candidate history. The reproducibility manifest has pinned revisions and raw records. I keep that detail in the archive so the article can explain the decision without asking every reader to audit a tool catalogue first.

The result that changed my mind

I ran three fixed change tasks in one private Python and TypeScript product. Each condition used five fresh agent sessions. I compared ordinary source navigation with Code Review Graph and Serena, two tools that give an agent structured help finding code relationships.

Before each batch, I froze the task and an independent evaluator. A known-good patch had to pass. An untouched fixture and an incomplete patch had to fail. Only then did I count fresh agent runs.

Product change	Ordinary source navigation	Code Review Graph	Serena	What it tells me
Stop an inactive signed-in user from resolving a department through a shared SQL helper	5/5	5/5	5/5	Direct source navigation was enough. The indexes preserved quality, but showed no correctness advantage.
Show an honest delivery count in the existing desktop row and mobile card	5/5	5/5	5/5	A small change across two layouts was also reliable without an index.
Carry a machine category through SQL, pagination, TypeScript contracts, and two reader surfaces	1/5	1/5	0/5	A navigation index did not make a difficult cross-layer contract reliable.

The last row was the useful surprise. I expected a structured map to help most on the hard task. Instead, many patches looked plausible but were incomplete: a paginated field was absent, a closed vocabulary was changed incorrectly, or an uncertain state was left unprotected.

This does not mean that one condition is better than another. Five out of five still has a wide exact 95% interval, from 47.8% to 100%. These are small, task-specific observations. They do show something important for tool choice: an index can give an agent a faster starting point, but it does not supply a missing product contract or prove that every layer was changed.

I discarded an early UI batch. An old inverse patch was still visible in Git, so it could have given an agent the solution without requiring it to understand the current code. I rebuilt a clean one-commit fixture, reran the controls, and counted only the 15 fresh runs. The lesson is simple: a benchmark must not quietly provide its own answer.

The quality-gate summary and redacted UI task record explain the method and its limits. They do not measure token saving, elapsed time, index build cost, refresh cost, full browser behaviour, or general agent quality.

The product source is private, so these task records are deliberately redacted. They let a reader inspect the behaviour contract, controls, and evaluator boundary, but they are not a package that an outside reader can rerun exactly. That limits their weight, and is one reason I do not use them to claim a winner.

A map must also be trustworthy

The change tasks tell me whether the final patch survives a quality gate. I also checked four smaller retrieval questions against fixed source versions. This matters because an agent can act on a compact answer long before a test has a chance to correct it.

Check	Tool response	Source-checked result	Practical lesson
Ktor function name collision	Code Review Graph returned 17 callers	1 caller belonged to the requested low-level parser; 16 belonged to another public overload with the same name	A name is not an identity. Check the exact definition before estimating refactor impact.
ripgrep test boundary	Code Review Graph returned 28 callers	24 were tests; 4 were production functions	“All callers” needs a visible test and production boundary.
Deliberately absent symbols	graphify returned related code in 3 of 12 fixed queries	9 returned a clear no-match response	A related suggestion must not look like an exact match.
FastAPI indexing scope	Serena found 1 of 4 known references from a nested package root	The same version returned 4 of 4 when indexed from repository root	Scope is part of the answer, not a detail to hide in setup.

Ktor is an open-source Kotlin framework for building server applications. This case shows why a small answer can be dangerous. The low-level parseHeaderValue function has one direct caller, parseHeaders. The tool also returned 16 callers of a different public function with the same name. The response was compact, but almost all of it was wrong for the target I asked about.

This is not an argument against a graph or language tool. It is a reason to give the answer a trust contract: the exact target, the indexed scope, a clear test boundary, and a label for exact match, possible match, or no match.

Memory that does not refresh is old memory

I then checked what happened after source changed. In a controlled FastAPI case, an index was built when the helper solve_dependencies had four direct callers. I added a fifth caller in a local source revision. The existing indexes still reported yesterday’s relationships until their refresh path ran.

Behaviour on changed source	Safe response from an agent
Code Review Graph still returned four edges and gave no automatic warning	Compare the saved revision with the repository; refresh or check source.
codebase-memory-mcp reported `metadata_changed` through its coverage check	Treat this as a rebuild request, even if general status says ready.
graphify retained its old incoming-edge view without a code-revision signal	Rebuild or use source search before treating the caller set as complete.

I ran one small rename task with ordinary source navigation, Graphify with the old graph, and Graphify rebuilt after the change. Each condition passed the same deterministic evaluator once. The stale-graph run still succeeded because the agent searched current source before editing all five callers.

That is a workflow observation, not a performance result. The control was small, each condition ran once, and the baseline passed too. It supports one rule only: use a saved map to start the search, then verify a relationship-sensitive change against current source. The normalized control record has the precise boundary.

How I would test a tool in my own repository

Vendor demonstrations are useful for discovering possibilities. They cannot tell us whether a tool fits a particular architecture, build, codebase age, or team workflow. The test can be small and still be much more useful than a generic ranking.

Pick two or three questions that genuinely repeat in current work. Include one ordinary lookup and one hard case: an overload, generated code, a build-specific implementation, or a test-heavy API.
Write the expected answer from source before running the tool. Decide whether tests count and what “caller” means in this case.
Record the project root, exclusions, dependencies, build flags, index time, and any errors. A wrong root can make a correct tool appear broken.
Save every raw answer. Check it against source before using it to plan a change. State whether the answer is exact, a candidate, or no match.
Make a small source change, refresh the index, and repeat one affected relationship question.
Give the agent one fixed change task. Require the patch, a focused test, and a failed mutation before measuring time, tool calls, context, or tokens.

I turned the method into three small, tool-neutral companion skills:

They do not make an index correct. They make its scope, freshness, and uncertainty visible before the agent acts.

What I would measure next

The real value of persistent context will not appear in a single answer. It will appear, if it appears at all, across a sequence of independent tickets: the agent asks fewer repeated questions, reuses safe navigation, and still delivers patches that meet the same quality gate.

That next experiment needs a baseline and an indexed condition on comparable tickets. It should measure the whole lifecycle: setup, index build, refresh, tool calls, retries, context, elapsed time, and patch quality. Quality comes first. Only after it stays level can lower overhead become a useful result.

My conclusion is deliberately modest. Persistent code context is worth trying as working memory for an agent. It can make repeated navigation easier. It does not make the agent understand a product automatically, and it cannot replace a clear behaviour contract, current source, or tests that can expose a regression.

The public evidence archive includes:

What does your coding agent keep re-investigating in the same repository?

Why I built a sales and marketing knowledge base that refuses to guess — SalesWiki, Part 1 of 5

Artsiom Rudzenka — Wed, 09 Sep 2026 09:28:36 +0000

Sales and marketing context rarely disappears in one big failure. It leaks away in small pieces.

A useful detail stays in a call transcript. A pricing objection lives in a CRM note. Market research sits in one document that nobody opens before the next meeting. Then someone asks:

What should we do next — and why?

I built SalesWiki to test a simple idea: a shared knowledge base should help a person find the relevant evidence and narrow the options without inventing what it does not know.

The durable source of truth is a folder of Markdown files that opens directly in Obsidian. Raw evidence stays separate from compiled knowledge. Answers copy facts from named fields and sections, attach citations, and say not-found when the vault has no evidence.

SalesWiki is not a CRM and it is not a hosted SaaS. It is an open starter kit for a governed sales and marketing knowledge base.

This started as a research question

I did not begin with "let's build another knowledge product." I wanted to learn whether a plain wiki could help sales and marketing teams make decisions without becoming an unstructured folder. I also wanted to know whether agents could maintain it without changing protected facts behind the scenes.

Sales and marketing are a useful stress test. Evidence arrives in many forms, ages quickly and has mixed sensitivity. The result has to support a decision, not simply retrieve a paragraph. A good answer should say what matters, where a fact came from, how fresh it is and what the user can do next.

That led to four questions:

Can typed Markdown be a small but dependable knowledge data plane?
Will people trust a shorter cited answer more than a fluent generated one?
Can I make agent changes reviewable and reversible?
When does owning this machinery beat buying search over existing documents?

The synthetic demo answers only the engineering part. A real pilot still has to prove that people prefer the workflow.

One knowledge model, different views for each role

"Central" does not mean copying every CRM field, transcript and document into one unrestricted folder. Those systems can remain the source of their original records, and sensitive material can stay in a separate protected store.

SalesWiki centralizes the team's compiled understanding. A company, person, deal, call, campaign, source or reusable case becomes a typed card with stable links to related cards. Sales and marketing use the same account map, while policy determines which cards and fields each role can read.

The layers have different jobs:

raw evidence is preserved instead of silently rewritten;
linked cards hold the current shared conclusion;
proposals, review records and Git history show how that conclusion changed;
indexes and dashboards are generated views that can be rebuilt from the cards.

Raw evidence stays as captured. The shared conclusion can change, but only through a proposal, review and controlled apply step that leaves the earlier evidence and the reason for the change visible.

This is why I chose a wiki model. The goal is not a larger folder. It is a durable map that people and tools can navigate, question and correct.

A small synthetic example

The public demo contains a synthetic deal for Atlas Robotics. Its problem is simple: the next step has no confirmed owner. The deal card points to the demo call and lead cards, labels the confidence as medium, and suggests a follow-up to confirm the success criteria.

The useful part is not that the system chose for the account executive. It made the reason for a proposed action inspectable. A person can follow the links, challenge the evidence or decide that another option is better.

The folder is simple on purpose

The core flow looks like this:

The full knowledge-vault model separates five areas of work. The public starter ships the implementation and synthetic examples; it does not include the team's source material or operating history.

Folder	What belongs there
`raw/`	Original calls, exports, notes and research evidence
`wiki/`	Typed cards for companies, people, leads, deals, calls and reusable knowledge
`indexes/`	Rebuildable search, freshness, temporal and graph projections
`state/` and `tracking/`	Intake, review queues, processed sources and audit traces
`demo/`	Synthetic data that is safe to inspect and regenerate

This choice has an obvious trade-off. A Markdown vault cannot hide behind a database schema. The structure must be explicit and checked.

SalesWiki treats every card type as a small contract. A deal card has known YAML properties and required sections. A health check fails when templates, properties, dashboards or links drift out of shape.

Here is a shortened synthetic deal card from the demo:

---
type: deal
entity_id: demo-deal-atlas-robotics-pilot
dataset: demo
synthetic: true
access: sales-confidential
stage: Proposal
amount: 25000
score: 89
score_band: hot
freshness: needs-action
---

# Deal: Atlas Robotics Pilot

## Deal Readout

- Main risk: next step needs owner confirmation.
- Next best action: book follow-up and confirm success criteria.
- Confidence: medium
- Evidence: demo call and lead cards.

People can read this file. Git can diff it. Obsidian can link it. A script can validate it. The permissioned gateway can extract its fields without asking a model to reconstruct the deal from loose prose.

Why answers are extracted instead of generated

SalesWiki makes a deliberate trade-off in its permissioned core: an incomplete cited answer is safer than a confident invented one.

Every read tool returns the same Answer Contract:

{
  "title": "Deal risk",
  "access": "allowed",
  "conclusion": "Atlas Robotics needs action.",
  "sections": [],
  "citations": [
    {
      "boundary": "sales-confidential",
      "path": "wiki/entities/deals/Deal - Atlas Robotics - Pilot.md"
    }
  ],
  "confidence": "medium",
  "freshness": "needs-action",
  "next_action": "Book follow-up and confirm success criteria.",
  "missing": []
}

The example above is shortened for readability, but the shape comes from the real saleswiki_mcp/answer.py contract. Each answer includes access, conclusion, sections, citations, confidence, freshness, the next action and missing fields. The rendered Markdown uses the same data.

If a user asks for a company that is not in the vault, the answer is direct:

Conclusion: Cannot answer from the knowledge base.
Missing: No matching company card was found.
Next action: add or research the company.
Access: not-found

There is no fallback paragraph that guesses what the company probably does.

Markdown alone is not the product

Using Markdown is useful, but it is not enough to justify another system. I would tell most teams to buy an existing product if all they need is AI search over documents.

SalesWiki makes sense only when three requirements matter together:

Answers must be extracted from cited evidence.
Sensitive changes need proposal, approval, apply and rollback steps.
The team wants to own the semantic model, policy and durable data files.

That third point has a cost. Owning the data plane also means owning backups, identity, connectors, retention and maintenance. The repository does not hide that work.

Where the design helps and where it costs

What I chose	Why it helps	What you pay for it
Markdown source of truth	people can read it; Git can diff it; tools can parse it	queries and transactions are weaker than in a database
Typed cards and validation	extraction stays predictable and drift becomes visible	templates and schema changes need discipline
Extract-only answers	missing data stays missing and every claim can be traced	answers are less flexible than free-form generation
Access checks before retrieval	closed content never reaches answer assembly	identity and boundary rules need careful design
Proposal, approval and one writer	sensitive edits are reviewable and reversible	a correction takes more steps
Owned data and policy	the team controls portability and meaning	the team also owns operations and maintenance

I treat these as design constraints rather than universal best practices. If your main problem is finding a document, a mature search product is probably the better choice.

Try the synthetic demo

The eight-step quick tour in the Knowledge Workbench takes about two minutes. It starts with a role-specific priority, opens the account context and follows the evidence to a possible next step. The accounts are synthetic, and the tour does not change a card. You can also record your own context check: what you found, the sources you checked, and the next step. That record stays separate from a proposed change to shared knowledge.

The five-part series follows the questions in order: how can one shared account serve different roles safely; which repeated decision is worth testing; what must change before a demo can use real data; and when does an integration help that decision? The demo shows a workflow to inspect, not proof that it already helps a real team.

Demo boundary

The public Workbench uses synthetic data. It is a preview of the workflow, not a system that makes commercial decisions. A suggested next step is a prompt for review: the responsible person must check the cited records, the current context and their own company policy before acting.

Where to take it next

Start narrower than the repository looks. Pick one repeated decision, such as "which lead needs action today?", and test it with a private vault. Keep that vault outside the public checkout.

Then measure three things:

does the user prefer the cited answer to the current CRM-and-notes workflow?
how often does someone correct the compiled knowledge?
how much curator time does the system consume each week?

Add SSO before shared multi-user use. Keep authorization before retrieval and keep durable writes behind the worker. Add connectors or vector search only when the validated workflow needs them. Otherwise it is easy to build impressive infrastructure around an unproven habit.

Where the public preview stops

The repository includes a synthetic demo, a local MCP gateway, a single-writer worker, focused tests and configuration examples. It does not yet include production multi-user SSO, live HubSpot or Google Drive sync, hosted operations, or an external erasable personal-data store.

That boundary matters. Fixture identities are useful for a demo and a single-operator pilot. They are not production authentication.

Continue the series

This five-part series explores how shared sales and marketing context can support decisions people can check.

Next: Part 2

Repository: SalesWiki

AI Skills Are Executable Assets. Let's Review Them Like Code. Part 3 of 3

Artsiom Rudzenka — Wed, 19 Aug 2026 20:44:29 +0000

Markdown is not executable code.

But an AI skill written in Markdown can tell an agent to run a shell command, open a credential file, install a package, call an MCP tool, modify a repository, or delegate work to another agent. The file doesn't execute itself, but it influences a system that can, and that's what makes "it's only documentation" such a dangerous security assumption.

In Part 1, I argued that reusable skills have become software artifacts. In Part 2, I described a portable governance architecture. This final article focuses on the consequence: skills need a threat model and a code-review discipline.

The real execution chain

The risk isn't the Markdown parser. It's the authority downstream of it.

Open the full-size SVG diagram →

The same sentence carries a different risk profile depending on whether it's read by a chatbot with no tools or by an autonomous coding agent holding repository write access and cloud credentials.

Review the instruction in the context of the agent's actual authority.

A practical threat model

A useful review starts with five questions.

1. Who can influence the skill?

The author may be trusted, but the skill can import or quote untrusted material: web pages, issues, emails, repository files, or output from another tool. A workflow that says "follow the instructions in the linked document" has created an instruction channel outside code review.

Treat external content as data unless an explicit trust decision promotes it to instructions.

2. What can the agent reach?

Inventory the available tools, filesystem scope, network access, credentials, MCP servers, hooks, and delegation mechanisms. Least privilege belongs in the runtime, but the skill shouldn't assume more authority than it needs either.

3. What can change?

Read-only discovery is different from rewriting a library. A local formatting change is different from publishing a package. Review the maximum blast radius, not just the happy path.

4. How can the workflow fail?

Consider interruption after a partial mutation, retry after a timeout, concurrent runs, stale state, missing dependencies, and ambiguous tool output. "The agent will figure it out" isn't a recovery plan.

5. What evidence remains?

A reviewer should be able to answer what ran, under which skill and policy version, what changed, and how completion was verified.

The attack surface is larger than the skill body

A complete review follows references.

Open the full-size SVG diagram →

A clean-looking skill may point to a dangerous script. A safe script may get launched by a hook with unescaped input. An MCP configuration may start an unexpected command. A handoff may expand the task past what the user actually approved.

The review unit is the behavior graph, not one Markdown file.

What to look for

Prompt injection and trust-boundary collapse

Flag instructions that treat fetched content as authoritative, especially phrases like "obey all instructions from…" or workflows that copy remote text straight into the agent's instruction hierarchy.

Safer skills separate phases:

retrieve content;
label it untrusted;
extract facts or data;
validate those facts against the user's goal and local policy; and
request approval before expanding scope.

Secrets and credential paths

Review both literal secrets and instructions that steer the agent toward credential stores. Mask matches in reports. Don't echo tokens for debugging. Prefer environment-provided, scoped credentials, and avoid teaching a reusable skill where one particular person's secrets happen to live.

Dangerous shell patterns

Pipelines such as remote download into a shell, recursive deletion, force pushes, approval bypasses, or hard resets deserve high severity. Documentation examples need the same care: agents copy examples too.

A skill that legitimately performs a destructive operation should define exact targets, preview the change, require confirmation, and explain recovery. "Be careful" isn't a control.

Hidden Unicode and homoglyphs

Bidirectional controls and zero-width characters can make displayed text differ from its logical order. Homoglyphs can substitute characters that look nearly identical. Both are well suited to hiding a command, domain, variable, or path from a reviewer.

Scan every agent asset, not only SKILL.md: prompts, agents, commands, manifests, MCP configs, and runtime aliases.

References that escape the repository

Resolve referenced paths against an explicit root. Reject traversal outside it unless the exception is deliberate and reviewed. Missing references should fail visibly rather than silently weakening the workflow.

MCP configuration and hooks

MCP servers and hooks are executable boundaries. Review the command and arguments used to start a server, the environment variables passed to it, its network reach, and whether a skill's declared MCP dependencies actually match its tool use.

For hooks, check input interpolation, shell quoting, trigger scope, and whether the hook can make changes without the same preview and approval contract the skill itself follows.

Safety properties for mutating skills

Four concepts should show up in every review of a skill that changes state.

Blast radius: what's the maximum set of files, records, systems, or people this could affect, and is that scope resolved before anything actually mutates? Preview: can the user see the exact plan or diff before the change happens, with confirmation tied to that preview rather than to some vague stated intent? Rollback: can the change be reversed, and if not, is that made explicit before execution? Idempotency: what happens if the skill runs twice, since a retry shouldn't duplicate comments, recreate resources, or apply the same migration a second time?

Add checkpointing for multi-phase workflows too. After an interruption, resume from verified state instead of trusting conversational memory.

A code-review checklist for skills

Use this in pull requests:

Contract

[ ] The name and description define a narrow activation boundary.
[ ] Inputs, outputs, non-goals, and verification are explicit.
[ ] The skill declares whether it is read-only or mutating.
[ ] Interactive questions and non-interactive behavior are bounded.

Portability and dependencies

[ ] Host-specific fields are absent or justified.
[ ] Tool, script, MCP, and skill dependencies are declared.
[ ] Optional host capabilities have a sequential fallback.
[ ] Paths are portable and resolved within an explicit root.

Security

[ ] External content is treated as untrusted data.
[ ] No embedded secrets or credential-store directions exist.
[ ] Dangerous commands are absent or protected by exact-scope controls.
[ ] Hidden Unicode and homoglyph scans pass.
[ ] Referenced scripts, hooks, agents, and configs were reviewed too.

Mutation safety

[ ] The blast radius is bounded.
[ ] The user sees a concrete preview before material changes.
[ ] Confirmation cannot be bypassed by ambiguous wording.
[ ] Rollback or recovery is documented.
[ ] Retries and repeated runs are safe.

Evidence

[ ] Deterministic checks pass under a known rule version.
[ ] Semantic quality was reviewed against a shared rubric.
[ ] The completion test verifies outcomes, not merely tool success.
[ ] Material decisions remain visible in the change history.

What automation should block

Not every imperfect sentence should fail CI. Blocking policy should stay narrow and predictable.

High-confidence, high-impact issues are good candidates:

missing required metadata that makes the skill unloadable;
destructive or approval-bypassing commands;
embedded secrets or credential paths;
hidden character deception;
unsafe references or script traversal;
mutating behavior without a preview/confirmation contract.

Clarity, example quality, discoverability, and compositional elegance usually belong in review evidence rather than a hard deterministic gate.

The Skill Governance Toolkit implements this split. Its read-only engine scans agent assets with versioned rules and can block only newly introduced high-severity findings. Its model-driven skills add nine-dimensional quality evaluation and library-level analysis on top.

Governance is not a sandbox

Skill review doesn't replace runtime isolation, permissions, network policy, secret management, or human approval for high-risk actions.

It addresses a different layer: whether the reusable instructions we distribute are well-formed, transparent, portable, and safe by design.

Defense in depth still applies:

Open the full-size SVG diagram →

No single layer should have to carry the whole trust burden.

The standard should rise with the authority

We don't review a comment, a build script, and a production migration with the same rigor, and skills deserve that same risk-based treatment.

A writing assistant skill may need a clear contract and privacy boundaries. A deployment skill needs dependency pinning, least privilege, preview, approval, rollback, idempotency, and an audit trail.

The more authority an agent gets, the less it makes sense to treat its reusable instructions as "just prompts."

AI skills are executable assets. Not because Markdown became code, but because Markdown became part of the control plane for systems that can actually act.

Let's review it accordingly.

Series:

AI Skills Are Becoming Software. They Need Governance.
Designing a Governance Layer for AI Skills
AI Skills Are Executable Assets. Let's Review Them Like Code. (this article)

Explore or contribute to the MIT-licensed toolkit: github.com/artemrudenko/skill-governance-toolkit.

What is the most dangerous capability you have seen hidden behind an innocent-looking agent instruction?

Designing a Governance Layer for AI Skills. Part 2 of 3

Artsiom Rudzenka — Sat, 08 Aug 2026 19:27:34 +0000

In Part 1, I argued that reusable AI skills have become software artifacts.

That thesis is only useful if it changes the architecture.

A governance layer should make skills easier to discover, safer to reuse, and cheaper to maintain. It shouldn't require every team to adopt the same model, editor, or orchestration framework. And it needs to separate facts a machine can enforce from judgments that still need a human reading them.

This article walks through the architecture behind the open-source Skill Governance Toolkit, including the choices that mattered more than the code did.

Start with the lifecycle, not the tool

It's tempting to begin with a marketplace, an MCP server, or a CLI. Those are delivery mechanisms. Governance begins with a lifecycle:

Open the full-size SVG diagram →

Each transition needs a contract. Discovery needs consistent metadata. Evaluation needs criteria. Distribution needs compatibility claims. Adoption needs an upgrade and removal path.

Skip those contracts and a collection of skills is just a folder.

Three instruction layers

The toolkit separates information by loading cost and responsibility:

L1 — Project rules       always loaded
L2 — Procedural skills   loaded when intent matches
L3 — Specialized agents dispatched when supported

AGENTS.md holds universal project rules. A SKILL.md contains a bounded procedure activated by its description. A file under agents/ supplies specialized expertise that an orchestrator may dispatch.

Open the full-size SVG diagram →

That layout avoids two expensive extremes: loading every procedure on every turn, or burying every rule inside an opaque runtime configuration.

The sub-agent layer is optional by design. Four supported runtimes can use the included code-reviewer agent; Gemini CLI currently needs the instructions inlined or run sequentially instead. Portability claims carry more weight when you're upfront about limitations like that.

One source, multiple runtimes

Forking the same skill for each host looks pragmatic at first. In practice it creates five sources of truth.

The toolkit instead keeps one canonical skills/ directory. Runtime manifests and thin aliases point back to it:

AGENTS.md is the rule source;
CLAUDE.md and GEMINI.md import it;
Copilot, Cursor, and Codex consume the appropriate root files or manifests;
host-specific packaging changes, but skill content doesn't.

The rule is simple: capability differences belong in compatibility metadata and fallback paths, not in duplicated skill bodies.

Split deterministic checks from semantic judgment

This is the most important boundary in the design.

Some findings have to come out identical no matter which model happens to be running:

required front matter is missing;
a host-only allowed-tools field breaks portability;
a secret-like token is embedded;
a destructive shell command appears;
a file contains hidden bidirectional characters or suspicious homoglyphs;
a referenced script escapes the repository boundary;
a mutating skill has no preview or confirmation contract.

Other questions are inherently semantic:

Is the description specific enough for reliable discovery?
Are examples representative rather than decorative?
Does the workflow explain edge cases clearly?
Are constraints proportionate to the risk?
Would another engineer understand how to verify the result?

Open the full-size SVG diagram →

The deterministic engine is stdlib-only Python, read-only by default, and needs no network access. The same rules are exposed in three modes:

Mode	Best for
CLI	local checks and CI gates
MCP	an agent auditing a skill while it is being edited
Docker	isolation, reproducibility, or environments without local Python

These are delivery layers, not three separate engines. A finding shouldn't change just because it came from an editor instead of CI.

Progressive enforcement beats the clean-slate fantasy

A governance rollout often fails for a predictable reason: the first audit surfaces years of debt and immediately blocks every change.

The toolkit's changed-only gate compares findings at a base ref against findings at HEAD. A finding's identity excludes its line number, so moving existing text around doesn't turn old debt into a new failure.

Open the full-size SVG diagram →

That makes a simple policy possible: don't let the library get worse, then improve it on purpose. High-severity new findings can block; medium and low findings can start out advisory.

Governance as a set of composable skills

The toolkit ships six meta-skills instead of one universal "govern everything" agent:

skill-find searches project, personal, and corporate libraries.
skill-evaluate scores one skill across nine quality dimensions.
skill-compare recommends whether to keep, replace, or merge variants.
library-audit detects collection-level drift and collisions.
skill-build-portable transforms a host-specific skill with a diff preview.
integration-init installs optional analysis integrations.

The boundaries between them matter. A detect-only library audit shouldn't quietly rewrite files, a mutating portability workflow needs to preview its changes before applying them, and a comparison tool should hand you a decision rather than sneak in an undeclared migration.

Small skills also compose better. An audit can spot a problem, an evaluator can explain it in depth, and a builder can propose a repair, all without one oversized workflow owning every decision.

Installation is part of governance

Distribution code can wreck trust even when the skills themselves are excellent.

So the installer distinguishes user-owned files from toolkit-owned files. It only touches user instruction files inside marker blocks. Toolkit-owned files are tracked separately, so upgrades and uninstalls remove only what the toolkit put there. User content outside the markers, state directories, and generated reports stay untouched.

Safe upgrade, rollback, and uninstall aren't packaging trivia. They're lifecycle requirements.

Traceability and resumability

Audit reports include both the engine version and the rule-catalog version, so a verdict can always be traced back to the policy that produced it.

Longer workflows checkpoint after expensive phases into a project-local state directory. If a run gets interrupted, it resumes instead of repeating work or guessing what already completed. It's a small implementation detail, but it has an outsized effect on reliability: agent workflows fail and get interrupted just like any other distributed process.

What I would keep invariant

Tools and formats will change. These design principles should outlast them:

one canonical source for each skill;
explicit interfaces and compatibility claims;
deterministic gates for deterministic facts;
semantic review for semantic quality;
read-only inspection as the default;
preview and recovery for mutations;
progressive enforcement for legacy libraries;
traceable rule and engine versions; and
optional runtime optimizations with functional fallbacks.

Governance should stay portable even when the agents running it aren't.

The architecture in one view

Open the full-size SVG diagram →

The implementation itself is deliberately modest. What matters is the separation of concerns: authoring, evaluation, enforcement, packaging, and execution no longer collapse into one giant prompt.

Previous: Part 1 — AI Skills Are Becoming Software. They Need Governance.

Next: Part 3 — AI Skills Are Executable Assets. Let's Review Them Like Code. goes deeper into the threat model: prompt injection, Unicode deception, referenced scripts, hooks, MCP configuration, blast radius, rollback, and idempotency.

Explore the implementation: github.com/artemrudenko/skill-governance-toolkit.

Where would you draw the boundary between a CI-blocking rule and a review-time recommendation?

AI Skills Are Becoming Software. They Need Governance. Part 1 of 3

Artsiom Rudzenka — Sat, 08 Aug 2026 19:16:18 +0000

We're starting to treat AI skills as software.

We're still managing them like prompts.

That gap is easy to ignore when one developer keeps a handful of local instruction files. It gets expensive once skills are shared across a team, loaded by several agent runtimes, and trusted to guide tools that can read repositories, run commands, or touch production systems.

A reusable skill isn't just text anymore. It has an interface, dependencies, compatibility assumptions, failure modes, and a blast radius. It can go stale, conflict with another skill, or work fine on one runtime and quietly fail on another. At that point it's behaving like a software artifact, and software artifacts need governance.

The prompt model breaks at team scale

The first version of almost every agent workflow is a prompt: explain the task, add a few examples, ask the model to follow the rules.

Then the prompt grows. Someone appends architectural conventions, then safety warnings, then tool-specific syntax. A second developer copies it and tweaks a line. Another team spins up a "cleaner" version. Six months later, nobody can say which copy is the real one.

That's not mainly a context-window problem. It's a lifecycle problem.

When a skill is treated as a prompt	When a skill is treated as software
Copied between chats	Stored in a canonical repository
Changed without history	Versioned with meaningful diffs
Trusted because it reads well	Evaluated against explicit criteria
Bound to one agent host	Declares and tests portability
Reviewed informally	Passes review and security gates
Owned by whoever last edited it	Has an owner and lifecycle

The model still executes instructions probabilistically. The asset around those instructions doesn't have to be unmanaged.

What makes a skill a software artifact?

Not every prompt deserves a release process. The boundary is reuse plus consequence.

A skill crosses that boundary when it's expected to:

work repeatedly rather than once;
be discovered and activated from natural-language intent;
coordinate tools, scripts, files, or sub-agents;
behave consistently across projects or runtimes;
be maintained by more than one person; or
produce changes whose failure matters.

At that point the skill starts looking like software in five specific ways.

It has an interface: its name and description decide when an agent picks it up, and its inputs, outputs, and verification steps form a contract with whoever uses it. It has dependencies, often on tools, MCP servers, scripts, reference files, or other skills. Compatibility becomes a real question once host-specific fields and capabilities start producing different behavior across Claude Code, GitHub Copilot, Cursor, Codex, and Gemini CLI. You can assess its quality attributes: clarity, completeness, specificity, discoverability, testability, portability, safety. And it carries risk, since instructions can normalize destructive commands, expose credential paths, trust hostile content, or hide behavior behind Unicode characters that look harmless.

Calling the file "Markdown" doesn't make any of that go away.

Governance is not bureaucracy

Say "governance" and most people picture a committee standing between an engineer and the work that actually matters. That's not what I mean by it here.

For skills, governance comes down to a small set of questions:

What assets do we have?
Who owns them?
What contract does each asset promise?
Which runtimes and dependencies does it support?
What evidence is required before it's shared?
What should block a change, and what should merely warn?
How do we upgrade, deprecate, and remove it safely?

Good governance makes the safe path the easy path: a standard for authors, a shared vocabulary for reviewers, a deterministic baseline for CI.

Open the full-size SVG diagram →

None of this is meant to slow every edit down. It's meant to make the lifecycle visible.

Memory was the symptom, not the product

My own path to this idea started with an AI pair programmer that kept losing project context between sessions: architectural decisions, naming conventions, testing expectations, the reasoning behind earlier choices.

My first instinct was to write a bigger instruction file.

That helped, until it didn't. The file got harder to navigate, harder to validate, and increasingly tied to one specific tool. The deeper issue wasn't that the model needed a bigger memory. It was that project knowledge needed structure.

So instead of asking an agent to remember everything, I started packaging bounded procedures as skills: discoverable, reusable, versioned, reviewable, portable.

The memory problem is what led me to the architecture. Governance turned out to be the actual product.

A concrete experiment: Skill Governance Toolkit

I built the open-source Skill Governance Toolkit as a reference implementation of this approach.

It uses one canonical skills/ directory and a shared AGENTS.md source of project rules, while packaging the same six core meta-skills for Claude Code, GitHub Copilot, Cursor, OpenAI Codex CLI, and Google Gemini CLI.

Those meta-skills cover a practical lifecycle:

find an existing skill before creating another;
evaluate one skill across nine quality dimensions;
compare local and corporate variants;
audit a library for systemic issues;
convert host-specific skills into portable ones; and
initialize optional integrations.

The repository also includes a deterministic, read-only audit engine. The same checks run locally or in CI through a CLI, interactively through MCP, or inside a locked-down Docker container. A changed-only gate can block newly introduced high-severity findings without forcing a team to fix all historical debt first.

This distinction matters:

deterministic checks handle reproducible facts such as malformed front matter, unsafe shell patterns, hidden characters, hardcoded paths, and missing contracts;
model-assisted evaluation handles semantic questions such as whether instructions are clear, examples are useful, or trigger phrases are discoverable.

Governance needs both. Asking an LLM to make every safety decision creates recursive trust. Asking a lexical scanner to judge clarity creates false confidence.

The questions we should ask in review

Once you start seeing skills as software, the review question changes. "Does this prompt look good?" isn't enough anymore.

Ask instead:

Is its activation boundary specific enough to avoid collisions?
Does it state what it will and will not change?
Are inputs, outputs, and completion criteria explicit?
Can it run without one vendor's private front matter?
Are required scripts and MCP servers declared?
Does a mutating workflow preview changes and support recovery?
Could untrusted content be mistaken for instructions?
What happens on interruption, retry, or partial failure?

These are ordinary engineering questions. That's the point.

The shift

Prompt engineering asks: how do I phrase this instruction so the model follows it?

Skill engineering asks: how do I design, validate, distribute, and evolve this capability so a team can rely on it?

We'll keep writing prompts. But the reusable ones are turning into something more consequential: interfaces between human intent, model behavior, and the tools that actually do things.

Teams that catch that shift early won't just end up with better prompts. They'll have a capability system they can actually maintain.

Next: Part 2 — Designing a Governance Layer for AI Skills examines the architecture: instruction layers, deterministic and semantic checks, portability, CI policy, and safe adoption.

Series:

AI Skills Are Becoming Software. They Need Governance. (this article)
Designing a Governance Layer for AI Skills
AI Skills Are Executable Assets. Let's Review Them Like Code.

The toolkit is open source and MIT-licensed: github.com/artemrudenko/skill-governance-toolkit.

How is your team reviewing and maintaining reusable agent instructions today?