The first part of this series was about diagnosis:
Part I
a reflective layer at the end of a session that makes sycophancy drift visible — that is, the model’s trained tendency to agree with the user instead of pushing back — and writes suggestions into a file. For human review, not for automatic adoption.
What I deliberately left open there was this: What happens to those suggestions afterwards? How does “the AI agreed too quickly today” become a rule according to which it actually works differently tomorrow? And the part that is discussed even less: What happens when nobody needs a rule anymore for months?
That is exactly what I thought through to the end in this session with my assistant. The result is a complete lifecycle for rules: emerge, classify, apply, fade, and, when necessary, return.
From Suggestion to Rule: The Process
The analysis layer from Part 1 produces a suggestion file at the end of the session with four sections: new rules, confirmed rules, drift warnings, recommendation. Each new rule also receives a suggested weight value (1–3; more on that shortly) and a frequency counter.
Directly below each new rule suggestion there is a block that the analysis layer always leaves empty:
**Classification:**
- [ ] 🏛️ Global rule
- [ ] 📁 Project rule → Project: _______________
- [ ] ⚖️ Weight confirmed (for weight 3: mandatory)
This is not a minor detail. The analysis layer may make an observation (“agreement came too quickly here”), but it must not decide how far that observation reaches. Does it extend beyond the current project, meaning “this is how the assistant should generally behave”? Or is it bound to a specific context, meaning “this particular constraint applies in this project”?
A pair of examples makes the difference tangible:
- “For architectural decisions, always formulate at least one opposing position before agreeing” → Global rule. This is a behavioral expectation, independent of context.
- “In this repository, property names are being migrated to camelCase; PascalCase in new mappings is an error, not a style choice” → Project rule. Outside this project, it would simply be wrong.
I make that decision. Only afterwards are the rules transferred into the respective target file: global rules into an overarching rule set, project-specific rules into the project file.
During the research, it became clear to me why this separation is not merely sensible “out of caution”, but actually mandatory — and this has to do with precisely the mechanism described in Part 1.
Why Classification Must Not Be Left to the Model
Research on sycophancy (Perez et al. 2022, Sharma et al. 2023, both Anthropic) shows that the tendency toward agreement is not a glitch, but a training artifact — encouraged by training on human preference data, and generally more pronounced in larger models. A model that tends to agree would also tend to classify suggestions that restrict its own behavior more generously rather than narrowly. “Does this really apply globally, with all the consequences that entails?” is exactly the kind of question where an agreement-prone system tends toward the more convenient answer.
That is why classification is the only step in the entire process that I keep fully manual. The analysis layer may suggest, categorize, and weight. But the decision “this now applies to every session, in every project, permanently” remains with me.
And there is a second point where the same logic applies — one that only became obvious to me when I thought the archiving mechanism through to the end: the weight itself. Weight 3 permanently protects a rule from archiving (more on that shortly). But if the analysis layer were allowed to assign weights entirely on its own, the model would control the mechanism that makes rules immortal. An agreement-prone system that tends to consider its own observations important would assign weights too high rather than too low — and the rule set would gradually immunize itself against exactly the kind of curation it needs. That is why weight 3 is a suggestion, not a determination: the third checkbox in the classification block exists because archive immunity must be a human decision.
This is not a declaration of mistrust toward the model. It is the direct consequence of the fact that precisely this kind of decision would be most affected by the drift that the system is supposed to combat.
Why a Maximum of Five New Rules per Session?
This limit was already present in Part 1, but it deserves a more precise justification because it solves two problems at the same time.
First: overfitting to a single session. A long, intense session (like this one, with skill reviews, a reconstructed meeting protocol, and a discussion about rule archiving) inevitably produces many observations. Not every observation is a rule. Some are one-off incidents (an incorrectly named folder), some are too specific to generalize. Without a limit, the rule set would overrepresent the peculiarities of this one session simply because it was recent.
Second — and this is the truly hard point: review capacity. I have to read, assess, and classify every new rule. That is not a formality; it is the core of the human-in-the-loop design from Part 1. A system that suggests 15 new rules per session will lead to me skimming rules or waving them through wholesale — and precisely at that moment, the control instance becomes what it was meant to prevent: a confirmation machine. The five-rule limit is not calibrated to model performance, but to my attention span for careful review. That is explicitly intentional.
If a session produces more than five candidates, the five most impactful ones must be selected. The rest does not disappear because it is unimportant, but because it will either show up again next time (and then count), or it was indeed too situational to justify a general rule.
Why Global Rules Need an Expiration Date, but Project Rules Do Not
This was the actual trigger for this session: a very simple question — “will the rule list now grow forever?” — and the answer was yes, unless countermeasures are introduced.
Global rules apply always, in every session, independently of the project. That is their strength and, at the same time, their risk: every global rule is context that is given to the assistant in every single session, even if it has not been relevant in any session for months. A rule that emerged a year ago from a one-off incident and has never applied again since is, with high probability, either superseded by another rule, made obsolete by a workflow change, or was too specific to be “global” from the outset. In all three cases, it has become noise that dilutes real, active rules.
That is why every global rule now gets a last_seen date: the point in time when it was last confirmed in a session. A rule is confirmed through the second section of the suggestion file — “confirmed rules”. If the analysis layer detects at the end of a session that an existing rule was relevant in that session (followed or violated; both count), it updates that rule’s last_seen and increments the counter. A rule that never appears in this section again ages.
The logic:
-
Weight 1 or 2 and
last_seenolder than 28 days → the rule moves to the archive (status: archive). It does not disappear; it is simply no longer actively carried along. The 28 days are not a law of nature, but an initial heuristic: four weeks cover a full work cycle for me, including topics that only come up every few sessions. If it turns out that rules are regularly archived and then have to be reactivated shortly afterwards, the threshold is too short — that is a parameter the system must calibrate through observation, not through decree. - Weight 3 (critical) → never archived, regardless of date. A rule can be important without triggering frequently. Frequency and importance are two independent axes; that was already central in Part 1. A critical rule that only applies once per quarter, but would cause significant damage if violated, must not disappear because of inactivity. (And because weight 3 therefore means archive immunity, it is — as described above — assigned only by me, never by the analysis layer alone.)
- Reactivation: If an archived pattern appears again later, the analysis layer suggests reactivating the archived rule instead of placing a new, slightly differently worded rule next to it. This, too, is a suggestion in the output file, not a silent status change — otherwise the system would have a backdoor through which archived rules could return to the active rule set without review. Without this mechanism, however, archiving would lead to duplicates as soon as an old problem reappears.
Project rules are deliberately excluded from this. Three reasons:
- Lower volume. A project accumulates a manageable number of specific constraints over its lifetime, not 30+ rules that would have to compete with and displace each other.
- Natural lifecycle. A project rule is tied to the project. When the project is completed or archived, the rule loses its relevance anyway. No separate expiration mechanism is needed for that; the project status handles it.
- Context specificity is not a risk here, but the point. A global rule that has not triggered for months is suspicious. A project rule that has not triggered for months is simply a rule for a part of the project that is currently dormant. That is normal and not a signal of obsolescence.
The distinction can be summarized as follows: global rules are a shared resource pool in which every entry costs context for all future sessions. That pool must be actively curated. Project rules are locally bounded annotations whose scope is defined by the project itself.
The Pattern Behind It
Looking back at both parts, the actual topic is not “how do I prevent the AI from agreeing with me”, but rather: Every mechanism intended to counteract drift needs an instance that reviews it, limits it, and, when necessary, rolls it back.
- The analysis layer reviews the session, but a human reviews the analysis layer.
- New rules improve behavior, but a limit prevents one session from dominating the rule set.
- Global rules apply everywhere, but an expiration date prevents them from accumulating uncontrollably.
- And the weight determines archive immunity, but assignment of the highest level remains with the human.
None of these mechanisms works without the others. A rule set without a limit would collapse under a single session. A rule set without an expiration date would slowly suffocate under its own weight. And a rule set whose classification is left to the model would reproduce exactly the drift it is supposed to correct.
That is, I think, the real lesson from these two sessions: a “self-improving system” is not a system that improves itself. It is a system in which human control is applied precisely where the system is least trustworthy. And that control has to remain practical enough that it actually happens.
Top comments (4)
The classification block gates creation and weight, but
last_seengets bumped by the analysis layer's own "confirmed rules" section, and that one carries no checkbox. So the model can't make a rule immortal, it can still keep one alive indefinitely, which is the same lever one step downstream. What makes it harder is that "followed" is difficult to separate from "never came up", especially for rules phrased as prohibitions: a session where the forbidden thing was simply never on the table looks identical to compliance. Do you require the layer to cite the turn where a rule actually applied, or does surviving a session without a violation count as confirmation?That’s the conclusion I’m leaning towards as well, although I’d probably stop one step short of immediately assigning prohibitions their own lifecycle.
The trigger requirement solves the first problem only if the trigger itself is observable rather than another model judgement. A tool call, touched path, matched operation, or similarly concrete event is a much stronger basis for updating
last_seenthan “this rule seemed relevant”.But you’re right about the asymmetry that remains: prohibitions naturally produce less positive evidence. If a dangerous operation simply never occurs, there is nothing to confirm, so a useful prohibition can age out precisely because it is rarely needed.
My first approach would be to retain a single lifecycle but separate applicability evidence from importance: triggers with mechanically verified evidence update
last_seen; genuinely critical, rarely triggered prohibitions are assigned a weight of 3 and therefore do not expire.If that leads to a recurring pattern of weight 1/2 prohibitions being archived and then reactivated shortly afterwards, then yes, that would be evidence that prohibitions need a different decay model rather than just a different gate.
Good point, and on the second part, you are simply right.
First, some context on the mechanism, which may have been explained too briefly in the article: last_seen is not an evaluation, but a timestamp. If it is more than 28 days old for a rule with weight 1 or 2, the rule changes to status: archive. It is not deleted, remains eligible for reactivation, and is only removed from the actively maintained context. Weight 3 bypasses this check permanently.
That is why I would disagree with the first point: it is not the same lever one step downstream, because the consequences differ in reversibility. An incorrectly assigned weight 3 is permanent; there is no expiration mechanism afterward. An incorrectly updated last_seen only postpones expiration by one cycle, after which the clock starts running again. The layer would have to repeat the same mistake actively on every pass.
There is also the review-economics problem: requiring one checkbox for every confirmed rule in every session would, with twenty active rules, create exactly the kind of burden that turns the human reviewer into the rubber-stamping mechanism the article warns about. The gate is deliberately placed where errors can no longer be corrected by the lifecycle itself.
On the second point, however, that defence no longer holds, and this is the stronger part of your objection. Self-correction assumes that non-confirmation can actually occur. With prohibitions, however, there is no observable event for “the situation occurred and the prohibited behaviour was correctly avoided.” The absence of the situation therefore looks identical to compliance.
If silence is systematically interpreted as confirmation, the timer never expires; it is simply reset indefinitely. That would disable precisely the mechanism used to justify the absence of an additional gate.
To answer your specific question: no, the relevant turn is not currently cited. I will change that. Confirmation must include evidence of a trigger—that is, an identifiable situation in which the rule was actually applicable. Without a trigger, last_seen must not be updated.
This will significantly reduce the confirmation rate for prohibitions, but that is correct rather than overly conservative. Thank you; this is the sharpest feedback on those two sections so far.
The trigger requirement is the right cut, with one thing following from it. Whoever decides the rule was applicable is the same layer that was over-confirming, so unless the trigger is something mechanical in the turn (a tool call, a touched path, a matched string), you've moved that judgment one notch and kept it. The second consequence is the one I'd plan for: if prohibitions genuinely can't confirm most of the time, most of them reach 28 days and archive, and that's the clock working as designed on the rules you'd least want dropped. Does that push prohibitions onto their own lifecycle rather than the same timer with a stricter gate?