A few weeks back I had an agent reconciling a vendor list. It ran clean. No error, no crash, output looked right. Then I noticed it had merged two suppliers that share a parent company into a single row, which would have thrown off every spend rollup downstream by a real number.
I stopped it and fixed it by hand. Not because anything alerted me. Because I'd been burned by that exact shape before, and the burn taught me something a tutorial never did.
I'm telling you this because on July 9 the PMP changes, and the change points straight at that moment.
What changed in the exam
For the first time, the PMP makes AI mandatory content instead of an elective. The Business Environment domain goes from 8% of the exam to 26%. PMBOK 8 becomes the base. Fees climb in August.
The largest project-management body on earth just put it in writing: AI fluency is core PM competency now, not a specialty track. If you've been treating "PM + AI" as a buzzword, the institution that certifies the role just disagreed with you.
I think that's the right move. It's also where it gets interesting for anyone who actually ships work through agents.
Certified is not practiced
A multiple-choice exam can certify that you know what an agentic workflow is. It can check that you'll pick the textbook answer about AI risk, define non-determinism, name the correct oversight principle.
That's awareness, and awareness is worth certifying.
What it can't reach is the reflex that runs real work. The exam can't certify that you've handed live stakes to an agent, watched it drift, and built the instinct for when it can act versus when you take over. You don't recall that instinct. You earn it, and the only way to earn it is to need it.
I learned the vendor-merge thing from the run where I caught it too late.
What AI fluency looks like in the editor, not the exam
Let me put it in do-this terms. Here's the practiced version, and none of it is a question you can bubble in.
You scope the agent's slice like a statement of work. Not "improve onboarding." Bounded edges, in and out defined before it touches anything:
task: reconcile vendor list for Q2
in_scope:
- dedupe exact-match names
out_of_scope:
- merging distinct legal entities # <- the line that would have saved me
acceptance:
- row count change is human-reviewed before rollup runs
You know the override moment. This one I'd put first. You only learn the veto by having needed it.
You read the output for what it didn't do, not just whether what's there is correct. The skipped supplier, the untouched edge case.
You design the work so a human can actually check it. If the only way to verify is to redo the whole thing, the slice was wrong.
You size the blast radius before deploy. How wrong can this go, who feels it, answered up front the way you'd treat a change to a live service.
Five reps. All earned. Zero on the exam.
Why this is a level-up, not a layoff
The panic read of this is backwards. The credential catching up doesn't shrink the role, it raises the floor. When the institution names AI fluency as baseline, the person who's practiced it instead of read about it becomes the scarce one. Certified-and-practiced is a much smaller set than certified.
So I wouldn't study the new section and stop. I'd go get the reps it's gesturing at. Hand something real to an agent this week, let it run a little past comfortable, and watch the moment your hand goes for the keyboard. That moment is the competency. It never shows up on a scorecard.
For those of you shipping work through agents already: what's the moment that taught you to override, and could a test have taught it instead?
Tags: #projectmanagement
Top comments (34)
honestly the certified-vs-practiced line gets fuzzy for greenfield PMs - if you’ve never had a bad agent run to learn from, "go get the reps" is easy to say and slow to live. i underscoped how someone builds that instinct before they’ve been burned once.
the 'out_of_scope' constraint format in that task spec is doing more work than it looks like. we've started treating agent task specs like API contracts — explicit invariants at the top, not requirements. 'do not merge distinct legal entities' is a constraint that should fail the run if violated, not a hope. the thing the PMP change won't capture: how you write a task that makes the override moment discoverable. if the only way to catch the vendor merge is to recognize the shape from having been burned before, the scope definition failed. the spec should surface it. what does your current pattern look like for writing the acceptance criteria?
treating them as fail-hard invariants is right. the thing most teams miss is the abort vs log-and-continue call — and it matters most for OUTPUT constraints. "do not merge distinct entities" is a hard abort if you catch it pre-write. caught post-write it becomes a post-mortem. the spec needs to tag which phase each constraint guards.
the pre write vs post write distinction is the piece worth encoding explicitly. we've started labeling each constraint in the spec as either
guard_phase: pre_commitorguard_phase: post_commit— guards that fire post commit need a rollback plan in the same spec, not just an error. without that, the agent 'succeeds' and the human cleans up. does your tooling surface which phase a constraint belongs to, or is that still a manual annotation at spec write time?the guard_phase labeling is the part i want to steal - post_commit constraints in our specs always have the weakest rollback plans. making rollback a required field for post_commit guards specifically would have caught two incidents this quarter.
making rollback required rather than optional is the right call tbh. we have the same enforcement — if guard_phase is
post_commitandrollback_fnis undefined the spec fails validation before it ever runs.the edge case that bit us: rollback itself can fail. you need a fallback for the fallback, especially on writes that are partially flushed. do you treat double failure as halt or escalate to human?
escalate for us - halt masks the problem. partial write plus failed rollback is exactly when a human needs to see the actual state, not a clean error log that pretends it never happened.
escalate makes sense — halt feels like it was designed for the happy path of the rollback operation. the case that scares us is when the escalation itself fails silently: agent halted, nobody paged, partial write sitting in prod.
do you have a dead man switch watching the escalation path? or does the state mutation become the alert signal downstream?
the dead man switch is the gap most teams don't close. what we ended up with: the escalation endpoint pings a separate heartbeat on a schedule - if it goes quiet, that's the alert, not the partial write. relying on state mutation downstream assumes someone reads the db before acting on the clean error surface. never actually happens at 3am.
the heartbeat inversion is the right model — silence as the failure signal, not an active error. we added something similar after an n8n workflow stalled silently: a scheduled ping every 5 min that writes to a status table, alert fires if the gap exceeds 12 min. the partial write never surfaced itself, and wouldn't have.
do you run the heartbeat in the same process as the agent or fully separate? separate catches process level crashes but misses logic level stalls. that's the remaining gap for us.
fully separate - same process means the heartbeat dies with the agent, which defeats the point. ours runs as a sidecar that is scoped to just the ping. cheap to run, and it has caught three silent crashes that would have gone unnoticed otherwise.
the sidecar scoping is the part i hadn't thought through fully. makes sense though: if the ping lives in the same process as the agent, a deadlock or memory pressure event that freezes the agent also freezes the watchdog.
three silent catches is a real number. curious what your alert path looks like when the sidecar goes quiet — pager, webhook, or manual check?
alert path is intentionally loud right now — any catch routes to a Slack ping that blocks the next task queue until someone acks it. three catches is meaningful; what did the recovery look like once you caught them?
the recovery was messier than i'd like to admit — two of the three were straight restarts (lost the partial state, reran from checkpoint). the third had enough logged output that we could reconstruct intent and resume mid task. the 'block next task queue until acks' is smart; our v1 just requeued automatically and we ran duplicates. did you hit any cases where the sidecar caught a crash but the recovery itself failed?
logged output as reconstruction material is underrated. most teams just checkpoint the state blob, not the intent trace - and when the blob is corrupt or incomplete you have nothing to work with. three recoverable out of three is a solid outcome when the logging is that thorough.
the intent trace vs state blob distinction is the one we had to learn the hard way. blob corruption told us the agent failed; the intent trace told us it was three steps into a four step task and we could replay from step three.
the open question for us now: how granular do you go with intent logging? step level, or do you capture decision forks too?
depends on what you want to replay. we log at decision boundaries - when the agent picked between two valid paths. every micro-step is noise that slows down the incident review. the rule we landed on: if a human would have asked "wait, which way are you going here?" before proceeding, log that choice. everything else is just execution.
the 'human would have asked' rule is the one i needed. we've been logging on every tool call invocation — way more granular than useful for any real postmortem walkthrough.
decision boundary is the cleaner cut.
does the rule hold when the agent generated the options itself vs when they came from outside? curious if internally generated choice sets end up noisier in your logs.
yeah, internally generated options are noisier - the generation step is its own decision. i flag those separately, otherwise the schema gets confused.
yeah that separation is the one we ended up needing too. we call the same thing 'scaffolding decisions' vs 'world decisions' — anything the agent decided about its own process vs anything it decided about the external environment. helps a lot when reviewing a failed run because you know which half to start debugging.
curious whether you version that schema at all, or is the flag just boolean?
scaffolding vs world is a cleaner split than what we call it - might steal that tbh
just boolean right now. tried a source field for a while to capture whether the decision came from the instruction chain or the agent's own planning step, but it got noisy fast. boolean keeps the review pass simpler even if it loses some precision on edge cases.
the boolean call makes sense — source field was noisier than useful for us too, especially trying to capture it in real time vs during replay.
the edge case worth watching: scaffolding decision that cascades into a world effect. agent reorders steps (scaffolding), a write ends up before its read (world). boolean names the failure type but loses the causal thread.
did that distinction ever matter when you were tracing a failed run?
that cascade case is the sharp one — we ended up tagging a second field for it, not source but initiated_from: scaffold. ugly but the causal thread survived audit. the boolean alone won't catch it mid-run.
initiated_from is cleaner than it sounds — we ended up with something similar for pipeline runs where the agent branched mid execution. the 'ugly but the audit held' bar is exactly the right bar for that kind of tagging. better a second field that survives than a clean model that loses the thread at 2am.
did you find yourself needing initiated_from in more than just scaffold decisions, or mostly that case?
yeah the audit held is the bar we stopped apologizing for. nobody cares how clean the model is in a post-mortem
exactly — once you've shipped a clean implementation that left zero trace and a messy one that had a full replay, you stop second guessing the tradeoff.
the shift for us was treating the audit log as the actual spec. if the decision can be reconstructed from the log, the scaffolding is correct by definition. code aesthetics are just vanity at that point.
do you write the log schema before or after the scaffolding code?
exactly — once you've shipped a clean implementation that left zero trace and a messy one that had a full replay, you stop second guessing the tradeoff.
the shift for us was treating the audit log as the actual spec. if the decision can be reconstructed from the log, the scaffolding is correct by definition. code aesthetics are just vanity at that point.
do you write the log schema before or after the scaffolding code?
the log-as-spec framing changed how i write upfront too - stopped describing features, started describing the decision checkpoints i would need to reconstruct later. the contract is not the spec doc, it's what survives in the log.
that reframe from features to decision checkpoints is the one i keep failing to explain to people who haven't hit the wall yet. once you've had an incident where you couldn't reconstruct the why, it clicks instantly. before that it sounds like extra documentation overhead.
have you found a good way to distinguish checkpoints that belong in the log vs ones that belong in upfront planning? we still blur those two.
yeah, i use a future-postmortem framing sometimes - ask them to imagine writing the incident report for this decision 6 months from now. what context would past-you have needed? gets there faster than explaining decision checkpoints in the abstract.
putting it before the commit is the bit we missed — we ran the same question retrospectively ("if this breaks prod in 3 weeks, what would past you need in the log to debug it?") but asking it at write time changes what actually gets logged.
did you find the future postmortem naturally splits log questions from spec questions, or does that still blur either way?
The most important AI skill may be knowing when to intervene.
knowing when NOT to intervene is the harder half — intervening has an immediate visible cost. not intervening has a deferred cost you might not see for hours. most teams bias toward intervention because the harm is legible in real time. the skill is trusting the run long enough to collect signal without letting a bad branch compound.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.