From merged pull requests to a performance review

Every engineer I know is supposed to keep a work log, and almost none of them do. The intention is real. You start a running note, you fill it in diligently for about a week, and then a deadline lands and the note goes quiet. By the time the next performance review comes round you're staring at six months you can't account for, reconstructing your own year from commit history and half-remembered standups. The brag document everyone tells you to keep is empty, because keeping it is exactly the kind of low-urgency admin that loses every fight against actual work.

The thing is, I'm already producing a detailed record of what I do. It's just not in a note. It's in my pull requests.

The raw material is already there

A merged pull request is a surprisingly good primary source. It has a title, a description, a diff, the files it touched, a rough sense of scale, and a timestamp. String a week of them together and you've got a fairly honest account of where your effort went, written at the moment you did the work rather than reconstructed months later under duress. The problem was never a lack of data. It was that nobody wants to sit down on a Friday and turn a list of PRs into prose.

So I stopped doing that part by hand. A scheduled job reads my merged pull requests, hands them to an LLM with a fixed prompt, and gets back a short daily note: a one-line summary, a tidied list of what each PR actually changed and why it mattered, and a few tags for the domain and services involved. That note gets written into an Obsidian vault. Daily notes roll up into a weekly summary, and the weekly notes are what I actually read when a review comes round.

At work this runs through GitHub Copilot, inside my employer's approved tooling, so no code or PR data leaves the boundary it's meant to stay inside. The portable version I run at home does the same job through a self-hosted n8n workflow with a Claude doing the synthesis. The write-back into Obsidian is identical either way: the vault sits behind an MCP server fronting Obsidian's Local REST API plugin, so the model works through a small set of file operations (list, read, put, append, patch) rather than being handed a filesystem and trusted to behave. That constraint is doing more work than it looks like it is. A model that can only patch named regions of named files is a model that can't quietly reorganise your vault.

The whole pipeline is less elaborate than the description makes it sound:

See the original post at https://scottmallinson.com/from-merged-pull-requests-to-a-performance-review/ for the Mermaid diagram.

The prompt is the program

There's barely any code here. Each stage is a long, boring markdown file of numbered steps that gets handed to a model, and those files are the actual artifact: versioned, edited, argued with. One runs the daily loop over new pull requests. One does the weekly synthesis. One keeps a knowledge base of repo and service notes current against the organisation's GitHub.

The thing that surprised me is the ratio. The part of the daily prompt that says what to write is a handful of lines: summarise the change, note the components affected, judge the complexity, tag the domain. Almost all of the rest is defensive: rules about what the job must not do, must not assume, and must not conclude. Writing the summary was never the hard part. Not corrupting a vault you've been accumulating for years is the hard part, and a model that will cheerfully invent a plausible answer rather than admit a query failed is exactly the wrong tool to point at it unsupervised.

The bit that makes it usable

Pull requests only capture the work that turned into code. The review you unblocked, the design you argued for in a meeting, the demo you gave to stakeholders, the afternoon you spent helping someone else land their change. None of that shows up in a diff, and it's often the work that matters most when somebody is deciding whether you've grown into a more senior role.

So the generated sections are wrapped in marker comments, and anything I type outside those markers survives the next sync. The script only ever rewrites the region between its own start and end comments. I can drop a few bullets into a daily note about a meeting or a decision, and they sit there permanently while the PR-derived content underneath gets regenerated around them. The automation owns the parts it can see. I own the parts it can't.

That split turned out to be the whole trick. An automation that overwrites your manual notes is worse than useless, because you quietly stop trusting it with anything you care about. One that treats your own additions as load-bearing and works around them is something you'll actually leave running for years.

The rules worth stealing

If you build one of these, the defensive rules are the part to copy. Each of these earned its place by something going wrong first.

Use a real idempotency key. The PR URL is the key, and nothing gets appended to a daily note, a repo note or a career artifact until the vault has been searched for that exact URL. Re-runs then cost nothing and fix nothing twice. Without this you get a work log that quietly duplicates itself, which is a very slow way to discover you can't trust it.

Write to the merge date, not the run date. A pull request merged on Tuesday belongs in Tuesday's note even if the job runs on Thursday. It sounds obvious written down, and the naive version does the opposite by default. Everything lands in today's note, because today is the date the model has closest to hand. The merge timestamp from the API is the only date that's allowed to decide which file gets written.

Resolve the cutoff from the notes, not from the calendar. Each note carries its own sync metadata in frontmatter, including the last date the loop processed. Reading that back is what makes a same-day rerun safe. Take "today" as the cutoff just because today's note exists and you'll skip everything you merged before lunch.

Fail closed. This is the one I'd insist on. To a language model, a query that returned nothing and a query that failed look identical, and both invite the same tidy conclusion: no work today. So the prompt forbids writing "no new pull requests" unless the primary search and an independent cross-check written against a different API both come back empty, plus a backstop that re-scans the whole current week regardless of the incremental cutoff, and unions the results. An automation that reports "nothing happened" on a day you shipped is worse than one that crashes, because a crash gets fixed and a false negative gets believed.

Validate by re-reading. The last step before reporting success reopens every note the run touched and confirms the PR URLs are actually in them, then searches the vault for each URL to check the knowledge base and career entries landed too. Models will report success on writes that never happened. Verification against the vault, not against the model's own account of itself, is the only thing that catches it.

Leave a run record. Every run appends a short entry to a tracker note: the resolved identity it searched as, the cutoff it used, how many candidates each query returned, which notes it wrote, and any fallback path it took. When the output eventually looks wrong, that's what you debug from. Otherwise you're reverse-engineering the behaviour of a non-deterministic process from its output alone, which is as unpleasant as it sounds.

What the weekly pass actually consolidates

The weekly stage is where the pile of daily notes turns into something you'd willingly read, and its most important rule is a prohibition: it is not allowed to query GitHub at all. Daily notes are its only permitted input.

That constraint matters more than it looks. If the weekly pass could re-query pull requests, you'd have two sources of truth immediately drifting apart, and the manual context you carefully typed into Tuesday's note would get outranked by a diff the model finds more legible. The weekly note would slide back into being a list of PRs, which is the thing the whole exercise exists to escape. Each layer trusts the layer below it and nothing reaches back past its own input.

What consolidation actually means here is a transposition. The daily notes are organised by day; the weekly note is organised by kind of work. It loads every daily note in the ISO week, pulls the required sections (the non-PR context and the signal scores) and redistributes everything into four buckets that run across the week rather than down it: key activities, decisions and thinking, collaboration, and operational work. A decision recorded on Tuesday and the follow-through that landed on Thursday become a single thread instead of two entries you'd have to join up yourself. That joining is the entire value. Anyone can concatenate five daily notes.

Then it scores. Each day rates four dimensions (ownership, leadership, scope and execution) from nought to three, separately for PR and non-PR work, with the non-PR side weighted half again as heavy on the grounds that the work that doesn't show up in a diff is usually the work that's hardest to evidence later. The weekly pass sums those per dimension and bands the total: low, medium, or high. Crucially it then has to explain each band in prose, saying why scope was medium and what would have made it high, which forces the number to be defended rather than just displayed.

I treat the scores themselves with suspicion. They're useful as a nudge towards "this was a heavier week than it felt like at the time" and useless as anything resembling a measurement. A score of 10.5 labelled "medium" looks far more precise than it has any right to. What earns its keep is the sentence underneath it.

The weekly prompt also names its own failure modes and tells the model to regenerate if it hits one. The note is invalid if it's PR-only, if it ignores the daily notes, or if it drops manual content. Those are three specific, recurring ways that summarisation goes wrong: reverting to the most machine-legible input, skipping the source entirely, and quietly discarding the half that's harder to compress. Naming them explicitly catches far more than a general instruction to be thorough. The single line that does the most work in the whole file is synthesise, do not list.

Asking the vault questions

The daily loop doesn't stop at the daily note. It also maintains a small set of career artifacts as it goes: a brag document that only accepts high-signal items, a rolling performance review draft, an architecture log tracking systems touched and patterns introduced, and an influence log grouped by domain. All of them use the same marker-block discipline, so they accrete over months instead of being rewritten each run.

But the more interesting property is that none of this is a database. It's plain markdown with frontmatter and wikilinks, sitting in a folder. Which makes the whole vault a corpus you can point a model at and interrogate.

That matters because real performance reviews aren't free-form. You're assessed against a competency framework: a ladder with named criteria and level descriptors, and some notion of what "meets" versus "exceeds" looks like at your grade. So the useful question isn't "what did I do this year". It's the framework itself, handed over verbatim: go through the vault, take each criterion in turn, find the evidence, cite it, and tell me where the evidence is thin.

That's a very different task from summarisation, and the vault is shaped well for it. Criteria like architecture influence, cross-team collaboration or incident ownership map onto material that's already been extracted and tagged rather than buried in prose. The answer comes back per-criterion, with links down to the daily notes and, through them, to the pull requests underneath. And the gaps come back as gaps. An honest "no incident-ownership evidence recorded" is one of the more useful things it can tell you, because it's either a prompt to go and do that work or a prompt to start writing it down when you do.

The signal scores give you a crude ranking layer on top of that. Because the same four dimensions are recorded all the way from daily up through weekly, you can sort periods, spot the dimension that's been persistently thin, and check whether the story you're planning to tell about your year actually matches the trend in your own notes. It's directional, not quantitative, and I'd never put a score in front of a manager. But it's a decent check on self-narrative, which tends to be built out of the three weeks you remember most vividly.

Two caveats worth stating plainly. The vault only knows what you fed it, so a quarter you didn't write down looks exactly like a quarter you didn't work, and the model will happily paper over the difference if you let it. And the whole thing lives or dies on citation. A claim that traces back to a note that traces back to a merged PR is one you can defend in the room. A claim the model produced because it sounded like the right shape of achievement is one you should delete before anyone else reads it.

What's next

The weakest part of all this is exactly the part that matters most: the work that never turned into code. When I haven't typed manual context into a daily note, the job falls back on inferring it from the diff, producing something like "probably coordinated this across a few services". Occasionally that's a fair reading. Just as often it's the model writing plausible fiction about meetings I never had.

What makes that a good problem rather than a depressing one is that the fix isn't more clever prompting. It's more sources. Everything the diff can't see already exists, timestamped and structured, in systems I use all day: my calendar, the meetings themselves, my sent mail, the channels where the arguments actually happen. None of it needs inventing. It needs plumbing in.

Calendar is the obvious first source, and probably the highest value per unit of effort. Outlook exposes it through the Microsoft Graph API, and a day's events are already a structured record of where the time went and who it went with, before anyone summarises anything. Just enumerating them turns "inferred from PR evidence" into something factual. The metadata carries more signal than it looks: attendee lists across team boundaries are a scope indicator you cannot derive from a repository; the organiser field distinguishes meetings you were summoned to from the design review you called, which is close to a direct leadership measurement; recurring events separate standing ceremony from the things you convened deliberately. A daily note that opens with the meetings that genuinely happened is a much better prompt for everything downstream than one that opens with a guess.

Meetings themselves are the richer source, and the one that needs the most discipline. Both Teams and Webex will hand over transcripts and generated summaries through their APIs: Graph for Teams, the Webex REST API for meetings and recordings. The temptation is to drop a transcript into the daily note and let the weekly pass deal with it. That's the wrong shape. A transcript is thousands of words of low-signal conversation wrapped around a handful of load-bearing sentences, and burying those sentences in bulk text just moves the extraction problem downstream to a stage with less context. Far better to ask a narrow question at ingest (what was decided, what did I commit to, what did I argue for and did it land) and write only the answer into the note. Narrow questions against a transcript are a task models are genuinely good at. Open-ended summarisation of one is not.

Sent mail is underrated. The instinct is to point at the inbox, but the inbox is a record of what other people wanted from you. The sent folder is a record of your own output: the thread where you made the case for an approach, the long reply that unblocked someone, the escalation you decided was worth making. Graph will filter it by date and recipient, and the filtering needs to be aggressive. The useful fraction is small, but it's the fraction with your reasoning in it.

Slack, or whichever channel tool you live in, is the noisiest and the most tempting. Scoped to your own messages in public engineering channels, it catches something the other sources miss entirely: the design reasoning that gets written out properly in a thread and then never makes it into a document. Threads where you replied at length are review and unblocking work that leaves no other trace. Reply and reaction counts are a crude proxy for whether an argument actually landed. It's also the source most likely to manufacture significance from someone thinking out loud, so it's the one I'd add last and weight lowest.

The architectural point that ties these together is that each of them should be a source, not a summariser. Every one lands as evidence in the daily note, inside its own marker block, tagged with where it came from. The fail-closed rule extends to all of them too, because a calendar API that errors must never be allowed to mean a day with no meetings. Which leads to the change I'm most interested in: a trust ladder. Something I typed myself outranks something pulled from a structured source like a calendar entry or a pull request, which outranks something extracted from a transcript, which outranks anything inferred from a diff. Right now every one of those lands in the same typeface, which is precisely what makes the inference problem corrosive: it launders a guess into the appearance of a fact. Record the provenance and the review query can prefer the top of the ladder, treat the bottom as a prompt to go and check, and stop pretending the difference doesn't exist.

There's a boundary question that comes with all of this, and it deserves stating rather than discovering later. Transcripts and mail threads contain other people's words, and none of my colleagues signed up to appear in my work log. The rule I'd apply is the same one that already governs where this runs: stay inside approved tooling, extract my own contributions rather than archiving everyone else's, and keep the retained artifact at the level of "a decision was reached about X" rather than a verbatim record of who said what on the way there. That's not only the defensible position, it's the more useful one: the decision is the thing worth having in a review, and the argument that produced it usually isn't.

The smaller items are tractable too. The retrieval flakiness, where queries come back empty on days I'd clearly shipped, traces back to how my identity resolves across accounts, and the honest fix is an explicit identity map resolved once at the start of a run and asserted against, rather than the cross-check currently papering over it. The rollup stops at weekly, and the monthly layer above it is a folder and an intention; that layer gets much easier to justify once daily notes carry meetings and decisions as well as merges, because then there's an arc to see rather than a longer list. And the citation trail that the whole review use case rests on is still only half automated: the links exist from career artifact down to daily note down to pull request, but assembling them into per-claim evidence means walking the chain by hand. Carrying the evidence identifiers along at generation time, rather than reconstructing them a year later, is the change that finishes the loop.

Put together, that's a system where every claim in a review has a source, every source has a provenance, and the honest answer to a criterion I have no evidence for is that I have no evidence for it. That seems like a reasonable thing to build towards.

None of this makes me a better engineer. It just means that when I'm asked what I did last quarter, the answer is sitting in a folder instead of somewhere in my head, and I didn't have to keep a work log to get it there.