As an Engineering Manager in a Platform team, I manage 10 engineers. I'm hiring more. I run weekly 1:1s, facilitate technical decision meetings, screen candidates, moderate retrospectives, and still need to keep up with the delivery of a platform spanning dozens of AWS accounts.
Besides the lack of time to focus on technical problems, the technical part is not even the real challenge. The less obvious problem becoming an Engineering Manager is: the skills you need as an engineering manager are fundamentally different from those that made you a great engineer, and there's no compiler or unit test to tell you when you're doing them wrong. The feedback loop is absent or very slow (and when you realise that, your team has already gone silent or become dependent on you because you are the main input and the main bottleneck).
Skills That Don't Come From Code
As a senior or staff engineer, you develop communication skills gradually. You present ideas, challenge others respectfully, summarise outcomes, and identify owners. You participate in technical deep dives and put candidates at ease while probing technical depth. These are valuable skills, and a good IC develops them over the years. But unless you start behaving like a brilliant jerk, they're secondary - your technical depth is still what defines you.
But as an EM, the game changes. You're not "the smartest person in the room" anymore, and increasingly, you shouldn't be.
You still have a broad context from all those alignment meetings and roadmap syncs, but you lose contact with the codebase week by week. If your organisation has principals or staff engineers, you're not even close technically anymore.
Your job is to give direction, create space for others to solve problems, and facilitate decisions, not to be the one with the answer.
This is hard. Especially when you used to be the one with the answer. The urge to jump in doesn't disappear just because your title changed.
And interviewing? Facilitation? Giving feedback that lands? These are entirely different muscles, ones you never systematically trained as an IC.
I learned some of these skills at Toastmasters International years ago - structured practice for communication and leadership in a safe environment.
But I know I still have a lot to learn. And the feedback loop in management is painfully slow: you facilitate a bad meeting and only realise it weeks later when the decision doesn't stick. You run a mediocre interview and only notice the pattern after three hires. You micromanage your top performer out of "helpfulness" and only connect the dots when they start disengaging.
The Opportunity: Transcripts + AI = Instant Feedback Loop
Recently, we moved to Google Meet, and Gemini now automatically transcribes meetings (those that we want to transcribe, so we can focus on the discussion rather than taking notes. We also adopted a new tool for the hiring pipeline that produces transcripts of the interviews.
In the last 10 months, I pushed myself to use AI for everything - partly out of genuine interest, partly as self-defence against burnout and context-switching (the series this article belongs to is the result of this journey).
So one day, after receiving the email informing me that meeting transcripts were ready, I tried something simple: I fed Kiro the transcript of a screening interview I'd just conducted and asked for honest feedback on my interviewing technique.
The result was unexpectedly useful. Not generic advice like "ask open questions" - specific, timestamped observations with reasoning and concrete suggestions for what to do differently:
"Your intro ran for over 3 minutes of near-continuous speaking. You explained the team structure, the product roadmap, and ongoing projects before asking a single question. By the time the candidate spoke, they'd been passive for minutes, losing energy and initial spontaneity. Target: 90 seconds. State your name, the role, and the interview format, then flip it to them. Everything else can come up organically during the conversation."
"At 38:52, you shared your own approach: 'I honestly like managed services. The more I give to AWS to manage, the less we have to take care of.' This signals to the candidate what the 'right' answer is. You'll never know if they genuinely agree or are just mirroring your preference back. Keep your technical opinions out until after the gate decision."
"At 22:14, you filled a silence with 'yeah, of course, that makes total sense' after the candidate gave a surface-level answer. This validates before probing. The silence was working - they were about to elaborate. Next time: count to five. Let them fill the gap."
But it wasn't only criticism. It also reinforced what worked:
"Excellent persistent probing at 11:21-13:47. When the candidate described security guardrails generically, you pushed for a specific example. When they couldn't provide one, you pushed once more with a concrete rephrasing. Only after they explicitly said they couldn't did you move on. This is exactly the right technique - push twice, accept the answer, don't rescue. You managed the transition gracefully, leaving the candidate at ease to stay honest for the rest of the interview."
"Good patience at 15:18 when the candidate misunderstood your question. You rephrased with a concrete scenario. No frustration visible."
This was the kind of feedback that a peer observer might give you - if they had the time, the attention to detail, and the willingness to be direct.
This is actually what happens at Toastmasters: during a talk, different people observe different things (movement, eye contact, tone of voice, content structure, engagement), and afterwards these observations are shared publicly, which is also great training for giving feedback openly. At work, during a technical meeting or an interview, this isn't happening - unless you did something so obviously wrong that a colleague feels the urge to tell you.
From One-Off Experiment to Structured Practice
After that first experiment, I created two Kiro steering files - structured instructions that turn the AI into a specialised evaluator:
The Interviewer Evaluator
This steering file knows my interviewing approach, my known patterns (talking too long in intros, answering my own questions, filling silence with validations), and a structured evaluation framework.
When I feed it a transcript, it produces:
- Questioning technique breakdown - counts how many spec-driven, behavioural/STAR, edge-case, direct, leading, and self-answered questions I used
- Signal quality assessment - for each topic, whether I got a strong signal, a moderate signal, or a contaminated/no signal
- Pattern tracking - am I improving on my known weaknesses? What's persistent?
- 5 C's extraction - did I actually assess Competence, Confidence, Communication, Character, and Culture?
- Gate decision quality - Did I get enough technical signal to confidently make a go/no-go call?
Since I keep all my Kiro feedback in a specific folder, I can use the steering file to track patterns across interviews and see my trend line.
Here's the core of the steering file - it's just markdown:
---
inclusion: manual
---
# Interviewer Evaluator
When given an interview transcript, evaluate the **interviewer**, NOT
the candidate. Track patterns across sessions.
## Instructions
Provide timestamped, specific feedback with exact quotes.
Compare against known patterns from previous sessions.
Do NOT evaluate the candidate.
## Personal Pattern Tracking
| Pattern | What to check |
|---|---|
| Intro length (target: <90 sec) | Measure from transcript timestamps |
| Answered own question | Count instances with quotes |
| Filled silence with validations | Count "of course", "yeah yeah" |
| Got a concrete STAR story | Yes/No - quote it if yes |
| Shared own solution/opinion | Count instances |
| Pivoted to candidate questions by 40 min | Check timestamp |
## Critical Rules
1. Be direct and specific - quote timestamps and exact phrases
2. Acknowledge improvement - if a pattern is getting better, say so
3. Don't mix candidate evaluation with interviewer feedback
4. If you see zero issues, your calibration is wrong - look harder
5. Flag "shared own solution" as a key anti-pattern
The full version also includes a questioning technique taxonomy, signal quality definitions, and an output template for consistent formatting across sessions.
The Meeting Facilitation Evaluator
Same concept, different skill. This one evaluates me as a facilitator - specifically for technical decision meetings. On many topics, I have strong opinions and although they are loosely held, I'm also aware that my being Italian makes my communication sound even stronger. On top of that, even in flat hierarchies, your title carries weight. Although I'm just expressing a thought, for some people and some cultures (see my post about The Culture Map) it might land as a directive.
The cardinal rule it enforces:
Facilitator ≠ Advocate. If you must share your view, announce the hat-switch explicitly.
After a recent architecture decision meeting (75 minutes, 10 participants, successful outcome), the AI caught four moments where I advocated without announcing it.
It also caught that someone else had to close the decision for me: I got so engaged in the discussion that I forgot my job was to CALL for the decision, not wait for it to emerge organically.
Again, it wasn't just about criticism. It also reinforced what worked:
"You let the person with the most resistance shape the solution. Their suggestion became the team's shared plan. You didn't prescribe the 'how.' This is excellent facilitation."
"Your summarisation is your superpower. You naturally paraphrase what people say to confirm understanding. This builds trust and shows you're listening."
Of course, I think there is some AI flattering in this feedback, but hey, sometimes it's good to hear.
Here's the core of the facilitation evaluator:
---
inclusion: manual
---
# Meeting Facilitation Evaluator
When given a meeting transcript, evaluate the **facilitator**, NOT
the meeting outcome or participants. Focus on process, not result.
## The Cardinal Rule
Facilitator ≠ Advocate. If you must share your view, announce
the hat-switch explicitly.
## Facilitator/Advocate Separation
Count and quote every instance where you:
- Endorsed a participant's position
- Argued for a position yourself
- Led with framing that implied the "right" answer
- Announced hat-switch properly
Score: count advocacy moments vs. hat-switch announcements.
Target: 0 unannounced advocacy moments.
## Decision Mechanics
- Did you explicitly call for the decision? (Yes/No)
- Did someone else have to close it?
- Were dissenting concerns recorded?
- Were the next steps assigned to owners?
## Critical Rules
1. The cardinal rule is always #1 in feedback
2. Outcome ≠ facilitation quality - evaluate PROCESS not RESULT
3. If you see zero issues, your calibration is wrong
4. Credit the hard stuff - facilitating when you have the strongest
opinion is much harder than neutral facilitation
The full version also tracks time management per phase, scope management (how fast you park tangents), psychological safety (did all participants speak?), and questioning technique (genuine questions vs. advocacy disguised as questions).
What Makes This Feedback Useful
I can get feedback immediately after the interview - while I debrief with the Talent Acquisition Manager and decide whether to move the candidate forward, I ask Kiro to evaluate me and prepare notes for what to practice next time.
The feedback is specific and honest. AI doesn't worry about hurting my feelings or preserving a working relationship. It's direct in a way that peers often aren't.
And running this evaluation is so fast that I can do it every single time. No bad weeks. No "too busy to reflect."
What the AI Can't Do
It can't read the room. A transcript doesn't capture the energy shift when a discussion stalls. It doesn't show body language when a candidate gets nervous. It can't tell you if someone's silence was thoughtful or uncomfortable. As with any framework, there will be cases where the suggestions don't apply. AI will flag it every time; you still need to decide if it was the right call.
It can't replace human feedback. Although I now get feedback from transcripts, asking my team "how was that meeting?" - publicly and in 1:1s - is still irreplaceable. I want to build a culture of trust, honest feedback, vulnerability, and psychological safety. I expect my team members to point out what I can do better.
The Setup
- Meeting transcription - Google Meet with Gemini, or any AI tool you can invite to your meetings.
- A structured evaluation prompt - A detailed framework with specific dimensions, pattern tracking, and your known growth areas, written in simple markdown (you've seen the snippets above).
- AWS Kiro or any AI assistant - I use steering files, but as a simple repeatable prompt or skill, it could work with Claude, Copilot, etc.
- A place to store the feedback - I keep mine in an Obsidian vault, organised by type (interviewing, facilitation) and date. This makes trend tracking trivial.
A note on privacy: The transcripts themselves are never stored. Especially for interviews, I treat them as ephemeral - I feed them, already anonymised, to the AI, get my feedback, and discard the transcript. Only the feedback about my performance stays in my vault. No candidate names, no answers, no personal data persists beyond the session.
From AI-Native SDLC to AI-Native Leadership
Most AI-for-engineering-leaders content focuses on technical leverage: code review, documentation, architecture decisions, ticket triage. I've written about all of that.
But the biggest surprise in my AI journey has been this: AI is equally powerful for developing the "soft" skills that leadership actually requires.
Interviewing, facilitation, communication, active listening, neutral moderation - these are the skills that determine whether your team trusts you, whether decisions stick, and whether you hire the right people. And they're the skills where most EMs are flying blind, learning only from rare, delayed, and often sugar-coated feedback.
With this approach, a meeting transcript and a structured AI evaluator give me immediate, detailed, honest post-performance analysis.
In the 8 years since I started my leadership journey, it's been very hard to get real actionable feedback. Sure, there are 1:1s with my Senior EM or Director, sure, there are quarterly performance reviews. But often these are too bound to OKRs and company goals - they don't focus specifically on the skills you need every day, the ones that compound over time and allow you to run a high-performing team smoothly.
And yes, EM roundtables, coffee chats and Agile Coffee exist, but sharing often stays shallow. People protect their image because in many orgs, today's peer EM is tomorrow's director candidate. Vulnerability feels risky.
What actually helps is a network outside your immediate org.:
- ex-colleagues who followed a similar path.
- community of practice (I'd love to find proper EM meetups beyond Slack groups).
- Reddit threads where anonymity unlocks honesty.
AI doesn't replace human connection - nothing can. But it gives you an interlocutor who doesn't judge. You can process a frustrating meeting, admit you handled a 1:1 poorly, or confess that you don't know what you're doing, and get honest, constructive feedback without feeling exposed. No politics. No sugar-coating.
Countering the Sycophancy Problem
AI can be too agreeable, and often it tells you what you want to hear. That's the opposite of coaching.
One way to fix that is to structure your evaluator to be explicitly critical.
My steering files include instructions like "do not validate, surface gaps", and "if you see zero issues, your calibration is wrong - look harder." The evaluation framework itself prevents flattery because it's looking for specific patterns and counting them.
For broader use, there are dedicated anti-sycophancy skills you can layer on top:
- grill-me - forces the AI to challenge rather than agree
- Independent Thought Challenger - pushes back on your reasoning
Or write your own anti-flattery instructions. This matters for the evaluator but also for any other agents or skills you use - when planning a new feature or fixing a bug, you want AI to challenge your thinking, not just tell you it was a great idea and a very efficient solution.
What's Next
Right now, I have two simple steering files. I'm planning to grow them into a proper skill or agent that I can use to:
- Pre-meeting coaching - feed it the agenda and participants, get facilitation reminders ("remember: you have a strong opinion on this topic, commit to not voicing it during the discussion phase")
- Practice specific scenarios - interjecting respectfully when someone is hijacking the meeting, steering a conversation that's stalling, probing further without putting too much pressure. For this, I'm finding a lot of value in voice prompting (I use Wispr Flow, but SuperWhisper or even the default Mac Dictation Mode work well and for free). I don't lose spontaneity by typing - Kiro throws questions at me, and I can just talk and practice how I would behave during the actual meeting.
- Share anonymized versions of the steering files in my ai-native-engineering-leadership repository so others can adapt them.
The pattern is the same every time: take a skill with a slow feedback loop, add transcription + structured AI evaluation, and compress months of learning into weeks.
Less toil. More growth. Still human.
Related posts:
Top comments (0)