DEV Community

Cover image for How Do You Evaluate People Who Are "Good at AI"? The ABCD2 Framework
Hichoi-Dev
Hichoi-Dev

Posted on

How Do You Evaluate People Who Are "Good at AI"? The ABCD2 Framework

Try asking this in your next team meeting: "Who on our team is the best at using AI?"

There will probably be a moment of silence. Someone thinks of the person who uses the tools the most. Someone thinks of the person who's good with prompts. Someone thinks of whoever recently produced something impressive-looking with AI. But none of these answers feels convincing. There's a gut sense that something is off — and yet nobody knows what the right criterion would be.

Nearly every company is standing in front of this wall right now. AI competence is rising to the top of hiring and performance criteria, and there's no yardstick to measure it with. This essay proposes one. The short answer is a set of five axes I call ABCD2. But before that, we need to fix the question itself.

The very idea of "good at using AI" is wrong

The title of this essay says "good at AI" — but that phrasing is itself the trap.

The word "using" carries a hidden frame: AI is a tool, and the person is a user. It puts AI on the same shelf as Excel and Photoshop — something you get skilled at operating. On top of that frame, evaluation naturally becomes a measure of tool proficiency. How many features do you know? How fluently do you operate it?

But AI is not a hammer. AI is an engine that judges and executes. It's less like a tool and more like a junior colleague you can delegate work to. And once you think of a junior colleague, the frame flips. Behind every junior who performs well, there is invariably a senior who makes them perform well — someone who explains the context of the work, lays out the criteria for judgment, makes it safe to ask about ambiguities, and verifies the output. That senior isn't "using" the junior. They are making the junior do the work well.

AI is exactly the same. The person who truly gets the most out of AI is not the person who is good at using AI, but the person who helps AI do the work well.

And what do they help with? Two things, broadly. One is knowledge building — preparing the ground AI stands on: what our organization's terms actually mean, what the criteria for judgment are, what separates a good deliverable from a bad one, all captured in a form AI can draw on. The other is context pipelining — designing the flow so that this knowledge reaches the AI at the right moment, in the right shape. Which context rides along with which task, what the output gets verified against, and where things route back to when verification fails.

Seen from this angle, what is a prompt? It's merely the last line of a much longer flow. A good prompt is the output of good knowledge building and good context pipelining — not the substance of the capability itself.

Sharp readers will have already caught it by now: this is where ontology becomes important. Structuring an organization's terms and concepts, the relationships between them, and its criteria for judgment into a form AI can draw on — that is precisely what an ontology is. Peel one layer off the phrase "knowledge building," and what you find inside is ontology. The more AI becomes an organization's execution engine, the wider the gap will grow between organizations that have an ontology for that engine to stand on and those that don't.

Why every existing yardstick misses

Hold on to that redefinition, and it becomes obvious at a glance why the evaluation methods in use today all miss the mark.

Start with the traditional yardsticks: coding tests, algorithm problems, certifications, credentials. What these have actually been measuring is execution ability — how accurately and quickly you carry out a well-defined problem. But execution is precisely the part that has gone to the AI. The object of measurement has moved wholesale to the AI's side while the measuring instrument still points at the human — of course the predictive power evaporates. There is no guarantee that someone who aces a coding test works well with AI. If anything, the more someone is optimized for solving well-defined problems fast, the clumsier they may be at defining problems and designing context.

What about the newer yardsticks, the ones supposedly built for the AI era? Two are common.

First, prompt-skill assessment. As we just saw, the prompt is the last line of the flow. Measuring only the last line is like hiring a chef based on plating alone. Worse, prompt techniques are tied to specific tools and specific model versions, so they have a short half-life. The better the models get, the less the tricks are worth — and what remains is the ability to design what to ask for and what context to provide.

Second, tool usage volume and time spent. The assumption is that whoever uses AI the most uses it best — and it may be exactly backwards. The person who has built a proper pipeline actually types less. The person who starts from scratch every time, manually grinding through long conversations, racks up the highest usage. Using a lot and making it work well are different things.

What the two yardsticks share is that both are measurements from the "use" frame. A tool-proficiency ruler cannot capture the ability to make something do the work well.

So what should we measure — ABCD2

"The ability to make AI do the work well" cannot be measured as a lump. To measure it, you have to decompose it. I decompose it into five axes: Comprehend, Abstract, Dissolve, Build, Describe. Taking the initials: ABCD2.

From an evaluator's point of view, each axis becomes a question.

Comprehend — does this person understand the world to be connected? Do they grasp what the task actually touches, what that word means in this organization, who is affected by the outcome? Without comprehension, they misunderstand the very thing they're asking the AI to do.

Abstract — does this person abstract the essence? Can they strip a workable structure out of messy reality? Good abstraction simplifies without discarding what matters. Without that sense of balance, the problem definition handed to the AI is warped from the start.

Dissolve — does this person break work into verifiable units? Delegate the whole lump and you can't tell where it went wrong. Do they cut the work along boundaries where AI output can be checked? This connects directly to the ability to catch AI's plausible-sounding wrong answers.

Build — does this person construct the flow? Do they weave the dissolved units into a single flow with verification and fallbacks? Do they build not a one-off request but a pipeline that is repeatable and can be handed to others?

Describe — does this person explain and take responsibility? Can they articulate what they connected, why, and how? And for the AI's output, do they stand as the subject of responsibility — "I designed it to judge this way"?

You may have noticed: nowhere among these five axes is "proficiency with a particular tool." Swap the tool, swap the model — the axes stay valid. That is the minimum requirement for any evaluation criterion worth adopting.

A case — same instruction, two people

Abstract axes alone don't quite land, so consider a scene. A manager gives two team members the same instruction: "Analyze this month's customer inquiries with AI and write it up."

A does this: downloads the inquiry data, pastes the whole thing into an AI, and asks, "Analyze these customer inquiries and summarize the main issues." After a few follow-up exchanges, a clean summary comes out — category breakdown, top complaints, even improvement suggestions. Time spent: 30 minutes. The report looks solid.

B does this: first checks where these inquiries come from. Learns that each channel has a different character, and that the CS team already maintains a tagging taxonomy (Comprehend). Takes that taxonomy as the base, but reorganizes it around three lenses that fit this analysis's purpose — deriving next quarter's improvement backlog: "recurring inquiries / new types / urgent" (Abstract). Instead of feeding the data in whole, splits it by channel and by week, so each batch's results can be checked against the source (Dissolve). Writes up the classification criteria, the company's term definitions, and examples of good classification into a single document that rides along with every request, and routes ambiguous cases to human review (Build). And at the end of the report, writes: "This analysis is based on the CS tagging taxonomy; channel X data is not included; the criterion for 'urgent' is one I defined as follows" (Describe). Time spent: half a day.

Now — looking at this month's report alone, who did better? Hard to tell. Honestly, A's report came out faster and doesn't look inferior. By the existing evaluation method — deliverable quality and speed — A might even win.

The difference shows up next month. When the same instruction comes down again, A starts over from scratch. The classification criteria have drifted from last month's, so trend comparison isn't even possible. B pours the new data into the flow already built, and that's it. The criteria are stable, so month-over-month trends accumulate — and even when B goes on vacation, someone else can run the flow. What A left behind is a report. What B left behind is an asset.

This is the crux. The existing evaluation looks only at "this month's deliverable," so it cannot separate A from B. ABCD2 looks not at the deliverable but at the five points in the process that produced it — and so it can.

In practice — how to ask in interviews and reviews

So how do you actually probe ABCD2 in an interview or a performance review? One example per axis:

Comprehend — Give a task with the context deliberately missing. "How would you analyze our service's churn rate with AI?" — without defining "churn" or describing the data. A strong candidate asks back before answering: how are we defining churn, and what data do we have? The one who launches straight into a plausible-sounding procedure without asking is the one who, on the job, will hand the AI a misunderstood problem.

Abstract — Give a messy real case and have them structure it. Throw them an exception-riddled business process and ask, "If you were delegating this to AI, what stages would you organize it into?" Watch whether they find the balance between the person who tries to cram in every exception and the person who throws away what matters.

Dissolve — Give them a plausible wrong answer from an AI. Show a deliverable that reads smoothly but has one factual error, and ask, "If you had to review this, how would you do it?" "I'd reread the whole thing" and "I'd cut it into verifiable units and check each against the source" are entirely different capabilities.

Build — Follow up with: "What if you had to repeat that every week?" See whether they can turn a one-off solution into a repeatable flow, and whether verification and exception handling are designed into the flow.

Describe — About an AI deliverable the candidate produced, ask: "If this result turns out to be wrong, whose responsibility is it?" The distance between "the AI produced it that way" and "I designed it to judge by these criteria, so it's my responsibility — and here's what I'd fix" is the distance of this axis.

You'll notice what these have in common: none of them are questions with a correct answer. They are questions that expose how a person handles a problem. The exact opposite direction from execution-ability testing.

Completing the frame — three axes

ABCD2 alone doesn't complete a hiring decision. Add two more axes and you get a frame you can actually use. One is personality fit — does this person have the temperament to bear that kind of thinking and ownership in the first place? The other is domain fit — is the world they must comprehend and connect precisely our domain? However high someone's ABCD2, without domain knowledge their Comprehend doesn't engage; and however well capability and domain align, a temperament that dodges ownership collapses their Describe.

Put together: ABCD2 (capability) × personality fit (temperament) × domain fit (context). These three axes are my answer to the question, "how do we hire and place talent in the age of AI?"

Closing — stop looking for people who use it well

Back to the title: how do you evaluate people who are "good at AI"? Now we can answer. Don't look for people who are good at using AI. Look for people who make AI do the work well. And that ability can't be measured as a lump — but decomposed, it can. Do they comprehend the world to be connected, abstract the essence, dissolve into verifiable units, build the flow, describe and take responsibility?

One thing is worth saying honestly in advance. Apply this standard seriously, and you'll find that far fewer people pass than you expected. That's not because the standard is wrong — it's because until now we've been measuring with the broad ruler of "execution." Change the ruler, and the map of talent gets redrawn. And the company that draws that map first takes the lead in the great talent reshuffle of the AI era.

This article is also post in my gist page : https://gist.github.com/casamia918/678cd716333a43f6b5fe539625b1e1ac#file-abcd2_framework_en-md

Top comments (2)

Collapse
 
alexshev profile image
Alex Shev

I like that this treats AI competence as judgment, not tool frequency. The strongest people are usually not the ones who prompt the most; they are the ones who know when to trust, when to verify, when to narrow the task, and when to stop using the model.

Collapse
 
vicky_acedia profile image
Vignesh Athiappan

Like the way of ABCD2 competence