DEV Community: Unlocked Consulting

The step got faster and the process got slower

Helkyn Coello — Mon, 02 Nov 2026 16:48:59 +0000

In a supplier's demo, each incoming case (an invoice, a customer email, an order) arrives, the system reads it, matches it against the records, and closes it. Then the next one, and the one after that. The supplier is not cheating. Those cases really do clear, and they really are the bulk of the volume. What nobody opens is the other folder: the cases the system passed over.

There is a familiar argument that the hours an automation saves never show up in delivery, because nobody decided in advance where they should go. This piece is about something harder than that. Automation can add work. The cases it cannot handle come back to a person, in smaller numbers, usually harder, and often without the context the old process gave them.

Where the work goes after the automated step gets faster

The mechanics are not subtle once you look for them. An automation is built against the routine shape of the work, because that is the shape you can describe in rules or train a model on. It takes the routine cases. What stays behind is everything irregular, and irregular cases were never evenly distributed across the old workload. They were already the ones that took longest.

Then two more things get added that did not exist before. Someone has to check a sample of what the automation decided, because nobody signs off on an unchecked machine making commitments to customers or suppliers. And some share of the automated decisions turn out wrong and come back, usually later, usually from the customer, which is the most expensive moment to find out.

As an illustration, take this example: invoice matching. The system matches invoice, purchase order and delivery note, and clears the ones where all three agree. What is left on the desk is the invoice where the quantity differs by two units, the one billed in a different currency, the one for an order someone changed by telephone and never wrote down. Those used to arrive mixed in with the simple ones. Whoever worked through the stack had seen forty invoices from that supplier that week and knew which of its branches habitually ships short. Now the hard one arrives alone, twice a day, with none of that around it. The time per invoice goes up, and the person handling it is working with less.

A second illustration, with the same shape: inbound customer email. The automation answers the standard requests well. What reaches a person is the angry one, the one with three unrelated questions in it, and the one that refers to an answer the automation already gave and got wrong, so the conversation has to be reconstructed before it can be continued. Add the daily sample check on the automated replies. Total human minutes can rise while the share of emails answered without a human also rises.

Both numbers are true at the same time, which is why step-level reporting does not catch it.

A step-level metric counts what the automation handled. It has no column for what it handed back, none for the sample check, and none for the case that came back from the customer three weeks later.

And the dashboard the supplier ships is built only from what the automation handled.

Where this has been measured

The place with hard measurement is software development, where one step, writing the code, got much faster and the steps after it did not. I went through those numbers in an earlier piece on why saved hours have nowhere to go, so I will add only one detail here. In 2025, DORA, the long-running research programme on software delivery, added a fifth metric called Rework Rate: the share of work spent fixing or redoing things, compared with work that moves something new forward. They added it because counting finished work had stopped telling them whether teams were really doing better. Rework is exactly the cost this piece is about, and it took a dedicated measurement to make it visible.

I have my own version of this, in a production workflow I run where AI produces the first version of each piece of work. It works. The first version used to take most of a day and now takes minutes. What did not get faster was my review, which still takes hours per piece, and which now has more arriving into it. So the limit on how much finished work comes out of that workflow is set entirely by how much reviewing I can do in a week. Adding another AI step to produce first versions would not produce one additional finished piece. The measurement that would have told me this earlier is how long one piece takes from arrival to finished, and I was not tracking that number, because the fast part felt like progress.

None of this is an argument against automating. In that workflow I would not go back, and the arithmetic holds whenever two conditions are true: the share of cases that leave the automated path stays small, and those cases are not much harder than the average case used to be. Plenty of office processes meet both, such as standard order entry, address changes and recurring monthly reconciliations. The point is that both conditions are measurable before you widen anything, and almost nobody measures them.

What to measure instead

There are three numbers to track, and a company of fifty people without an IT department can collect all of them.

How long one case takes from start to finish, from the moment the request arrives to the moment it is done and nobody will touch it again, including the exceptions, the sample checks and the cases that came back. Measure it for a few weeks before you automate, and the same way afterwards. A clock and a spreadsheet are enough.
The share of cases that leave the automated path, and how much time your team spends on them each week, compared with the time the whole process took before.
Rework: how many finished cases are reopened, and how late. Late reopening is the one that costs real money, because by then a supplier has been paid or a customer has been told something.

Give those three numbers one named owner, the person who already answers for that process day to day. That should not be a committee, or the supplier who sold you the tool.

Before renewing the licences or putting two more people on the system, run that measurement for one month. If the automated step got faster and the time per finished case did not move, the work has shifted into the exceptions and the checking, and that is where to go looking for it.

Original article: https://unlockedconsulting.ai/blog/the-step-got-faster-and-the-process-got-slower

What Companies Actually Want From AI

Helkyn Coello — Fri, 23 Oct 2026 11:43:38 +0000

Forty-seven senior AI job descriptions, and not one of them asks you to build AI.

Over the past weeks I read more than forty senior AI and technology leadership job descriptions published across Europe. Not the company culture pages. The requirements, the responsibilities, the third bullet down where the real job hides.

I expected to find companies hiring people to build AI. That is what the market noise suggests: models, pipelines, fine-tuning, the whole apparatus of construction. It is what the conference talks are about and what the vendor decks promise.

That is not what the job descriptions say.

Of the thirty-five postings I can quote directly, not one asks the candidate to train a model. None mention fundamental research. None ask for a pipeline built from scratch. One comes close, and it asks for applied research rather than original work.

What they ask for instead is a different job entirely.

The verbs

Twelve of the thirty-five ask explicitly for governance, guardrails or compliance as a core requirement.

One asks for "guardrails and observability" around emerging agentic use cases. Another wants "architectural guardrails that allow teams to move fast without compromising long-term maintainability." A third asks for "security and certification as an engineering property, not paperwork." A fourth wants guardrailed agentic tooling as the foundation that every new team inherits on day one.

These are not the verbs of construction. They are the verbs of containment.

Which points at something that has already happened inside these organisations. They are not at the beginning of an AI programme. They are somewhere in the middle of one, and they have discovered that using AI turned out to be the easy part.

The hard part is knowing who is using it, against what data, at what cost, and with what exposure.

The gap they are hiring to close

Nine of the thirty-five describe the same specific problem in almost the same words.

One asks the candidate to take a programme that is "already live" and turn it from a set of promising experiments into the way the function actually runs. Another asks for the full lifecycle "from PoC to fully operational and scaled solutions." A third asks to prioritise high-impact use cases, show early results within ninety days, and scale what works while cutting what does not.

Two separate companies, in unrelated sectors, with no apparent connection, both used the phrase production AI depth, not proximity to it.

That is not a coincidence of language. It is a shared problem being described by people who have each arrived at it independently.

The requirement nobody was writing a year ago

Eight of the thirty-five ask for something that would have looked strange in a job description eighteen months ago: the judgement to decide what not to build.

The most explicit asks for ownership of build-versus-buy decisions across every product line, including where AI features get bought, built, or refused. Another asks for the ability to "cut through hype and identify the most promising and impactful use cases while sorting out noise." A third simply says: cut what does not work.

Refusal has become a job requirement. That only happens after an organisation has been burned by something it should have refused.

What the surveys miss

I have spent months reading the 2026 enterprise AI research, and the picture it paints is broadly consistent. ServiceNow's maturity index found that 59% of organisations are using agentic AI, while only 9% report meaningful progress toward autonomous multistep workflows — progress, not operation. The same study found that only 20% have implemented AI testing, auditing and risk-assessment processes, and that no organisation surveyed had built a cross-functional agentic operating system.

That last set of numbers is the one that matters here. Capability is being deployed considerably faster than the discipline underneath it, and ServiceNow's own conclusion is that moving from assisted to autonomous is a governance decision rather than a technical one.

The job descriptions confirm this from a different direction, and that is what makes them worth reading.

A survey tells you what an executive says when asked. A job description tells you what an organisation is willing to pay a senior salary to fix. One is a stated intention. The other is a budget line with a start date attached.

When both point at the same gap, the gap is real.

Read the third bullet

One practical observation, since I read these closely enough to notice it.

Job titles in this space have stopped carrying information. One senior technology title turned out to describe a revenue role measured on new customer acquisition. Another asked for a decade of product management experience, including pricing strategy. A third required an engineering degree in a discipline unrelated to software. A fourth carried AI in the title and described industrial maintenance and ERP integration in the responsibilities.

The title tells you nothing. The third bullet of the responsibilities tells you everything.

That is useful if you are reading these as a candidate. It is more useful if you are writing one, because a title that does not describe the work will attract several hundred people who are not the person you need.

What this means if you are the one hiring

If you are about to open a role like these, the pattern in the market suggests three things.

The first is that your candidate pool is not the one you think it is. Almost nobody in it has trained a model, and you do not need them to. The scarce skill is architectural judgement applied to a technology that most organisations adopted before they governed it.

The second is that the person who has killed an AI project is more useful to you than the person who has only launched them. Somebody who has cancelled a programme with internal momentum, and can explain the reasoning, has demonstrated the exact judgement your job description is asking for in the abstract.

The third is that if your posting says agentic — and fourteen of the thirty-five did — it is worth being specific about what you mean. Most of the ones I read were not. Agentic has become a positioning word, and positioning words attract candidates who match the word rather than the work.

The shift

The market has moved. It has moved from building to managing, and the managing part is what companies now want most.

That is a less exciting sentence than the one the industry has been telling itself, and it is better news than it sounds. It means the constraint is no longer access to the technology. It is the discipline around it: knowing what runs where, what it costs, who approved it, and what happens when it is wrong.

That discipline is learnable, auditable, and mostly unglamorous. It is also, judging by what these organisations are writing into their senior roles, the thing they are finding hardest to hire.

Methodology: 47 senior AI and technology leadership job descriptions published in Europe were reviewed. Counts in this piece refer to the 35 for which I hold the complete original text. Categories were assigned by reading the stated requirements and responsibilities; where a posting was ambiguous, it was not counted. This is a reading of a sample, not a statistically representative survey.

Original article: https://unlockedconsulting.ai/blog/what-companies-actually-want-from-ai

Shadow AI is a demand signal. Treat it like one.

Helkyn Coello — Mon, 19 Oct 2026 09:13:18 +0000

EY asked 975 C-suite leaders about responsible AI for its Responsible AI Pulse survey. One finding stands out: 32% of companies prohibit employees from building their own AI agents, and among those companies, 12% admit they cannot see whether it happens anyway. Among the companies that allow it, only 60% have a formal framework covering the whole organization, and only half say they have good visibility into what people actually do.

So even among companies with a ban, one in eight admits it cannot tell whether the ban is followed. A ban is a statement of intent, and on its own it tells you nothing about what people actually do.

The same applies to the wider case, usually called shadow AI: AI tools people use at work without the company’s approval. Shadow AI follows the work. People rarely reach for an unapproved tool out of curiosity alone; they reach for it because a job is taking too long. Picture someone in sales with several proposals due by the end of the week, someone in support working through long email threads, or someone in finance trying to understand a contract clause. Each of those is a job that needs help and is not getting it from anything the company has bought.

ServiceNow and ThoughtLab, in a survey of 4,500 executives, found that only 20% have testing, auditing and risk-assessment processes in place for AI. Most governance today covers part of the ground, and unapproved use grows in the part it misses.

before you count anything

As soon as you start recording which AI tools each named person uses, you have a record of how someone works. In the EU that is personal data under the GDPR, and in countries with works councils, such as Germany, employee representatives usually have a say before a system like that starts. The details differ by country, so settle them with your data protection officer, your employment counsel and, where you have one, your works council.

The agreement does not need to be long, and four terms are usually enough:

Purpose: the record exists to govern AI use, and is not used for performance evaluation.
Access: who may look at the data, and in which situations.
Content: what people type is not read, and names are replaced by codes by default.
Time: how long the record is kept, and the date it is deleted.

Settling those four before you pull any data is faster than settling them afterwards, when the data already exists and positions have hardened. It also makes the exercise easier to explain to the people whose usage you are about to count, and that decides whether they keep using the tools in the open.

what the records already show

Once that is agreed, you can look at what people actually use. A survey is the obvious way to ask, but it asks people to report the tools they think they are not allowed to use, so the answers will be incomplete. The records your company already keeps are more reliable.

Two sources cover most of what you need, and neither requires a project. The first is the log your internet connection keeps of which outside websites and services company devices connect to. Whoever runs your network, one person internally or the provider on your support contract, can export that list with how often each was reached. The second is the list of apps people have connected to their company accounts. Every time somebody clicks “sign in with your work account” to try a new tool, that permission is stored, and the administrator of your company’s email and office platform (Microsoft 365 or Google Workspace, for most companies) can list them all. Both exports are usually a short job.

The list will be longer than you expect. Read it before you act on it, and read it for repetition. One person who tried a tool once tells you little. Three people in the same team using the same tool every week tell you which job is underserved and roughly how much it matters to them.

answer the request

For each of those groups, the response is one approved tool that does that job at least as well, and the measurement is how many people use it in a given week, not how many licences you bought or training sessions you ran. If that number does not move within about four weeks, your approved option is slower or clumsier than what people found themselves, and they will keep using their own. You can mandate it anyway, and you will end up with the same blind spot you started with.

For tools still under review, blocking is usually the wrong move for a mid-sized company. It is better to offer the tool in a controlled way: an account the company pays for and manages, with the setting that keeps your data out of the provider’s model training switched on, plus a plain rule about what may be pasted in. That gives you visibility and most of the protection while the review runs.

A hard block is still right in one place, and it should be enforced by the system, not just written into a policy document. There is data you are contractually or legally not allowed to put into an outside system: client files under a confidentiality clause, personnel records, and anything covered by a data processing agreement (the contract that lists which outside providers may handle personal data for you). Keep that data behind technical controls that stop people from copying it out in bulk or uploading it to tools outside the approved list.

Write the result on one page: which tools are approved for which jobs, what may never be pasted anywhere, who to ask about a tool that is not on the list, and how quickly they will get an answer. Keep that answer fast, because a slow approval teaches people to skip the question and use the tool without permission.

Start this week with the export of apps connected to your company accounts, and before anyone attaches a name to a line in it, agree on purpose, access, content and time with the people who need to be involved.

Original article: https://unlockedconsulting.ai/blog/shadow-ai-is-a-demand-signal

The visual platform tax: why your no-code automation grows expensive faster than you think

Helkyn Coello — Mon, 19 Oct 2026 09:12:47 +0000

Somewhere in your company there is probably a workflow built on a no-code platform. Someone sensible built it, it solved a real problem, and it still runs. When the subscription comes up for renewal, the question on the table is whether the platform still does the job, and the answer is almost always yes.

That is the wrong question. The licence is the smallest cost of a visual platform. The larger costs build up quietly in other places, and the largest one, leaving, never appears in the business case at all.

what nine months on a visual platform taught me

I learned this on a side project: a multi-step publishing workflow, built in evenings and on weekends, with a website and six social channels at the end of it. The first version ran on Make.com, with a spreadsheet as the database underneath. Both were defensible choices for a testing project, and the combination worked.

Getting it to work reliably took about nine months. Most of that time went to edge cases: each platform had its own limits and its own formatting rules, one step in the flow would fail silently and leave the spreadsheet half written, and connection errors needed automatic retries built from more steps, which then needed their own error handling. Those were evenings and weekends, so nine months of calendar time covers far fewer hours of actual work than it sounds like. The grind was real either way.

Then I rewrote the whole thing as ordinary code, with an AI coding assistant helping. It took about six weeks of the same side-project hours. The new version is faster, cheaper to run and easier to change, and when something breaks, the error points to the problem.

where the cost actually builds up

Nobody compares platforms on the thing that costs the most. The comparison people run is time to the first working version, and on that measure the visual platform wins almost every time. It should win. That is what it was designed for.

The money goes somewhere else. First, to fixing things when they break, which does not get cheaper over time. In code, a failure tells you the exact line that broke and what changed since it last worked. In a visual flow you get a stopped run and a long diagram to search, and what you learned from the last failure does not help with the next one, because each connection to an outside service fails in its own way.

The second place the money goes is the limit of what the platform can handle. That limit does not announce itself in the first week. It arrives as a slope that gets steeper once the straightforward part of your system is built. A simple “if this, then that” choice grows into a sprawling diagram. Something the system needs to remember ends up as a spreadsheet row that two flows try to update at the same moment. By the time the slope is obvious, months of your business rules live in a format nobody on your team can read or review.

The last of the three, and the largest, is migration: the cost of getting off the platform. It gets left out of the business case entirely, because at evaluation time it is years away and belongs to someone else’s budget.

There is a halfway move that feels clever, and I tried it. Before migrating fully, I kept the old flows running and called them from the new code. Technically it works. It also keeps the platform in the middle of the system, keeps the subscription running for a tool you no longer want, and gives each of those connections a reason never to be revisited: nothing is broken, and the backlog always has something louder on it. Bridges like that tend to become permanent. If your team takes that route, put a removal date on each connection and treat it as a real deadline.

the same shape at company scale

ServiceNow and ThoughtLab’s 2026 maturity index, a survey of 4,500 executives, found the share of organizations reporting streamlined, integrated workflows across business functions fell from 30% in 2025 to 16% in 2026. That is a fourteen-point drop on a measure that was supposed to be climbing. The authors attribute the drop to companies still carrying fragmented platforms, now with a layer of AI agents spread on top of them.

What happened to my side project happens to companies, with more platforms involved. No individual platform failed there either. Each one was bought for a sensible reason, solved the problem it was bought for, and stayed. What changed is that there are now many of them, each holding a slice of a process, each with its own connections and logins, and the work of keeping them joined up costs more each year, because there is more of it.

why leaving costs less than you think

Two things made that six-week rewrite of my side project possible, and only one of them is AI.

The first is that I knew precisely what to build. Nine months on the visual platform had produced a complete specification in my head: every edge case, every limit, every formatting rule. That knowledge could not be shortcut, and AI did not generate it. Had I started in code on day one, I would still have had to learn all of those things. The nine months were the price of learning them.

The second is that, with the specification already settled, writing the code took little time, and an AI assistant did much of it. Those two things together are what change the arithmetic of leaving. A rewrite used to mean a budget line and a steering committee, because the building was most of the work. For a system the size of my side project, it was six weeks of evenings. Your workflows are larger than mine, but the same two conditions apply: the people who run them every day already carry the specification, and the code now gets written faster than it used to. If your estimate of what leaving a platform costs was formed before AI coding assistants arrived, it is probably far too high, and too high in the direction that keeps you on the platform.

when the visual platform is still the right call

None of this means you should never use one. My first version was a sensible choice for something that might not survive its first month. A visual platform is a good fit for a prototype you may throw away, a small flow that will stay small and stable, or a team with nobody who can maintain code. The trouble starts when something built for one of those reasons grows, step by step, into a system the business depends on, and nobody asks about the exit while it is still cheap to answer.

the one-day test before you renew

So before you renew your no-code platform, ask a different question. The platform will handle what you need today; the demo already showed you that. What you need to know is what leaving looks like once it stops handling what you need, and whether you can afford it on the day you find out.

Here is how to check. Take the two or three most complicated flows in the system: the ones with retries, partial failures and data shared between systems, not the tidy one from the demo. Ask your IT team or an external developer to rebuild them as code against your real systems, and give it one day. Then compare the two: how much of it they rebuilt, how easy it is to test, and how easy it is for someone else to read. It costs one day, and it tells you what leaving would cost before you have nine months invested.

Original article: https://unlockedconsulting.ai/blog/visual-platform-tax-no-code-automation-cost

Europe cares most about AI sovereignty and depends on it least

Helkyn Coello — Mon, 12 Oct 2026 21:32:05 +0000

A renewal questionnaire came back from a client's procurement team with a question I had not seen on that form before: in which jurisdiction is the model executed, and who operates the hardware it runs on. It was between a question about SOC 2 evidence and a question about password rotation. Answering it properly took the better part of a week. The call went to a vendor, who passed it on to another company that actually ran the model, on hardware in a region someone had picked eighteen months earlier for latency reasons and never written down. Part of the final answer, after all that tracing, was "we believe so."

Nobody on that project was making an argument about European digital policy. The question existed because one of the client's own customers had put it into a contract, and contracts propagate downward. Sovereignty has become a field on a vendor form, sitting between data residency and incident response SLAs, a text box somebody has to fill in before the deal moves.

The gap between wanting AI sovereignty and depending on foreign stacks

The measurement backs this up. Deloitte's State of AI in the Enterprise, published January 2026 across 3,235 director-to-C-suite respondents in 24 countries, found 77% now factor an AI solution's country of origin into vendor selection and 58% build their AI stacks primarily with local vendors. Eighty-three percent rate sovereign AI at least moderately important to strategic planning, 43% very or extremely important, and 66% report at least moderate concern about relying on foreign-owned AI. Worth noting that the survey screened organisations already running AI daily, so these are operators answering, not spectators.

Then the figure that reorders the whole conversation, from the same survey: 11% of companies in the Americas rely on foreign-sourced solutions for the majority of their AI stack, against 32% in EMEA.

Roughly three times the dependence, in the region with the strongest stated preference against it.

That number settles two arguments at once. The European concern is attached to a real exposure, and the exposure is still sitting in production today. What remains is an architecture and supply problem, with a measurable gap between what people say they want and what they actually run.

Now let's break the regional averages into individual countries and we will get a more concrete picture. In France, only 42% of executives say clients are happy with AI-enabled experiences, against 63% in the US, according to the ServiceNow and ThoughtLab Enterprise AI Maturity Index 2026. Customer resistance is a real cost line. It shows up in adoption rates and in how much hand-holding a rollout needs, long before anyone writes it into a governance review. In Germany the same study flags legal liability and intellectual property exposure as a heavier concern than elsewhere, which it is attributed to the regulatory environment and claimant-friendly courts. McKinsey's State of Organizations 2026 puts a number next to that: regulatory, ethical and legal concerns were named as a barrier to adopting externally developed AI by 48% of European respondents and 56% of German ones, against 44% in North America.

The same survey has 72% of leaders reporting that geopolitical uncertainty is already affecting their organisations. That is the concern that the new procurement questions are trying to address: if a supply route or a legal arrangement can change on short notice, buyers want it on paper which country and which company their models actually run in.

For a Mittelstand board in Munich, none of this reads as abstract. A director who already thinks in terms of a duty to organise and a duty to document looks at a system where nobody can say which entity performed the processing, and sees an evidentiary problem they will have to solve under time pressure, with the burden of proof pointing at them.

Answering the question, system by system

Three answers per AI system make the questionnaire survivable:

First: where inference happens: naming the actual executing entity and region rather than the brand on the invoice.
Second: where the data rests: including embeddings, logs and whatever the vendor retains for abuse monitoring, which is the one people miss.
Third: what the replacement path costs: if the first two answers become unacceptable.

The third answer is what turns a documented procurement position into something more than a stated preference, and it is usually the one missing. The first two take an afternoon of chasing documentation. The third requires knowing how much of your system is coupled to a specific provider's behaviour: prompt formats tuned against one model's quirks, function-calling schemas, a retrieval pipeline built around one embedding model's dimensions, evaluation results that only exist for the current vendor. If moving providers means re-tuning every prompt and re-running every evaluation, the replacement path costs six weeks and a quality regression nobody has budgeted. If your own code calls the model through a single wrapper you wrote, with a test suite that checks output quality independently of who the provider is, switching is a configuration change and a week of evaluation.

Decoupling what your business depends on from who runs the hardware

I learned the value of that boundary on a migration that had nothing to do with AI. We had 300 clients on a system built in Microsoft Access, and a mandate to move all of them to a modern web stack. The conventional path was to redesign the interface properly while we were in there. Our main competitor did exactly that, shipped something genuinely better looking, and then spent an enormous amount of time and money retraining their entire client base. We did the opposite and made the new web system look exactly like the old one: same screens, same labels, same flow, with a completely different engine underneath. The migration ran about a year in batches, required zero additional training, and produced a negligible number of migration-related support tickets. The reason it worked is that the surface the users depended on was decoupled from the infrastructure we needed to replace.

Same mechanic, different layer. If the thing your business depends on is "a model that answers these questions at this quality level," and the thing you might need to change is "which company's hardware produces that answer," those two need to be separated by an interface you control before the day you need to use it. Build it afterwards and you are doing architecture work during a procurement crisis, which is the most expensive time to do it.

The normal reflex is to skip all of this by running everything on your own hardware. That is a legitimate option with a cost profile most people underestimate, and I have written separately about what on-premise inference actually demands once you move past the demo. On-premise is a different answer to the same three questions, carrying its own capital expenditure and its own staffing requirement, and the architecture work stays on your plate either way.

The work here is unglamorous and finite. Take the three AI systems closest to customer data or regulated decisions, and write the three answers for each: executing entity and region, every place data comes to rest, and the number of weeks plus the quality risk involved in switching providers. Two of those answers come out of existing documentation in an afternoon. The third is the one that stalls, and the systems where it stalls are the ones to price properly, because that number is what you will be carrying into the next renewal, and somebody's procurement team will eventually ask for it in writing.

A fractional CAIO can own the vendor and roadmap decisions.

Original article: https://unlockedconsulting.ai/blog/europe-ai-sovereignty-dependence-gap

Your AI programme looks better from where you are sitting

Helkyn Coello — Mon, 05 Oct 2026 19:43:14 +0000

Second day of the AI summit in Barcelona, in a room with maybe two hundred people in it, a governance speaker asked how many of us had the authority to stop an AI system we oversee, not merely flag a concern about it. Nobody raised a hand. Then she sharpened it: "human in the loop" has become a checkbox, and the question worth asking is which human, with what authority.

Every organization in that room would answer yes to a survey question about human oversight. The documentation exists. Someone is named in it. And in the room, with no consequences attached to putting your hand up, the real answer was different. Both facts are true at the same time, and only one of them ends up in a report that reaches a board.

And the gap between them leans the same way every time: the further you sit from the work, the better it looks.

Thirty-five points apart, same companies, same week

Valtech surveyed 1,003 professionals who already use AI at work in May 2026, about 60% of them in the EU and most of that in DACH. Among directors, 60.1% described AI as integrated across workflows, tools and teams. Among C-level respondents it was 55.9%, and across the whole workforce sampled, 25.2%. Same companies, same instrument, same week, thirty-five points apart. Valtech's own framing for this is a leadership visibility gap: leaders see pockets of integration that the wider organization does not experience.

Measured from the other end, it looks like impatience. In the ServiceNow and ThoughtLab maturity index for 2026, 57% of US employees said leadership is not keeping up with fast-changing trends. In Japan, six in ten employees said leadership was "not up to the task." I would not put much weight on either number as a fact about leadership. They are useful as evidence that the two groups are describing different objects and calling them by the same name.

Two opinions disagreeing is still two opinions. What makes the direction interesting is that the more senior source is also the more confident one about things it can verify least.

Deloitte's 2026 Global Technology Leadership Study surveyed 662 technology leaders, 87% of them C-suite. 81% said they were confident in their organization's ability to deploy and govern AI capabilities at scale. In the same study, 75% agreed their operating models and processes have to change within twelve to eighteen months to get more value out of AI. Deloitte titled the slide "Confidence is ahead of readiness." When a Big Four firm puts that label on its own respondents, it is reporting what those respondents said about themselves: in one answer, a capability they are confident about; in the next, the machinery underneath it that they know does not work yet.

Nobody is lying in any of this. The mechanism is simpler than dishonesty.

The answer depends on which question you ask

Deloitte's State of AI in the Enterprise, January 2026: 42% rate their organization highly prepared on strategy. Ask the same population about risk and governance and it drops to 30%. McKinsey's State of Organizations 2026, with 10,018 respondents, reports that 86% of leaders feel their organizations are not very prepared to adopt AI in day-to-day operations. More than half of that sample are middle managers rather than executives, which makes the figure harder to dismiss: it is bad news from the people closest to the work.

The closer you sit to a thing, the more accurately you can rate it. A strategy document sits on your desk: you wrote it or approved it, so you can judge it honestly. Day-to-day operations are hundreds of small handoffs happening three floors down, and nobody senior is watching them, because that is what seniority means.

Cloudera's Data Readiness Index, 1,270 IT leaders, has the cleanest version of this. 85% say their data strategy is clearly defined and tied to business objectives. 84% are confident in the accuracy and completeness of their data. Then: 18% say their data is fully governed, and 30% say their sources are fully integrated. The high numbers describe the plan, the low numbers describe the plumbing, and both sets came from the same person in the same sitting.

I had lunch in Barcelona with a former professor of mine, now running regional IT for Western Europe at a multinational in healthcare. He told me two things an hour apart without connecting them. First, his company uses very little AI and moves slowly, although he sees it as a structural change. Second, that inside a large organization, even when he is certain his proposal is right, he sometimes has to take the long way round: ask instead of tell, let someone else own the idea, and accept that being right early is worth nothing on its own. Nobody surveys that. If you asked his organization about AI strategy maturity, you would get an answer about the strategy. The bottleneck: convincing costs more than building, and the person who knows what to do spends his week on the long route.

The one time somebody used a clock

METR ran a randomized controlled trial with 16 experienced open-source developers on 246 real tasks in repositories they had worked in for about five years on average, between February and June 2025. Before starting, the developers forecast a 24% speedup. After finishing the work, having done it themselves, they estimated a 20% speedup. The measured result was a 19% slowdown.

The study has not replicated cleanly since, and I would not build a policy on the sign of that number. What survives is the spread inside those 16 people: perception and clock pointed in opposite directions, and the second estimate came after first-hand experience. Doing the work did not correct the error. If domain experts cannot self-report a speedup on their own keyboards, a director three levels up is not going to produce a reliable integration figure for a company of four thousand.

Survey against logs, and what your AI programme actually reports

Every figure so far comes from asking people. That is the limit of all of them, including the thirty-five-point gap this article opened with.

Survey against survey gives you a disagreement. Survey against logs gives you a measurement.

The comparison is available in almost every organization and almost nobody runs it. Compare seats assigned against seats with activity in the last thirty days. Count the tools that were opened once and never again. Then take the workflows a steering committee listed as automated and count how many tickets still arrive for them through the manual channel. Anthropic's own December 2025 study on how AI is changing work states plainly that it cannot determine where reported time savings were reinvested — and that is the vendor with the best telemetry in the market declining to claim it.

The size of the gap between the reported number and the logged number is itself the finding. It tells you how far your reporting line has drifted from the work, it is specific to your company rather than to a benchmark population, and it is the one measure you can compute about yourself without asking anyone a question.

I argued in an earlier piece that the EU AI Act assumes a logging and oversight floor most companies never built. This is the companion problem: from the top of the organization, you have no way to see whether any of that logging exists, because the instrument reporting on it was assembled by people who have never watched a single ticket move.

Before signing the next round of AI investment, find out which instrument produced the case for the last one. If it was a survey, a maturity self-rating or a slide in a steering committee, the number was measured in the wrong place. Replace it with one observed number. Take a request that actually arrived last week (a customer onboarding, a quote, a support escalation) and follow it end to end with timestamps: when it landed, when each person touched it, when it left. Compare the elapsed clock time to what the slide says it takes. Two weeks of that gives you a baseline nobody in the building can argue with, including you.

Original article: https://unlockedconsulting.ai/blog/your-ai-programme-looks-better-from-where-you-are-sitting

The first four weeks of AI governance

Helkyn Coello — Mon, 05 Oct 2026 19:42:54 +0000

Ask a company for the list of AI systems it runs and you get a procurement export: licensed tools, seat counts, renewal dates. It is an accurate answer to a question nobody asked. The marketing assistant somebody wired into the CRM through a no-code connector is not on it. Neither is the Python script an analyst wrote in March that calls a model API on a schedule and mails the output to four people. Neither is the AI feature in the helpdesk product bought in 2023, which the vendor switched on in a quarterly update and announced in a changelog.

That absence is the whole problem, and it is also where the work starts. Four steps, in a fixed order: find every system that uses AI, classify each by the risk category its use falls into, name the risks that category actually carries, and put a person's name against every mitigation. Most governance material describes the destination and skips the route, which is why leaders who genuinely want to start have nowhere to put their hands. Four weeks is enough for a first pass at all four, and roughly half of that will go to the first one.

Week one of AI governance is detective work

The inventory has to be assembled, because no single function holds it and no single function can hand it to you on request. Four places to look, and you need all four.

IT software-approval records give you the sanctioned tools. Procurement contracts give you the AI clauses buried in third-party software that nobody flagged as AI when it was bought, because in 2022 it wasn't. Privacy and legal give you the Data Protection Impact Assessments (DPIAs): anything processing personal data left a paper trail, and that trail is often the only written description of what a system actually does. Then the code repositories, searched for ML library imports and calls to model API endpoints. That last search is the one that reveals the systems with no owner, no budget line, and no ticket.

What comes out sorts into three categories: built internally, bought from a vendor, or embedded in something you already own — Copilot, Slack, Google Workspace, the CRM, the ticketing system. The third category is the one that surprises people, and it is usually the largest by headcount exposure, because every employee has it and nobody chose it. A structured workflow audit is one way to force all four searches to happen in the same fortnight rather than across four separate quarters.

Two numbers explain why one function cannot supply the list. Gartner's Hype Cycle for Agentic AI 2026 puts business-unit funding at nearly 30% of AI initiatives, which means roughly a third of the estate was never procured through the channel you are searching. And in Gartner's Predicts 2026 note, 56% of IT leaders strongly agree that IT cannot drive generative AI adoption on its own. One caveat I will state plainly, since this is an article about governing what you can actually see: the reprint of that note has its evidence section collapsed. It lists no population, no sample size and no fieldwork dates. Treat it as a 2025 Gartner survey and weight it accordingly.

The ServiceNow and ThoughtLab maturity index from 2026 puts the shape of the gap in one pair: 59% of organisations report using agentic AI, while 9% report meaningful progress on autonomous multistep workflows. Whatever the definitional differences behind those two figures, no company is going to reconcile them from a procurement report.

Two names per row

Every row in the AI inventory needs two names (and this is the part almost nobody writes down): Who owns the system, and who owns the data it touches. They are rarely the same person. The system owner decides what it does. The data owner decides whether it is allowed to do it with that data — consent basis, retention, residency, who else can see the outputs. Skip the second name and you have an asset register that tells you nothing about your actual exposure. It is the same gap that shows up when nobody can say whose credentials an agent is running under.

The strongest evidence I have for this comes from that same Gartner note—with the exact same caveat. Only 14% of application leaders are confident that their current data is secure and well-governed enough to provide substantial value to both AI and human interactions. Meanwhile, 77% plan to prioritize spending on AI-ready data. That means five times as many companies are paying for their future state as those who can actually guarantee the safety of the systems they run today. Gartner projects that this confident group will reach 35% among large organizations by 2030, but keep in mind that's just a forecast, so treat it as one.

ServiceNow's index makes the operational version of the same point: assessing the data requirements for multi-step workflows forces you to update ownership and privacy policies, and those decisions determine which autonomous capabilities can be deployed at all. So treat data governance as a column in the inventory from the first week*.*

Week two: tier, and expect most of it to come back low

The EU AI Act attaches obligations to categories of use, not to technical sophistication. A retrieval system over your own documentation can be architecturally involved and carry no obligations. A simple scoring rule applied to job applicants carries a lot. Most internal tooling lands in minimal risk, and saying so out loud is the point of this step.

The failure mode here is over-classification: a company applying high-risk controls to a meeting summariser, burning six weeks and all its political capital, and arriving at the system that genuinely needed a conformity assessment with nothing left in the tank. I sat in a room in Grünwald last week where a lawyer, presenting to a dozen Mittelstand directors, called the alternative Compliance mit Augenmaß: a mid-sized company does not have hundreds of high-risk areas, it has six or eight, and concentrating there is what makes the obligation affordable in time and money. His subject was corporate liability generally, not AI. But the structure transfers exactly, though it only works if the logging and oversight floor the Act assumes is already there underneath.

Week three: name the risks the tier carries, not risks in general

Generic risk lists are why governance programmes stall. "Bias, hallucination, privacy" applied uniformly to forty systems produces forty identical registers and zero decisions.

Take each system apart four ways instead: what it is used for and in what environment, what data feeds it, what judgement it automates, and what happens in the real world as a consequence.

The fourth question is where the exposure actually lives. A model that predicts churn and a model that cancels accounts based on the prediction are the same accuracy and entirely different liabilities. A model that drafts a rejection letter and a model that sends it are separated by one integration that somebody enabled on a Tuesday. When people say AI risk is hard to assess, they are usually stopping at question three.

Week four: mitigations with a name on each

The inventory and the risk register are different artefacts, and conflating them is a common structural mistake. The inventory says what exists: system, owner, data owner, version, model type, category. The register says, for each one, what threatens it, how likely, how bad, what control was applied, and what residual risk is being accepted.

That last field needs a person's name attached to it.

Residual risk with no name on it has not been accepted by anyone. It has been noticed and left alone, and that reads very differently in an investigation.

Ship it at eighty percent

Governance functions stall waiting for the inventory to be complete. But that is a trap: it will not be complete, because someone is standing up a new agent this week. An eighty-percent inventory in week four beats a perfect one in month six, for the plain reason that the systems keep running in the meantime and the incomplete register still catches the two or three that matter most.

Run it centralised at first. One team making the tier calls keeps the judgements consistent while the pattern is still forming, and you cannot delegate a classification standard you have not exercised yourself. Federate to the business units once the calls have stopped surprising you. Gartner's read is that high-AI-maturity organisations centralised strategy, governance, application development and data management practices first, which matches what happens when you try the reverse: eight business units producing eight incompatible interpretations of "limited risk."

The scale of the unfinished work is not in dispute across sources. Deloitte's January 2026 survey of 3,235 leaders found 23% using agentic AI at least moderately today, 74% expecting to within two years, and 21% reporting a mature governance model for autonomous agents. ServiceNow puts organisations with implemented AI testing, auditing and risk-assessment processes at 20%. Gartner projects that successful agentic AI governance could cut regulatory compliance costs by around 70% by 2028, a projection with a number attached to it, and an argument for starting early rather than a measured return. The recurring cost is the logging, attribution and guardrails, not the build.

Four weeks of this does not make a company compliant, and selling it internally as compliance is how these programmes lose their budget in year two. What it produces is narrower: a list of what exists with two names on every row, a tier for each item, the specific risks that tier carries, and a named owner against every mitigation and every accepted residual.

So when a supervisory authority, a customer's procurement team, or your own board asks which of your systems they should be worried about and who is accountable for it, someone can answer in the meeting. Most companies start building that answer the week the letter arrives, and by then the answer takes three months and reads like it.

Original article: https://unlockedconsulting.ai/blog/first-four-weeks-of-ai-governance

The hours AI saved you have nowhere to go

Helkyn Coello — Mon, 14 Sep 2026 20:33:06 +0000

A manager at the company started building with AI last year. He described what he wanted, iterated with the model, clicked through the result, and fed back the cases he had forgotten until the behaviour was right. An afternoon of that produced something that ran. He handed the branch to engineering and considered the work finished.

The branch sat for months, even though nobody was obstructing it. Review capacity hadn't changed, the architect's week hadn't changed, and the definition of "ready to publish" was the same as it had always been, with the same queue sitting in front of it.

An afternoon of work entered a pipeline, and the pipeline did what it was designed to do: absorb it at its own pace. The manager was measuring whether the thing worked. Engineering was measuring whether it could be operated, secured, tested and supported. Both were right. And the afternoon he saved bought the company nothing, because the step that got faster was never the step that set the delivery date.

That is the problem this article is about: AI accelerates the step in front of the person using it, and the steps behind that person stay exactly as slow as they were. The aggregate numbers say it is happening everywhere.

spend doubled while integrated workflows halved

Two numbers from the same survey. ServiceNow and ThoughtLab asked 4,500 executives across nineteen countries for their 2026 maturity index, and the third edition lets them compare against last year.

AI spend rose 110% in a single year. The share of organizations reporting streamlined, integrated workflows across business functions fell from 30% to 16%.

Spend doubled. Integration halved. The report's own explanation is fragmented platforms carrying a new wave of agent sprawl on top of them, which is a polite way of saying the tools multiplied faster than the plumbing.

It is the only integration figure in this year's research that has a previous year to compare against, and it runs backwards.

where the saved hours go

Follow it one step at a time, because each step is defensible on its own and the outcome is still near zero.

You buy licences and hand out tools. Reasonable enough; the per-seat cost is trivial against a senior salary. An individual who used to need three days to produce a piece of work now needs three hours. Also real; I have watched it happen at my own desk. But the workflow still allocates three days, and the ticket still waits for the weekly planning call. Two approvals sit in front of deployment, both sized for the latency of a human who had to read a document and think about it. So the individual, done by Tuesday, picks up the next ticket, and then the one after that. More work arrives at those two approvals every week than before. The approvals process it at the same speed they always did. Everything they cannot absorb sits and waits and becomes a backlog.

The company paid for speed and got a longer queue.

BCG's 2026 survey of 11,749 employees puts a number on the gap: 61% have limited or no guidance on what to do with the time AI saves them, and 45% are not reinvesting it into anything strategic. The hours are real. They just have nowhere to land.

In the same ServiceNow data, 59% are past piloting agentic AI and only 9% report meaningful progress on autonomous multistep workflows. Zero percent have built anything resembling a cross-functional, self-improving agentic operating layer. Only 16% have replaced fragmented legacy systems with an integrated platform, and 41% still name siloed data as a major obstacle. Cloudera's survey of 1,270 IT leaders (vendor-run, so weigh it accordingly) has 79% saying they cannot access all the data their initiatives need and only 30% with fully integrated sources. Adoption figures like these are worth reading carefully, because three studies measuring agentic adoption this year returned 17%, 23% and 59% by counting three different things.

Everything above comes from the people who bought the tools. Valtech asked the people who use them: a thousand professionals already working with AI daily. Asked where the most value would come from, 4.3% said more pilots. Integration of tools and data ranked first.

what 95% actually bought

Five percent of organizations in the ServiceNow data are redesigning work. The rest are pointing agents at workflows that already existed, in the shape they already had.

Deloitte's January 2026 survey of 3,235 leaders lands on the same spot from a different angle: 84% have not redesigned jobs or the nature of work around what AI can now do. My favourite pair in the whole report: 53% considered fewer management layers and smaller, more autonomous teams, and 16% actually moved to them. The idea got as far as a slide.

McKinsey have two surveys on this. Their State of Organizations report, with 10,018 executives: 88% deploying, 81% reporting no meaningful bottom-line gain. In their mid-2026 follow-up, which asks the same questions as last year, use rose to 89%. The share saying AI moved their profit at all stayed at 37%. The share getting significant profit from it stayed at 6%. What separates that 6% is not budget. Roughly three-quarters of them redesigned their workflows, against one-quarter of everyone else. McKinsey's own summary: conviction in AI is growing faster than the returns anyone can attribute to it.

All of that is self-report, and self-report on productivity is generous. Which is why the software engineering data matters more than it looks.

Faros AI, which sells engineering analytics and has a stake in this conclusion, instrumented 22,000 developers and compared each team's lowest-adoption quarters against its highest. Developers finished 34% more tasks. Each piece of work then waited 441% longer for someone to check it before it could go out. And the speed at which finished work actually reached users did not change.

That is the individual getting faster and the organization standing still, measured with telemetry rather than a questionnaire. Writing the code got cheap. Review, QA and deployment stayed exactly as expensive as they were, and everything that got written piled up in front of them. Faros is blunt that tightening review is the wrong response, because review was never the thing that changed.

DORA, a non-commercial research programme surveying close to five thousand developers, states the mechanism in one line: AI amplifies what is already there. Their 2024 edition put a number on the downside: for every 25-point increase in AI adoption, teams shipped 1.5% less and what they shipped broke 7.2% more often. More AI in an unchanged process did not mean more delivered. It meant slightly less, arriving slightly worse.

three moves that change the process

Decide where the saved hours go before you buy the tool. If an analyst saves six hours a week, somebody has to say what those six hours are now for. More time with clients. The project that has been waiting since March. One fewer hire next year. If nobody names it, the hours get absorbed back into the same job and nothing changes. "We got more productive" is not an answer to where the time went.

Count the steps, don't just speed them up. A three-hour task inside a five-approval chain is still a two-week task. The cheapest redesign move is deleting a handoff, and it is almost always resisted more than a six-figure platform purchase, because a handoff belongs to somebody.

Make each item cheaper to review. If the queue in front of review is the constraint, the fix is not more reviewers or faster ones. It is that each thing arriving needs less looking at. Put a human on the plan while the model is still writing it, so the implementation starts from something somebody already shaped. Have the repository check every incoming branch against its own architecture rules and refuse what does not pass. By the time a branch reaches a person, it has cleared both. Same reviewer, same week, far less to read per item. That is what got the manager's afternoon of work into production.

where the hours AI saved should land

The pattern in all of this is that AI speeds up the step a person does, and everything that step feeds into stays exactly as slow as it was. The spending is real, the individual gains are real, and the process is still shaped for the pace it had before.

So take one workflow you have already put AI into and answer two questions. Where are the saved hours supposed to go, and who decided that? If nobody can answer this, you bought speed for one step and a longer wait for everything after it.

That question is most of what an AI workflow audit does in its first week, and you can run it yourself on a whiteboard before anyone signs anything.

Original article: https://unlockedconsulting.ai/blog/hours-ai-saved-have-nowhere-to-go

Everyone Measures AI Usage. 70% Can't Measure What It Returned.

Helkyn Coello — Fri, 11 Sep 2026 23:15:07 +0000

Anthropic surveyed 132 of its own engineers about Claude Code. Merged pull requests per day rose 67 percent. Daily use of the tool climbed from 28 to 59 percent. Self-reported productivity gains ran between 20 and 50 percent. But then someone checked the organization's delivery dashboard and saw that the delivery metrics had not moved.

That gap is the whole subject of this piece. A tool can be used constantly, rated highly by the people using it, and leave no trace on the numbers a business actually runs on. The measurement problem underneath it is bigger than one company's coding assistant. McKinsey found that 30 percent of leaders could say where the time AI freed up actually went. The other 70 percent could not. Seven out of ten organizations have people spending less time on tasks and no idea whether that turned into anything. Ask the question from the other side and the answer is just as thin: in Gartner's 2025 survey, 22 percent of leaders said their AI tools had returned significant value, a share that lands where McKinsey, Deloitte and ServiceNow each arrived measuring it their own way.

Usage and tokens are costs, not returns
Two numbers get reported as if they answered the ROI question, and neither does.

Adoption tells you whether anyone is using the thing. It is a leading indicator and a useful one, but a tool with high adoption and no measured outcome has produced activity, not value. Token spend tells you what the tool costs to run. It belongs in the calculation, on the cost side, and watching it closely tells you nothing about whether the work it produced was worth having.

Both are easy to pull from a dashboard, which is exactly why they get reported. The number that matters sits one step further out and takes real work to produce: whether the company made or saved a defensible dollar. Everything below is how you get to that number without lying to yourself on the way.

So when a business case lands on a desk claiming a figure in saved dollars, the honest question is not whether AI helped. It is whether the number is real. In most cases I have looked at, it is inflated, and usually in two separate places at once.

Inflation one: soft hours counted as hard dollars
Here is the calculation almost every AI business case runs. The tool saved each person two hours a week. Multiply the hours by the hourly cost of those people, add it up across the team, and report the total as money saved.

The hours are usually real. The dollars usually are not, because the budget did not change. Nobody was let go, no contractor was dropped, no line item fell. What happened is that a group of salaried people have slightly lighter weeks, and the company pays them exactly what it paid before.

Freed hours become real money in three situations: the time is redeployed onto work that generates value, or it lets you avoid a hire you were about to make, or it lets the same headcount produce more of something you sell. If none of those is true, the saving is soft, and soft savings do not survive contact with a CFO who can see the budget did not drop.

This is where the 70 percent from the opening returns. If seven in ten leaders cannot say where the freed time went, then most of the "hours saved times hourly rate" figures in circulation are soft hours nobody traced to an outcome, dressed up as hard dollars. The fix is not complicated. Label every dollar of claimed value as hard or soft, and report the two separately. The number gets smaller, and much harder to dispute.

Inflation two: calendar time counted as labor time
The second inflation is subtler, and I see it most in engineering cases. A feature "took three weeks" before and "takes one week" now, so the case dollarizes two weeks of saved time at an engineer's rate.

The problem is that three weeks was never three weeks of work. Some of it was a ticket sitting in a queue, some was waiting on a review, some was a dependency that had not shipped. Cycle time, the calendar span from request to delivery, is not the same as labor time, the hours a person actually spent. Converting the calendar span to dollars at an hourly rate invents labor that no one performed.

Speed is still worth reporting, but as a rate: this class of work now moves through 40 percent faster. It becomes money only when moving faster captures something real, most often revenue that arrives earlier because the thing shipped sooner. When it does not, a faster cycle is still a genuine improvement worth reporting as speed, but it is not a number you convert into dollars.

There is a related trap in trusting self-reported speed at all. The one controlled study I know of that timed the same developers with and without AI, METR, found a measured slowdown of 19 percent against a self-reported gain of 20 percent. The developers were sure they were faster. The clock disagreed. Whatever you build your ROI on, it should not be a survey asking people how much time they think they saved.

What actually produces dollars
Strip out the inflations and there are only two mechanisms by which an AI tool produces money, and each converts to dollars through a different bridge.

The first is acceleration: someone does a task they already did, in less time. The bridge is hours saved multiplied by the hourly cost of that person. If a task dropped from four hours to two and a half and happens eighty times a month, that is 120 hours a month, and at their hourly cost you have a real figure. Then you subtract rework, because generated output a person has to redo never saved the time it appeared to.

The second is avoided work: a task stops happening at all. A support ticket the knowledge base resolves is a ticket a human never touches. The bridge here is not an hourly rate, it is the full cost of one whole interaction: the total monthly cost of the function divided by the number of interactions it handles. If support costs 20,000 a month and handles 2,500 tickets, each avoided ticket is worth 8 dollars, and that 8 already carries the tooling and overhead, not just one agent's wage. Using the acceleration bridge here, an hourly rate, would undercount it.

Both of those are illustrative arithmetic, not figures from any client. The point is the shape: pick the mechanism, pick the matching bridge, and do not mix them.

The denominator nobody writes down
A return needs a cost to divide by, and this is where the tokens finally belong. The total cost of an AI tool per month is the build cost amortized over the months it will run, plus token spend, plus infrastructure, plus maintenance, plus any human review of its output. A build that took 150 hours and will run two years is not a 9,000-dollar hit this month; it is a few hundred a month spread across its life. Token cost is simpler than it looks: measure the average number of tokens one output consumes, multiply by the price per token, and you have a stable cost per output to hold against the value that output produces.

A project can carry many metrics, but it has a single ROI. Each metric adds its own slice of value to the same numerator, and every slice divides by that same total cost. The one discipline this requires is avoiding double-counting: if a token cost already sits inside a per-unit figure, it does not also go in the denominator, and if two metrics describe the same saved dollar from two angles, you keep one of them.

The math is deterministic, which is why it gets skipped
I argued in an earlier piece that AI belongs at the edges of financial analysis and never in the arithmetic itself. Measuring your own AI's return is the same shape seen from the other side. The calculation is deterministic: a subtraction, a multiplication, an hourly cost, a total. No model is required, and none should be trusted with it.

The reason it gets skipped is not difficulty. It is that the honest number is almost always smaller than the inflated one, and smaller numbers are harder to carry into a budget meeting. A defensible small number survives scrutiny and an impressive large one does not, and the second time a leader is caught reporting soft hours as hard dollars, the whole program's credibility pays for it.

But isn't agentic AI supposed to be exempt from ROI?
There is a serious version of the opposite argument, and Gartner makes it: early agentic AI is experimental, and organizations that demand a proven business case before they will touch it risk being outpaced by the ones that treat it as something to iterate on. That is right, as far as it goes. You do not gate a two-week experiment behind a formal ROI model, and pretending you can forecast the return on something genuinely new is a guess dressed as a forecast.

But two different claims get folded together under that banner. "Do not require a business case before you experiment" is defensible. "Do not measure what it returned" is not, and the first is routinely used to justify the second. An experiment you never measure is not an experiment, it is a purchase with no follow-up. The reason for not gating early work behind ROI is to buy yourself room to find the value, which only means something if you then check whether you found it. Exempting agentic AI from a business case at the start is reasonable. Exempting it from measurement forever is how the share of companies that see real value stays stuck at one in five.

Where to start
Capture the baseline before you deploy anything, because once the old way is gone you cannot reconstruct how long it used to take. Measure one real unit of work end to end, from request to delivered, including the waiting, and see whether that number moved. Stop reporting seats deployed, tasks completed, prompts submitted and self-reported speed, and start reporting the one figure that ties to a customer or a budget. Label every claimed dollar hard or soft, and report the net dollars alongside the percentage, because the percentage moves with whatever you put in the denominator while the net figure does not.
_
Doing that across every process you run, and writing down what each would need in order to actually change a budget line, is a workflow audit. It takes longer than pulling a usage chart. It is also the only version of the exercise that produces a number you can defend.

Questions this raises
How do I put a dollar value on time saved?
Multiply the hours saved by the hourly cost of the person who saved them. Count it as a real saving only if that time is redeployed, avoids a hire, or produces more sellable output. Otherwise it is a soft number and should be labelled as one.

Should token spend count as part of ROI?
Yes, as a cost, in the denominator, and never as a benefit. Track cost per unit of useful output if you want a figure to hold against value, and make sure you are not counting the same tokens in two places.

Can one project have several ROIs?
No. A project has one ROI. Several metrics can each add a slice of value to the same numerator, divided by the same total cost. If you end up with two ROIs for one tool, you have either double-counted or mixed two projects together.

https://unlockedconsulting.ai/blog/everyone-measures-ai-usage-70-can-t-measure-what-it-returned

Financial Analysis Needs AI at the Edges. Never in the Math.

Helkyn Coello — Sat, 05 Sep 2026 11:38:15 +0000

A colleague recently asked me how he would run financial analysis with AI over the books of an accounting system: the ledgers, the trial balances, the P&Ls. He had three options on the table and no strong preference among them. Generate the PDF reports the system already produces and hand those to a model as input. Point the AI directly at the relational tables and let it query. Or skip the AI and compute the analysis with deterministic code.

The question sounds like a tooling choice. It is an architecture decision, and it happens to be one I had already answered a couple of months earlier, in an independent prototype: a system that computes the 33 financial metrics I consider core, reads accounting PDFs from any source system, and confines AI to the two edges of the pipeline. It never touches the math.

Every metric is a formula
Financial analysis has a property that most AI conversations skip past: it is probably the most deterministic workload in the building. Every metric in my catalog is an arithmetic expression over named sums. The current ratio is current assets divided by current liabilities. Working capital is a subtraction. Margins are divisions. There is no judgment inside any of these, no interpretation, nothing that benefits from a model's flexibility. A formula that is allowed to be flexible has stopped being a formula.

That settles the third of my colleague's options first: the computation itself is code, and it should not be anything else. I made the general version of this argument in June: if you need the same answer twice, dont't use AI. A trial balance is the purest case of that rule I know. Ask the system for the current ratio twice, and anything other than the identical number to the cent is not intelligence showing initiative. It is a defect.

The other two options fail for more interesting reasons. Handing exported PDF reports to a model as the analysis input means taking numbers that were structured data a second ago, flattening them into a document, and asking a language model to re-derive them with no guarantee it reads every line the same way tomorrow. Pointing the model at the relational tables sounds more rigorous and carries the same flaw one layer down: the queries it writes and the aggregations it chooses can drift between runs, and in accounting, drift between runs has a technical name. It is called an error.

Two numbers that prove the rule
All 33 metrics compute the same way: by formula, with no exceptions and no heuristics in between; 14 of them carry thresholds that can raise findings. Most of those thresholds flag things worth investigating, not emergencies: a liquidity ratio drifting below its floor, receivables aging out too far. Two conditions sit in a different class, the only ones in the system marked as critical. And calling them the most important metrics would miss what they are. They are the alarms.

The first is trust liability coverage: cash held in trust minus the deposit liabilities it is supposed to cover. If that number goes negative, client money is not where it must be. That is not an analysis finding; that is a legal problem. The second is the balance sheet check: assets minus liabilities minus equity, adjusted for the year's income and expenses. Accounting says that expression equals zero. If it does not, the emergency is of a different kind: the source data contradicts itself, which means the other 32 metrics were computed from numbers that do not agree with each other.

Only two conditions in the system count as emergencies: client money not covering deposits, and books that do not balance. They are exactly the numbers no model should ever estimate.

The entrance: reading, then auditing the reading
So where does AI belong here? At the entrance, doing the one job deterministic code is genuinely bad at. Every accounting platform prints its own PDFs: different layouts, different column orders, different names for the same concepts. The classic answer is a rigid import template, which means every company adapts its exports to your tool before the tool does anything for them. The prototype I built inverts that. AI reads whatever the source system produced, extracts the figures, and writes them into the database. Interpreting messy, heterogeneous documents is precisely where non-determinism is a feature, because there is no formula for "understand this layout you have never seen."

The payoff is practical: 25 of the 33 metrics compute from the trial balance alone. One file, whatever it looks like, and most of the analysis lights up.

But the design does not even trust the reading. Before anything is ingested, a second, cheaper model call audits the first one: it compares the parsed table against a sample of the raw source and flags the classic misparses, a column shifted one position over, a subtotal ingested as if it were an account, a period detected wrong.

A deterministic rule decides when that second reader runs. For PDFs it always runs, because PDFs are where parsing goes wrong most often. For spreadsheets it runs only when the code itself finds a reason to doubt the parse: too many source rows dropped, or a value column that came back empty. Each of those calls costs money, so the same rule that guards quality also controls the cost: the system pays for a second opinion only when the numbers justify one.

Then the deterministic gates run. Before a single formula executes, the trial balance identity has to hold, receivables and payables have to tie to their aging reports, and periods have to be continuous, all within a tolerance of one dollar. A misread cell does not flow quietly into a ratio; it fails loudly, before the analysis exists. If the AI service is down, nothing stops: the file is still ingested, the metrics still run, and the system records that this particular check did not happen. The pipeline works without the models; what they add is checking, not dependency.

The exit: prose over a case file the code built
The other place AI earns its seat is at the exit, and this is where I expected to give the model the most freedom and ended up giving it the least. A current ratio below its threshold is just a finding; it is only useful once someone explains it. That explanation is not the model's to invent. Before the model writes a word, code gathers the evidence: up to six months of history for the metric that breached, the other metrics that breached in the same month, and, when a general ledger is loaded, the transactions behind the affected accounts. The model writes prose over that evidence and may cite only figures inside it; a validator then checks that every number in the sentences traces back to a value the code computed. The AI writes the sentences. Every figure inside them was calculated somewhere else.

The same discipline decides the month's verdict. Whether a period is clean, has issues, has critical issues, or failed validation is assigned by rules, never by the model; the model narrates the state it is handed. The design notes for the system state it as a flat rule: AI reads, suggests, and narrates; code verifies and computes.

My favorite consequence is what happens when nothing is wrong. Zero findings do not produce silence. They produce a verdict, computed by code, that the books came back clean, every accounting check passed, all 33 metrics inside their configured ranges, with the model writing the short summary a person reads. In the months when nothing is wrong, that verdict is the deliverable.

What the prototype refuses to do
When data is missing, the system says so instead of estimating. A metric whose source report has not been uploaded comes back empty, with the reason attached; growth rates simply do not exist until there are two months to compare. A model sitting in the middle of the pipeline would have filled those gaps fluently, and that is exactly the problem: here, the absence of an answer is information, and the system preserves it.

Two planned capabilities, cash forecasting and fraud detection, stay switched off until they can be built with the same discipline. And I should be equally plain about status: this is a prototype. It has processed the documents I have fed it; it has not run anyone's production books. What I am confident in is the pattern, and the pattern is the point.

In June I wrote that the shape is AI in the data, code in the process, and called it the architecture that actually scales. A summer of building later, I would sharpen the claim. The mature shape is not two layers but a guarded pipeline: AI reads at the entrance and a second model audits the reading, deterministic code computes everything in the middle, and AI explains at the exit from evidence the code assembled. AI at the edges, code at the core. It still scales. More to the point, it is an architecture you can answer for.

https://unlockedconsulting.ai

You cannot fire your AI agents

Helkyn Coello — Sat, 29 Aug 2026 18:24:39 +0000

A branch came in for review with about sixty commits on it, every one authored by someone on the team. He hadn't written them. Claude Desktop had, running on his laptop, signing commits with the git identity we configured during setup. As far as the repository was concerned, the work was his. As far as blame, audit and every code-ownership convention we had, the work was his. Nobody could separate the four or five decisions he had actually looked at and accepted from the fifty-odd changes the model produced while he clicked through the result to see whether it worked.

We moved the whole thing off his machine: the model runs server-side now, the working copy is provisioned per ticket in an isolated environment, and what comes back is a URL. That solved the port conflicts and the dependency drift, which was why we did it. It did not solve the attribution problem. It relocated it. Now a service account commits, and the service account is one identity shared by every run, for every person, on every ticket.

That is the shape of the thing arriving at enterprises considerably faster than most access-management programmes are ready for.

a new hire, a printer, and an agent

Take a new hire in their first week. They have a unique identifier that will never belong to anyone else, a set of permissions somebody requested by name, a login trail, and an offboarding procedure that takes an afternoon. Four things: who they are, what they can reach, what they did, and how you get rid of them. Hiring, permissions, audit, firing.

Now the printer on the third floor. It has an asset tag, it sits on a network segment that lets it reach the print server and nothing else, it logs every job, and you can unplug it. Same four things. Nobody is impressed by the printer, but the printer is fully accounted for.

Now the agent your team stood up last month to triage tickets, read the CRM and post summaries into Slack. Who it is: it uses a key minted from a human account, probably belonging to whoever built the thing. Every log in every downstream system records that person, not the agent. What it can reach: whatever they could, which for an engineer with admin rights is usually everything. What it did: indistinguishable from what the human did. How you fire it: you rotate the key, and you find out what else breaks.

Nobody decided the agent could write to the CRM. It inherited the permission from the account it borrowed.

where the answers get uncomfortable

Identity means a credential belonging to this agent and only this agent, distinguishable in every downstream system's logs from the human who deployed it and from the other eleven agents your company is running. A name in a config file does not qualify. Whether you can have this depends entirely on the platform. API-native services handle it well: Anthropic, OpenAI, Stripe and others will issue a key with its own permissions and its own name, no seat required. The business systems where most agents actually work are the problem. Salesforce, HubSpot and most CRMs model access around seats occupied by people, so a separate identity per agent means paying for a seat or sharing a credential, and sharing is what teams choose.

Scope is what it can touch, expressed as operations rather than systems. "Read from the ticket store, write to one Slack channel, call the pricing endpoint" is a scope. "Has an API key for Salesforce" tells you nothing about what the thing can actually do. The trade-off is real and worth stating plainly: narrow scopes break more often, and every break lands on the person who understands the permission model, which is never the person operating the agent. Broad scopes are cheaper to run and quieter, right up until the day they aren't.

Attribution needs three facts on every action rather than one: which agent did it, which run it belonged to, and which human or system event authorised that run. What we ended up doing was giving every run its own number, and making sure that number appeared everywhere the run left a mark: in the commit, in each API call, in the ticket it came from. So anything the agent touched could be traced back to one specific run and to whoever started it. It is unglamorous plumbing, and it is the difference between a twenty-minute investigation and a week of reading diffs, which is the recurring governance cost that shows up after the agent is already built.

Revocation is where the situation exposes itself, because revocation and identity turn out to be the same problem seen from opposite ends. Try it. Pick an agent running in production and cut off its access without touching anything else. If the credential is shared you cannot, so you rotate the key instead, and the rotation takes down the nightly export, two internal dashboards and something somebody wired up in March, because they all use the same key and nobody wrote down that they did. The rotation gets scheduled, then deferred, and the agent you wanted to stop keeps running for another eleven days.

That delay is the test. If cutting off one agent requires a change-management window, what you have is not a revocation path. It is an intention to revoke, subject to scheduling.

the parts exist, the whole does not

Every piece of this problem has been solved somewhere, and none of the solutions cover the ground you need. Cloud providers can issue an identity to a running process and rotate it automatically, which works cleanly as long as everything stays inside one cloud. Credentials that expire in hours rather than years are normal practice in infrastructure and almost unknown in the business tools where agents actually operate. And there is no single place to go and switch one agent off across Salesforce, your warehouse and a third-party API, because each of them keeps its own list. You will build this out of parts, and it will not be elegant.

What you can do now costs engineering time once rather than exposure forever. Mint a distinct credential per agent per purpose, even when the platform makes you buy a seat to do it, because the licence fee is cheaper than the incident. Keep a register of which agent holds which credential, with an owner's name on each row, because the thing that makes revocation slow is not the API call, it is not knowing what breaks. Give every credential an expiry date when you create it, so that the ones nobody remembers stop working on their own rather than living forever. And stop letting agents inherit a human's permission set as a starting point, because a permission set assembled over four years of a career is the worst possible baseline for a process that only needs to read tickets. This is the same logging, testing and oversight floor most companies never built, arriving now with agents attached.

verifiable agent identity is coming, and retrofitting is expensive

In June 2026 Estonia approved a framework giving AI agents a verifiable digital identity, with permissions that are scoped, auditable and revocable, tied to the eIDAS 2.0 ecosystem. It is an approved framework rather than a law in force, and that distinction matters. What requires no guessing is the direction: the first government to treat agent identity as identity infrastructure rather than a vendor feature has now done so.

Plan around it. At some point an agent acting on a company's behalf will be expected to have a verifiable identity, and someone will have to be able to say which one acted and on whose authority. Retrofitting attribution onto a fleet of agents that all share one service account is going to be a genuinely expensive project.

One thing worth doing, and it takes an hour. Take an agent already running in production and try to switch off its access alone, in a staging environment that mirrors how your credentials are set up. Time it, and write down what else stopped working.

That is your answer to how you would fire that agent. Ask the same question in a meeting and you will get "we'd rotate the key," which is a description of an action rather than an outcome. The hour tells you what the action actually costs. That is the number worth knowing before you need it.

faq
Why can't we just give each AI agent its own account in every system?

Sometimes you can, and where you can you should. API-native platforms will issue a key with its own permissions and no seat attached. The obstacle appears in business systems priced per user, which have no concept of a non-human account, so a separate identity for each agent means buying a seat. Buy the seat. It costs less than the incident.

What does agent attribution actually require in the logs?

Three facts on every action: which agent performed it, which run it belonged to, and which human or system event started that run. The minimum that makes an investigation possible is giving each run its own number and carrying it through every call the agent makes.

Isn't narrow scoping more trouble than it's worth?

Narrow scopes break more often, and every break lands on the person who understands the permission model rather than the person operating the agent. Broad scopes are cheaper and quieter until the day they aren't. The trade-off is real; it is a decision to make deliberately rather than by default.

How do we know if our revocation path works?

Test it. Pick an agent running in production, switch off its access alone in a staging environment that mirrors your credential setup, and count what else stopped working. If doing it for real would need a change-management window, you don't have a revocation path yet.

https://unlockedconsulting.ai/

Adoption Is a Latency Problem: The Zero-Backlog Policy

Helkyn Coello — Mon, 17 Aug 2026 07:15:32 +0000

Notes from an AI video project, and the release cadence that turned one team into the project's owners

The most common artifact of an enterprise AI rollout is a dead feedback channel. It gets created in week one with genuine enthusiasm, collects a burst of suggestions in the first month, and goes quiet by the third. Not because people ran out of opinions, but because nothing they said ever came back as a release.

The industry numbers say access is no longer the problem. Deloitte's latest survey puts sanctioned AI access at close to 60% of workers, and fewer than 60% of those use it in their daily work. The gap between having a tool and using it is where adoption lives, and I want to describe the project that taught me what actually closes that gap. It was not the training plan. It was the release cycle.

The project

The tool generated training videos for our training team. We started with a deliberately narrow scope: simple functions, one team, one job. The build ran in stages of two to three weeks at most, and every stage had to end in something the team could actually use. Not a demo. Not a prototype behind glass. A working increment in their hands.

The day the first increment shipped, we opened a dedicated channel with everyone on the team who used it. Feedback arrived daily: bugs, friction points, and, more valuable than either, feature ideas from the people doing the work. The AI team had one standing instruction: respond the same day. Every day we held a short meeting to go through what had come in, with the lead of the using team in the room, and whatever we decided got built the same day or the next. Releases went out daily, and they carried not only fixes but the users' own ideas.

Zero backlog as a promise

We called it a zero-backlog policy: nothing you tell us gets parked. A suggestion either ships within a day or two, or you hear the same day why it won't.

Here is why I now treat this as the mechanism of adoption rather than a nice-to-have. Buy-in decays at the speed of your release cycle. A suggestion implemented within a day is visible proof that the user shapes the tool; the same suggestion sitting in a quarterly backlog is proof of the opposite. Both signals are received loud and clear. Only one of them produces owners.

And ownership is what we got. The team stopped talking about "the AI tool" and started talking about the features they had asked for. They corrected each other's usage in the channel. When something broke, they reported it the way you report a problem in something that is yours. Without anyone assigning the role, they had become the project's evangelists inside the company.

Let the users do the presenting

We showed the project at every company-wide meeting, and made one deliberate choice: the people presenting were the users, not us. A training-team colleague explaining what the tool changed in their own week produces a categorically different effect than an AI team presenting slides about capabilities. One is a testimonial; the other is a pitch, and everyone in the room can tell them apart.

Every demo carried the same metric, tracked from day one: finished videos per week. Not prompts sent, not logins, not a satisfaction score. The number the business already cared about before the project existed. When the metric moved, nobody had to be convinced that it mattered.

The taper

When the tool stabilized and adoption was no longer in question, we let the cadence relax: releases every other day, then weekly.

This part matters more than it looks. Zero backlog is a launch regime, not a way of life. It is expensive, it consumes the AI team, and it is worth paying for during exactly one window: while the organization is still deciding whether this tool is theirs or something being done to them. Once that question is settled, you can taper. Taper before it is settled and the channel dies like all the others.

One decision that looked unrelated

Traceability and audit were built in from the first increment, not added later. This looked like a compliance preference at the time. In practice governance never showed up as a separate, later layer of friction: it was simply how the tool worked from day one. A governed path that shows up after adoption shows up as a slowdown, and any governed path slower than the ungoverned one gets bypassed. Built in from the start, nobody ever experienced it as a gate.

The common failures are latency failures in disguise

A big-bang project that spends six months before anyone can touch it means feedback cannot even start until the window for winning people over has already closed: the suggestion board meets once a quarter, and by the time an idea ships, the person who proposed it has stopped expecting it. So ninety days of latency produces the same adoption as never shipping at all. The rollout whose entire change plan is a training session and a comms deck has no release cycle at all for feedback to flow into, so the loop never exists in the first place — which goes some way to explaining why "change management" has the reputation it has. ServiceNow's 2026 maturity index found that 74% of its top-scoring organizations run change management programs, against 7% of everyone else. I doubt the difference is that the 74% write better emails.

What I would compress it to

Start with a scope small enough that a two-week stage produces something usable. Release fast enough that a suggestion and its implementation live inside the same week. Ship the users' ideas, visibly, so the tool becomes partly theirs. Put the users on stage, not the project team. Track one business metric from day one, the one the business already watched. Keep the audit trail on from the first build. And treat the daily cadence as an investment with a defined end: its job is to settle the ownership question — whose tool is this — while it is still open, because it does not stay open forever.

Adoption is a latency problem. Every item on that list is a way of lowering the latency.