DEV Community: Helkyn Coello

80% of Companies Are Already Out of Step With the AI Act and Don't Know It

Helkyn Coello — Mon, 20 Jul 2026 17:59:18 +0000

The tool worked. That was never the problem. A business manager had specced it with AI, straight from the business need, and the result did what he asked: it ran, it produced output, it looked finished. What it did not have was a single log line. No trace of why it produced a given answer. Nothing that could explain, later, a bad call made on real employee data. Nobody had asked for any of that, so the AI built none of it.

That absence is the gap nobody priced in. Scaled across a company and pointed at the wrong category of system, it puts you out of step with the EU AI Act without a single person acting in bad faith.

The AI Act assumes an engineering floor you may not have built

Here is the collision worth sitting with. The AI Act (Regulation (EU) 2024/1689) does not ask high-risk system operators to start testing their AI. It assumes they already do. Read the high-risk obligations and you will notice they are written on top of an engineering discipline that is taken for granted. Article 12 and Article 19 require automatic record-keeping and logging, with logs retained at least six months. Annex IV and Article 11 define what the technical documentation must contain; Article 18 requires keeping it at the disposal of authorities for ten years after the system is placed on the market. Article 14 requires human oversight designed in, with real intervention and stop capability, and it clearly calls out automation bias — the tendency to over-trust a machine's output — which tells you the drafters understood exactly how people behave around confident machines. Article 15 requires testing for accuracy, robustness, and resistance to adversarial manipulation.

Every one of those clauses assumes a working baseline: that you log what your systems do, that you test them against errors, that a human can see inside the decision and pull the lever when it goes wrong. The Act regulates on top of that floor. It does not build the floor for you.

Now hold that next to the data. The ServiceNow and ThoughtLab Enterprise AI Maturity Index for 2026 surveyed 4,500 executives and 2,000 employees across 19 countries and 12 industries. The number that matters: only 20% of organizations have implemented AI testing, auditing, and risk-assessment processes. Read that the way an engineer reads a red alert on a dashboard. Four out of five organizations have not built the discipline the high-risk regime stands on.

So the inference — and I want to be precise that it is an inference, not a directly measured compliance figure — is that roughly 80% of companies are structurally out of step with what the Act assumes, for any system that falls in the high-risk tier. The obligations exist on paper; the engineering they depend on does not.

Almost everything written about the AI Act treats it as a legal question: what does "high-risk" mean, what exactly does Article 14 require. Read it that way and the hard part is understanding the text — as if the logging, testing, and oversight would somehow build themselves once the reading is done. They do not build themselves. The Act is an engineering problem wearing legal language, and the harder question is whether the infrastructure it takes for granted — the logging, the testing, the oversight — exists in your organization at all. For most companies the honest answer is no, and they have never checked.

Scope this before you panic

The alarm only applies to a specific subset of systems, and saying so is the difference between analysis and fear-mongering. The Act tiers systems by risk. Most internal tools — the meeting summarizer, the draft-email assistant, the thing that reformats your reports — sit in the minimal-risk tier and carry none of these obligations. You can run those with a clear conscience and a thin paper trail.

The obligations attach to the high-risk tier, defined largely by Annex III: systems used in employment and worker management, access to essential services, credit scoring, critical infrastructure, biometric categorization, and a handful of others. A tool that screens job candidates, ranks employees, or gates someone's access to a service is in a different legal universe than one that summarizes a PDF.

So the first move is not legal. It is classification. You cannot assess your exposure until you know which of your systems are in scope, and in my experience companies have done this for zero of them. They know the AI Act exists. But they have not mapped their own deployments against it. The business manager with the AI-specced tool could not have told you which tier it fell in, because the question never came up, and depending on what that tool actually decides, the answer changes everything about what he owed under the law.

The reason capability outran the floor

The same maturity report explains how we got here. 59% of organizations are past piloting agentic AI, but only 9% have working autonomous multistep workflows. Capability is being deployed far faster than the governance layer beneath it. Only 16% have replaced fragmented legacy systems with an integrated platform; 41% still cite siloed data as a blocker. You cannot log and audit a decision cleanly when the data feeding it lives in six disconnected systems and the workflow stitching them together is held up with visual no-code duct tape.

I have lived the small version of this, spending months building AI automation pipelines in visual tools. They produced output. But when a node failed silently, I had no clean trace of what happened or why, and debugging meant clicking through visual steps trying to find which one died. I eventually rewrote the core of it as proper code, and what changed was not speed. I could finally see inside it. Now imagine that same opacity at a corporate level sitting under a system that decides who gets a loan or who keeps a job. That is the engineering gap the Act assumes you closed before it ever showed up.

There is a cultural tell in the data too. 57% of employees think leadership is not keeping up with fast-changing trends. That is what the governance gap feels like from below: the people closest to the work can sense that the controls are not there, even when the official version says they are.

What this actually requires of you

Stop treating this as a question of whether you have read the regulation correctly. The exposure for anything you run in the high-risk tier comes down to one thing: whether the testing, logging, and oversight discipline — the one the entire regime takes for granted — actually exists under your systems. If it does not, you are not one clause away from compliance. You are one engineering layer away, and that layer takes months to build and tune, not an afternoon with a lawyer.

The work is two concrete steps, in order. First, classify your AI systems by tier so you know which ones carry high-risk obligations and which ones you can leave alone. Most of your tools will fall out of scope, and that is the point. You want to spend your effort where the law actually bites. Second, for the systems that remain, check whether the assumed baseline is there: automatic logging you retain, documentation you can produce, testing against accuracy and adversarial failure, and a human who can both see the decision and stop it.

When that human can pull the stop lever in time and understands what they are overriding, you have oversight. When the data feeding the decision lives across four disconnected systems with no trace of what changed, you have a documentation requirement you cannot satisfy and an audit you would fail. A structured workflow audit is one way to surface exactly where those traces break down.

Most companies have done neither step. They have not classified their systems, and they have not audited the foundation under the ones that matter. Start with the first. Pull a list of every AI-assisted system that touches employment, access, credit, or essential services, and ask one question of each: if a regulator asked us to show six months of logs and explain a single decision, could we. The systems where the answer is no are your real exposure — and that one hour of asking will tell you more than another month of reading the Act.

https://unlockedconsulting.ai

Is Your Knowledge Base Actually Thinking, or Just Retrieving?

Helkyn Coello — Mon, 20 Jul 2026 17:57:15 +0000

A while back I was able to build a working knowledge base in less than an hour. Fifty-five minutes to be accurate. I started by opening VS Code with Claude Code in the side panel. I had documentation scattered across roughly ten training manuals, a video archive nobody watched, support ticket histories, enhancement docs, and meeting transcripts. The knowledge existed. It was just spread across systems with nothing connecting them.

I described the islands to Claude the way I'd describe them to a new engineer. Then I asked for a chat interface backed by a vector database with RAG-style retrieval from plain-language questions. At around minute 45, it was running and returning accurate answers from the actual docs. I know it was not yet production software, but it was a real prototype on real data.

But I had slipped one extra requirement into that prompt, almost without thinking about it: client-specific information should be flagged and surfaced only when a query came from that client or was about that client. We had training recordings and support history tied to individual accounts. That requirement is the whole article. The moment you say "surface this when the query is about that client," you have stepped outside what retrieval by similarity can do. Similarity answers one question: which stored text resembles the wording of the query. But a support ticket and a training recording tied to the same client may share no vocabulary with each other, or with the question being asked. What links them is not wording; it is the client they both belong to. That link is a relationship, and a relationship is something a similarity search has no way to represent.

My prototype got away with it in a demo. In production, where real users ask cross-source questions every day, that shortcut stops holding.

What embeddings actually do, and where they stop

Embed-and-retrieve works like this: you chunk the documents, run each chunk through an embedding model, store the vectors, and at query time you embed the question and pull back the nearest neighbors by cosine similarity, a measure of how close two texts sit in meaning. It works. For a large class of questions — "what does our refund policy say," "how do I configure SSO" — it works well, and you can have it running in an afternoon.

The ceiling shows up the moment an answer lives across sources that don't look alike.

Take a question we actually get internally: "Why is this client's month-end close behaving differently from the others?" The answer is not in one chunk. It's in a custom configuration noted in a support ticket from two years ago, a behavior described in a training recording for that specific account, and a feature flag mentioned in an enhancement doc. None of those three documents are textually similar to each other, and none of them are similar to the question. An embedding search returns whatever happens to share vocabulary with "month-end close." That points straight to the generic manual, which is the one document in the set that does not contain the answer.

Similarity is a proxy for relevance. It's a decent proxy when relevant things tend to use the same words. It collapses when the relevant things are connected by something other than vocabulary: a shared client, a shared subsystem, a cause-and-effect chain that nobody wrote down in a single place.

This is the part that gets skipped in the market. A team sets up RAG, declares "the knowledge base is done," and stops exactly there. You end up with a search engine that talks. Useful, but it answers "what does the document say," and the questions that actually justify the project are the ones that ask "what does the situation mean."

The relationships are the knowledge

I was reminded of all this at Milano AI Week, where someone framed it cleanly: the real intelligence in a knowledge base isn't in the chunks; it's in how the segments relate, the graph rather than the index.

Concretely, that means a second representation alongside your vectors. You run entity extraction over the same corpus — clients, features, configurations, people, error conditions, dates — and you model the edges between them. Ticket 4471 references client Aurora. Aurora runs the custom close configuration. That configuration was introduced by enhancement request 882. The training recording from March covers that workflow. Those are explicit edges in a graph, and they are traversable regardless of whether any two of those nodes share a single word.

Now the same question (why is this client's month-end close behaving differently) gets answered along a different path. The system maps "this client" to the Aurora entity, walks the edges out to its configurations, tickets, and recordings, and assembles a candidate set that no similarity search would ever produce, because the documents have almost nothing lexically in common. The graph answers what is actually related; the embeddings answer what looks similar. You want both: similarity finds the entry points and the unstructured passages, and the graph traversal gathers everything connected. Either one alone leaves capability on the table.

I'm not arguing embeddings are wrong. I'm arguing that "chunks plus embeddings" is a floor people keep mistaking for a ceiling.

The cost is real, and it is not catastrophic

Here's the honest trade-off, because the graph approach is not free.

Entity extraction has to be tuned. The first pass over-extracts, treats every capitalized string as an entity, and cannot tell "Aurora the client" apart from "Aurora the internal name of a feature." Entity resolution — deciding that three different spellings refer to the same client — is where most of the real effort goes. You need a schema for what entities and relationships matter to your business, and that schema is opinionated, which means it requires someone who understands the domain to define it. You'll iterate it several times before it behaves. This is the same tuning loop I've hit building constraint surfaces for AI tools internally: the gap between how you described the rule and how the system interpreted it is always wider than you expect at the start, and you only close it by running real queries and watching what breaks.

Operationally you're now maintaining two stores and a pipeline that keeps the graph in sync as documents change. That's more moving parts than a single vector index. Budget for it.

But the cost difference between the two architectures is incremental, not an order of magnitude. The embed step is shared. The extra work is the extraction pipeline, a graph store, and the retrieval logic that combines the two. The capability difference, by contrast, is enormous. A chunk-only system can answer questions confined to a single document. The graph version can answer questions whose answer requires reasoning across sources that were never connected by an author. For an enterprise knowledge base, that second class of question is usually the entire reason the project was funded.

If your knowledge base is meant to serve support engineers, account managers, or anyone diagnosing a situation that spans history, the chunk-only version will demo beautifully and disappoint in week three, when the first cross-source question comes in and it confidently returns the generic manual. This is exactly the kind of representation decision we work through when we design a knowledge base meant to survive contact with real questions.

What to check before you sign off on an architecture

There's a simple test. Take five questions your team actually asks, the hard ones, the ones that make someone sigh, and for each one, ask yourself whether the full answer lives in a single document or has to be stitched together from several. If most of them need the stitching, similarity retrieval will not get you there, and no amount of better embeddings or bigger context windows fixes a representation problem.

That test is also the dividing line between a search engine and a knowledge base. A search engine retrieves text. A knowledge base models the entities your business cares about and the relationships between them, then reasons over that structure. The prototype I built in fifty-five minutes was the former with one relationship bolted on by hand. The production version of that same system is the latter, and the difference is entirely in how seriously you model the graph.

Before you approve any internal knowledge base design, ask the team building it one question: when the answer lives in three unrelated documents, what mechanism finds the other two? If the only answer is "similarity," that tells you two things: the team hasn't seen the ceiling yet, and the budget you're about to approve will buy a system that stops exactly where your hardest questions begin. Decide whether that's enough before the cost moves downstream into the maintenance no one can see.

https://unlockedconsulting.ai

The On-Premise LLM Lottery

Helkyn Coello — Mon, 20 Jul 2026 17:53:51 +0000

I lost a full long weekend to OpenClaw once. Friday to Sunday, trying to get it running locally with an on-premise LLM on my personal machine. The machine was reasonably capable, though nothing close to a datacenter. I had watched something like forty hours of tutorials first, so I went in confident. Every video made it look like four steps: download, install, connect a model, start automating.

By Sunday evening nothing worked. The wall I hit had nothing to do with my configuration. It was a real limitation in the tool itself, on the hardware I had, with no path around it from the user side. The clean take in the tutorial had been produced in a controlled setup, and the hours of debugging that preceded it never made it into the video.

That weekend taught me something I keep coming back to when someone tells me they want to run their models on-premise. I hadn't picked the wrong app. I'd treated "run an LLM locally" as a task with a fixed answer, when it is actually a search.

There is a whole genre of content that presents local inference as a tidy checklist. Download the app, check your RAM, pick a model size that fits, done. And for the audience those guides are written for — one person, one laptop, one chat window — that genuinely is the whole job. They are answering a smaller question than the one an enterprise is asking, and they answer it well. The problem starts when someone in a regulated company reads that checklist and assumes their version is the same problem with more zeros in the hardware budget.

It isn't. In the enterprise version, your hardware is already fixed by procurement decisions made before you arrived. Your runtime is constrained by what your ops team will agree to support. Regulatory and compliance requirements narrow which models you can even consider before you benchmark a single one. The checklist collapses under those constraints, and what's left underneath is a matchmaking problem with several moving axes.

The axes nobody puts in the checklist

After that weekend I stopped looking for the right app and started running an actual search. I tested roughly twenty models before one fit. That number wasn't diligence for its own sake. There are many possible combinations of model, quantization, and runtime, and most of them simply don't work on any given hardware, so finding one that does takes that many attempts.

Those combinations vary along a few axes, and the axes interact with each other. Model size is the obvious one, but the parameter count on the label tells you almost nothing until you pair it with quantization. A model that won't load at full precision runs comfortably at a lower quantization, with a quality cost you have to measure rather than assume. Then the runtime layer sits on top of that: I settled on vLLM and Ollama depending on the case, because the runtime is what turns "the weights exist on disk" into "the thing answers a request at an acceptable latency." And all of that has to land on hardware you don't get to choose.

The published guides almost always cover exactly one point in this space and present it as universal. A tutorial that shows a specific model at a specific quantization on a specific GPU is accurate. It is also useless the moment your GPU is different, which it always is. The tutorial isn't lying, it just isn't your situation, and the gap between a working demo and a working installation in your environment is the entire job.

Gemma turned out to be the fit for my hardware, and the reason had little to do with rankings: at the quantization and runtime I could actually deploy, it gave me acceptable output at a latency I could live with. Someone else, on different hardware, with a different tolerance for latency, lands somewhere else entirely. There is only the best model for a specific combination of circumstances, and you find it by running the combinations.

Why teams quit before they finish

The reason this matters commercially is that most teams give up partway through the search and don't realize that's what happened. They test three or four models, hit the same kind of wall I hit that weekend, and conclude either that local inference doesn't work or that they need to spend more on hardware. Sometimes more hardware is the answer. Often it isn't, and they've just stopped short of the combination that would have worked on what they already own.

The search costs real engineering time, and that's the part leadership doesn't budget for. Benchmarking a model isn't downloading it and asking it a question. It's loading it at several quantization levels, measuring throughput and latency under something resembling real load, checking output quality against your actual tasks, and doing that across enough candidates to be confident you've found a fit rather than the first thing that didn't crash. On a fixed hardware target, that is days to weeks of an engineer's time before you write a line of application code.

There is also a quieter failure mode. A model that loads and answers in a demo can fall over under concurrent requests, or produce output that's fine for a chat toy and unacceptable for the compliance-sensitive task you actually need. The runtime tuning I did — adjusting for responsiveness so the thing felt usable rather than technically functional — was its own cycle of test, measure, adjust. That work never shows up in the four-step version because the four-step version was never asked to serve more than one person..

The case for doing the search anyway

None of this is an argument against on-premise inference. For a healthcare or finance client who can't send patient records or transaction data to a third-party API, on-prem is the constraint, not a preference. When that's your situation, the matchmaking phase stops being optional overhead and becomes the foundation the rest of the project stands on.

The teams that finish the search end up with something durable. They know, concretely, which model runs on their hardware, at which quantization, under which runtime, at what latency, for which tasks. That knowledge is specific to their environment and expensive to reproduce, which is exactly what makes it a moat. A competitor who wants the same capability has to run the same search on their own hardware. There's no shortcut around it, which is the whole point.

A team that hasn't run the search and claims they can run AI on-prem is describing a future promise, not a current capability.
The distinction sounds pedantic until the deadline arrives and someone has to explain to leadership why "just run it locally" turned into a month of benchmarking. I've watched that conversation happen. It goes better when the month was budgeted up front as part of the strategy rather than discovered halfway through as a surprise.

So before you commit to on-prem as a strategy, run a scoped version of the search first. Pick your three or four most plausible model-and-quantization combinations for the hardware you already have, benchmark them against a real task, and measure how far you got and how far you have left. If your team can't tell you which combination fits their hardware today, that's the gap to close before anyone promises a timeline. In our workflow audits, that scoping exercise is usually the first thing we do, because it's the difference between a plan and a wish.

https://unlockedconsulting.ai/

The Hidden Mental Model That Determines Whether AI Tools Help You

Helkyn Coello — Mon, 20 Jul 2026 10:03:16 +0000

I built a working knowledge base while I was in a one-hour company meeting, waiting for my turn to give my briefing. Even I was surprised at how fast it came together. And it was not a mockup. It was a complete chat interface that connected to nearly everything the company had — training manuals, forgotten documentation, enhancement request docs, support ticket histories — all indexed into a vector database. When I tested it, it answered my plain-language questions and surfaced the right context every time. Fifty-five minutes was all I had to do it. And by the time I needed to do my briefing, the tool was already running.

I want to be precise about why this was possible, because the obvious explanation is normally wrong: the tool wasn't the reason. Yes, VS Code with Claude Code inside was the tool I used to pull this together, but the real reason it took under an hour is that I already knew what the right solution looked like. I knew how the pieces had to fit, where the data belonged, what would break if I got it wrong. That judgment was already in my head. The AI didn't supply it. It executed against it at speed.

Same tools, different results

Those same tools I used could produce completely different results, depending on who's using them and how. So even though the tools are the same, what makes the difference in the output is the specific technical judgment behind it. Put the same setup in front of someone who doesn't carry that technical judgment, and the AI runs just as fast — except toward a structure they have no way to evaluate. The output looks plausible, but the cost surfaces later, somewhere harder to see. That gap is the whole point, and it has almost nothing to do with the tool.

What the mental model actually is

So what is that judgment, concretely? It's a mental model of how the parts of a system fit together: where data lives, how one piece talks to another, how something gets from your machine to where it actually runs. Not the code, the shape of it. You can have all of that without writing a single line yourself.

Two things tend to get mistaken for it. The first is the ability to write code. Those aren't the same thing; plenty of people who can write a function still carry no picture of the system that function lives in. The second is knowing the business — the company, the clients, the domain. That knowledge helps, and it certainly made me faster in that meeting, but it isn't what's missing when AI output goes wrong. What's missing is the structural picture: a sense of where a piece of data should originate, what should talk to what, and what breaks downstream when those choices are wrong.

That picture matters because of what it lets you do with the AI's output: judge it. When you're holding an accurate model, a suggestion that doesn't belong stands out right away, because it doesn't fit the shape already in your head. Strip that model away and the same suggestion looks as reasonable as everything else, since there's nothing to measure it against. That's why the model, not the tool, is the real variable. AI amplifies whatever you bring to it — good judgment moves faster, and so does the absence of it.

The model, not the tool, is the real variable. AI amplifies whatever you bring to it — good judgment moves faster, and so does its absence, just toward a structure you'll pay for later.

Why "which AI tool should we use" is the wrong opening question

That's why "which AI tool should we use" is usually the wrong place to start. Over the past few months I've run a systematic evaluation across my team: Cursor, Claude web, Claude Desktop, Claude Code, Copilot in VS Code, AugmentCode, and various combinations. The finding was the same every time. There's no best tool, only the best tool for a specific situation. What decides it is the project, the stack, who maintains the code afterward, the deployment target, the cost model, and above all whether the person using it is a developer or not. Change any of those and the right answer changes with it.

"Which tool" is a tempting question because it's answerable. You can compare features, watch demos, pick a winner, and feel like you've made progress, and it's the one question every vendor is glad to answer for you. But it skips the harder one: does the person using the tool understand the architecture underneath — how the parts actually connect? That question has no vendor, no demo, and no clean answer, which is exactly why it gets avoided.

The cost of skipping it doesn't announce itself. Hand a capable tool to someone who can't see that architecture, and nothing breaks on day one. It shows up later, in maintenance, where it's hardest to trace back to its cause.

What to fix when productivity gains are uneven across your team

If you lead a team, you've probably seen this already: the same tools landed, and a few people got dramatically faster while others barely moved. The reflex is to standardize: pick one tool, roll out training, make everyone consistent. That treats it as a tooling problem, and it usually isn't. What to do next depends on who's in front of the tool.

For the people who aren't developers, don't try to turn them into architects. That's the slow path, and most of them don't need it. Give them a setup where the hard parts — where the code lives, how it connects, how it ships — are already decided and kept out of their way. They get the leverage of the tool without having to carry the model themselves, because it's built into the environment around them.

Your engineers and technical leads need the opposite investment. For them the architecture model is the multiplier, so the time spent making sure they can see how a system fits together pays back more than any tool upgrade. That understanding is what the tool runs on.

And if you want a cheap way to find out where each person actually stands, skip the survey. Take your fastest and your slowest person on the same tool, hand them a whiteboard, and ask them to draw the system they're working on — where the data comes from, what talks to what, where the code lives, how it gets deployed. The drawing tends to track the productivity gap closely; often you'll see it before the marker's back in the tray. It's the cheapest diagnostic you'll run, and it measures the one thing that actually matters here: whether they can see the system they're building on.

https://unlockedconsulting.ai/blog/hidden-mental-model-ai-tools

Can a Non-Coder Become a Coder Just With AI?

Helkyn Coello — Mon, 20 Jul 2026 10:00:37 +0000

My company decided to lean into AI — to let people who had never written code build real software with it. I thought that was the right call, and I still do.

The logic behind the decision seems airtight: tools like Claude Desktop are built to be accessible to non-developers. They abstract the complexity. They turn natural language into working software. But put those tools in front of a capable non-developer, and what tends to follow is weeks of confusion, cluttered repositories, and architectural decisions that look functional but rest on shaky foundations.

I've watched it happen. And the question "can a non-coder become a coder just with AI?" has a more useful answer than yes or no. Yes, they can produce working code. No, they cannot produce maintainable code without understanding how the parts fit together.

The infrastructure reality check

The non-developer gets a real developer's environment: GitHub, Python, Node.js, Visual Studio Code, a database server, an AI assistant on top. The premise is that having that setup is most of the battle. The expectation is immediate productivity.

What actually happens: Git becomes the first obstacle. Branches, commits, pushes, pulls, clones — all of it opaque. The AI makes changes, generates commits, pushes to the repository. But without understanding what's being committed or why, the repository fills with stray commits mixed into legitimate work. Unexpected changes land in branches they shouldn't. The history becomes hard to follow.

Then the local environment problems. Port conflicts. The UI can't reach the API because the ports aren't configured correctly. Services that have to be started, stopped, restarted. Every working session begins with fifteen minutes of troubleshooting that a developer handles automatically, by reflex, but that a non-developer has to solve consciously, from scratch, every time.

The mental model problem

The deeper problem isn't technical. The non-developer doesn't know what an API is, what a database does, where the UI ends and the backend begins. Without that mental model, every AI suggestion looks equally plausible — including the wrong ones. There's no way to evaluate the output, because there's no way to see the architecture.

When the AI suggests connecting the frontend directly to the database, it gets implemented. When it recommends storing user passwords in plain text environment variables, that happens too. The AI isn't being careless. It's responding to prompts that don't include the constraints a developer would apply automatically.

Here is the floor — the five things a non-technical person needs to understand before AI tools help instead of hurt. None of them is about writing code. All of them are about how the pieces relate.

What a database is. Information lives in a structured store, and you don't connect the front end straight to it — for the same reason you don't hand customers the keys to the warehouse.

What an API is for. It's the control layer between the user and the data: it checks who's asking and enforces the rules. When the AI proposes skipping it, that alone tells you something is wrong.

What runs where. Part of any app runs on the user's device and part runs on your servers. Whatever sits on the user's device, the user can see and change — so anything sensitive has to stay on your servers.

What version control is for. Every change has a reason, experiments stay separate from the live system, and what customers use keeps working. Lose that discipline and you lose the record of what changed and why.

Where secrets live. Passwords and keys belong apart from the code, and they differ between a test setup and the real one. Put them in the wrong place and they leak.

These five take a few hours to learn, not months. But without them, AI tools become code generators that produce expensive technical debt.

Necessary, not sufficient

That mental model gets you to better prompts. It doesn't get you to safe output on its own. Even with solid foundations, a non-developer using AI will still face decisions they aren't equipped to make — and left to their own judgment, they'll make those decisions anyway.

The other half of the answer is a constrained environment: guardrails that make the bad decisions impossible rather than merely unlikely — schemas that can only change through migrations, endpoints that must validate input, frontends that can't hold secrets. Building that surface is its own undertaking, and a separate conversation. What matters here is that the mental model comes first. Without it, even the best guardrails are fighting a user who can't see why the rails are there.

The decision in front of a technical leader

The ambition my company started with was the right one. The choice that actually matters comes a step later: how much infrastructure you're willing to build around the tools. That choice isn't binary.

The cheapest path is raw AI tools with light training. Cheap upfront, expensive later: working prototypes that become maintenance liabilities, with technical debt compounding until someone rewrites everything properly.

The middle path is structured tools with foundational training and a constrained environment. Higher upfront, lower long-term — the output is maintainable from day one because the constraints prevent the unmaintainable patterns before they happen.

The third path, and often the most efficient for smaller organizations, is to keep development centralized and use AI to accelerate the team you already have, rather than distributing it across people who can't yet evaluate what it produces.

The path that doesn't work is the first one, chosen in the hope it will somehow work out.

The real question

So the real question isn't whether non-coders can become coders with AI. It's whether your organization can afford the infrastructure that makes it safe — the foundational training, the constrained environment, the deployment and monitoring around it. If you can build that, then yes, non-coders can ship maintainable software with AI. If you can't, you're better off keeping development centralized and making your existing developers faster.

If you do nothing else, do this: train the mental model first, in a few focused sessions, before handing out any AI tools. Five concepts, concrete examples from your own systems. The investment pays back immediately, because every prompt the user writes afterward is a better prompt. They ask for input validation instead of assuming it's optional. They name the environment variables they need instead of hardcoding credentials. They think in endpoints instead of wiring everything to everything.

AI amplifies what's already there. If the user doesn't understand the basics, AI amplifies that gap straight into the systems they ship.

https://unlockedconsulting.ai

AI in the Data, Code in the Process

Helkyn Coello — Mon, 20 Jul 2026 09:57:59 +0000

If you need the same answer twice, don't use AI.

I learned this the hard way during a project where headquarters mandated to use OpenClaw - an AI orchestration platform and workflow automation tool that had been making the rounds in tech publications. The requirement wasn't driven by the problem we were solving. It was driven by visibility. Decision-makers had heard this name everywhere.

The actual work was straightforward: automate a set of multi-step tasks with clear inputs, clear outputs, and deterministic logic. Nothing ambiguous. Nothing requiring interpretation or context-aware responses. Just reliable, repeatable processing that needed to happen the same way every time.

The specified architecture would have required dedicated hardware installations, per-machine configurations, and staff training on platforms they didn't need to understand. The overhead was significant. The justification was probably to some extend trend-driven.

I restructured the project. Removed this tool from the design. Replaced it with clean deterministic code — written efficiently with AI assistance during development, but deterministic in execution. No orchestration platform. No hardware purchases. Everything moved to the cloud with a simplified workflow.

The project shipped faster, ran cheaper, and required less maintenance. But the resistance was still there. Removing AI-powered tools felt like a step backward to stakeholders who equated sophistication with AI agents handling every step in the process.

Where AI Actually Belongs

The confusion comes from treating AI as a universal architecture instead of understanding where non-determinism is a feature versus where it's a liability.

AI belongs where you need interpretation. When I built a knowledge base during a company meeting — connecting training manuals, support tickets, and recorded sessions into a searchable system — the value was in AI's ability to understand natural language queries and synthesize information across disconnected sources. The same question asked in slightly different ways should return contextually relevant answers. Non-determinism was exactly what I needed there.

But business logic, transactional flows, and validation rules require the opposite characteristic. If a payment processing workflow returns different results for identical inputs, that's not intelligence — it's a bug. If an inventory calculation varies based on the AI agent's "interpretation" of the same data, your accounting is broken.

The Enterprise AI Maturity Index shows this misapplication at scale: 59% of organizations use agentic AI, but only 9% have autonomous workflows. The gap represents companies that deployed AI agents everywhere without asking which processes actually benefit from non-deterministic behavior.

The Development Versus Execution Distinction

The most productive approach separates AI assistance during development from AI execution in production. AI accelerates the creation of deterministic tools. It doesn't replace the need for these deterministic tools.

When I need to write validation logic, database queries, or API endpoints, AI helps me write them faster. But the output is still deterministic code that behaves predictably in production. AI assisted in the creation process. But it doesn't run the process.

This distinction matters because failure modes are different. AI assistance during development fails gracefully — you review the code, test it, and fix what doesn't work. AI execution in production fails unpredictably — the same input might produce different outputs depending on model behavior, token limits, or service availability.

Companies treating agentic AI as the universal solution are designing instability into their own systems. They're putting non-deterministic behavior in places where determinism is a business requirement.

The Cost of Misplaced Intelligence

I've seen teams spend months building AI agents to handle workflows that could have been solved with fifty lines of deterministic code. The agent approach requires prompt engineering, error handling for model failures, retry logic for API timeouts, monitoring for drift, and ongoing tuning as the model's behavior changes over time.

The deterministic approach requires writing the logic once and deploying it. The maintenance burden is lower. The behavior is predictable. The failure modes are known.

The economic argument is clear when you run the numbers. AI agents consume tokens, require specialized monitoring, and introduce operational complexity that scales with usage. Deterministic code runs at computational cost — orders of magnitude cheaper for repeated operations.

Systems need predictable behavior in their core processes. Customer data, financial calculations, and operational workflows cannot behave differently on Tuesday than they did on Monday for reasons nobody can explain.

The Data Layer Is Different

Data retrieval, analysis, and synthesis represent the opposite case. This is where AI's non-deterministic nature becomes valuable rather than problematic.

When someone queries a knowledge base, they're not looking for the same answer every time. They're looking for the most relevant answer given their specific context and phrasing. When AI analyzes customer support tickets to identify patterns, slight variations in output based on different analytical approaches can reveal insights that deterministic queries might miss.

The pattern that works is to use AI at the data layer and deterministic code at the process layer. Let AI interpret, classify, and synthesize information. Use deterministic code to act on that information in predictable ways.

A practical example: AI can analyze customer behavior data and flag accounts that might churn. But the automated response to those flags — sending specific emails, adjusting pricing, or triggering sales outreach — should follow deterministic rules. The intelligence is in understanding the data. The reliability is in how you act on that understanding.

Evaluating Where AI Belongs

When evaluating where AI fits in your system, ask which characteristic the workflow actually needs. If the same input must produce the same output, you need deterministic code. If you need interpretation, classification, or context-aware response, you need AI.

The most common architectural mistake is using AI agents to do work that should have been deterministic code — built fast with AI assistance during development, but deterministic in execution.

This evaluation changes how you approach AI implementation. Instead of asking "How can we add AI to this process?" ask "Does this process benefit from non-deterministic behavior?" Most processes don't. Most data analysis does.

The companies that get AI implementation right understand this distinction. They use AI where interpretation adds value and avoid it where consistency is required. They build AI-assisted tools quickly but deploy deterministic systems reliably.

AI in the data, code in the process. That's the architecture that actually scales.

https://unlockedconsulting.ai/

AI Doesn't Think For You, It Thinks Like You

Helkyn Coello — Mon, 20 Jul 2026 09:54:02 +0000

A few weeks ago a Business Manager handed me a 50-page AI-generated technical spec. The document was impressive. The perspective behind it was the problem.

I had talked to him a few days earlier about a new internal tool the company needed. We discussed the use case and the requirements in depth. Later on, I discussed those same requirements with my IT team and assigned them the job of making the technical specification.

But before my team finished, the Business Manager handed me his own spec. A spec ready to be executed. The document was generated with AI assistance and it was impressive — fifty pages long, detailed feature breakdown, implementation timeline, cost projections. Everything looked professional. The AI had done exactly what it was asked to do.

I read the whole document and noticed a problem. Not with the quality of the document but with the perspective that shaped it.

The spec called for a manual Excel-based workflow with several manual steps and validations in-between. All seemed clean and manageable, matching the Business Manager's mental model of how that business workflow should work.

When I checked the spec my team was working on using the same AI assistance tools, I noticed they had produced something completely different: automated data ingestion, real-time dashboards, API integrations with existing systems.

Both specs addressed the same business need. Both were technically sound. Both could be built in roughly the same timeframe. But they were fundamentally different architectures, shaped by fundamentally different perspectives on how work should happen.

The Business Manager's version was optimized for control and visibility — he could see every step, every piece of information and approve everything manually at any stage. The technical lead's version optimized for efficiency and scale — minimal manual intervention, automated error handling, designed to handle 10x the current volume without breaking.

Neither person was wrong. But the AI amplified their existing perspectives without challenging them. The Business Manager got back a business-shaped solution. The technical lead got back a technical-lead-shaped solution. The AI didn't think for either of them. It thought like them.

The Perspective Filter

This dynamic shows up everywhere once you start looking for it. A marketing director using AI to design a customer onboarding flow produces something heavy on touchpoints and engagement metrics. An operations manager designing the same flow produces something optimized for throughput and error reduction. A customer success lead produces something focused on retention signals and satisfaction measurement.

The underlying business requirement is identical in all three cases: get new customers productive as quickly as possible. But the AI output varies dramatically based on who wrote the prompt, because the AI inherits the prompter's definition of what "productive" means and what "quickly" optimizes for.

The dangerous part isn't that these perspectives exist (they always have). The dangerous part is that the AI output feels objective. When someone shows you a fifty-page specification that an AI helped generate, the instinct is to evaluate it as a neutral technical document. But it's not neutral. It's a reflection of whoever wrote the prompts, processed through a layer that makes their bias harder to see.

I've seen this pattern derail projects. A business stakeholder uses AI to spec out a technical solution, presents it to the engineering team as "the requirements," and expects implementation to be straightforward. The engineering team looks at the spec and immediately sees the architectural problems — missing error handling, no consideration for scale, integration points that don't exist in the current system. But because the spec was "AI-generated," there's an assumption that it must be technically sound. The AI becomes a shield against technical pushback.

The Authority Transfer Problem

The most insidious version of this is when AI output gets treated as authoritative simply because it came from AI. "The AI said this is the best approach" becomes a conversation-ender, as if the AI has some independent perspective on what "best" means.

But AI doesn't have opinions. It has training data and pattern matching. When it recommends an approach, it's recommending based on the patterns it learned and the way the prompt was framed. If you prompt it from a cost-optimization angle, you get cost-optimized recommendations. If you prompt it from a user-experience angle, you get UX-optimized recommendations. If you prompt it from a compliance angle, you get compliance-heavy recommendations.

The choice of angle isn't made by the AI. It's made by whoever writes the prompt. And that choice shapes everything that follows.

I saw this also play out in a project where the initial AI-generated architecture called for a microservices approach. The recommendation looked sophisticated, included proper service boundaries, even had a reasonable deployment strategy. But when I dug into the prompts that generated it, the framing was entirely around scalability and modularity. No one had prompted the AI to consider the team's actual experience with microservices, the operational overhead, or the debugging complexity for a team that had been running monoliths for five years.

The AI gave exactly what it was asked for: a scalable, modular architecture. But it wasn't asked to consider whether that architecture matched the team's ability to operate it. The prompter's bias toward modern architectural patterns shaped an output that would have been expensive to maintain and difficult to debug for that specific team.

Multiple Perspectives, Better Decisions

The solution isn't to stop using AI for architectural decisions. The solution is to recognize that a single perspective using AI produces a single-perspective architecture, no matter how sophisticated the output looks.

The same prompt from different organizational roles produces different answers. Neither is necessarily right, but the combination produces something better than either alone.

When working on AI projects now, I run it through multiple perspectives — typically business, technical and security. Each prompt is framed for that particular role's priorities and constraints. Then I look at where the outputs diverge.

The AI output from each perspective is always different. Not slightly different — fundamentally different. The business stakeholder gets a solution that fits the budget and timeline. The technical lead gets a solution that won't create technical debt. The security team gets a solution that won't create compliance problems.

None of these perspectives alone produces a complete architecture. But taken together, they surface the trade-offs that a single-perspective approach would miss. Without that contrast, you don't have an architecture decision. You have a single-perspective recommendation that looks complete.

The conversation shifts from "this is what we should build" to "here are the different ways we could build it, and here are the trade-offs between them."

The Bias Is in the Prompter

The fundamental insight is that AI suggestions are not opinions of the AI. They are reflections of the prompter back to themselves, processed through a layer that gives them a veneer of objectivity. The output looks neutral but the input never was.

This matters for how technical leaders should handle business-side AI-generated specs. Treat them as input, not directive. The spec represents one perspective on the problem, shaped by one set of priorities. It's valuable input. But it's not complete requirements.

And it matters for how business leaders should think about AI-assisted decision-making. "The AI said so" doesn't close the architectural conversation. It opens it. The question isn't whether the AI's recommendation is correct. The question is whether the perspective that shaped the prompt captured all the constraints that matter.

The leverage comes from using AI to rapidly explore multiple perspectives on the same problem, not from using AI to validate a single perspective more convincingly. The bias isn't in the AI. The bias is in the prompter. The AI just makes it harder to see.

So when someone comes to you with an AI-generated solution, the first question shouldn't be "is this technically sound?" The first question should be "what perspective shaped this output, and what perspectives are missing?"

https://unlockedconsulting.ai/