Every organization scaling past two or three AI agent projects hits the same fork in the road eventually. Keep letting each team build its own model access, its own evaluation setup, its own observability, and you get duplicated effort, inconsistent security practices, and five different teams solving the same authentication problem in five slightly different, none-of-them-quite-right ways. Centralize everything under a platform team, and you get consistency, but often at the cost of the speed and domain-specific judgment that made those individual teams effective in the first place.
Neither extreme works well for long. The organizations handling this well have found a specific, deliberate split, and it's worth understanding what actually belongs on which side of it.
Where Centralization Genuinely Pays Off
Model access is an obvious candidate. Negotiating API contracts, managing rate limits across the organization, handling failover between providers, and keeping a consistent view of spend across every team using AI models is genuinely more efficient centralized than duplicated five times over by five separate teams, each managing their own vendor relationships and often overpaying or hitting avoidable rate limits because nobody has visibility into aggregate usage.
Identity and access management follows the same logic. A single, well-audited system for how agents authenticate, what permissions they're granted, and how those permissions get revoked when something goes wrong is a genuine security improvement over five teams each rolling their own credential handling, with five different levels of rigor and five different blind spots.
Observability and evaluation infrastructure belong centralized too, for a slightly different reason: consistency of measurement. If every team builds its own way of tracking agent performance and logging agent behavior, comparing outcomes across the organization becomes nearly impossible, and leadership ends up making resourcing decisions based on incompatible, apples-to-oranges reports from different teams describing success in entirely different terms.
FinOps for AI spend, tracking cost per model call, per agent, per team, and attributing that back to business outcomes, is another strong candidate for centralization, mostly because it requires a consistent methodology applied organization-wide to actually mean anything when compared across teams.
Where Centralization Actively Hurts
Domain-specific agent logic, the actual reasoning, tool selection, and workflow design for a specific business problem, is where centralization tends to slow things down without adding real value. A platform team sitting between a fintech team and their agent's actual decision logic, requiring every change to route through a centralized review process, adds latency and friction to iteration without meaningfully improving safety or quality, because the platform team usually doesn't have the deep domain context to evaluate that logic as well as the team building it directly.
The same applies to evaluation criteria specific to a domain. A logistics team knows what "good" looks like for their exception-handling agent in a way a centralized platform team generally doesn't, and forcing that team to use a generic, one-size-fits-all evaluation framework built for the whole organization often means missing the specific failure modes that actually matter for their particular use case.
Where People Get This Wrong
The failure pattern shows up in both directions, and it's worth naming both because organizations tend to overcorrect from one straight into the other. Under-centralization looks like five teams each building redundant infrastructure, each with slightly different security postures, none of them getting the benefit of shared learning across the organization. Over-centralization looks like a platform team that's become a mandatory gatekeeper for every agent decision, slowing domain teams down so much that they start building workarounds outside the platform entirely, which quietly recreates the original fragmentation problem while also adding an extra layer of unnecessary bureaucracy on top.
The better split centralizes shared infrastructure and shared risk, model access, identity, observability tooling, cost tracking, while explicitly preserving domain-team autonomy over the actual agent logic, evaluation criteria specific to their use case, and day-to-day iteration speed. That's a genuinely harder organizational design to get right than either extreme, because it requires the platform team to resist the natural pull toward expanding its scope and control over time, which most platform teams, left unchecked, tend to do anyway.
The Riyadh Angle: Centralization Pressure in a Government-Led Digital Push
Saudi Arabia's digital transformation is unusually centralized by design, driven top-down through Vision 2030 mandates, the Digital Government Authority, and large state-backed entities coordinating technology strategy across sectors in a way few other markets attempt at this scale. That creates a specific pull toward over-centralizing AI platform decisions that's worth naming directly, because it's structurally different from the pressure seen in more organically-grown tech markets.
A national or giga-project-level platform team overseeing AI initiatives across multiple domains, healthcare, logistics, financial services, faces real temptation to standardize everything centrally, given the top-down mandate driving the whole initiative in the first place. That instinct makes sense for model access, identity, and observability tooling. It works far less well applied to domain-specific agent logic, where a healthcare use case and a logistics use case genuinely need different reasoning patterns, different evaluation criteria, and different domain expertise that a single centralized team is unlikely to hold deeply across every sector simultaneously.
An AI development company in Riyadh advising on this kind of large-scale platform buildout is well positioned to push back on full centralization specifically because of that structural pressure, arguing for the shared-infrastructure-with-preserved-autonomy split even when the natural instinct in a top-down digital strategy environment leans toward centralizing everything by default. Toadster's view on this, shaped by watching similar centralization patterns play out elsewhere, is that the platform teams succeeding in these large national-scale initiatives are the ones drawing a clear, deliberate line between what genuinely needs central control and what needs to stay close to the domain team actually solving the problem.
A Reasonable Starting Split
Centralize anything where inconsistency creates real risk or waste: model access and vendor management, identity and permissions, observability standards, and cost attribution methodology. Leave domain-specific agent logic, tool selection for a particular workflow, and evaluation criteria tailored to a specific use case with the teams who actually understand that business problem deeply. Revisit the split periodically, because the right balance shifts as an organization matures. What needs central control at five agent projects is different from what needs it at fifty, and neither the fully centralized nor fully decentralized instinct scales cleanly on its own past a certain point.
FAQ
Should every AI agent project go through a centralized platform team?
Not for everything. Shared infrastructure like model access, identity, and observability benefits from centralization. Domain-specific agent logic and evaluation criteria usually work better when left with the team that actually understands that business problem.
What's the biggest risk of over-centralizing AI platform engineering?
Slowing down domain teams to the point where they start building workarounds outside the official platform, which quietly recreates the fragmentation the centralization was meant to prevent, while adding unnecessary process overhead on top.
How do I know if my organization needs a dedicated AI platform team yet?
If you have more than two or three teams independently building agent infrastructure, especially around model access, identity, or observability, that's usually the signal it's worth centralizing the shared pieces before the duplication and inconsistency compound further.
What should stay decentralized even after a platform team is established?
Domain-specific agent reasoning and tool logic, along with evaluation criteria that reflect what "good" actually looks like for a specific business use case, should generally stay with the team closest to that problem.
Top comments (0)