DEV Community: Azrty

Confidential computing AI inference in the UAE: GPU TEEs

Azrty — Wed, 28 Oct 2026 05:34:39 +0000

Residency says where your data sits. Confidential computing says who can read it while the model runs. Inference on a rented or colocated GPU in a UAE data centre gives you the first and not the second. Here is what a GPU TEE protects, what it costs, and what to demand from a vendor.

If a vendor tells you "your data stays in-country", they are answering a question about geography. They are not answering the question your risk register actually asks: who can see the data while it is being used? Confidential computing for AI inference in the UAE is the answer to that second question. It runs the model inside a hardware trusted execution environment (TEE) on the GPU itself, so the data centre operator, their engineers and anyone with hypervisor or management-plane access cannot read your prompts, your retrieval context or your model weights while they are in use. Between the prompt arriving and the answer leaving, that data otherwise sits in plaintext in server RAM and GPU memory. On a rented or colocated GPU in a UAE data centre, it is all in principle readable by the host.

The GPU TEE is the technology that closes this gap, and it is now practical: NVIDIA has shipped it since the H100, the tooling is public, and the measured overhead for inference sits in the low single digits on a well-configured stack. This article sets out what a GPU TEE protects, the stack to demand from a vendor, the performance numbers to budget for, and the answers that should already be in your procurement and due-diligence packs.

What does confidential computing for AI inference mean in the UAE?

Residency answers where. Confidentiality answers who.

Data residency is about where data is stored and processed. Confidentiality is about which people and processes can observe data in use. They are different controls, and in the UAE the second is now the harder of the two.

The regulatory backdrop pushes regulated organisations towards in-country processing but says much less about in-use confidentiality:

UAE Central Bank outsourcing rules require banks to keep regulated data, including the master record, within the UAE, and set due-diligence expectations covering data centre location, encryption and access control. Health-sector rules point the same way.
DIFC Regulation 10, part of the Data Protection Regulations enacted on 1 September 2023, governs the processing of personal data through autonomous and semi-autonomous systems, meaning AI. It drives an AI data protection impact assessment, a risk register and certification for high-risk processing, with enforcement and audit expectations running through 2026 (summary of the current enforcement position).
In Saudi Arabia, SDAIA's personal data protection regime and the Regulation on Personal Data Transfer Outside the Kingdom make cross-border movement of personal data the exception rather than the rule.

None of this prevents the operator of your in-country GPU from reading your prompts. An AI DPIA or a Central Bank outsourcing assessment that stops at "the data centre is in Dubai" has answered the easy half of the question. When an auditor or counterparty asks "who inside the hosting arrangement can access this data, and how do you know?", residency alone has no answer. That is the gap a TEE fills.

What does a GPU TEE actually protect?

The NVIDIA H100 was the first GPU with hardware confidential computing, and the feature now spans the Hopper and Blackwell generations. When the GPU boots in CC-On mode, a chain of trust is established: the GPU firmware performs a secure, measured boot from an on-die hardware root of trust; the driver running inside a CPU confidential VM establishes an SPDM session with the GPU; and the GPU produces a cryptographically signed attestation report that you verify before trusting it. All data crossing the PCIe bus between the CPU TEE and the GPU passes through encrypted bounce buffers, and command buffers and kernels are encrypted and signed in transit. CUDA applications run unchanged, as NVIDIA documents in its H100 confidential computing technical blog.

Two details decide whether the control is real:

The CPU TEE is not optional. A GPU in CC mode is only meaningful when the driver runs inside a CPU confidential VM: Intel TDX (Emerald Rapids or Granite Rapids) or AMD SEV-SNP (Milan, Genoa or Turin). The NVIDIA Confidential Computing Deployment Guide (v7.1, April 2026) lists the supported combinations, including Blackwell single and multi-GPU modes and Hopper multi-GPU Protected PCIe mode. A vendor offering "GPU confidential computing" without a CPU TEE is offering the shell without the contents: prompts are still plaintext in host memory.

CC mode is a state you can check. Inside the confidential VM it is queryable:

nvidia-smi conf-compute -f    # CC status: ON
nvidia-smi conf-compute -e    # CC Environment: PRODUCTION
nvidia-smi conf-compute -q    # CPU CC Capabilities: AMD SEV-SNP or INTEL TDX

Protected: prompts, retrieval context, KV cache and model weights while in use, against the host, the hypervisor and anyone with physical or administrative access outside the measured TCB. Not protected: the GPU's HBM, which NVIDIA does not encrypt because it is deemed secure against everyday physical attack tools, and network transit, which remains your own TLS concern. The GPU also refuses work until attestation has passed and the ready state is set, which is the enforcement hook that makes the architecture auditable.

What should you ask a vendor for? Composite attestation, then gated key release

Attestation is where most deployments quietly fail. The pattern that actually delivers confidentiality has two gates, and both must be verified, not assumed.

Gate 1: composite attestation of CPU and GPU. The CPU TEE proves it is a genuine SEV-SNP or TDX confidential VM with the expected firmware. The GPU proves it is genuine NVIDIA silicon in CC mode with approved driver and VBIOS versions. NVIDIA's Remote Attestation Service (NRAS) is the primary verifier for the GPU side, with a local verifier for air-gapped deployments. The attestation quick start shows how little code this takes:

from nv_attestation_sdk import attestation

client = attestation.Attestation()
client.add_verifier(attestation.Devices.GPU,
                    attestation.Environment.REMOTE,
                    "https://nras.attestation.nvidia.com/v4/attest/gpu", "")
result = client.attest(client.get_evidence())
token = client.get_token()  # JWT with per-GPU claims

The token's claims are concrete and pin-able: driver version, VBIOS version, secure boot state, certificate revocation status. Your release policy should pin the ones you care about, so a driver upgrade changes the measurement and fails closed until you approve it.

Gate 2: attestation-gated key release. Model weights ship encrypted at rest (AES-256-GCM is the standard pattern), and the data encryption key is wrapped by a key encryption key held in an HSM-backed KMS. The KMS releases the key only to a workload that presents a passing attestation. Microsoft's confidential LLM inferencing tutorial documents the full build on an AMD SEV-SNP VM with an H100, including the Key Vault release policy:

{
  "version": "1.0.0",
  "anyOf": [
    {
      "authority": "https://sharedeus2.eus2.attest.azure.net",
      "allOf": [
        { "claim": "x-ms-isolation-tee.x-ms-attestation-type", "equals": "sevsnpvm" },
        { "claim": "x-ms-isolation-tee.x-ms-compliance-status", "equals": "azure-compliant-cvm" }
      ]
    }
  ]
}

In that pattern the CPU evidence gates the KMS, and the GPU is verified in-guest before any key is used. The composition matters: no verified CPU measurements, no key; no verified GPU state, no model. A vendor that cannot walk you through both gates, with the evidence artefacts, is selling a feature flag, not a control.

What does confidential inference cost in performance?

The honest performance story is two-sided. Inside the GPU nothing changes: raw compute and HBM bandwidth are at par with non-confidential mode. The cost sits on the interconnect. NVIDIA measures the encrypted CPU to GPU path at roughly 4 GB/s, limited by CPU encryption performance, so workloads that are compute-heavy relative to data movement amortise well, while workloads that hammer host-to-device transfers or serve short, prefill-heavy exchanges pay more. NVIDIA's own published CC-mode overhead for LLM inference is in the low single digits, roughly 2 to 5%. Independent H100 measurements give you the range to plan against:

Study	Platform	Models	Headline CC-mode result
Zhu et al. (2024)	AMD SEV-SNP (EPYC 9V84) with H100 NVL, vLLM 0.5.4	Llama-3.1-8B, Phi-3-14B, Llama-3.1-70B (4-bit)	Throughput overhead under 7%, below 5% for typical queries, near zero for the 70B model; TTFT overhead up to 19% on the 8B model
H100 under Intel TDX benchmark (2026)	Intel TDX confidential VM, 1x H100 80GB	Mistral-7B, Qwen3-30B-A3B	TTFT up 21.8% and 27.8%; global token throughput down 17.7% and 21.1%; closed-loop throughput gaps of 11.5 to 20.2%

Three operational readings fall out of these numbers:

Sizing, not feasibility, is the issue. The TDX study's own guidance is to reserve 15 to 25% extra throughput capacity on Hopper-class single-GPU deployments and validate with your own traces, because the penalty is stable but the saturation knee arrives earlier: Qwen3-30B-A3B saturated at concurrency 32 in CC mode versus 40 without it.
Decode-heavy long-context serving degrades differently from raw compute. Inter-token latency moves a few percent; it is time to first token and aggregate prefill throughput that absorb the encrypted interconnect. Short, prefill-heavy exchanges are the worst case, and longer decode phases amortise the cost.
A datasheet percentage is not your number. Published low-single-digit overheads hold when the serving stack is tuned for CC mode. Measure TTFT, throughput and the saturation concurrency on your own prompt lengths and concurrency mix before you size the estate.

Worked example: a long-context assistant on colocated H100s in Dubai

Take a Dubai-based insurer deploying a claims-summarisation assistant: a Qwen3-30B-class model on a colocated 8-GPU H100 node, prompts of a few thousand tokens of policy and claims text, outputs around a thousand tokens. The colocation provider's staff hold host access, which is precisely the threat a TEE addresses, and the model weights are commercially licensed, so they are encrypted at rest with the data encryption key wrapped in the insurer's HSM.

Sizing, using the TDX benchmark's single-GPU figures scaled across the node:

Non-confidential baseline: about 994 tokens/s aggregate per GPU, so roughly 7,950 tokens/s for the node.
CC mode: about 820 tokens/s per GPU, so roughly 6,560 tokens/s for the node.
Peak-hour demand of 7,000 tokens/s fits the non-confidential node and does not fit the confidential node.

The insurer has three options, all arithmetic on the same 21% gap: run a second node during peak windows (more GPU-hours, but no queueing), accept that peak concurrency caps earlier (the knee at 32 instead of 40), or trim context length. None of them is a reason to skip confidentiality. All of them belong in the capacity plan before the pilot, not after the first month of queueing complaints.

Latency is the smaller problem: at the study's 5 requests/s, TTFT rises from roughly 395 ms to roughly 505 ms, which no internal user will notice. The operational surprises are elsewhere. The attestation handshake is a one-time startup event, so it adds nothing to per-request latency, but a confidential VM cannot simply reboot: the deployment guide notes that rebooting terminates the VM under CPU CC limitations, and the GPU scrubs its state on reset. Restarts are measured in minutes, so autoscaling and maintenance windows need designing around that. Driver persistence mode must be enabled or the SPDM session tears down and the GPU needs a full reset before it will work again. And every driver or VBIOS upgrade changes the attested measurements, so your release policy must be re-pinned deliberately, in a change window, rather than drifting with apt upgrade.

Finally, confidentiality only holds end to end: the application inside the enclave must not be quietly calling a third-party moderation, guardrail or embedding API, because that exports every prompt out of the TEE in plaintext. The enclave boundary is where your architecture review starts.

What do GPU TEEs not fix?

Anti-pattern	Why it fails
GPU CC mode without a CPU TEE	Prompts and driver state sit in plaintext host memory; the host can read them
CC-DevTools left enabled	Matches CC workflows with security protections disabled and performance counters on; it is a debug mode, not a deployment mode
Attestation performed but not enforced	If the KMS hands the model key to any booted VM, the measurement exercise is theatre
Verifying the CPU but not the GPU	Half a composite proof is no proof; the Microsoft pattern verifies both before it unwraps keys
Third-party API calls from inside the enclave	Confidential computing does nothing for prompts that leave the TEE to an external service
Unpinned TCB	Without version pinning in the release policy, a routine driver update silently changes the trust boundary
Assuming HBM is encrypted	It is not; it is deemed secure against everyday physical attack tools, which is a different claim

Two further limits deserve plain statement. First, model weight licensing is a contract question a TEE does not touch: running a model inside an enclave does not create rights the licence does not grant, and some provider licences restrict deployment context, tenancy or geography regardless of the hardware. Second, attestation introduces a dependency you must own: NRAS is an online NVIDIA service, with local verification available for air-gapped estates, and the revocation and reference-manifest data behind local verification can go stale. If your threat model includes a fully offline deployment, test that path specifically.

Which questions belong in the procurement pack?

When a provider claims "confidential AI" or "your data never leaves your control", these questions separate the two tiers:

Question	What a good answer looks like
Which CPU TEE, which GPU, which modes?	Named SKUs: Intel TDX or AMD SEV-SNP with Hopper or Blackwell GPUs in CC mode, per the supported combinations in the deployment guide
Show me the attestation flow	Composite evidence: CPU TEE report and GPU attestation token, plus the policy that pins driver, VBIOS and firmware versions
What gates the model key?	HSM-backed KMS with a release policy bound to the attested measurements; no passing proof, no key
What is the measured overhead on our workload shapes?	Numbers for TTFT, throughput and saturation concurrency on representative prefill and decode mixes, not a vendor datasheet percentage
What leaves the enclave?	An explicit list of outbound calls from inside the TEE, ideally empty
How do reboots, driver upgrades and node replacement work?	A documented cold-start and TCB-repinning procedure, with expected timings

What to do next

Split your AI estate into two tiers. Tier one: workloads where residency is sufficient. Tier two: workloads where prompts or weights are confidential (customer data, regulated data, proprietary models), which require a CPU TEE plus GPU CC mode, composite attestation and attestation-gated key release. Write the standard once and apply it in due diligence.
Benchmark before you size. Reserve 15 to 25% extra capacity on Hopper-class hardware as a planning heuristic, then measure TTFT and the saturation knee on your own prompt lengths. Budget for the interconnect, not for the compute.
Put the enclave in the compliance paperwork. An attestation report is exactly the kind of artefact an AI DPIA, a risk register or an outsourcing due-diligence file under DIFC Regulation 10 or Central Bank expectations is short of today. "We can show the enclave, and here is the evidence" is becoming a procurement differentiator in this region.
Review the application, not just the platform. Audit what the code inside the TEE calls out to, and pin TCB versions in a managed change process.

At Azrty we design, build and run this kind of AI infrastructure for organisations in the UAE: the GPU platform, the gateway in front of every model, and the access and control layer around both (AI infrastructure). For teams that need agents and models running on their own GPU servers, each workload in its own isolated space and all managed from one console, our Kuvryn AI platform is built for that shape of deployment. We are based in Dubai, and our own team does the deployment and the support.

If you take one thing from this article: next time a vendor says your data stays in-country, ask them who can see it while it is in use, and ask for the attestation evidence. The hardware can now answer that question. Whether your vendor's stack does is a different matter, and it is entirely checkable.

Originally published on Azrty.

Using AI in a Dubai clinic: what DHA rules allow, and what they require

Azrty — Wed, 28 Oct 2026 05:14:40 +0000

Clinic owners in Dubai ask us the same question every month: can we use AI for bookings, reminders, insurance pre-authorisation and doctors' notes without falling foul of the Dubai Health Authority (DHA)? The answer is yes, and the rules that shape the project are clearer than most people expect. Four requirements do most of the work: Arabic for anything the patient sees, a licensed clinician's signature on anything clinical, health data that stays inside the UAE, and audit logs you can produce when an inspector asks. This piece sets out what the documents actually say, picks the three uses worth starting with, and lists the questions to put to any vendor.

The rulebook is clearer than you think

There is no single "DHA AI licence" for private clinics. What exists is a small stack of documents that, read together, tell you exactly where the lines are. DHA takes AI seriously on its own side of the fence: it has a Chief Artificial Intelligence Officer, Fatima Al Khaja, and during Dubai AI Week in April 2025 it launched an AI training programme covering everything from an executive diploma with ESCP Business School to prompt-engineering training, while encouraging private healthcare facilities to participate. A month later it put AI-driven privacy monitoring, Imprivata's Patient Privacy Intelligence, inside the NABIDH platform itself, to detect unauthorised access to patient records in real time.

Here is the stack, and what each layer means for your project:

Document	Who it binds	What it says that touches AI
DHA Artificial Intelligence Policy	DHA staff, contractors and vendors; the clearest available statement of the regulator's expectations	Arabic for patient-facing AI, clinician verification of AI clinical content, UAE data centres for AI inputs and outputs, "default deny" for new tools
DHA Telehealth Standards v2.0, January 2024	Licensed telehealth providers	Patient records on Dubai-based servers, bilingual (Arabic and English) patient information, consent obtained by the physician, NDAs for staff with data access
DHA Patients' Rights and Responsibilities Policy v1.4, June 2023	All DHA-licensed facilities	Patient data protection in line with Dubai Law No. 26 of 2015 and Dubai Law No. 46 of 2006
Federal Law No. 2 of 2019 on ICT in health	All health ICT in the UAE, including free zones	Health data must not leave the UAE without a health authority resolution; fines of AED 500,000 to 700,000 for breach; minimum 25-year record retention
NABIDH Policies and Standards, Audit Policy	Facilities connected to NABIDH	Privacy and security audits, audit logs kept 3 years, quarterly log reviews, annual external audits

A note on scope, because it matters: the DHA AI Policy formally governs how DHA itself and its vendors use AI, not a licensing regime for private clinics. But it is the regulator's own standard, it is unusually specific, and it is what any vendor building for you will be measured against the day a private-sector equivalent is formalised. Treat it as your design brief.

Four rules that shape any clinic AI project

Rule 1: Arabic for anything the patient sees

The DHA AI Policy, Article 8.3, requires patient-facing AI interfaces to display information in Arabic, with support for text recognition and cultural context alignment. The Telehealth Standards go further for anything remotely clinical: physician credentials must be available in both Arabic and English (Appendix 5.2.5), and any information a patient needs before a virtual consultation must be provided in both languages (Appendix 5.2.6).

What this means in practice: your WhatsApp booking agent defaults to the patient's language, with proper Arabic (right-to-left rendering, formal register, correct transliteration of clinic names) as a first-class output, not a machine-translated afterthought. Test it with Arabic-speaking front-desk staff before any patient sees it. A 2026 Dubai practitioner guide makes the same point bluntly: patient-facing AI in Dubai requires Arabic support, and generic translation is where most implementations fail.

Rule 2: AI clinical text is a draft until a clinician signs it

Article 8.4 of the DHA AI Policy is unambiguous: any AI-generated clinical content must be clearly labelled, and reviewed and verified by a qualified licensed healthcare professional. Article 3.1 reinforces the point: AI is a supportive tool, and liability does not transfer to it.

This is the rule that makes the "AI doctor's notes" project safe if you design it correctly. The note the model drafts is a draft. It sits in a pending_signoff state, with the model name, prompt version and reviewer recorded. It does not sync to NABIDH, does not go to an insurer, and does not appear in the patient's file until a licensed clinician has opened it, corrected it and signed it. That workflow is not a burden; it is the regulatory argument for the use case.

Rule 3: health data does not leave the UAE

Two layers here. Article 8.7 of the DHA AI Policy requires that all AI inputs and outputs, including patient data, be stored and processed exclusively within UAE data centres. Above that sits Federal Law No. 2 of 2019: Article 13 bars health data related to services provided inside the UAE from being stored, processed, generated or transferred outside the country except by resolution of the health authority in coordination with the Ministry. Article 24 sets the penalty for breaching Article 13 at AED 500,000 to 700,000, and Article 20 sets minimum record retention at 25 years. The Telehealth Standards add the Dubai-specific layer: the telehealth platform must be capable of storing all patient records on Dubai-based servers (Appendix 7.1.6).

The practical consequence for your architecture: nothing carrying a patient name, Emirates ID, diagnosis or lab result goes to a hosted model endpoint outside the UAE. Not in the prompt, not in the logs, not in the vendor's training pipeline. Ask for region pinning at the API level, and ask what happens to prompts after processing.

Rule 4: log everything, and keep it for the auditor

The NABIDH Audit Policy requires audit logs to be retained for 3 years and reviewed at least quarterly to detect improper use of patient information, with external audits at least annually. The log must be able to answer specific questions: which users accessed a given patient's record, which patients' data a given user accessed, every break-glass event, and every access to data marked sensitive. DHA audits the NABIDH platform and its vendors annually, and since April 2025 the platform itself watches for anomalous access patterns with AI.

Your AI systems should sit inside the same logging discipline. Log the generation event (case ID, model, reviewer, sign-off state), the consent event, and every export.

Two governance notes round this out. Article 4.2 of the AI Policy is a "default deny": AI tools are denied by default until a compliance assessment is completed, so keep a one-page assessment on file for every tool you switch on, including the ones your doctors install themselves. And Article 8.2 requires explicit patient consent where patient data is shared for AI-driven diagnostics or telehealth, explaining how the AI will be used, its risks and its limitations. Your intake form gets a new paragraph.

The three uses worth starting with

Not every AI use case carries the same regulatory weight. Here are the three that pay for themselves fastest under these rules, in the order we recommend:

Use case	Arabic required	Clinician sign-off	Data involved	Regulatory risk	Where the payback is
Bookings and reminders (WhatsApp, SMS)	Yes, patient-facing	No, but symptom escalation rules needed	Name, phone, appointment type	Low	Recovered no-shows, front-desk hours
Insurance pre-authorisation packets	Internal tool, but insurer forms are often bilingual	Coordinator reviews; clinician confirms clinical content	Clinical notes, lab results, procedure codes	Medium	Approval speed, denial rate, coordinator hours
Draft consultation notes	Arabic for patient letters; English acceptable internally	Yes, licensed clinician signs	Full clinical record	Medium-high	Doctor minutes per consultation

Start with reminders: lowest risk, and the numbers move immediately. Pre-authorisation is the strongest first pilot for a clinic with meaningful insurance volume: document-heavy, high-volume, financially significant, and it still allows human review before anything is submitted, which is how one Dubai healthcare vendor frames it in its published guidance on agentic AI for Dubai hospitals and clinics. Notes come third, once the audit trail and sign-off workflow are proven on lower-stakes work.

Worked example: pre-authorisation at a four-doctor clinic

Take a typical outpatient clinic: two GPs, two specialists, roughly 1,100 consultations a month. About 60 per cent of visits are insurance-covered, and on a typical outpatient mix (imaging, minor procedures, some referrals) around 160 cases a month need pre-authorisation.

Baseline. A revenue-cycle coordinator handles each pre-authorisation manually today: pulling the doctor's notes, attaching lab results, completing the insurer's form, checking the document list and chasing missing attachments. Assume 55 minutes of human work per request. That is 160 x 55 = roughly 147 hours a month, close to a full coordinator FTE.
With the agent. The agent assembles the packet: it reads the consult note and the requested procedure, checks that insurer's document requirements, verifies that the lab results and imaging reports are attached, and flags what is missing. The coordinator reviews and submits. Assume 15 minutes of human review per case, in the same range as the vendor scenario where preparation drops from up to 48 hours to around 15 minutes. That is 160 x 15 = 40 hours. About 107 hours a month released.
Value the time. Assume a coordinator at AED 12,000 a month, loaded at 1.4x for benefits and overhead, over roughly 170 working hours: about AED 100 an hour. 107 hours x AED 100 = AED 10,700 a month, or about AED 128,000 a year of recovered capacity.
Denials. Assume 12 per cent of submissions bounce today for documentation gaps: about 19 cases a month. If the agent's completeness checks cut that to 6 per cent, about 10, then 9 cases are saved. At 2.5 hours of rework each, that is another AED 2,250 a month. Each bounced case also delays a typical AED 1,200 reimbursement by about 30 days, so 9 cases is AED 10,800 of cash pulled forward.
Total. Around AED 13,000 a month in measurable time and rework, plus roughly AED 11,000 of reimbursement permanently one month earlier. A 2026 Dubai guide prices a focused single-workflow pilot in the market at roughly AED 150,000 to 300,000 for a proof of concept, so the measurable payback lands at 12 to 24 months, before you count the softer wins: patients who do not abandon procedures because approval took three days instead of a fortnight.
Layer on reminders. The same platform runs the reminders agent. At a 15 per cent no-show rate, 1,100 visits produce about 165 missed appointments a month. Recovering just 20 per cent of them, at an average AED 350 per visit, is 33 recovered visits: AED 11,550 a month, about AED 139,000 a year, from the lowest-risk use case on the list.

The agent never decides clinical necessity, never invents justification, and never submits anything. It gathers, sorts and checks; a human submits. That is what keeps it on the right side of Article 8.4.

An architecture that passes inspection

Two technical decisions carry most of the compliance load: where the model runs, and what you log.

Routing. Put a gateway in front of your models so that patient-identifiable traffic is pinned to UAE-hosted infrastructure and nothing else. Our FastLLM Proxy does exactly this: an OpenAI-compatible gateway with routing, budgets and access control in one place. The configuration principle, whatever gateway you use:

# Illustrative gateway policy: patient data never leaves the UAE zone
routes:
  - id: preauth-drafting
    match:
      header:
        x-data-class: phi          # set by the integration layer
    providers:
      - name: uae-selfhosted-llm
        endpoint: https://llm.clinic.internal/v1
        model: qwen2.5-14b-instruct  # UAE data centre, self-hosted
    fallback: reject                 # no external fallback for patient data, ever
    log: full                         # prompt + completion retained 3 years
    budget:
      monthly_usd: 400

  - id: marketing-copy
    match:
      header:
        x-data-class: public
    providers:
      - name: hosted-llm             # fine: no patient data on this route
    fallback: reject
    log: minimal

Audit records. Every AI generation tied to a patient case should produce a record an inspector can read:

{
  "event": "ai_generation",
  "workflow": "preauth_packet",
  "case_id": "OPD-2026-09127",
  "actor": "agent:preauth@clinic",
  "model": "uae-selfhosted/qwen2.5-14b",
  "output_status": "draft_pending_signoff",
  "reviewer": "revenue.cycle@clinic",
  "consent_ref": "CN-2026-04-118",
  "timestamp": "2026-10-26T08:14:22+04:00"
}

Add the two labels the rules ask for: AI-generated clinical content carries a visible "AI draft, pending clinician review" tag until it is signed, and consent is captured per Article 8.2 with an explanation of how the AI is used.

What to ask a vendor

Print this section and take it to your next vendor meeting:

Where, at the API level, is our data processed? Give us the region pinning, not the marketing page.
Do you train on our prompts or outputs? Where do they go after processing, and for how long?
Show us the Arabic output. Not a translation API demo: the actual booking and reminder flows, reviewed by native speakers, with right-to-left rendering in WhatsApp.
How does the sign-off workflow work? Can a draft reach a patient file, NABIDH or an insurer without a licensed clinician?
Export your audit logs in a format we can hand to an auditor: who accessed which record, when, and what changed.
What happens when you change the model or prompt version? Will you notify us, and can we pin versions?
Which of your deployments have been through a DHA or NABIDH-related audit? Can we speak to a reference?
Will you sign a data processing agreement with UAE data residency, a breach notification commitment, and alignment with our 25-year record retention obligations?

If a vendor cannot answer 1, 4 or 8 without checking with someone, they are not ready for a Dubai clinic.

What to do next

Pick one workflow, not four. For most clinics that is the reminders agent: low risk, quick to build, immediate numbers. In parallel, write the one-page compliance assessment per tool, add the AI paragraph to your intake consent, and set audit log retention to at least 3 years. Run a 4-week pilot with fixed baseline numbers (no-show rate, coordinator hours, denial rate), then expand to pre-authorisation and notes on the same platform and the same logs.

This is work we do with clinics in the UAE from our base in Dubai: our own team handles the consulting, the deployment and the support. If you want the architecture and the agents built on your infrastructure with these controls from day one, that is our AI engineering service. Start with one workflow, prove it, and let the audit trail you build there carry the next three.

Originally published on Azrty.

AI document processing for freight forwarders: bills of lading, invoices and the customs sign-off

Azrty — Wed, 28 Oct 2026 05:14:39 +0000

A Dubai forwarder's AI project that pays for itself: read the document pack into structured records, flag what does not reconcile, and keep the customs declaration with a licensed human. Here is how to scope it, what it costs per page, and what accuracy to demand before you go live.

Freight forwarding in the Gulf is document work wearing a logistics badge. Every consignment arrives as a pack: commercial invoice, packing list, bill of lading or air waybill, certificate of origin, delivery order, and finally a WhatsApp photo of the signed delivery note. Someone re-types the fields your systems need; a licensed person files the customs declaration. Reading that pack into structured data is the clearest AI return a forwarder can get this year: it removes typing hours per consignment, catches mismatches before they reach customs, and leaves the legally sensitive step with a person. Vendors are pitching exactly this across Dubai in 2026, from document automation to WhatsApp status bots (Soluvide, Runbook). This piece covers the pipeline, the engines and their published per-page prices, the accuracy bar to demand before go-live, how "where is my shipment" gets answered from live data, and a scope you can hand straight to an IT partner.

What AI reads, and what a person still signs

"Reading" a shipping document is three steps, and the second one is where the money is:

Extraction. Pull the fields: shipper, consignee, container or AWB number, description, weights, values, incoterms, HS codes where present. Every document type has its own schema. A bill of lading carries carrier, ports and references; a commercial invoice carries lines, quantities and values.
Validation. Check what was extracted against what you already know. Does the container number on the B/L match the packing list? Does the consignee match the customer record? Do the line totals add up? Is the HS code plausible for the description?
Routing. Clean records go into your shipment record or TMS. Anything that does not reconcile goes to a review queue, with the discrepancy highlighted and the source page attached.

Extraction alone only moves errors faster. Extraction plus validation catches the mismatched container number before it reaches the declaration, which is where the cost lives: a wrong identifier in a customs filing means amendment fees, delay, and often storage or demurrage on top. The DCSA makes the same point about the bill of lading: it is the most important trade document in container shipping, and its electronic B/L standard exists so B/L data flows straight through without manual re-entry.

The boundary is fixed, and vendors who claim otherwise deserve caution. AI prepares and checks data. A licensed person submits the customs declaration and carries the declarant's responsibility, through the relevant system, whether that is Dubai Customs' Mirsal 2 or the equivalents in other emirates and in Saudi Arabia. Check the current guidance from the authority you file with (Dubai Customs). The same rule covers anything with legal weight: issuing the B/L as a carrier, approving invoices and credit notes, committing to delivery dates your data does not support, and signing anything.

The pipeline: extract, validate, flag, approve

The build that works has four stages, and it is mostly unglamorous code:

Intake. Email attachments, portal uploads and WhatsApp media land in one queue. Classify each file: B/L, invoice, packing list, certificate of origin, delivery order, or unknown.
Extraction. OCR the page, then have the model emit a JSON record that conforms to a strict schema. Version the schema and pin the model version: change either and you re-run the test set.
Deterministic validation. Rules in your code, not in the model. Cheap, transparent, and they never hallucinate.
Exception routing. Every field carries a confidence score. Below threshold, or any failed rule, and the pack routes to a person with the source image and the extracted value side by side.

The schema is where you decide what a "field" means. A short Pydantic model for the invoice side of the pack:

from typing import Annotated, Literal
from pydantic import BaseModel, Field, StringConstraints

ContainerNumber = Annotated[str, StringConstraints(pattern=r"^[A-Z]{4}\d{7}$")]

class LineItem(BaseModel):
    description: str
    hs_code: str | None = None          # suggestion only, never final
    quantity: float | None = None
    unit: str | None = None
    net_weight_kg: float | None = None
    gross_weight_kg: float | None = None
    unit_value: float | None = None
    currency: Literal["AED", "USD", "EUR", "GBP", "CNY", ""] | None = None

class ShipmentPack(BaseModel):
    document_type: Literal["bl", "invoice", "packing_list",
                           "coo", "delivery_order", "other"]
    bl_number: str | None = None
    container_numbers: list[ContainerNumber] = Field(default_factory=list)
    incoterm: str | None = None          # closed list, checked downstream
    consignee_name: str | None = None
    lines: list[LineItem]
    flags: list[str] = Field(default_factory=list)

Then the checks the model never gets to vote on. Container numbers carry an ISO 6346 check digit, so a wrong digit is provable in ten lines of Python:

def iso_6346_valid(container: str) -> bool:
    if len(container) != 11:
        return False
    vals, v = {}, 10
    for ch in "ABCDEFGHIJKLMNOPQRSTUVWXYZ":
        if v % 11 == 0:      # 11, 22 and 33 are skipped by the standard
            v += 1
        vals[ch] = v
        v += 1
    body = container[:10]
    total = sum(
        vals[c] * (2 ** i) if c.isalpha() else int(c) * (2 ** i)
        for i, c in enumerate(body)
    )
    expected = "0" if total % 11 == 10 else str(total % 11)
    return container[10] == expected

Beyond format checks, cross-document rules do the real work: container numbers must agree across the B/L, packing list and delivery order; parties must match the customer record; line values times quantities must recompute; gross weight must be at least net weight; the B/L date must precede arrival; incoterms must come from the 2020 set and currencies from ISO 4217. Any miss raises a named flag on the pack, and the pack stops.

Choosing the extraction engine

OCR is a commodity. The interesting choice is what turns OCR text into fields. Four viable patterns, at the list prices published on each vendor's pricing page (check them, they move):

Approach	Example tools	Published price	Where it wins	Watch-outs
General OCR plus LLM structured output	Google Document AI Enterprise Document OCR ($1.50 per 1,000 pages) feeding a hosted LLM	roughly $20 per 1,000 pages all in (see the worked example)	messy, varied layouts, five or more document types, bilingual pages	token costs and vendor data terms; keep validation in your code
Specialised invoice parsing	Textract Analyze Expense ($0.01 per page), Google's pretrained invoice parser ($0.10 per 10 pages)	$10 per 1,000 pages	high-volume commercial invoices	tuned on consumer-style invoices, not carrier B/L layouts
Form and table parsing	Textract Forms ($0.05 per page) plus Tables ($0.015 per page), Google Form Parser ($30 per 1,000 pages)	$65 per 1,000 pages for Textract combined	fixed templates, clean scans	brittle across carriers and customers
Open-weights vision models on your own GPUs	vision variants from families such as Qwen2.5-VL, Llama 3.2 and Granite	no per-page fee; GPU and operations cost instead	data locality and PDPL posture	you own the MLOps

The usual right answer for a forwarder is the first row. Document AI and Textract both charge pennies for raw OCR ($1.50 per 1,000 pages for Google's Enterprise Document OCR, $0.0015 per page for Textract's Detect Document Text), and a hosted LLM handles the messy part: carrier-specific B/L layouts, stamps over printed fields, mixed Arabic and English. The specialised parsers are cheaper per page but tuned for invoices, so you end up running them alongside something else anyway. If consignee data cannot leave your estate, run open-weights vision models on your own GPUs and pay in operations instead of per page.

One pattern worth copying: run the system inside your own cloud account, under your name, and go live only after it passes a test set built from your hardest real consignments and frozen before the build starts.

Worked example: 2,500 consignments a month

A mid-sized Dubai forwarder clearing imports through Jebel Ali and Dubai South. Assumptions: 2,500 consignments a month, an average pack of 10 pages, so 25,000 pages a month (300,000 a year), arriving as a mix of clean PDFs, scans and WhatsApp photos.

Model spend. OCR on Google Enterprise Document OCR at $1.50 per 1,000 pages: 25 x $1.50 = $37.50 a month. LLM extraction: roughly 2,000 input tokens per page (OCR text plus schema and instructions) and 800 output tokens, so 50M input and 20M output tokens a month. At $3 per million input and $15 per million output, a typical mid-tier hosted model price: $150 in, $300 out, $450 a month. Total model spend about $490 a month (AED 1,800), under $0.02 per page. The same volume on Textract Forms plus Tables ($0.05 plus $0.015 per page) would be $1,625 a month; the specialised expense parser ($0.01 per page) $250.

Current manual cost. Assume the documentation team spends 12 minutes re-typing and checking each pack: 30,000 minutes, about 500 hours, roughly 3 full-time people. At AED 9,000 a month fully loaded per documentation officer, AED 27,000 a month (about $7,350).

After go-live. Expect 60 to 70 per cent of packs to pass clean if the mix is mostly clean PDFs, fewer if WhatsApp photos dominate; take 60 per cent straight through. The other 40 per cent, 1,000 packs, need about 4 minutes each in the review UI: about 67 hours, 0.4 of an FTE, AED 3,600 a month. Post-go-live total: about AED 5,400 a month (models plus a part-time reviewer) against AED 27,000. The saving is roughly AED 21,600 a month (about $5,900), before you count faster quotes or the wrong-container incidents that never happened. A build costing six figures of dirhams pays back inside a year at those numbers. The review queue never disappears; it gets much shorter.

The accuracy bar: the gate before go-live

Vendors quote "95 per cent accuracy", which means nothing without definitions. Demand these instead:

Field class	Examples	Gate before go-live	Enforcement
Identifiers	container, B/L, AWB, invoice, booking	99 per cent or better exact match on the golden set; 100 per cent format check	any format failure stops the pack
Arithmetic	totals, quantities, weights, values	100 per cent recomputed, no silent tolerance	mismatch means exception
Closed lists	incoterms, currencies, countries, ports	99.5 per cent or better; anything off-list flags	enum failure means exception
Tariff	HS codes	suggestion only, quoting the tariff line	a human classifies, always
Free text	goods descriptions, marks	no numeric gate; confidence threshold	low confidence routes to review

Three definitions make those numbers real:

The golden set. 300 to 500 of your real documents, spanning every intake channel: mostly clean PDFs, a third scans, a fifth phone photos, including the bilingual Arabic-English ones. Include your worst cases, the consignments your team argued about. Freeze it before the build, and re-run it after every prompt, model or schema change, so a second team can be measured on the same set.
Field-level accuracy. Exact match per field per document type, reported separately for identifiers and free text. A 95 per cent blend is a way of hiding a 60 per cent identifier failure rate.
Silent errors. The metric that matters at the end: an extracted value that is wrong, passes every check, and reaches the declaration unflagged. A typo in a goods description is cosmetic. A wrong container number on a customs filing is a claim.

The gate to set before go-live: zero silent identifier errors across two consecutive golden-set runs, and a straight-through touch rate that has stopped improving week on week. Pin model versions, log every approval (who, when, which model version, which prompt), and keep a human sign-off screen between the extracted record and anything that touches customs, a bank or a customer's money.

Answering "where is my shipment?" on WhatsApp

The status question is not a document problem, it is a data problem, and the model's job in it is small. The answer lives in your TMS or freight system and in carrier feeds; the assistant only turns the question into a lookup and the record into a reply. The design that works:

Identity first. The customer's WhatsApp number maps to a customer record; only then may shipment details be shared. Sharing consignee shipment data with an unverified number is a data protection failure, not a bot quirk.
Verified facts only. The assistant replies with milestones your systems actually contain (vessel arrived, clearance in progress, out for delivery, with timestamps and source). It does not invent an ETA the data does not support, and if the customer is chasing a delay or disputing a charge, it hands the conversation to an operator with the history attached.
Tool calls enforced server-side. The customer context is injected in your code, never chosen by the model:

{
  "name": "shipment_status",
  "description": "Return verifiable milestones for one shipment owned by the verified customer.",
  "parameters": {
    "type": "object",
    "properties": {
      "reference": {"type": "string", "description": "BL, booking or PO number"}
    },
    "required": ["reference"]
  }
}

Proactive updates through templates. Milestone messages go out as approved WhatsApp templates on your Business API number. Meta's template rules and the customer service window change, so check the current policy when you scope rather than trusting a vendor's screenshot.

The prerequisite is honest: a status bot is only as current as your shipment events. If milestones live in a shared drive and a WhatsApp group, wiring the bot to live data is the actual project, and the document pipeline above is what fills that data in the first place.

Compliance: PDPL, data residency and the declarant

Three rules keep this build safe in the UAE.

Personal data applies to shipping documents. Consignee names, addresses and phone numbers are personal data, and Federal Decree-Law No. 45 of 2021 (the PDPL, effective 2 January 2022) covers controllers and processors in the UAE and anyone processing the data of subjects inside the State. It imposes purpose limitation, accuracy, security measures and retention limits, and it leaves free zones with their own regimes, such as DIFC and ADGM, so confirm which regime applies to your licence before you pick a model API. Sending consignee records to an overseas model endpoint is a cross-border transfer: prefer in-region endpoints or self-hosted models, and paper the data processing terms before the first page is read.

The declarant stays liable. AI can draft the declaration and check the pack against the documents, but submission and the declarant's responsibility stay with your licensed staff. Put that in writing in the design, and make the sign-off screen impossible to bypass.

Standards pay twice. Align your internal shipment record with the DCSA electronic B/L standard (open APIs for straight-through B/L data, aligned with IMO, ISO and UN/CEFACT). Carrier eBL data then flows into the same record your document pipeline fills, and you enter it once instead of three times.

The scope to hand an IT partner

Copy this, fill the brackets, and you have a pilot scope an engineering team can quote against:

AI document processing for freight forwarding: pilot scope
1. Volume: [N] consignments/month, [M] pages/month, channels: email, portal, WhatsApp.
2. Document types: B/L, air waybill, commercial invoice, packing list,
   certificate of origin, delivery order, declaration draft.
3. Fields: [list per document type; ~40 on the invoice].
4. Extraction: OCR + LLM structured output; schema versioned; model versions pinned.
5. Validation: ISO 6346 check, cross-document container and party match,
   totals recomputed, incoterm/currency/country closed lists, date sanity.
6. Exceptions: named flags; queue UI with source page beside extracted value; SLA [4] hours.
7. Outputs: shipment record into [TMS/FMS]; declaration draft, never submitted; API/CSV.
8. WhatsApp: status lookup on [number]; identity via [CRM]; template list; human hand-off rules.
9. Compliance: PDPL review; data residency [UAE or self-hosted]; retention [X] months; DPAs signed.
10. Acceptance: golden set of 300+ documents frozen before build; gates per field class;
    zero silent identifier errors in two consecutive runs; licensed person submits every filing.

What to do next

Pull 300 real documents across your channels and freeze the golden set. Define your field lists and validation rules. Pick the engine and the data residency. Pilot one lane with the review queue running in parallel with manual entry. Only then widen the funnel and switch on the WhatsApp status replies. Time spent at the front on the golden set and the rules buys a pipeline your documentation team will actually thank you for, and a customs sign-off that never once left human hands.

If you want the review queue and the integration built around your own systems rather than bolted onto a vendor's, that is our kind of work: Azrty builds AI engineering projects on your infrastructure and connects them to your systems, from our team in Dubai, with your developers able to extend what we leave behind. Where several model providers are in play, one gateway in front of them (FastLLM Proxy) keeps routing, budgets and access control in one place instead of scattered across services.

Originally published on Azrty.

AI CV screening in the UAE: write the rules, then let good candidates through

Azrty — Wed, 28 Oct 2026 05:09:39 +0000

A single Dubai job advert can pull several hundred CVs in days, so employers switch on AI screening inside the ATS or buy a GCC assessment tool, often before anyone has written down what a good candidate looks like. The result is a filter that quietly drops the people you needed. Here is how to define the scoring rules first, test them against your past hires, put a person on every rejection near the cut-off, and hold a vendor to the standards the UAE already expects.

A single Dubai job advert can pull several hundred CVs in days. Recruiters in the UAE told the Khaleej Times in 2026 that one vacancy now attracts "several hundred applications within a matter of days", and one founder who spoke to the same publication put numbers on the shift: a role that used to draw 200 applications now draws 2,000. The response has been predictable: switch on the AI screening already sitting inside the applicant tracking system (ATS), or buy an assessment tool built for the Gulf. What far fewer employers do is decide what "a good candidate" means for that specific role, write it down, and check who the software is filtering out. That gap is where good candidates disappear, and where the UAE's data protection law starts to apply to a hiring decision.

What AI CV screening is actually doing to your vacancy

Three layers do the filtering in a typical UAE hiring stack today, and they behave differently. A 2026 review of AI assessment tools for UAE hiring compares the main options now on the market.

Parsing and ranking in the ATS. The careers page or job board feed receives the CV, extracts name, work history, skills and education, matches keywords from the job description and ranks candidates. MakeMyCV's 2026 guide to ATS use in Dubai describes the failure modes plainly: multi-column layouts, graphics, images saved as JPEG, key details in headers and footers, and non-standard headings all break the parse. The important nuance is that an ATS usually does not auto-reject. It ranks, and a CV ranked low is never opened, which has exactly the same effect as a rejection with no human ever making a decision.
Assessment platforms. Tools such as Evalufy sit alongside the ATS and run structured tests (cognitive, behavioural, technical, language) plus AI-scored video responses, producing scorecards before a recruiter spends time.
Conversational screening. Newer GCC-built tools such as OVI's Milo agent run an audio conversation with the candidate against a rubric and score the transcript. Some tools score video, expressions or tone instead. That design choice has legal consequences in the UAE, covered below.

The pressure behind all three is volume against the number of CVs a person can actually read. When a role draws hundreds of applicants and a recruiter gets through perhaps 20 an hour, most CVs are never seen. That is where the cost of a wrong rejection rises: the candidate you filtered out takes an offer elsewhere and you never know you lost them.

There is a quieter problem too. Avinav Nigam of TERN Group told the Khaleej Times that many screening systems "prioritise keywords over actual capability", so a candidate who knows how to optimise a profile for an algorithm outranks someone better at the job. If your screening is still a keyword match against the job description, you are selecting for CV-writing skill. Semantic matching helps; it does not fix the underlying issue, which is that nobody has defined what good means, so the tool infers it from the advert.

The UAE rules that already apply to an automated rejection

The UAE has no separate "AI hiring law", but two existing instruments cover most of what matters, and a third is worth following as the standard for how to evaluate these tools.

Federal Decree-Law No. 45 of 2021 on Personal Data Protection (the PDPL). In employment, a rejection is a decision with real impact on a person, so treat the screening step as automated processing of personal data with legal consequence. Three articles matter:

Article 13 gives the data subject the right to know the types of data processed, the purposes, and the decisions made based on automated processing, including profiling.
Article 18 gives the right to object to decisions resulting from automated processing that have legal impact or adversely affect the person, and Article 18(4) obliges the controller to include a human element in reviewing those decisions at the data subject's request.
Article 21 requires a data protection impact assessment before processing that involves a systematic and comprehensive assessment of people using automated processing or profiling where there are legal consequences or serious impact, and requires you to review that assessment regularly.

In practice that means three duties before you enable auto-scoring on a live vacancy: tell candidates the software is used and how it works, keep a working human review route, and run a DPIA you revisit whenever the rules change.

Federal Authority for Government Human Resources (FAHR) guidance. FAHR's guidelines on the use and evaluation of AI in talent assessment, produced with SHL, are written for federal government entities but are the clearest UAE-side statement of how to evaluate an AI assessment. The headline position is that no AI assessment should make a decision without human oversight. The framework turns on six questions (training data, transparency, bias, validity, finality, candidate disclosure) and flags three risk classes: legal risk from data protection and bias; public relations risk from candidates who feel the process was unfair or intrusive; and effectiveness risk from a tool that does not work on your population. Those six questions are the spine of the vendor checklist below.

Emiratisation. AI can rank CVs; it does not understand Emiratisation obligations, visa eligibility or MOHRE reporting. As Innovations Global's 2026 employer guide notes, firms with 50 or more employees face recurring penalties for unfilled Emirati quota positions, and that exposure does not disappear because sourcing was automated. Screening rules that penalise career breaks or non-linear CVs can quietly work against Emirati candidates. Check.

Write the scoring rules before you switch the tool on

The scoring rules belong to the hiring manager, not to the vendor and not to HR alone. Software can only implement what somebody wrote down. Here is the minimum artefact you need, in the order you need it.

1. Gates before weights. Split criteria into pass/fail gates (eligibility, licence requirements, minimum experience, availability) and weighted criteria (depth of relevant experience, domain, named systems, evidence of results). A weighted score should never compensate for failing a legal or eligibility gate.

2. Weights that add to 100, in a file anyone can read. Keep the rubric as configuration, versioned per role, so you can show a candidate or a regulator exactly what was applied. Example:

# screening-rules.yaml: Fleet Operations Supervisor, Dubai
gates:
  - eligibility: visa_status in [residence, golden, transferable] or uae_national
  - availability_within_days: 30
  - relevant_experience_years: ">= 2"
  - cv_parse_quality:
      rule: ">= 0.95"
      on_fail: manual_queue   # never auto-reject on a bad parse
weights:
  relevant_experience_years: 25
  freight_or_3pl_last_role: 20
  named_wms_tms_systems: 15
  measurable_results_in_cv: 15
  currently_based_in_uae: 15
  tenure_stability: 10
cut_bands:
  auto_advance_min: 70        # person still confirms
  human_reads_every_cv: 55    # 55-69: a person reads it
  auto_reject_below: 55
audit:
  random_reject_sample: 0.10
  recalibrate_if_override_rate_above: 0.10
  appeal_sla_working_days: 5
  appeal_mailbox: "careers-review@yourcompany.ae"

3. Reason codes, not just scores. Every CV should carry a score plus the reason codes that produced it (missing: named systems; gap: no logistics experience in the last role). This is what lets you answer an appeal, satisfy Article 13, and spot a systematic error such as the parser failing on Arabic-language CVs.

4. Candidate disclosure. Put it on the application form, in English and Arabic:

When you apply, our recruitment software scores your CV against criteria set by the hiring manager for this role. Our recruitment team reviews the results. You can ask a person to review your application at any time by writing to careers-review@yourcompany.ae.

5. A cut-off band, not a cut-off line. The single most important design choice. Any candidate within a defined margin of the cut gets a human read of the full CV. Rejections far below the cut can be automated with reason codes and sampling; rejections near the cut cannot. This is what "letting good candidates through" means in operational terms.

Worked example: 412 CVs for one Dubai logistics role

A Dubai logistics employer posts a Fleet Operations Supervisor role. 412 CVs arrive in 9 days. The numbers below are illustrative; the sequence is the part that matters.

Step 1: the hiring manager writes the rubric (half a day). The panel agrees gates (right to work, availability within 30 days, 2+ years in logistics customer service, CV must parse cleanly) and the weights in the config above. "Measurable results in CV" is weighted 15 deliberately, because it is harder to game than keywords.

Step 2: back-test on past hires before go-live (one day). Pull the last 12 people hired into this role and 48 CVs rejected in past rounds. Run the rules over all 60.

Version 1 scored only 8 of the 12 past hires above the cut: 67 per cent recall. Unacceptable. Below 90 per cent, your rules describe something other than your own definition of good.
Diagnosing the misses: three hires had image-based or multi-column CVs that parsed badly (they came in by referral, so nobody noticed) and one had short average tenure but strong performance. Fixes: a bad parse sends the CV to the manual queue instead of scoring it; tenure weight drops from 15 to 10; referrals and internal applicants are hard-coded to always get human review.
Version 2 scores 11 of 12 past hires above the cut (92 per cent). The one miss is the internal transfer, which the new rule catches. Of the 48 previously rejected CVs, 5 now land in the human-review band, which tells you the old manual process was dropping people at the margin too.

Step 3: run the first live round with bands. Of 412 CVs: 55 score 70 or above (auto-advance, recruiter still confirms each one), 88 fall in the 55 to 69 band (a person reads every full CV), 269 score below 55 (auto-reject with reason codes, 10 per cent sample re-read).

Step 4: check the time maths. Reading all 412 CVs at 3 to 4 minutes each is about 24 hours of recruiter time. The new process reads 143 CVs plus 27 audit reads: about 10 hours. You save roughly 60 per cent of screening time while making the deliberate, budgeted choice that 88 borderline CVs get human judgement. That band is the cost of not losing good candidates. Do not let anyone "optimise" it away in round three.

Check every round: who got filtered out

The rules are a hypothesis about your talent market. Each round, test the hypothesis with a short audit pack.

Metric	What to record each round	Trigger for action
Override rate	How often the recruiter's shortlist differs from the tool's ranking	Above 10 per cent: re-run the back-test, the rules no longer match the hiring manager
Grey-band yield	Share of the 55 to 69 band invited to interview	If the band never produces interviews it is too wide; if it produces most of them the cut is too low
Audit-sample error	Good candidates found in the 10 per cent sample of auto-rejections	Any find: pause auto-rejection and read that round's rejections in full
Appeals and outcomes	Volume, response time, how many upheld	An upheld appeal is a rule defect, not a one-off favour
Parse failure rate	CVs sent to the manual queue	Spikes often mean a new CV format or an Arabic-language parsing gap
Score spread by group	Where voluntary data allows, spread across nationality, sex, age band	Unexplained gaps: bias review, FAHR question 3

Keep the audit pack for at least the recruitment cycle. If a candidate objects under Article 18 and asks for human review, your five-working-day SLA and your reason codes turn that request into a 20-minute task instead of a panic.

The questions to ask a vendor before it touches a live vacancy

FAHR's six questions are the right spine for any demo or RFP, in any sector. Add the data and commercial questions underneath.

Training data (FAHR Q1). Was the model trained on CVs from this region, in which languages and dialects, for which occupations? Ask what happens to an Arabic-language CV, or to an Egyptian, Levantine or Khaleeji accent if audio is scored. Ask whether the training data included outcomes (who got hired, who performed) or only CV text.
Transparency (Q2). Can you show, per candidate, which inputs drove the score? Ask for the technical manual. If the answer is a black box with a single "fit score", you cannot explain a rejection to a candidate or to a regulator.
Bias (Q3). What adverse impact analysis has been run, by which groups, and what were the results? What does the tool do when demographic data is missing (it usually is)?
Validity (Q4). What validation evidence exists, and is it content, construct or criterion related? Ask the vendor to validate against your historical hires before you trust the scores.
Finality (Q5). Can we configure cut scores? Can recruiters override? Is auto-rejection on or off by default? FAHR's position is that AI assessments should inform human decisions, not make them.
Candidate disclosure (Q6). Do you supply English and Arabic notice text, and where must it appear?
Data. Where is candidate data hosted, for how long, who are the sub-processors, is our data used to train your models, and can we export and delete everything at exit? Ask how the tool supports the Article 21 impact assessment. Claims such as "PDPL compliant" need a document behind them, not a website badge.
Commercial. What is the cost at our volume, what counts as a billable unit, and what does exit look like?

What the market looks like in 2026

Prices and posture as published or reported this year. Several figures below come from vendors writing about their competitors, so verify everything in writing before signing.

Route	Examples	Published price (2026)	Arabic and localisation	Human oversight design
Ranking inside your existing ATS or job board	Enterprise ATS ranking modules	Bundled in the ATS subscription	Varies by system	Ranking only; you decide who reads what
GCC assessment platform	Evalufy	Freemium: 15 assessments per month, 20 invites, 1 seat. Core and Pro: quote-based	Arabic-first, bilingual tests, regional dialect support claimed	Multi-evaluator scorecards, custom weighting, proctoring on Pro
GCC conversational screening	OVI (Milo)	Starter $99 per month (200 interview minutes, roughly $2.50 per five-minute interview); Growth $450 per month; free tier 50 credits	UAE-built, audio conversations, transcript-only scoring, no biometrics	Human-in-the-loop decision support, configurable rubrics
Global enterprise assessment	HireVue, SHL	HireVue reported at roughly $35,000 to $75,000 per year in one comparison, another citing $50,000 to $150,000 plus. SHL: quote required	SHL offers Arabic tests with global norms; HireVue Arabic workflow to be confirmed	Mature science and audit reports; confirm UAE PDPL documentation directly

A few observations from reading the material closely. Evalufy publishes a genuine free tier (15 assessments per month, no credit card), enough to test on a live requisition, and claims PDPL compliance with ISO 27001-certified MENA data centres: ask for the certificates. Published descriptions of Evalufy's headquarters also disagree between reviews, which is exactly the kind of detail to confirm with the vendor rather than assume. OVI's comparison of the four tools is useful but it is OVI's own material, and OVI sells a competing product, so treat its pricing for HireVue and SHL as a starting point for quotes rather than fact. SHL co-authored the FAHR guidance, which is a fair reason to include it on a shortlist for psychometric-heavy roles. And notice the design split that matters most under the PDPL: transcript-based scoring (what the candidate said) carries far less risk than tools scoring face, voice or expression, which is closer to sensitive personal data and a heavier Article 21 exercise.

What to do next

Pick one live or upcoming vacancy and write the gates and weights. Half a day, hiring manager owns it, HR facilitates.
Back-test the rules on your last 12 hires and a sample of rejected CVs. Do not go live below 90 per cent recall.
Configure cut bands, with a person reading every CV within 15 points of the cut, plus a 10 per cent audit sample of auto-rejections.
Turn auto-rejection off by default in your ATS until the bands and reason codes exist. Most systems will let you rank without rejecting.
Run the data protection impact assessment (PDPL Article 21) and put the disclosure text and appeal mailbox on the application form, in English and Arabic.
Put the six FAHR questions plus the data questions in writing to every vendor, and require answers before they see a single live CV.
Review the audit pack after round one and recalibrate anything the triggers flag.
Keep the rules per role, versioned, and revisit them whenever the role or the market changes.

If your leadership team cannot agree the rubric for a role, that is a strategy problem before it is a tooling problem. Our AI strategy work, run by our own team in the UAE, starts exactly there: we look at how your business runs, from processes and systems to data, tell you plainly where AI will pay off, what it takes and what to do first. Screening is one of the highest-volume, highest-risk AI decisions most UAE employers run, and it is worth getting right before you scale it.

Originally published on Azrty.

The best Arabic LLM for your enterprise is the one that passes your eval, not the leaderboard

Azrty — Wed, 28 Oct 2026 05:09:38 +0000

Four serious Arabic models now come out of the Gulf: Falcon-H1-Arabic from Abu Dhabi, ALLaM from Saudi Arabia, Fanar 2 from Qatar and Jais 2 from the UAE. None of them can be picked from a leaderboard, because the Open Arabic LLM Leaderboard measures academic Arabic, not your tickets, your policy documents or your customers' dialect. The two things that do transfer to your organisation are both measurable in about a week: a 200 to 500 item eval set built from your own material, and the tokenization economics that decide what each Arabic conversation really costs in money, latency and GPU memory.

If you are building Arabic-language customer service, a government services assistant or an internal knowledge bot in the GCC, you face a decision that looks like a model choice but is actually two measurements. First: does a regional open-weight model score within your tolerance on your content, with your product names, place names, dialects and policy language? Second: what does an Arabic token actually cost you, in API spend, time to first token, KV-cache footprint and concurrent sessions per GPU? Answer both and the decision makes itself. Skip them and you will either overpay a frontier API for years or ship a model that fails quietly on Gulf dialect.

The field in late 2026

Four model families are credible candidates for enterprise Arabic work today. All of them are open weight unless noted, which means you can run them on your own GPUs and keep data in-country.

Model	Developer	Sizes	Context	Licence	What the evidence says
Falcon-H1-Arabic	TII, Abu Dhabi	3B, 7B, 34B	128K (3B), 256K (7B, 34B)	Open weights (check the model card for your size)	About 71.5% (7B) and 75.4% (34B) on the Open Arabic LLM Leaderboard, beating 70B-class transformers including Llama-3.3-70B and Qwen2.5-72B. Hybrid Mamba-Transformer architecture, runs on vLLM.
ALLaM	SDAIA, now run by HUMAIN, Saudi Arabia	7B open checkpoint; 34B flagship powers HUMAIN Chat	4,096 (7B)	Apache 2.0 (7B Instruct preview)	The 7B Instruct preview is trained from scratch on 4T English tokens plus 1.2T mixed Arabic/English, with two published revisions (v1.27.2.25 and v2). ALLaM 34B backs HUMAIN Chat, hosted end to end in the Kingdom and PDPL-compliant.
Fanar-2-27B-Instruct	QCRI, Qatar	27B	32,768	Apache 2.0	Continually pretrained from gemma-3-27b-pt on roughly 166B Arabic and English tokens over 75,000 H100 hours. Explicit MSA plus Gulf, Levantine and Egyptian dialect support, native Arabic reasoning traces, tool calling and hallucination mitigation.
Jais 2	Inception (G42), Cerebras and MBZUAI, UAE	8B, 70B	8,192	Apache 2.0	Trained from scratch on 600B+ curated Arabic tokens plus 1.6T of web, math and code, with a custom 150,272-entry Arabic-centric vocabulary. Leading results among evaluated open models on OALL2 and AraGen. Note: the hosted API was retired on 21 August 2026; weights remain on Hugging Face.

Primary sources: the Falcon-H1-Arabic announcement and release blog, the Fanar-2-27B-Instruct model card, the ALLaM-7B-Instruct-preview model card, the HUMAIN Chat launch and the Jais 2 documentation.

The fourth candidate in every real procurement is a frontier API model prompted in Arabic. It belongs in the comparison, because on raw Arabic quality it often wins. The question is by how much, and what that gap costs.

Why the leaderboard cannot pick your model

The Open Arabic LLM Leaderboard is genuinely good work. It runs reproducible evaluations through the EleutherAI lm-evaluation-harness Arabic task set, so a score is a score, not a vendor claim. TII even documents that its own vLLM-based runs land within about one point of the leaderboard's numbers.

But read what it measures: knowledge, reading comprehension and reasoning over web-derived Arabic text, mostly Modern Standard Arabic. Your workload is not that. It is a customer writing "ما أقدر أرسّع الفاتورة من التطبيق" (I can't pay the bill faster from the app, in Gulf dialect), a code-switched complaint ("الـ bundle خلص، كيف أعمل top-up؟"), and questions about a product name that appears nowhere in any pretraining corpus. Leaderboard rank measures none of the things that will make your assistant succeed or embarrass you.

This is not an argument against the leaderboard. It is the screening stage: it tells you which models are worth a week of your team's time. The final stage happens on your data.

What transfers: build a 300-item eval set in three days

The single highest-leverage artefact in an Arabic LLM project is a small, versioned eval set built from your own material. Two hundred items is a floor, five hundred is plenty; beyond that the marginal information per item collapses.

Where the items come from:

Real tickets and chat logs (redacted). Take them verbatim, dialect and typos included. Never rewrite them into MSA, because your users will not.
Policy and product documents. Turn paragraphs into questions: "What is the fee for an early exit from this plan?"
Named entities. Product names, service tiers, districts, mall names, government portals. Models hallucinate most confidently where pretraining data is thinnest, and nothing is thinner than your internal naming scheme.
Deliberately hard slices. Aim for roughly 40% formal MSA, 30% Gulf dialect, 30% code-switched Arabic/English, in the proportions your traffic actually shows.

A concrete mix for a 300-item set, drawn from a telecom support case:

90 items from the last two years of Arabic chat logs, sampled by intent cluster (billing, top-up, coverage, cancellation), including Emirati and Saudi dialect items
60 questions whose answers live only in your policy PDFs
60 entity-grounded questions where the correct answer must quote your exact product or tariff name
45 dialect and code-switched items, verbatim from logs, never normalised
45 adversarial items: questions about services you do not offer, questions that mix a real product name with a wrong assumption, and date and amount arithmetic from a bill

That is one analyst-day to sample, one subject-matter-expert day to write gold answers, one more to review. Gold answers should be short and checkable, not essays.

The scoring sheet and the judge

Score each response on five criteria, 0 to 2 each, so the maximum is 10 per item and a 300-item set yields a 3,000-point scale:

Criterion	0	1	2
Correctness	Wrong answer	Partially right or right with wrong entity	Fully right, correct entity and amount
Grounding	Invents a policy, product or price	Vague, not traceable to source	Every claim traceable to your documents
Dialect handling	Misunderstands dialect or code-switching	Understands but answers in stiff MSA	Registers the register, answers naturally
Format and language	Wrong language, broken Arabic	Usable but unnatural	Natural, correctly punctuated Arabic
Completeness	Misses a step or an explicit caveat	Usable	Complete, including edge cases

Two ways to score: a human reviewer (expensive, about 1 to 2 minutes per item, so a full pass on 300 items is a day) or a stronger model as judge. If you use a model as judge, check agreement first: have the human score a 50-item overlap sample and compute percent agreement on the Correctness criterion. Below roughly 85% agreement, your judge is not measuring your rubric, and you must fix the prompt before trusting it at scale.

Here is a copy-paste judge prompt that has the properties that matter: it forces evidence, it forbids generosity, and it outputs a parseable score.

You are evaluating a customer-service assistant's reply in Arabic.

QUESTION (verbatim from a real user, may contain Gulf dialect or
Arabic/English code-switching):
{question}

GOLD ANSWER (written by a domain expert):
{gold_answer}

ASSISTANT'S REPLY:
{reply}

Score each criterion 0, 1 or 2. Be strict: a score of 2 must be
clearly earned, not merely plausible.

1. correctness: 2 = matches the gold answer's facts, entities and
   amounts; 1 = partially right; 0 = wrong.
2. grounding: 2 = every claim is traceable to the gold answer or the
   supplied policy text; 1 = vague; 0 = invents facts.
3. dialect: 2 = correctly understands the question's register and
   replies naturally in it; 1 = understands but replies in stiff MSA;
   0 = misunderstands the dialect or code-switching.
4. language: 2 = natural, correct Arabic; 1 = awkward but usable;
   0 = broken Arabic or wrong language.
5. completeness: 2 = covers steps and caveats; 1 = usable with gaps;
   0 = misses something essential.

Reply ONLY with JSON:
{"correctness": n, "grounding": n, "dialect": n, "language": n,
 "completeness": n, "total": n,
 "fatal_error": "<one line, or 'none'>"}

Run every candidate model, including the frontier API baseline and its Arabic-prompted variant, on the identical set with the identical retrieval context if you use RAG. Report the mean total, the per-criterion breakdown, and the fatal-error rate. The per-criterion breakdown matters more than the total: a model that loses two points per item entirely on dialect handling might be fixable with prompt work, while one that loses them on grounding needs retrieval surgery, and one that loses them on correctness needs to be dropped.

For dialect specifically, published per-dialect evaluations such as DialectalArabicMMLU show how much scores move between MSA and regional varieties, and TII reports AraDice dialect sub-scores for the Falcon-H1-Arabic family. Your own dialect slice settles the question for your dialect, which is the only one that matters.

The second thing that transfers: tokenization economics

Arabic text costs more tokens than the equivalent English text on English-centric tokenizers, and that premium appears in four places at once: your API bill, time to first token, KV-cache memory and the number of concurrent Arabic sessions a GPU can hold.

The peer-reviewed evidence is blunt. Petrov and colleagues showed that the GPT-4 tokenizer encodes the same content in Standard Arabic with roughly 3.04 times as many tokens as English, and the ChatGPT tokenizer with about 4.40 times as many, with direct consequences for cost, latency and effective context length (arXiv:2305.15425). The original Jais team measured the same pattern at the word level: GPT-2's vocabulary fragments Arabic into about 4.17 tokens per Arabic word against roughly 1.1 per English word, while the bilingual Jais vocabulary achieves about 1.05 (fertility figures summarised here). Newer multilingual vocabularies have narrowed the gap, but a 1.5 to 2 times premium on an Arabic-inefficient tokenizer is still normal, and nothing on your invoice labels it.

Measure it on your own traffic before believing any ratio. Ten lines of Python gives you fertility, tokens per word, on a sample of your real conversations:

from transformers import AutoTokenizer

text = open("sample_arabic_conversations.txt", encoding="utf-8").read()
words = len(text.split())

# Verified repo IDs; add the other candidates from their model cards
for repo in ["QCRI/Fanar-2-27B-Instruct",
             "humain-ai/ALLaM-7B-Instruct-preview"]:
    tok = AutoTokenizer.from_pretrained(repo)
    fertility = len(tok(text).input_ids) / words
    print(f"{repo}: {fertility:.2f} tokens per Arabic word")

One caution: studies count words and normalise text differently, so compare ratios within your own sample rather than against published absolutes. Your sample is the number that will hit your bill.

Worked example: what the premium actually costs

Take a 400-word Arabic support answer and compare a model whose tokenizer gives 2.4 tokens per Arabic word against an Arabic-optimised model at 1.5. Same content, same job.

Spend. The answer is 960 tokens on the first tokenizer, 600 on the second, a 1.6 times premium on every request. At an illustrative $5 per million output tokens, one million such answers costs $4,800 versus $3,000, a $1,800 difference before you have written a word of prompt engineering. Multiply by your volume. And note the trap when comparing providers: a model priced 30% lower per token that fragments Arabic 1.6 times more is not 30% cheaper for you, it is roughly 12% more expensive.

Latency. Prefill is compute-bound on prompt length, so time to first token scales with the token count of your system prompt plus retrieved context. Decode is per token, so an answer fragmented into 1.6 times the tokens streams 1.6 times longer at the same hardware speed. Prompt tuning cannot remove this; it is in the vocabulary.

KV cache and concurrency. Per-token KV-cache size for a 7B-class model with 32 layers, 8 grouped-query KV heads and 128-dimension heads at fp16 is 2 x 32 x 8 x 128 x 2 = 131,072 bytes, about 128 KiB per token. A 4,000-token Arabic conversation at the efficient tokenizer holds about 500 MB of KV per session; the same content at 1.6 times fertility holds about 800 MB. On a GPU with 60 GB free after model weights, that is roughly 120 concurrent sessions versus 75. Tokenization, not parameter count, decided your concurrency.

Context. A 256K context window (Falcon-H1-Arabic 7B and 34B) holds far more Arabic at a low fertility than a longer window holds at a high one. This is why the Jais 2 team built a custom 150,272-entry Arabic-centric vocabulary and report it makes inference efficient: the efficiency is not marketing, it is arithmetic you can reproduce with the snippet above.

The decision rule

Now the choice reduces to one comparison on one spreadsheet.

Run all candidates on your eval set. Suppose the frontier API scores 9.1 out of 10 and Falcon-H1-Arabic 7B scores 8.6. The gap is 0.5 points.
Set your tolerance in advance, in the currency of your use case: acceptable fatal-error rate on billing questions, acceptable dialect-mishandling rate. Do not set it after seeing the scores.
If the open model is inside your tolerance: run it on your own GPUs. You keep every conversation in-country, you pay a fixed infrastructure cost instead of a token bill that scales with Arabic verbosity, and you are not exposed to an API being retired (Jais 2's hosted endpoint went away in August 2026; its weights did not). A regional open model that scores within tolerance on your data is strictly the better enterprise choice: cheaper per request, in-country data, and no dependency on someone else's pricing page.
If the gap exceeds your tolerance: do not pay for it blindly. Quantify the frontier model's advantage in cost per resolved request: with the 1.6 times tokenization premium on an English-centric tokenizer and your ticket volume, what is the API bill for the extra 0.5 points? Then price the alternative: the same eval gap might close with a better Arabic system prompt, stronger retrieval into your policy documents, or three days of fine-tuning the open model on 2,000 of your own tickets. Very often the fine-tune is cheaper in month four than the API premium is in month one.

A practical note on keeping this sustainable: whatever prompts and gold answers survive the eval, version them. The Arabic system prompts that work, the dialect-handling instructions, the few-shot examples, these are assets, and they are exactly the kind of asset that walks out the door in a Markdown file. We keep them in a shared, versioned prompt library (PromptForge), and when we build and run these assistants for clients, from eval set through deployment, that is our AI engineering work, done by our own team here in Dubai.

What to do next

This week: pull 300 real items using the mix above. It is three days of one analyst and one domain expert, and it is reusable for every model you will ever evaluate.
In parallel: run the fertility snippet on 100 of your real conversations for every shortlisted model, including the frontier API's tokenizer. You now have the two numbers that transfer.
Next week: run all candidates through the judge prompt, with a human agreement check on 50 overlapping items. Build the table: eval score, fatal-error rate, tokens per request, cost per resolved request.
Then decide with the rule above. If the regional open model is inside tolerance, it wins on data residency and unit economics, and GCC regulators and your security team will thank you. If it is not, you now know exactly what the gap costs per month, which turns a vague preference into a negotiation.

The best Arabic LLM for enterprise is not a fixed answer, because "enterprise" is not a fixed workload. It is whichever of Falcon-H1-Arabic, ALLaM, Fanar 2, Jais 2 or a frontier API passes an eval set built from your own tickets, at a token cost you have measured rather than assumed. Both measurements take a week. The alternative is guessing, and guessing is the most expensive part of any AI project.

Originally published on Azrty.

vLLM vs SGLang vs TensorRT-LLM: how to choose a self-hosted inference engine in 2026

Azrty — Fri, 23 Oct 2026 05:09:54 +0000

You have already decided to run your own GPUs. Now someone is asking you to pick a serving engine, and every deck in your inbox quotes a single throughput number. Those numbers are real, and they are nearly useless to you: the right engine for your platform is decided by the shape of your traffic and the strictness of your latency SLO, not by a leaderboard. This post gives you the decision table, the operational costs nobody quotes, and a benchmark you can run in a day to settle the question on your own hardware.

The short answer: vLLM vs SGLang vs TensorRT-LLM

If you want the decision in one table before the reasoning, here it is.

Your situation	Pick	Why
Long shared system prompts, RAG context blocks, multi-turn agents, tree search	SGLang	RadixAttention reuses cached prefixes across requests; independent harnesses measure 30 to 40% higher output throughput on prefix-heavy traffic
Strict p99 TTFT or ITL SLOs at scale, one stable model for months	TensorRT-LLM	Lowest measured TTFT at every concurrency tested; fused kernels and CUDA graph capture compress the tail
Many models, frequent model swaps, mixed hardware, thin platform team	vLLM	200+ architectures, no build step, broadest quantisation support, largest community
You genuinely cannot tell which of the above you are	vLLM first, measure, then specialise	It is the fastest path to a correct measurement of your own traffic

Everything below explains why that table holds, where the published numbers come from, and how to verify it yourself on your own infrastructure in one working day.

What the audited numbers actually say

Start with the only cross-engine numbers that are externally audited: MLPerf Inference v6.0, released by MLCommons on 1 April 2026 (announcement, benchmark definitions). This round added a GPT-OSS 120B benchmark with three scenarios, including a latency-bounded Server scenario with demanding constraints: p99 time to first token under 3 seconds and p99 time per output token under 80 milliseconds, with input lengths ranging from 600 to 15,000 tokens.

On GPT-OSS 120B running on 8×B200 systems, the audited spread is instructive:

A vLLM stack (Red Hat OpenShift AI with the llm-d scheduler and vLLM) submitted 93,071 output tokens/s offline and 71,588 tokens/s server, which Red Hat reports as about 8% ahead of the nearest comparable B200 submission in the throughput-oriented Offline scenario (Red Hat's submission write-up).
TensorRT-LLM based submissions led the latency-bounded Server scenario on the same model and hardware class by roughly 22%, while trailing on raw offline throughput.
NVIDIA reported that TensorRT-LLM and Dynamo optimisations delivered up to 2.7x higher token throughput on the same GB300 NVL72 hardware compared with six months earlier, and notably used vLLM for its Qwen3-VL multimodal submission (NVIDIA's write-up).

Two operational facts matter far more than who won a given bar chart. First, the audited spread on identical hardware is single-digit to low-double-digit percentages, not the 2x to 5x figures routinely claimed in vendor marketing blogs. Second, 20 submitters used vLLM in this round, up from 5 submitters in v5.1. That surge illustrates where open-source ecosystem gravity has settled.

Compare that audited reality with un-audited benchmarks. An independent cross-engine harness published in June 2026 (methodology and results) reports SGLang's RadixAttention advantage as "up to 5x" on extreme synthetic prefix sharing, yet measures a much more modest 1.3 to 1.4x on a realistic enterprise RAG workload with a 4K shared system prompt. It also notes that repeated runs carry a variance band of plus or minus 15 to 20%. Both realities coexist: the caching mechanism is genuinely powerful, but the marketing multiplier depends on synthetic assumptions that rarely reflect production traffic.

The takeaway is not to ignore benchmarks. It is to benchmark your own traffic shape, at your concurrency, under your strict latency service level objectives (SLOs).

Where the engines actually differ

All three engines implement continuous batching, paged or blocked KV cache management, chunked prefill, and FP8 execution. The architectural differences that alter your production throughput and latency are much narrower than vendor marketing suggests.

Dimension	vLLM	SGLang	TensorRT-LLM
Core memory design	PagedAttention: KV cache split into fixed-size blocks with copy-on-write prefix sharing (docs)	RadixAttention: a radix tree of cached prefixes keyed by token sequence, matching the longest prefix (docs)	Paged KV cache with tight memory layout and fused kernels; cross-request prefix reuse more limited by default
Prefix handling	Automatic prefix caching via `--enable-prefix-caching`	Enabled by default; `--disable-radix-cache` to test; configurable via `--radix-eviction-policy lru/lfu/slru/priority`; `--enable-cache-report` returns cached tokens in `usage.prompt_tokens_details` (server arguments)	KV cache reuse exists but is not the headline mechanism
Model coverage	200+ Hugging Face architectures, multi-modal, embeddings, 2000+ contributors	Broad and fast support for new open weights, strong MoE and hybrid attention coverage	Narrower; prioritises models specifically optimised by NVIDIA
Quantisation	FP8, MXFP8/MXFP4, NVFP4, INT8, INT4, GPTQ/AWQ, GGUF, compressed-tensors	FP8 and related formats via runtime flags, such as `--quantization fp8`	FP8/INT4 via ModelOpt quantisation; per-configuration artefacts
Startup	~60 s (weights download and memory allocation dominate)	~58 s	Compiled engine path measured at ~28 min build for Llama 3.3 70B FP8 on H100; PyTorch backend now loads Hugging Face checkpoints directly
API surface	OpenAI-compatible server, Anthropic Messages API, and gRPC	OpenAI-compatible server, plus native Anthropic API support	`trtllm-serve` providing OpenAI-compatible endpoints
Measured edge	Best generalist engine; wins high-concurrency chat in several harnesses	30 to 40% higher throughput on shared-prefix RAG traffic	Lowest TTFT p50/p95/p99 at every concurrency tested (Spheron H100 runs)

Consider the final row carefully against your specific workload. In Spheron's single-H100 benchmark using fully unique prompts (Llama 3.3 70B FP8, vLLM 0.18.0, SGLang 0.5.9, TensorRT-LLM 1.2.0), SGLang and vLLM finished within 4% of each other on raw throughput because RadixAttention had no shared tokens to reuse. In the alternative harness where incoming requests shared a 4K system prompt, SGLang pulled 30 to 40% ahead. Identical serving engines reached opposing conclusions purely due to differences in prompt structure.

Choose by traffic shape, not by leaderboard

Three distinct traffic profiles cover the vast majority of enterprise deployments in the UAE and wider GCC region.

1. Prefix-heavy traffic

This pattern includes retrieval-augmented generation (RAG) platforms with lengthy system instructions, legal or regulatory compliance assistants carrying mandatory policy blocks, multi-turn conversational agents, and agentic workflows running structured evaluation loops.

This is where SGLang's radix tree structure excels. Instead of discarding KV cache entries or relying solely on block hashing, SGLang maintains a tree structure in memory. When a request arrives, the engine finds the longest matching token prefix and reuses the computed KV states immediately. The prefill phase for those tokens is skipped entirely.

Empirical field testing indicates an operational threshold of around 60% prefix overlap. When shared prefix tokens exceed 60% of total input tokens across incoming requests, SGLang delivers a substantial throughput and latency advantage. Below 60%, vLLM with --enable-prefix-caching active closes most of the performance difference. Multi-turn chat agents represent an ideal workload for SGLang, as every consecutive turn resends the entire conversational history as an extended prefix.

2. Strict tail-latency traffic

This profile covers interactive voice agents, real-time code completion, customer-facing search copilot integrations, and any deployment governed by strict contractual latency SLOs.

TensorRT-LLM led across every measured latency percentile in both independent benchmarks. In the Spheron evaluation at a concurrency of 50 requests:

Time to first token (TTFT) p50 reached 340 ms for TensorRT-LLM, compared to 360 ms for SGLang and 380 ms for vLLM.
The TTFT p95 divergence was wider: 620 ms for TensorRT-LLM versus 680 ms for SGLang and 720 ms for vLLM.
On a 4×H100 cluster running conversational chat at concurrency 64, TensorRT-LLM clocked a TTFT p99 of 265 ms, against 340 ms for SGLang and 410 ms for vLLM.

If your technical team has committed to an internal SLO requiring p99 TTFT under 300 ms on a 70B parameter model, that 145 ms margin is the difference between compliant production operations and an SLA breach. TensorRT-LLM achieves this tail compression through custom fused kernels, specialised memory scheduling, and CUDA graph execution that avoids CPU launch overheads.

3. High model churn and broad catalogue requirements

This profile describes platforms supporting multiple business units, regular model fine-tuning cycles, experimental agent pilots, LoRA adapter routing, and mixed GPU fleets combining older Hopper nodes with newer hardware.

In this environment, vLLM remains the premier operational choice. Supporting over 200 Hugging Face model architectures, vLLM allows platform engineers to onboard a new model checkpoint simply by updating a Kubernetes manifest or configuration argument. Quantisation formats like FP8, AWQ, or GPTQ are applied dynamically with command-line flags such as --quantization fp8, without requiring offline compilation scripts.

Furthermore, vLLM provides the most mature ecosystem support, the deepest Prometheus metrics instrumentation, and the broadest developer community. If your platform engineering team is lean, vLLM minimises operational maintenance overhead.

Worked example: what a million output tokens costs on one H100

Consider a realistic deployment scenario: a bilingual banking copilot built for a financial institution in Dubai, serving English and Arabic inquiries. The workload runs Llama 3.3 70B Instruct at FP8 precision on a single NVIDIA H100 SXM 80GB GPU. Model weights occupy approximately 70GB in FP8, leaving the remaining memory for KV cache allocation using --max-model-len 8192 and --gpu-memory-utilization 0.92.

Each incoming request carries a 3,000-token system prompt containing regulatory policies, formatting guidelines, and retrieved documentation blocks. The end user provides 200 tokens of input, and the model generates 250 output tokens. Steady peak operating volume is 50 concurrent requests.

Step 1: Measure prefix overlap

Before selecting an engine or tuning hardware flags, measure the exact prefix reuse of your production traffic. Launch an SGLang instance configured with --enable-cache-report, dispatch 200 production sample requests, and read the usage.prompt_tokens_details.cached_tokens field in the API response.

If roughly 3,000 of your 3,200 input tokens return as cached on warm requests, your workload exhibits 94% prefix overlap. RadixAttention will yield immediate gains.

Step 2: Benchmark at operating concurrency

Benchmark the engines at your realistic concurrency level of 50, rather than at saturation limits. Drawing from published single-H100 measurements with unique prompts:

Engine	Output throughput	TTFT p50	TTFT p95
TensorRT-LLM 1.2.0	2,100 tok/s	340 ms	620 ms
SGLang 0.5.9	1,920 tok/s	360 ms	680 ms
vLLM 0.18.0	1,850 tok/s	380 ms	720 ms

Step 3: Translate throughput into infrastructure costs

The conversion formula from raw output tokens per second to cost per million output tokens is:

# cost per 1M output tokens = node $/hour / (output tok/s * 3600) * 1e6
rate_usd_per_hour = 2.64  # Spheron H100 SXM on-demand rate, October 2026
engines = {
    "TensorRT-LLM": 2100,
    "SGLang": 1920,
    "vLLM": 1850,
}

for name, tps in engines.items():
    cost_per_million = (rate_usd_per_hour / (tps * 3600)) * 1e6
    print(f"{name}: ${cost_per_million:.2f} per 1M output tokens")
# TensorRT-LLM: $0.35 per 1M output tokens
# SGLang: $0.38 per 1M output tokens
# vLLM: $0.40 per 1M output tokens

Under these baseline parameters, generating 250 output tokens costs less than $0.0001 in compute time per answered query across all three engines.

Step 4: Apply the traffic shape correction

The baseline figures above reflect unique, non-overlapping prompts. Once we evaluate the actual workload structure (a 4K shared system prompt with 8K total context across concurrency 256 on a 4×H100 cluster billed at $3.50 per GPU-hour), independent measurements demonstrate how traffic shape alters unit economics:

SGLang: $0.49 per million output tokens
TensorRT-LLM: $0.64 per million output tokens
vLLM: $0.70 per million output tokens

For an enterprise cluster serving 750 million output tokens daily, SGLang requires roughly $367 per day, compared to $480 per day on TensorRT-LLM and $525 per day on vLLM. Over an annual operational cycle, choosing an engine aligned with your prefix pattern yields over $57,000 in direct compute savings on a single cluster.

Step 5: Formulate the decision

Because this specific copilot workload demonstrates 94% prefix overlap and carries no contractual sub-300 ms p99 TTFT requirement, SGLang represents the optimal, cost-efficient deployment choice. vLLM with --enable-prefix-caching serves as a viable fallback. If product managers later impose a strict p99 TTFT threshold under 250 ms, the engineering team should pivot the deployment to TensorRT-LLM to safeguard tail latency.

A benchmark you can run in a day

To avoid vendor bias, establish an internal reproducible benchmark. Fix the technical parameters that benchmark marketing routinely obscures: exact Hugging Face checkpoint revision hash, precision configuration (FP8 versus INT4), physical GPU model, tensor-parallel degree, maximum context window (max_model_len), host OS kernel, NVIDIA driver, and CUDA toolkit version.

Evaluate four key operational metrics under your target production concurrency:

TTFT p50 and p99: The critical indicator of perceived interactive responsiveness.
Output token throughput: Total generation speed at target concurrency.
KV cache utilisation and prefix hit rate: Memory efficiency during sustained load.
Cost per million output tokens: True operational expenditure calculated at your realistic traffic rate.

Both vLLM and SGLang bundle native client benchmarking utilities designed to capture end-to-end metrics at the network edge:

# vLLM: Measure throughput and latency percentiles against a running instance
vllm bench serve \
  --backend vllm \
  --model meta-llama/Llama-3.3-70B-Instruct \
  --dataset-name sharegpt \
  --dataset-path ./sharegpt.json \
  --request-rate 8 \
  --num-prompts 400 \
  --percentile-metrics ttft,tpot,itl \
  --metric-percentiles 50,99 \
  --output-json results_vllm.json

# SGLang: Run steady-state benchmark with custom operational traces
python3 -m sglang.bench_serving \
  --backend sglang \
  --max-concurrency 50 \
  --num-prompts 400 \
  --dataset-name custom \
  --dataset-path ./production_trace.jsonl

For TensorRT-LLM, run the identical load generator against the trtllm-serve OpenAI-compatible endpoint. Standardising the test harness across all engines ensures parity. As vLLM's official documentation highlights, metric definitions vary significantly across distinct benchmarking packages (benchmark CLI documentation).

Follow three rules to ensure reliable data:

Replay authentic traces: Avoid synthetic pseudo-random token streams. Random strings suppress prefix cache hits and distort memory allocators. Collect 500 to 1,000 anonymised production requests and replay them sequentially.
Warm up before recording: Execute 60 seconds of unrecorded traffic to prime memory allocations, CUDA graphs, and caching trees. Record three subsequent five-minute test passes and record median values.
Sweep concurrency bounds: Test load at 0.5x, 1x, and 2x expected operational concurrency. Engine performance degrades along non-linear curves as KV cache memory saturates.

The operational costs nobody puts in the slide deck

Engine bake-offs frequently disregard platform operational overhead. Factor these practical engineering costs into your architectural plan.

TensorRT-LLM compilation overhead

Historically, compiled TensorRT-LLM engines required substantial compilation time: approximately 28 minutes for Llama 3.3 70B FP8 on a single H100 (5 minutes for weight quantisation and 23 minutes for engine graph generation). Building artefacts across multiple GPU counts and quantisation profiles easily demands 1 to 5 hours of engineering effort before serving the first request. Spheron's cold-start benchmarks recorded 28 minutes to readiness for compiled paths, compared to approximately 60 seconds for vLLM and SGLang. For Kubernetes clusters autoscaling from zero, compiled engines introduce unacceptable pod startup delays.

The legacy build path has been removed

Platform engineers must note a major architectural change: NVIDIA's TensorRT backend removal guide confirms that the legacy TensorRT engine backend has been removed. PyTorch is now the sole execution backend. Legacy utilities including trtllm-build, trtllm-refit, trtllm-prune, and standalone convert_checkpoint.py scripts have been deprecated and retired.

Current versions of TensorRT-LLM load Hugging Face checkpoints directly through the PyTorch runtime backend. This transition eliminates prolonged compilation delays, but historical automation scripts and CI/CD pipelines calling trtllm-build will break on modern releases. Any architectural assessment must be verified against current PyTorch-backend documentation.

Silent performance regressions during upgrades

High-velocity open-source engines release frequent updates. A minor version bump in vLLM or SGLang can introduce double-digit shifts in p99 latency or alter default memory allocators. Engineering teams should pin container image tags strictly in production and run automated benchmark verification scripts in staging environments prior to promoting updates.

Memory preallocation trade-offs

Serving engines allocate KV cache pools upon process startup. Engineering teams that proactively calibrate parameters such as gpu-memory-utilization in vLLM and --mem-fraction-static in SGLang consistently recover 20 to 30% in throughput. Conversely, improper configurations trigger out-of-memory crashes during unexpected context spikes. Memory settings must be tuned systematically per model checkpoint and context length.

Multi-replica tail latency

In scaled Kubernetes deployments, tail latency is frequently dictated by cluster ingress and pod scheduling rather than kernel execution. As demonstrated in the Red Hat MLPerf submission, integrating the llm-d scheduler with queue-aware routing compressed p99 TTFT from 3.4 seconds down to 2.1 seconds purely through intelligent dispatching based on KV cache state and pod queue depth.

Keep the serving engine replaceable

Because all three engines expose OpenAI-compatible REST endpoints at /v1/chat/completions, treating the inference engine as an immutable, tightly coupled dependency is an architectural anti-pattern. The engine should function as a swappable component sitting behind an enterprise routing gateway.

This decoupling allows organisations to operate a hybrid strategy:

Deploy vLLM for internal innovation sandboxes and long-tail departmental models.
Deploy SGLang for customer-facing RAG pipelines and multi-turn autonomous agents.
Deploy TensorRT-LLM for mission-critical core models bound by rigid latency service level agreements.

Adopting this architecture requires three capabilities at the gateway layer: unified OpenAI-compatible routing across both self-hosted nodes and external providers, fine-grained access control, and dynamic usage quota management to protect GPU resources from runaway query loops.

At Azrty, this architectural pattern is standard practice. We deploy FastLLM Proxy as an OpenAI-compatible gateway in front of self-hosted LLM clusters and hosted APIs, managing routing, user access, and team budgets from a single plane. When an enterprise requires an end-to-end platform to orchestrate agents, models, and GPU hardware as a managed internal service, we implement Kuvryn AI. For organisations planning their cluster architecture, our team provides comprehensive AI infrastructure engineering. Azrty is a UAE company based in Dubai. Our own team in the UAE does the consulting, the deployment, and the support for our clients across the region.

What to do next

Quantify your prefix overlap: Deploy SGLang with --enable-cache-report or monitor vLLM's prefix cache hit metrics against an active trace. If shared prefixes exceed 60%, prioritise SGLang.
Formalise concrete latency SLOs: Define target thresholds explicitly (for example: "p99 TTFT under 250 ms at 64 concurrent requests"). Without explicit figures, engine trade-offs remain ambiguous.
Execute the one-day benchmark: Deploy your candidate model on target hardware, sweep concurrency levels, and calculate unit economics using the cost formula provided above.
Audit operational friction: Include model onboarding time, cold-start latency, memory tuning, and container upgrade maintenance within your total cost of ownership.
Decouple applications with an inference gateway: Shield upstream applications behind an abstraction layer. When your inference engine sits behind a unified API gateway, changing serving engines requires a routing adjustment rather than an application refactoring.

Serving engines will continue to iterate rapidly. Maintaining a measurement-driven evaluation process and decoupling your infrastructure ensures your platform captures performance advances without incurring technical debt.

Originally published on Azrty.

How to evaluate AI software before buying: a hotel guide

Azrty — Mon, 19 Oct 2026 05:09:21 +0000

Every hotel AI vendor demos well, because the vendor chooses the examples. The questions that decide whether a guest messaging or pricing tool is safe to deploy are narrower and harder: how does it handle your real guests, your seasonal tariffs, and your mixed Arabic and English threads? This article sets out a two-week evaluation framework built from 50 past enquiries from your own communication logs, an operational scorecard that distinguishes right, wrong and made-up answers, a strict source traceability check, and pass criteria agreed in writing before testing begins. You can run it with any vendor and walk away from tools that fail.

To evaluate AI software before buying, stop scoring vendor demonstrations and test competing tools on 50 historical enquiries drawn from your own operational logs. The process requires three steps: establish reference answers with supporting source documents before testing begins, evaluate responses against strict pass gates for accuracy and source grounding, and disqualify any tool that fabricates commercial commitments such as room rates or policies. Whether evaluating guest messaging for a city hotel in Dubai or an enterprise workflow across the GCC, your own past data is the only reliable benchmark.

Hotel buying committees are already moving. A study by Canary Technologies of over 400 hospitality professionals responsible for technology purchasing decisions found that 71% say AI is having a significant or transformative impact, 82% expect AI usage to increase in their organisation within the next year, and 85% expect to allocate at least 5% of their IT budget to AI tools this year. Guest communications sit at the top of the wish list. That is precisely where the risk sits too, because a message to a guest is a promise, and an AI tool that invents a room rate, a cancellation policy or a facility makes commitments your property never authorised.

The question every GM, owner and technical lead is asking is how to evaluate AI software before buying with confidence. The answer is identical whether you run a 300-key city hotel in Dubai, a luxury desert resort in Abu Dhabi, or a boutique property group across the GCC: stop scoring the vendor's rehearsed examples and start scoring your own.

Why do vendor demos and published benchmarks fail to reflect reality?

Vendor demos use rehearsed inputs. Published benchmarks use someone else's inputs. OpenAI's SimpleQA benchmark, released on 30 October 2024, comprises 4,326 short fact-seeking questions with human-verified answers. The SimpleQA paper is honest enough to note that GPT-4o scores under 40% on it, and that the dataset itself carries an estimated 3% inherent error rate. Google DeepMind's FACTS Grounding benchmark, released on 17 December 2024, evaluates 1,719 document-grounded examples (860 public, 859 held out) where responses are judged across three frontier models: Gemini 1.5 Pro, GPT-4o and Claude 3.5 Sonnet, specifically to reduce family bias.

These benchmarks are rigorous instruments for measuring model capability under laboratory conditions. Yet they say almost nothing about whether a customer-facing messaging product will quote your Deluxe King weekend rate with your exact cancellation terms in colloquial Gulf Arabic to a guest messaging on WhatsApp at 11pm.

Worse, the marketing figures vendors quote are often unstable. Digital Applied published an AI hallucination rate study in April 2026 and then formally withdrew its model rankings, error rates and confidence intervals on 4 October 2026 because the claimed 5,000-prompt study could not be substantiated with run-level records. Read that retraction before reading any vendor slide showing an arbitrary hallucination percentage. The enduring value of that page is method rather than static measurement: define the denominator, report abstention alongside errors, and independently verify that a cited source passage actually supports the claim.

That points to the true method for evaluating AI software before you sign a commercial contract. Freeze an evaluation set of your own past enquiries with verified reference answers. Agree the pass marks in writing before anything runs. Score competing tools on identical data. Preserve the run-level audit trail. Everything that follows applies this discipline to hotel guest messaging and pricing.

What does an AI hallucination actually cost a hotel?

Braintrust's guide to hallucination evaluation splits hallucinations into three distinct failure modes: factually incorrect against reality, unsupported by retrieved reference documents, and inconsistent with earlier context in the same conversation. In hotel operations, each failure mode carries a direct operational or commercial price tag:

Factually wrong. The tool quotes AED 850 for a Friday night stay when your current seasonal tariff specifies AED 990. The guest completes the reservation based on that figure, forcing your front desk team to either absorb the loss or provoke an escalation on arrival.
Unsupported by your documents. The answer sounds natural but exists nowhere in your rate sheet, banquet pack or policy schedule. This is the fabricated answer: confirming early check-in at 10am without charge, promising an airport shuttle service you discontinued last year, or inventing a flexible 6pm cancellation window. The guest treats this as a binding operational promise, and your staff must manage the friction.
Inconsistent within the conversation thread. The system approves a late checkout at 2pm in message three, then claims checkout is strictly 12pm in message seven. The guest captures screenshots of both messages, eroding trust and compromising the front desk team.

A critical lesson from the Braintrust evaluation guide is that an answer can pass an automated groundedness check and still fail on factuality. Groundedness merely checks whether the output reflects whatever context the retriever fetched. If the retrieval engine pulls an outdated 2024 PDF instead of the active 2026 rate schedule, the model can generate a perfectly grounded answer that is factually wrong for the guest. You must evaluate factual accuracy and retrieval grounding as separate metrics.

How do you build a test set from real past enquiries?

Extract 50 genuine guest enquiries from the last 90 days across your property management system (PMS), CRM, WhatsApp Business logs and central reservations inbox. Avoid synthetic prompts generated by an LLM, and do not let vendors supply test scenarios. Use real historical messages with guest names, phone numbers, room numbers and reservation codes scrubbed clean.

A balanced distribution for a GCC city hotel or resort:

Block	Count	What it tests	Concrete examples
Routine	20	Recall of stable published operational facts	Breakfast operating hours, high-speed wifi access, valet parking, pool timings, basic late checkout policies
Money	15	Rates, supplemental fees, deposits and commercial terms	Net room rate for specific weekend dates, non-refundable deposit terms, early check-in fees, tourism dirham and VAT inclusions
Awkward	10	Contextual judgement, policy trade-offs and thread memory	A noise complaint combined with a date alteration in one message, mid-thread date changes, complex family room allocation requests
Unanswerable	5	Safe refusal, boundary detection and human escalation	Real-time room availability without an active PMS sync, unreleased future packages, unlisted custom corporate rates

Three operational rules keep this test set robust:

Write the reference answer first. For every enquiry, have an experienced reservations or front office supervisor author the exact response that should be sent. Attach the specific source document: the dated tariff sheet, the room policy PDF, the banquet menu, or the corporate contract terms. If your internal team cannot agree on the correct answer, the question is either unanswerable or reveals an internal documentation gap. Both outcomes provide immediate operational value.

Separate practice enquiries from scored enquiries. Allocate 10 of the 50 enquiries as practice material (5 routine, 3 money, 2 awkward) and exclude them from final scoring. The vendor uses this initial batch to index your knowledge base and calibrate retrieval chunking. The scored set comprises the remaining 40 items: 15 routine, 12 money, 8 awkward, and all 5 unanswerable scenarios. Record the source channel, timestamp and selection criteria for every enquiry, following the benchmark transparency principles outlined by Digital Applied.

Keep Arabic conversations in native phrasing. Ensure roughly 20 of the 40 scored threads are conducted natively in Arabic, with the remaining 20 in English. Retain genuine regional phrasing: Gulf dialect terms, Egyptian or Levantine expressions commonly used by staff and guests, mixed Arabic-English code-switching, and Arabizi (transliterated Arabic in Latin characters). The reference answer must be drafted directly in Arabic by a fluent bilingual team member, rather than translated mechanically from an English master. An AI agent producing flawless Modern Standard Arabic while misinterpreting colloquial Gulf hospitality vocabulary has failed your operational test.

How should you score AI responses: right, wrong, made up or refused?

Every evaluation run must receive one of four explicit verdicts. This framework adapts the taxonomy of correct, incorrect and not attempted used in SimpleQA, while introducing the specific failure mode that matters most to hospitality risk: fabricated claims.

Verdict	Definition	Hotel operational example	Required remediation	Standard pass criteria
Right	Matches reference answer without contradiction	Accurately calculates room rate and cancellation cut-off for requested dates	None	Minimum 90% on routine; minimum 85% overall among attempted
Wrong	Contradicts an authentic source present in the index	Quotes superseded rate from an archived tariff sheet still present in the vector store	Clean index, fix metadata filtering and update retrieval weighting	Small tolerable margin; rectify and re-evaluate
Made up	Fabricates an entity, rate, policy or promise absent from all sources	Claims executive lounge access includes complimentary laundry services	Enforce strict retrieval grounding or disqualify tool	Exactly 0 on the money block
Refused and handed off	Recognises missing data, declines guessing, and escalates to human	States live room availability is unavailable and transfers conversation to reservations	Verify routing mechanism, chat state preservation and agent notification	5 of 5 on unanswerable block

To ensure the resulting figures reflect operational reality, apply two measurement practices:

Score at the atomic claim level. Break each generated response into standalone factual claims, such as the room rate is AED 1,100, breakfast is included, and free cancellation applies until 48 hours prior to arrival. Verify each individual claim against the source documents. This approach aligns with modern long-form factuality evaluation methods. Validate two conditions for every citation: existence (does the referenced source document and page exist?) and entailment (does the cited excerpt directly substantiate every element of the claim, including dates, occupancy limits, room categories and tax assumptions?). A citation pointing to a generic policy page that lacks the specific detail quoted represents a made-up claim masked by a real link.

Publish coverage alongside accuracy. An AI system can artificially inflate its accuracy score by refusing to answer difficult questions. As highlighted in the Digital Applied methodology notes, if a test of 100 questions results in 70 correct responses, 10 incorrect responses and 20 refusals, the error rate is 10% over the full population but 12.5% among attempted answers, with an operational coverage rate of 80%. When comparing competing vendors, always inspect coverage, accuracy across attempted responses, and the raw made-up claim count.

A concise Python script ensures scoring consistency across evaluation runs:

import csv
from collections import Counter

# Input format: enquiry_id, language, run, verdict, source_shown, source_supports
rows = list(csv.DictReader(open("trial_results.csv")))
verdicts = Counter(r["verdict"] for r in rows)
total = len(rows)
attempted = total - verdicts.get("refused", 0)

coverage = attempted / total if total > 0 else 0.0
error_attempted = (verdicts.get("wrong", 0) + verdicts.get("made_up", 0)) / attempted if attempted > 0 else 0.0
made_up_share = verdicts.get("made_up", 0) / total if total > 0 else 0.0
right_attempted = verdicts.get("right", 0) / attempted if attempted > 0 else 0.0

print(f"Total runs: {total}")
print(f"Operational coverage: {coverage:.1%}")
print(f"Accuracy among attempted: {right_attempted:.1%}")
print(f"Error rate among attempted: {error_attempted:.1%}")
print(f"Made-up claim share of total runs: {made_up_share:.1%}")

# Thresholds agreed in writing prior to commencing evaluation
PASS_CRITERIA = {
    "max_made_up_on_money_block": 0,
    "min_right_on_routine": 0.90,
    "min_right_among_attempted": 0.85,
    "min_claims_source_supported": 0.95,
    "min_unanswerable_handoff_rate": 1.0,
}

How do you verify that every answer traces back to a source?

Source traceability is frequently resisted by sales teams, yet it represents your strongest contractual safeguard.

For every scored response, mandate that the software display the document identifier, publication date, version number, and exact text excerpt that informed the output. Conduct two verification checks:

Existence: Confirm that the cited tariff sheet, operations manual or F&B guide exists in your active document repository.
Entailment: Verify that the cited passage logically entails the complete statement made to the guest. If an AI agent states that valet parking is complimentary for visitors attending an event, the cited document must explicitly mention both event visitors and complimentary status, rather than general guest parking fees.

Beware of the document versioning trap. In hospitality environments, rate sheets, promotional packages and seasonal restaurant schedules update frequently. An AI retrieval system that fails to enforce strict timestamp filtering will inadvertently surface an archived 2025 Ramadan buffet price or an expired summer staycation package when responding to a 2026 booking enquiry. Require the vendor to demonstrate how document expiry dates are handled, what underlying foundation model is deployed, and what notification procedure occurs when model versions are updated. If a vendor cannot supply verifiable document references, the software cannot be audited, your reservations team cannot supervise it, and your property cannot defend its statements in a commercial dispute.

How to evaluate AI software before buying: the two-week schedule

Running this evaluation requires approximately two person-days of focused internal effort distributed across two weeks, using two property staff members.

Days 1 and 2: Freeze data and agree acceptance gates. Lock the 50 enquiry scenarios, the reference answers, and the underlying source documents. Formalise the pass criteria in a written schedule shared with all participating vendors: zero fabricated statements on the 15 money enquiries, at least 90% accuracy on routine enquiries, at least 85% accuracy among attempted answers, at least 95% of factual claims supported by verifiable sources, and 100% handoff on unanswerable enquiries. Pass criteria established after testing begins are invalid.

Day 3: Index knowledge base. Supply the vendor with a clean document bundle: active room tariffs with effective dates, cancellation terms, standard operating procedures, spa and F&B menus, and property amenity guides. Confirm in writing the specific files indexed into the vector store.

Day 4: Practice validation. Execute the 10 practice enquiries. Use this phase to identify internal information omissions, such as unwritten pet policies or ambiguous late-checkout surcharges. Rectify missing documentation in the index before proceeding.

Days 5 to 8: Scored evaluation runs. Submit the 40 scored enquiries in their authentic language format. Execute each enquiry twice across separate sessions, yielding 80 total runs. Running duplicate queries is essential: frontier models exhibit variance, and repeated generations highlight unstable answers when a model is guessing. Braintrust's evaluation methodology underscores this self-consistency approach for open-ended generative responses.

Days 9 and 10: Blind independent review. Two bilingual reviewers independently evaluate all 80 responses against the reference answers and source files. Avoid averaging out split decisions; resolve discrepancies by examining source documentation and conversation context.

Day 11: Quantitative calculation. Consider a representative outcome across 80 runs: 68 attempted responses and 12 human handoffs (10 on unanswerable scenarios, 2 on complex edge cases). Of the 68 attempted responses, 61 are verified right, 5 wrong due to retriever version confusion, and 2 made up, with both fabrications occurring on pricing queries in the Arabic test track. The resulting metrics:

Operational coverage: 68 / 80 = 85.0%
Accuracy among attempted: 61 / 68 = 89.7%
Combined error rate among attempted: 7 / 68 = 10.3%
Fabricated response rate across total runs: 2 / 80 = 2.5%

Days 12 to 14: Final procurement decision. Evaluate outcomes against the pre-agreed acceptance thresholds. While the overall accuracy figure (89.7%) exceeds the 85% threshold, the product failed the zero-tolerance requirement for fabricated answers on the money block. Fabricating room rates and payment terms exposes your hotel to financial and reputational liability. This outcome represents a contractual failure. You can either reject the vendor or require an architectural fix and a full re-evaluation of the money block before signing. Never select a vendor based solely on aggregate accuracy percentages; enforce the boundary conditions.

What contractual terms, data protection rules and running costs matter?

A successful technical trial proves the model works under controlled conditions. The commercial contract determines whether your hotel can safely operate it at scale.

Data residency and regulatory compliance. Guest messaging inevitably processes personally identifiable information (PII), including guest names, contact details, payment references and travel dates. In the UAE, personal data handling is governed by Federal Decree-Law No. 45 of 2021 on the Protection of Personal Data, which took effect on 2 January 2022. Articles 22 and 23 regulate cross-border personal data transfers, allowing transfers either to countries recognised as having an adequate level of protection, or where contractual safeguards, standard contractual clauses or explicit guest consent apply. Because free zones such as DIFC and ADGM operate under distinct data protection statutes, and broad statutory adequacy lists remain unissued, clear contractual guarantees under Article 23 are the practical standard for SaaS deployments in the UAE.

Insist on written contractual clarity on these points:

Physical data storage and inference processing locations, including backup archives and diagnostic log stores.
Named third-party sub-processors, particularly underlying foundation model providers and vector database hosts.
Explicit contractual clauses prohibiting the vendor or underlying model providers from training models on your proprietary hotel data or guest transcripts.
Defined retention schedules, user data deletion rights, and prompt written certification of data destruction upon contract termination.

Commercial and technical terms. Ensure the vendor contract specifies the exact model versions powering the system, with a mandatory advance notification requirement before any model switch or parameter modification takes effect. Attach the performance thresholds from your two-week trial as ongoing service level agreements. Outline fallback protocols for system outages, establish clear contract termination terms, and secure full export access to conversation logs in standard formats (such as JSON or CSV) upon contract conclusion.

Calculating true operational cost. With 85% of hospitality organisations committing 5% or more of their technology budget to AI initiatives, total cost of ownership must be calculated accurately. Look beyond headline software licensing fees. Calculate the fully loaded cost per room per month or cost per completed conversation, including messaging API charges, infrastructure surcharges and ongoing internal supervision overhead. Plan for 30 to 60 minutes of weekly supervisor auditing to inspect live production transcripts using your evaluation rubric. Language models and operational documentation evolve continuously; production accuracy must be monitored systematically rather than evaluated once.

What should you do next before signing an AI contract?

Before scheduling another vendor software demonstration, complete three actions:

Extract 50 genuine past guest enquiries from your messaging logs and have your reservations team draft reference answers linked to verified source documents.
Establish formal evaluation thresholds, including a zero-tolerance policy for fabricated pricing statements, and share these criteria with prospective vendors as pre-requisites for your procurement trial.
Subject every shortlisted vendor to the identical two-week evaluation schedule, scoring responses across identical data using the four-tier rubric.

If your leadership team requires specialist support to run this evaluation, Azrty can assist. We are a UAE company based in Dubai, and our own team delivers AI consulting, technical evaluation and deployment support for regional clients. Through our AI strategy and readiness engagement, we review your operational processes, data assets and systems to show you exactly where AI will deliver measurable value, what technical prerequisites are necessary, and how to evaluate vendors safely. Any vendor can deliver an impressive demo. A rigorous internal evaluation ensures you select the system that genuinely protects your property.

Originally published on Azrty.

AI Governance That Does Not Slow You Down: The Minimum Operating Model for UAE Enterprises

Azrty — Fri, 16 Oct 2026 05:00:53 +0000

Most UAE organisations run AI in production without a named owner, a clear intake route, or proof of why models access sensitive data. A 40-page policy will not fix this. Four small, load-bearing artefacts will, giving technical leaders provable governance that aligns directly with ISO/IEC 42001 and the UAE AI Charter.

Most UAE organisations now run artificial intelligence in production in some form: a customer service assistant on the web portal, a document summarisation pipeline in legal, or a coding assistant deployed across engineering teams. Yet when asked three basic questions that a board, an enterprise client or an auditor will pose, most technology leaders cannot answer with verifiable proof. Who is the named individual accountable for this system? Why is this specific model permitted to process this specific dataset? Where is the immutable audit log confirming that policy was enforced?

The typical response to this uncertainty is to commission a 40-page AI governance policy, assemble a committee, and consider appointing a Chief AI Officer. For most enterprises in the UAE and wider GCC, both instincts are counterproductive. A dense policy document that developers and product owners do not read does not mitigate operational risk. When a failure occurs, an ignored policy serves only as documented negligence.

What engineering leaders and risk officers actually need is a lightweight, enforceable operating model. You can establish provable AI governance in about six weeks using four load-bearing artefacts: a one-page use-case intake form with an objective risk scoring table, a named accountable owner per system paired with a 45-minute monthly review, an architectural enforcement layer that intercepts model traffic, and a live model and agent register.

This operating model satisfies the governance expectations of regional regulators and international counterparties. By designing these four artefacts intentionally, the exact same document set maps directly to clauses 4 through 10 of ISO/IEC 42001 and the core principles of the UAE AI Charter.

The gap is operating roles, not appetite

The narrative that regional organisations are hesitant to adopt AI is contradicted by industry data. A global study by the Dubai Future Foundation and IBM found that roughly 20% of UAE organisations have AI governance platforms in place, compared with 12% globally, alongside strong regional leadership across core AI adoption metrics. Executive research cited by CDO Magazine indicates that 100% of GCC chief executive officers are either investing in AI today or plan to invest within twelve months.

The regional market does not suffer from a lack of tooling or ambition. The breakdown occurs in operational execution: enterprise platforms are acquired, but everyday accountability remains unassigned. Teams deploy workloads using shared API credentials, use cases bypass formal intake, and nobody can produce a definitive list of models operating across internal environments.

This gap becomes urgent when external scrutiny arrives. Enterprise clients in banking, government and healthcare are embedding detailed AI assurance questionnaires into procurement tenders, demanding proof of human oversight and data residency before signing contracts.

Simultaneously, the regulatory posture across the Gulf is maturing. The UAE Charter for the Development and Use of AI establishes 12 core principles, prioritising human oversight, accountability, data privacy and transparency. Regulatory analysis by Latham & Watkins highlights that UAE authorities are systematically moving from voluntary, principles-based frameworks toward binding expectations in critical sectors, with public services and regulated industries leading the transition.

Voluntary guidelines do not withstand commercial procurement or compliance audits. Building an agile operating model now prevents governance from becoming an expensive, disruptive crisis later.

The four artefacts at a glance

Enterprise governance must not become an administrative bottleneck. The framework below replaces slow committee approvals and unverified rules with four concrete artefacts:

Artefact	Structural format	Creation effort	Operational replacement
1. Intake form and tier table	One-page document: scope, data classification, autonomy level, blast radius, risk tier	Half a day to draft	Ad-hoc approvals in messaging channels that cannot be audited
2. Named owner and monthly review	Single accountable person per system, 45-minute monthly cadence with fixed agenda	1 hour per month	Multi-departmental committees that meet twice yearly
3. Enforcement layer	Model gateway with team-scoped access tokens, token quotas, and structured logging	2 to 4 days of infrastructure setup	Unenforced acceptable use policies buried in employee handbooks
4. Model and agent register	Tabular register: unique ID, owner, data categories, assigned tier, next review date	1 day to populate	Tribal knowledge that disappears when a lead developer resigns

These four artefacts require approximately one working week of collective effort to establish. If an external consultancy proposes a six-month governance design phase before implementing basic intake and technical controls, they are delivering administrative overhead rather than operational resilience.

Artefact 1: the one-page intake form with a stop/go tier table

Every AI deployment, including shadow tools already running in engineering or marketing teams, must pass through an intake form. This document must remain strictly one page, take under ten minutes to complete, and capture six mandatory fields:

System function: a concise summary of what the system does, written in plain business English.
Data categories touched: explicit classification of input data (for example: public information, internal operations, personally identifiable data, electronic medical records, or payment card details).
Decision autonomy: the precise level of human intervention required before an output triggers an action.
Blast radius: the tangible downside if the system produces hallucinations, biased output, or erroneous decisions.
System owner: the named individual accountable for ongoing operation and maintenance.
Calculated risk tier: derived deterministically from the scoring rubric below.

To prevent subjective disputes over whether a use case is dangerous, use an objective scoring formula. Evaluate three dimensions on a 1 to 3 scale: Data Sensitivity, System Autonomy, and Blast Radius.

Assigned points	Data sensitivity	System autonomy	Blast radius
1 point	Public data or synthetic datasets	Advisory output only: a human reviews and manually acts on every output	Internal operational impact: errors are immediately reversible with zero external visibility
2 points	Internal confidential data: non-personal commercial documentation	Supervised execution: the system drafts an action that executes upon one-click human approval	Customer-facing impact: errors reach clients or partners but remain technically recoverable
3 points	Personal data, sensitive health records, or regulated financial data	Autonomous execution: the system takes direct action via APIs without prior human sign-off	Irreversible harm: direct financial loss, regulatory breach, or severe safety impact

Sum the scores across all three dimensions to assign the operational tier:

Total score	Classification	Authorisation route	Operational turnaround
3 to 4 points	Tier 1: Green	Named system owner approves; entry is recorded in the register	Within 48 hours
5 to 7 points	Tier 2: Amber	Evaluation at the next monthly review; operational conditions assigned	Up to 30 days
8 to 9 points	Tier 3: Red	Deployment prohibited until technical controls are verified and executive sponsor signs off	Strict freeze until controls exist

Worked deployment scenarios

Consider a private healthcare operator in Dubai seeking to deploy an LLM assistant to draft responses to incoming patient inquiries in its contact centre.

Data Sensitivity: 3 points (touches patient inquiries containing identifiable medical symptoms).
System Autonomy: 1 point (the system generates a draft response inside the CRM; human staff read, edit, and send the message).
Blast Radius: 2 points (an erroneous message sent to a patient could cause confusion, though clinical staff can issue immediate corrections).

Total score: 6 points (Tier 2, Amber).

At the monthly review, the project is approved subject to three mandatory conditions: model traffic must route exclusively through an internal gateway with zero data retention by external API vendors, prompts must pass through an automated de-identification filter before leaving the local boundary, and 5% of all agent-approved messages must undergo weekly quality sampling. Intake to conditional clearance takes 18 days. A year earlier, this exact project would have languished in legal limbo for months because there was no transparent mechanism to evaluate its risk.

Now consider the same healthcare provider evaluating an AI agent that automatically adjusts patient appointments directly inside the electronic health record system without human mediation.

Data Sensitivity: 3 points (direct access to patient scheduling records).
System Autonomy: 3 points (writes directly to production databases via automated functions).
Blast Radius: 2 points (a scheduling error displaces real clinical visits, impacting patient care).

Total score: 8 points (Tier 3, Red).

The table halts unmediated production deployment immediately. The engineering team is directed to re-architect the service into a Tier 2 workflow: the model may propose schedule optimisations to a human coordinator, but direct database writes are locked until automated rollbacks and audit safeguards are proven.

This intake rubric aligns conceptually with the govern, map, measure, and manage core functions outlined in the NIST AI Risk Management Framework. Because enterprise buyers widely recognise this taxonomy, internal tier assignments translate cleanly when international clients conduct vendor risk assessments.

Artefact 2: a named owner per system and a 45-minute monthly review

Quarterly steering committees do not govern software effectively. By the time a quarterly agenda is finalised, engineering teams have already shipped code or abandoned the project. Real governance requires a short, monthly operating cadence.

The operating model requires exactly two roles, adapting guidance on agile AI oversight structures from Trustible:

The Executive Sponsor: a single executive (such as the CTO, CIO, or Chief Risk Officer) who maintains institutional authority over the tier table and serves as the final escalation authority for Tier 3 projects. The sponsor attends only the opening ten minutes of the review.
The System Owner: a designated individual (not a team alias or department) assigned to every AI asset. The system owner is personally accountable for completing the intake form, verifying operational constraints, monitoring accuracy, and reporting incidents. If an AI workload lacks an accountable human owner, it is decommissioned.

The monthly review is scheduled for 45 minutes on a fixed recurring date, chaired by an operational lead:

Minutes 0 to 10: Triage new intakes. Review all Tier 1 and Tier 2 submissions logged over the preceding month. Confirm or adjust assigned tiers (the Executive Sponsor participates in this segment).
Minutes 10 to 20: Scheduled Tier 2 reviews. Examine running systems approaching their mandatory six-month reassessment. Review changes in underlying foundation models, prompt changes, or shift in data inputs.
Minutes 20 to 35: Incident evaluation and audit logs. Review reported anomalies, prompt injection attempts, output degradation, or gateway policy violations. Determine required mitigations.
Minutes 35 to 45: Register hygiene. Flag systems with expired review dates, identify unmaintained prototypes, and issue termination notices for orphaned systems.

Meeting minutes are recorded directly into the model register. This practice prevents governance drift, ensuring that risk registers and operational realities remain identical.

Artefact 3: an enforcement layer, not a written rule

The failure of traditional IT governance lies in treating policies as compliance reading material rather than technical guardrails. If a company policy states that external models must not process proprietary customer data, but engineers hold raw API keys in local development environments, that policy provides zero operational security.

Technical rules must be enforced at the network and runtime layers. The core architectural component is a unified model gateway: an OpenAI-compatible proxy situated between internal software applications and all downstream model providers (whether hosted externally or deployed in private data centres).

This architecture is precisely what FastLLM Proxy implements: routing, tenant token budgets, model allowlists, and end-to-end request logging consolidated into a single control plane. Engineering teams configure their SDKs to point to the internal gateway URL, keeping credentials entirely out of application source code.

# Production gateway policy configuration
team_identifier: contact-centre-ops
api_key_ref: env:CONTACT_CENTRE_GATEWAY_TOKEN
routing:
  default_provider: self-hosted-gpu
  allowed_models:
    - internal/llama-3.1-70b-instruct  # On-premise infrastructure; data remains in-country
    - cloud/gpt-4o-mini                 # Permitted strictly for de-identified general drafting
quotas:
  monthly_spend_limit_usd: 3000
  rate_limit_requests_per_minute: 120
  alert_threshold_percentage: 80
security_enforcement:
  pii_scrubbing_mode: redact_before_egress
  log_prompt_payloads: false          # Prevents recording sensitive health or personal data
  log_structured_metadata: true        # Records timestamp, user context, token counts, latency

With an enforcement proxy operational, compliance rules become automated routing logic:

The rule stating sensitive records must remain within national borders is enforced by restricting the team API key to self-hosted models running on domestic infrastructure.
The rule capping experimental AI spend is enforced via automated token quotas that throttle traffic when budgets are exhausted.
The requirement to provide auditable evidence to regulators is satisfied by structured metadata logs that record every invocation without persisting sensitive payload text.

The second half of technical enforcement involves standardising prompt configuration. If system instructions reside inside arbitrary application scripts, prompt updates cannot be peer-reviewed. Deploying a version-controlled repository such as PromptForge ensures prompt engineering changes are tracked through pull requests, tested against benchmarks, and deployed consistently.

Storing approved architecture patterns, tier criteria and registry data in an accessible system like Kryton lets human engineers and automated tools reference identical operating parameters. Governance artefacts that are isolated from developer workflows will be bypassed.

Artefact 4: the model and agent register

The model and agent register serves as the central inventory of all artificial intelligence assets across the enterprise. It provides immediate answers during board reviews, customer security audits, and regulatory inquiries.

Rather than maintaining static spreadsheets that quickly become obsolete, define the register in structured text formats that can integrate with continuous integration checks:

# Enterprise Model and Agent Register Entry
- system_id: sys-2025-cc-drafter
  system_name: Contact Centre Patient Message Assistant
  system_owner: a.almansoori@enterprise.ae
  operational_tier: Tier 2 (Amber)
  approval_date: 2025-05-10
  mandatory_review_date: 2025-11-10
  data_classification:
    - patient_medical_inquiries
    - synthetic_reference_templates
  allowed_endpoints:
    - internal/llama-3.1-70b-instruct
  gateway_team_scope: contact-centre-ops
  enforced_controls:
    - automated_pii_redaction
    - outbound_gateway_only
    - weekly_human_audit_sample_pct: 5
  incident_history:
    - date: 2025-07-22
      summary: Hallucinated clinic opening hours
      resolution: System prompt updated in PromptForge with canonical schedule reference

This register provides continuous visibility. If a model identifier surfaces in gateway logs that does not exist in the register, an unapproved system has been detected automatically.

In enterprise deployments across the UAE, runtime isolation tools such as Kuvryn AI simplify this tracking. By hosting autonomous agents and internal model servers within segregated workspaces on sovereign GPU clusters, infrastructure metrics automatically mirror the system register, eliminating manual reconciliation.

One document set, two assurance frameworks

Establishing governance around these four artefacts provides immediate regulatory alignment. The structure satisfies the dual requirements of international certification under ISO/IEC 42001 and local compliance with the UAE AI Charter without requiring duplicative documentation.

ISO/IEC 42001:2023 represents the international management system standard for artificial intelligence, establishing auditable requirements across clauses 4 through 10. Industry timelines published by accredited certifiers like Konfirmity show that achieving ISO 42001 certification typically spans 4 to 9 months, with significantly compressed timelines for enterprises that already maintain an active ISO 27001 Information Security Management System.

The table below demonstrates how the four small artefacts satisfy mandatory ISO clauses while fulfilling the operational principles established by the UAE government:

Operational artefact	ISO/IEC 42001 clause mapping	UAE AI Charter principles fulfilled
1. One-page intake form and tier table	Clause 4 (Context of the organisation) Clause 6.1 (Actions to address AI risks) Clause 6.2 (AI objectives and planning) Clause 8.1 (Operational planning and control)	Human Oversight Data Privacy Safety and Resilience
2. Named owner and monthly review	Clause 5.1 (Leadership and commitment) Clause 5.3 (Roles and responsibilities) Clause 9.1 (Monitoring and measurement) Clause 9.3 (Management review)	Accountability Human Oversight Fairness and Equity
3. Gateway enforcement layer	Clause 7.5 (Documented information) Clause 8.2 (AI risk assessment implementation) Clause 8.3 (AI risk treatment controls)	Data Privacy Transparency Security and Robustness
4. Model and agent register	Clause 7.2 (Competence and awareness) Clause 9.2 (Internal audit baseline) Clause 10.1 (Nonconformity and corrective action)	Accountability Transparency Inclusiveness

When a commercial partner or statutory auditor requests evidence of your AI governance framework, you do not need to hand over an abstract policy manual. You provide the intake criteria, the system register showing clear human ownership, the gateway configurations enforcing security boundaries, and the minutes of recurring monthly reviews.

If your enterprise requires an initial baseline assessment to map these controls across legacy workloads, Azrty's AI strategy & readiness consulting provides an accelerated diagnostic. We evaluate how your organisation operates across systems, workflows and data pipelines, identifying precisely where artificial intelligence yields measurable commercial returns, what infrastructure changes are required, and what controls must be established first. Azrty is a UAE company based in Dubai. Our own team in the UAE does the consulting, the deployment and the support for our clients.

The six-week implementation roadmap

To establish this operating model without stalling delivery, execute the deployment as an agile, six-week sequential project:

Week 1: Foundations & Sponsor Alignment
  └── Finalise intake form & 3×3 risk scoring table
  └── Secure Executive Sponsor sign-off (CTO / Head of Risk)

Weeks 2–3: Enterprise Discovery & Registration
  └── Scan gateway/network logs for existing AI traffic
  └── Survey development leads to compile undocumented workloads
  └── Populate baseline Model & Agent Register

Week 3: Ownership Assignment
  └── Assign named individual owner to every active system
  └── Flag orphan workloads for immediate deprecation

Weeks 3–4: Gateway Infrastructure Deployment
  └── Stand up unified proxy (FastLLM Proxy)
  └── Migrate client API credentials to team-scoped gateway tokens
  └── Configure token limits, logging policies, and PII filters

Week 4: The Inaugural Monthly Review
  └── Convene 45-minute review session
  └── Classify legacy systems into Tier 1, 2, or 3
  └── Define binding remediation milestones for Tier 2 systems

Weeks 5–6: Operational Hardening
  └── Ingest production prompts into versioned library (PromptForge)
  └── Publish registry and documentation into Kryton knowledge base
  └── Lock production CI/CD pipelines to enforce gateway routing

Immediate priorities

Week 1: Define the intake document and risk scoring table. Drafting requires approximately two hours; securing formal approval from the Executive Sponsor requires a single one-hour briefing.
Weeks 2 to 3: Inventory current operational systems. Review existing firewall and API logs to identify external model providers currently receiving queries. Distribute a single-question inquiry to software and business team leads to capture unlisted workloads. Construct the initial draft of the register.
Week 3: Enforce operational accountability. Assign a specific, named human owner to every active deployment. If a workload has no identifiable owner willing to assume accountability for its behaviour, schedule it for retirement.
Weeks 3 to 4: Deploy the gateway layer. Implement the proxy endpoint, update application configuration files to route through the gateway, invalidate direct provider API keys, and implement baseline logging and budget caps.
Week 4: Conduct the inaugural monthly review. Dedicate 45 minutes to tiering the existing inventory. Establish remediation requirements for any workloads processing customer or sensitive data.
Weeks 5 to 6: Consolidate operational workflows. Move production prompts into a versioned library, publish the completed register to your internal engineering knowledge base, and formalise upcoming review dates.

Once these six weeks are complete, resist the urge to draft additional theoretical documentation. Allow the operating model to run for three consecutive review cycles. The empirical data recorded in your gateway logs and system register will reveal precisely which operational risks require further refinement. Practical governance is built through operational visibility, unambiguous personal accountability, and deterministic technical enforcement.

Primary sources and regulatory references

UAE Federal Legislative Portal: The UAE Charter for the Development and Use of Artificial Intelligence
Regulatory Overview: Latham & Watkins on UAE AI Regulation and Authority Oversight
International Standards Organisation: ISO/IEC 42001:2023 Information Technology: Artificial Intelligence: Management Systems
Certification Guidance: Konfirmity on ISO 42001 Implementation Timelines and Prerequisites
Regional Adoption Benchmarks: Dubai Future Foundation and IBM Study on Regional AI Governance Leadership
Operating Roles in Practice: CDO Magazine on Practical AI Governance Roles and Scaling Accountability
Committee Structures: Trustible on Establishing Effective AI Governance Committees
Risk Framework Reference: National Institute of Standards and Technology (NIST) AI Risk Management Framework

Originally published on Azrty.

AI acceptable use policy for employees: what works

Azrty — Fri, 16 Oct 2026 05:00:53 +0000

Somewhere in your organisation this morning, someone pasted a client contract, a candidate CV or an unreleased financial projection into a personal ChatGPT account. This is not a hypothetical risk to manage next quarter: 66% of office professionals admit using AI tools at work even when they believed their employer did not permit it. The reflex response, a blanket ban, does not work. It simply pushes confidential data onto personal smartphones and unmonitored home networks. This article sets out the working alternative for a 20 to 200 person firm: a two-page AI acceptable use policy you can write this week, a sanctioned tool under company accounts, and a private gateway for the work that must stay inside your perimeter. With the numbers.

An effective AI acceptable use policy for employees sets clear data classification boundaries, specifies sanctioned corporate tools, and establishes a no-blame incident reporting path. Rather than attempting an outright ban, a workable policy gives staff approved, low-friction alternatives so confidential company and client data never enters public AI models. For a 20 to 200 person organisation, this means combining a two-page policy document with company-managed accounts and a self-hosted private gateway for sensitive workloads.

The urgency is driven by frontline behaviour. A PagerDuty survey published in June 2026 found that two-thirds (66%) of office professionals have used AI tools at work despite believing it was not permitted under company policy (PagerDuty). The same survey found 88% have shared work-related information with public AI tools such as ChatGPT, Claude or Gemini: 43% shared emails, 40% meeting notes, 34% customer data, and 31% financial information or confidential company documents.

If your instinct as a leader is to issue a strict ban or block AI domains at the perimeter firewall, the evidence suggests that the ban has already failed. Worse, it actively increases your exposure. When staff need to summarise twenty pages of meeting notes before a 2:00 PM executive call, they do not abandon the technology; they open ChatGPT on a personal smartphone, tether to a 5G hotspot, or forward the document to a personal webmail address.

Here is what actually works, what it costs, and how a 20 to 200 person organisation can put a working system in place within 30 days.

Why do employees ignore AI bans and leak company data?

The PagerDuty Shadow AI Survey was conducted by Wakefield Research among 1,250 office professionals in non-IT roles at organisations with annual revenue of USD 500 million or more, across the United States, United Kingdom, Australia and Japan (PagerDuty). While your organisation in the UAE or GCC is likely smaller, there is no evidence that staff behaviour differs when deadlines loom. The direction of travel is unambiguous.

Two operational figures matter most to technical and business leadership:

First, the volume of confidential material entering public models is accelerating rapidly. Figures compiled by Unio Digital from vendor security research put the share of corporate data entering AI tools that is sensitive at 34.8%, up from 10.7% two years earlier (Cyberhaven, 2025). The same review highlights that 77% of employees paste data directly into generative AI prompts, and 82% of those pastes originate from personal accounts rather than corporate workspaces (LayerX, 2025).

Second, the financial and operational cost when that exposure triggers a security incident is severe. IBM's Cost of a Data Breach 2026, as compiled in the Unio Digital review, found that security incidents involving shadow AI had more than doubled to 43% of organisations, averaging USD 5.39 million per incident (Unio Digital).

The lesson for a mid-market firm is not that you will necessarily face a USD 5.39 million breach bill; those samples skew towards multinational enterprises. The immediate takeaway is that shadow AI use is near-universal, it directly involves your most sensitive commercial material, and it is happening right now through unmanaged personal channels that your IT and security teams cannot audit.

Why do blanket AI bans fail in practice?

Blocking external AI domains at your office firewall feels decisive during an executive committee meeting. In practice, it creates a dangerous illusion of security.

Start with how your team actually works. Because 82% of corporate data pastes into generative AI tools originate from personal accounts (Unio Digital), the corporate network perimeter is largely irrelevant. Consumer ChatGPT accounts retain chat history by default and use prompt inputs to train and refine foundation models unless an individual user navigates deep into settings to opt out. A consumer account is not covered by your master service agreements, your non-disclosure agreements, or enterprise data protection commitments.

Furthermore, consumer account credentials represent an active secondary risk. Over 225,000 OpenAI credentials captured by infostealer malware were listed for sale on dark web illicit marketplaces in 2024 (Unio Digital). When an employee uses their personal account on an unmanaged personal device, any paste containing client confidential data sits inside an account that may already be compromised by infostealer malware. A firewall ban does not remove the data from these insecure environments. It simply removes your security team from the conversation, pushing usage to mobile browsers and home Wi-Fi networks where data loss prevention (DLP), conditional access, and audit logging cannot reach.

Bans also inflict direct damage on talent retention. In the PagerDuty study, 77% of office professionals stated that their employer's AI restrictions limit their professional growth, while 75% said they would be likely to look for a new role that offered better AI skills development (PagerDuty). Forcing high-performing analysts, software engineers, and project managers to work with one hand tied behind their backs creates resentment.

The practical alternative is not unconstrained permission. It is a three-part operating model: a concise, practical acceptable use policy, an approved commercial tool provisioned under company accounts, and a self-hosted private gateway for the sensitive workloads that cannot leave your perimeter.

How to write an AI acceptable use policy for employees

An AI acceptable use policy for employees should never be a 40-page legal dissertation. Lengthy governance manuals are rarely read, harder to remember, and virtually impossible for frontline managers to enforce. A policy that works must fit on two pages, present a clear mental model, establish a sanctioned path that is more convenient than personal workarounds, and provide a clear procedure for when someone makes a mistake.

The policy should centre on a strict three-tier classification model that tells staff exactly where different types of data may go:

Red (Restricted): Client personal data, Emirates ID numbers, bank details, unreleased financial projections, payroll files, intellectual property, and anything covered by an NDA. This tier must never be submitted to a hosted multitenant AI tool. It is routed exclusively to your internal private gateway.
Amber (Internal): Internal strategy memos, project roadmaps, non-sensitive source code, and business context that contains no personal identifiers or confidential financial metrics. Hosted commercial tools under corporate accounts are permitted only after redaction.
Green (Public): Public marketing collateral, standard boilerplate text, generic coding syntax queries, synthetic test data, and general brainstorming. Any sanctioned corporate tool is approved.

Below is a functional policy template. Customise the bracketed operational fields, keep the length under two pages, and issue it immediately:

ACME AI Acceptable Use Policy (v1.0, dated DD Month YYYY)

1. Purpose. This policy defines how we use artificial intelligence tools
   at work. Its goal is to enable productive, safe AI adoption, not to
   prohibit it.

2. Sanctioned tools. Staff may use AI exclusively via corporate accounts:
   - ChatGPT Business (under the ACME corporate workspace)
   - Microsoft Copilot Chat (authenticated with your corporate Entra ID account)
   - The ACME Internal AI Gateway at https://ai.internal for Red and Amber data
   Using personal accounts, consumer logins, or unapproved third-party AI
   extensions for company work is strictly prohibited.

3. Data classification rules. Before submitting information, classify it:
   RED: Client PII, national identity numbers, financial records, HR files,
   unreleased company metrics, NDA-governed material. Gateway only.
   AMBER: Internal memos, architectures, source code. Sanctioned hosted
   tools permitted ONLY after redacting identifiers.
   GREEN: Public content, generic drafting, open syntax questions.
   Sanctioned tools permitted freely.

4. Redaction protocol. When preparing Amber content for hosted tools:
   replace client and employee names with placeholders such as [Client A];
   strip account numbers, phone numbers, and addresses; retain only the
   abstract structure required to complete the task.

5. Human accountability. You remain personally responsible for deliverables.
   Verify all facts, calculations, statutory citations, and code outputs
   before sharing with clients or deploying to production.

6. Transparency. Inform clients or counterparties when AI has generated a
   substantial part of a deliverable whenever contracts or regulations require it.

7. No-blame incident reporting. If you inadvertently paste Red data into an
   unapproved or hosted tool, notify [Head of IT] within 24 hours. Prompt
   reporting carries no disciplinary sanction. Concealing a known data leak
   is a disciplinary offence.

8. Governance. This policy is reviewed quarterly by [Head of IT].

Two operational principles ensure this policy succeeds:

First, your sanctioned path must deliver superior performance: low-latency models, generous usage caps, and zero friction. If the internal gateway is slow or difficult to access, staff will revert to consumer accounts.

Second, enforce network boundaries pragmatically. Block consumer AI login pages on managed laptops via your secure web gateway or endpoint management profile. Configure endpoint DLP rules to detect patterns like UAE Emirates ID numbers (784-XXXX-XXXXXXX-X) and international bank account numbers (IBANs) in browser clipboard events. You cannot control personal mobile devices directly, but you can remove friction from the corporate path and maintain an auditable record of compliance.

Which AI tools should your company policy sanction?

The fastest risk-reduction step an organisation can take is migrating employees from unmanaged personal accounts to a corporate workspace. The contractual and technical differences between consumer and enterprise tiers are fundamental.

OpenAI does not use data submitted to ChatGPT Business, ChatGPT Enterprise, or the OpenAI API to train its models by default (OpenAI enterprise privacy). Workspace administrators retain administrative oversight, user provisioning, and retention controls, with deleted conversations purged from OpenAI operational systems within 30 days. ChatGPT Business is priced at USD 25 per seat per month on monthly billing, or USD 20 per seat per month on annual billing, with an upgraded Premium seat at USD 125 monthly or USD 100 annually, specifically accommodating organisations sized from 2 to 200 seats (OpenAI Business pricing).

For organisations running on Microsoft 365, Microsoft Copilot Chat accessed via an Entra ID corporate account provides commercial data protection at no additional cost (Microsoft Learn). Prompts and responses do not cross the Microsoft 365 tenant boundary, are not inspected by humans, are not used to train foundation models, and inherit corporate audit logging and eDiscovery retention policies.

When developing custom internal tooling using the OpenAI API, inputs have not been used for model training by default since 1 March 2023 (OpenAI platform data controls). API operational logs are retained for abuse monitoring for up to 30 days, and organisations handling highly regulated workloads can apply for zero data retention (ZDR).

Route	Direct cost	Used for model training	Governance and controls
Personal accounts (status quo)	USD 0 direct	Yes, by default unless opted out	Zero administrative visibility; personal email logins
ChatGPT Business	USD 25/seat monthly; USD 20/seat annual	No, by default	Central admin console, SSO, seat lifecycle, 30-day purge
Microsoft Copilot Chat (work account)	Included with Entra ID corporate plans	No, protected	M365 tenant boundary, corporate audit logs, eDiscovery
OpenAI API (custom applications)	Usage-based token pricing	No, default policy since March 2023	30-day abuse monitoring logs; zero retention on approval
Private self-hosted gateway	GPU compute leasing	Never, runs isolated weights	Complete data residency, raw prompt audit logs, local weights

A critical constraint for GCC organisations to evaluate: major providers such as OpenAI do not offer an in-country UAE data processing boundary for these software-as-a-service plans. For standard drafting and day-to-day administrative tasks, this trade-off is often acceptable when paired with strict Amber redaction rules. However, for sensitive client files, national identity data, and regulated records, cross-border data transfer without explicit legal grounds introduces significant compliance exposure.

When is a private AI gateway worth paying for?

Consider a typical GCC operational structure: an 80-person professional services consultancy in Dubai operating on Microsoft 365. The organisation comprises advisory, marketing, and client delivery teams. Approximately 25 staff members routinely handle Red or Amber material: financial models, audit schedules, M&A due diligence files, and human resources records. The remaining 55 staff members conduct general analysis, drafting, and research.

A private gateway consists of dedicated GPU hardware running open-weight inference models behind a unified reverse proxy. The proxy handles user authentication, per-department token allowances, and immutable local logging. Your staff interact with standard web interfaces or internal scripts, but the inference processes execute on private hardware hosted within a local data centre. Prompts and responses remain strictly within your controlled perimeter.

We deploy and manage these environments for clients within our AI infrastructure service, placing FastLLM Proxy in front of model instances. FastLLM Proxy acts as an OpenAI-compatible gateway that centralises request routing, enforces department quotas, and manages access controls across private GPU instances and external hosted endpoints.

Three triggers indicate that your organisation requires a private gateway:

Data residency mandates: Client agreements, public sector contracts, or regional data protection statutes requiring that personal and financial information never leave the UAE.
High volume of confidential data: When sensitive workflows are executed daily, human redaction discipline inevitably slips under pressure. Technical segregation prevents leaks by design.
Regulatory auditability: Regulated entities in financial services, legal, and healthcare sectors must produce verifiable audit trails detailing who issued each prompt, what model generated the response, and where the compute was located.

To evaluate the operational expenditure, consider indicative pricing from GPU providers. Using Hyperstack published rates as an industry benchmark:

NVIDIA L40 (48 GB VRAM): USD 1.00 per hour on demand, translating to approximately USD 730 per month running 24/7 (roughly AED 2,700). An L40 runs OpenAI's open-weight gpt-oss-20b model (requiring approximately 16 GB VRAM) or high-performance 32B parameter models in 4-bit quantisation with ample headroom.
NVIDIA H100 (80 GB VRAM): USD 2.50 per hour on demand, amounting to roughly USD 1,825 per month (approximately AED 6,700), or from USD 1.75 per hour on reserved terms (roughly USD 1,280 per month). OpenAI's deployment guidelines show that gpt-oss-120b requires approximately 60 GB of VRAM, running on a single H100 instance (OpenAI cookbook).

The production inference layer is powered by vLLM, an open-source, Apache 2.0 licensed serving engine featuring continuous batching, PagedAttention memory management, and an OpenAI-compatible API structure supporting hundreds of model architectures.

The service commands for standard open-weight architectures illustrate the operational simplicity:

# Launch gpt-oss-120b on a single 80 GB GPU instance (requires ~60 GB VRAM)
vllm serve openai/gpt-oss-120b --max-model-len 32768

# Launch gpt-oss-20b on an L40 instance (requires ~16 GB VRAM)
vllm serve openai/gpt-oss-20b --max-model-len 32768

Internal developers and chat user interfaces connect to the gateway by modifying the base URL in the standard OpenAI client:

import os
from openai import OpenAI

# Direct internal tools and agent workflows to the secure internal gateway
client = OpenAI(
    base_url="https://ai.internal/v1",
    api_key=os.environ["AI_GATEWAY_KEY"],  # Department-specific key with proxy budget enforcement
)

Now compare the direct operational cost across potential setups for this 80-person firm:

Architecture	Monthly compute cost	Model training risk	Data residency	Audit logging control
Unmanaged personal accounts	USD 0 direct	High: default training enabled	No control	None: inaccessible to IT
ChatGPT Business (80 seats)	USD 1,600 (annual billing)	Zero: enterprise commitment	SaaS cloud boundary	Enterprise console metadata
Private gateway (L40 48 GB)	~USD 730 hosted	Zero: self-hosted instance	UAE perimeter	Full prompt and response logs
Private gateway (H100 80 GB)	~USD 1,825 on demand	Zero: self-hosted instance	UAE perimeter	Full prompt and response logs

The strategic approach is a blended model: provision ChatGPT Business seats for the 55 employees engaged in public or generic research (55 seats at USD 20 equals USD 1,100 monthly), while directing the 25 sensitive-work analysts to the private gateway (USD 730 to USD 1,825 monthly).

For a total monthly budget between USD 1,830 and USD 2,925, the firm eliminates shadow AI, provides high-speed tooling for general staff, and guarantees that sensitive client data never leaves an audited, local GPU instance. Compared against the IBM benchmark cost of USD 5.39 million per shadow AI incident (Unio Digital), the business case is clear.

How do you roll out an AI policy in 30 days?

Deploying this framework across a mid-sized organisation requires an orderly progression over four weeks:

Week 1: Policy sign-off and baseline tooling

Finalise your organisation's two-page acceptable use policy using the template provided. Secure executive sign-off from legal, operations, and IT leadership. Procure an initial pool of five to ten ChatGPT Business seats or confirm Microsoft Copilot commercial protections in your Entra ID tenant. Implement standard web filtering rules blocking consumer generative AI sign-up pages on managed endpoints, and configure endpoint DLP to alert when UAE Emirates ID numbers or IBAN strings appear in web paste events.

Week 2: Private gateway staging and integration

Spin up your dedicated GPU instance. Deploy vLLM with your chosen open-weight model weights and configure the reverse proxy layer. Configure departmental API keys with strict monthly token quotas, log retention policies, and rate limits. Onboard the 25 staff members handling sensitive workflows to test latency, prompt responsiveness, and usability.

Week 3: Departmental practical training

Policy adoption fails when training is limited to mass email announcements. Conduct one-hour interactive sessions with each department. Review real-world emails and documents from their daily workflow, demonstrating how to categorise items into Red, Amber, and Green. Teach practical redaction techniques, verify that users can access their sanctioned accounts, and reiterate the non-punitive incident reporting policy.

Week 4: Production cutover and exception management

Decommission ad-hoc developer access and unapproved API trial accounts. Move the remaining corporate staff onto their sanctioned commercial accounts. Publish the official directory of approved AI tools on the intranet alongside the two-page policy, and schedule the first quarterly governance review. The audit logs generated by your gateway and administrative consoles now form your operational compliance baseline.

UAE AI compliance: what does the PDPL require by 2027?

For organisations operating in the United Arab Emirates, establishing AI governance is no longer just a cybersecurity best practice; it is an immediate statutory requirement.

Federal Decree-Law No. 45 of 2021 on the Protection of Personal Data (PDPL) requires full compliance by 1 January 2027 (WCR Legal). The UAE PDPL is a consent-first framework. Unlike the European Union General Data Protection Regulation (GDPR), the UAE federal statute does not recognise "legitimate interests" as a lawful basis for processing personal information.

Relying on informal assumptions while employees feed client or employee data into public machine learning systems exposes the organisation directly to statutory sanctions. Administrative fines under the PDPL reach up to AED 5 million (approximately USD 1.36 million) for severe violations, and regulatory compliance guidance explicitly instructs organisations processing UAE personal data at scale to maintain documented AI usage policies covering approved tools and permitted use cases (WCR Legal).

In financial free zones, strict supervision is already in effect:

DIFC: DIFC Data Protection Regulation 10 has been active since January 2026, establishing mandatory requirements for AI impact assessments, transparency disclosures, and rigorous record-keeping for high-risk data processing, backed by statutory fines between USD 25,000 and USD 50,000 per violation (WCR Legal).
ADGM: The Data Protection Regulations 2021 (DPR 2021) closely mirror international standards, requiring strict technical safeguards, documented transfer mechanisms, and verifiable audit trails for automated processing.

A practical corporate posture recognises that the two-page acceptable use policy, commercial workspaces, and a private inference gateway are not separate initiatives. Together, they constitute the exact compliance package an auditor or regulatory authority will examine: a defined policy, sanctioned tooling, boundary protections against unauthorised processing, and verifiable logs demonstrating where data was processed.

What should your organisation do next?

Take these operational steps immediately:

This week: Adapt the acceptable use template to your organisation's reporting lines and get it approved. Enable commercial data protection in your Microsoft 365 environment or purchase a starter tier of ChatGPT Business seats. Review your standard client engagement contracts to identify existing data processing restrictions.
This month: Evaluate your sensitive data exposure. If contractual data residency, high-volume confidential processing, or regulatory compliance mandates apply to your firm, evaluate your private gateway deployment. Size your compute capacity using published rates: an L40 at USD 1.00 per hour for standard models, or an H100 at USD 2.50 per hour for 120B parameter architectures (Hyperstack pricing).

Azrty is a UAE company based in Dubai. Our own team in the UAE does the consulting, the deployment and the support for our clients, configuring local GPU infrastructure and integrating reverse proxy software to establish secure, audited AI endpoints. If you need a fully compliant, private AI gateway operating within your perimeter, explore our AI infrastructure services to begin.

Originally published on Azrty.

Why is my AI API bill so high? Where tokens go

Azrty — Sun, 11 Oct 2026 05:05:58 +0000

Your AI bill is climbing because total cost is unit count multiplied by unit price, and agents expanded unit counts by 10 to 30 times while unit prices merely halved. A chatbot makes one model call per question; an autonomous agent makes 10 to 20, re-sending system prompts, tool schemas, and conversation state on every turn. The remedies are practical and proven: cached static prefixes, intelligent routing of simple tasks to lightweight models, and gateway budgets that prevent runaway spend. Here are the numbers, an enterprise case in UAE dirhams, and five questions for your technology lead this week.

Why is my AI API bill so high? Unit prices fell, unit counts exploded

If you are asking why is my AI API bill so high when frontier model prices keep dropping, the answer is straightforward: unit prices fell, but unit counts exploded. Total spend equals unit count multiplied by unit price. While providers cut per-token rates by up to 98%, autonomous agents increased the number of tokens required per task by 10 to 30 times. The savings from cheaper inference were quickly erased by multi-turn loops, repeated context transmission, and unmonitored tool calls.

The headline economics of artificial intelligence look deceptively attractive. Average cost per million tokens across major providers fell from roughly $10 to $2.50 in a single year, per Ramp enterprise spending data cited by Cockroach Labs, whose own analysis calculates that the per-token cost of frontier intelligence fell by about 98% since early 2024. Goldman Sachs, cited by both TechCrunch and PointFive, projects global token consumption multiplying 24 times by 2030, with semiconductor advances driving underlying compute costs per token down by 60% to 70% a year.

Crucially, compute cost is what cloud providers spend, not what appears on your corporate invoice. Providers do not pass every operational saving through to enterprise subscribers, and consumption patterns have fundamentally altered.

The financial divergence occurred when corporate systems transitioned from simple chatbots to autonomous agents. Gartner analysis from March 2026, cited by Cockroach Labs, reveals that agentic workflows consume 5 to 30 times more tokens per task than a standard conversational interface. While a chatbot processes a single prompt and returns a single completion, an autonomous agent executes an iterative loop: it decomposes objectives, inspects environment state, retrieves external context, executes API tools, parses structured schemas, and conducts self-correction passes. A workflow that once required a single inference call now triggers 10 to 20 consecutive calls, with each iteration compounding context size.

The enterprise consequences are now evident across quarterly balance sheets:

Uber depleted its entire 2026 artificial intelligence coding budget by April. Developer adoption of autonomous coding agents climbed from 32% to 84% across a 5,000-engineer department in roughly three months. Spend averaged between $150 and $250 per engineer per month, while active power users approached $2,000 per month.
Microsoft Experiences and Devices division cancelled substantial portions of internal Claude Code licences during the same period to arrest unbudgeted consumption.
A Priceline engineering lead confirmed to TechCrunch that a routine seat-based renewal for Cursor escalated by a factor of four to five upon contract renegotiation.
DoiT and Sapio Research surveyed 500 enterprise buyers in February 2026, finding that 79% suffered material AI cost overruns in the preceding 12 months.
KPMG AI Pulse research in Q2 2026 revealed that only 36% of active organisations maintain direct token caps or programmable usage policies.
The FinOps Foundation State of FinOps 2026 survey shows 98% of practitioners now manage AI compute spend, up from 31% two years ago, identifying granular monitoring of tokens, request paths, and GPU utilisation as their single highest tooling priority.

Gartner's 2026 hype cycle estimates that 40% of enterprise agent deployments will be abandoned by 2027 purely on financial grounds: not because the software failed technically, but because operational margins collapsed. For financial controllers, chief technology officers, and engineering directors across the UAE and the wider GCC, understanding where these funds disappear is the prerequisite to regaining control.

Where does your AI API spend actually go?

When engineering teams audit client inference bills, token leakage routinely clusters within five architectural patterns. In order of fiscal severity:

1. Context re-transmission

Context re-transmission represents the largest invisible cost in multi-turn architectures. In standard ReAct (Reasoning and Acting) loops, an agent re-submits its entire conversational history, system instructions, tool definitions, and prior environmental observations with every single model invocation.

Stanford Digital Economy Lab research on agent cost attribution found that re-sent context accounts for 62% of aggregate agent inference expenditure. The organisation pays repeatedly for the model to re-read static instructions that never changed between steps. Retrieval-augmented generation (RAG) compounds this cost: dynamic document lookups expand input context windows by 3x to 5x on each step, according to analysis from PointFive.

2. Unbounded retries and cyclic loops

When an external API endpoint lags, returns invalid JSON, or yields stale records, an unmanaged agent defaults to self-correction. It attempts alternative queries, modifies formatting, or repeats the tool execution. Because every retry carries the cumulative context history, a three-retry sequence on a minor database query triples token consumption for that specific subtask.

Without rigid loop detection, agents can enter catastrophic cycles. In one documented incident, a healthcare provider running three production agents saw monthly inference expand from $12,000 to $68,000 in six weeks. A faulty retrieval parameter fetched clinical PDF attachments eight times larger than intended. Because each individual step appeared syntactically valid in isolation, application logs failed to raise alerts while cloud credits evaporated.

3. Frontier-model over-provisioning

A 15-step agentic sequence rarely requires frontier-grade reasoning across all 15 stages. In typical business workflows, at least half the intermediate steps consist of simple text extraction, sentiment classification, entity disambiguation, or JSON formatting.

Routing these routine chores to flagship reasoning models is commercial waste. One enterprise team that audited token consumption by task type and re-routed simple parsing steps to lightweight models reduced monthly API invoices from $40,000 to $24,000 without altering application output or accuracy: pure routing hygiene.

4. Supporting infrastructure overhead

Model inference constitutes roughly 20% of the comprehensive cost of ownership. The supporting software ecosystem introduces substantial recurring fees:

Vector database operations and embedding generation contribute an additional 3% to 8% above visible inference billing.
Enterprise vector storage hosting contributes 5% to 12%.
Knowledge base synchronisation demands recurring document re-embedding, an operational overhead that requires roughly 20% of the baseline monthly budget.
Automated evaluation frameworks using LLM-as-a-judge patterns demand between $0.01 and $0.10 per benchmarked response, and a production agent suite can easily execute 100 test variations per release.

5. Idle capacity and over-provisioned throughput

Organisations running self-hosted models or committed cloud instances often reserve dedicated compute to handle theoretical peak traffic. An audit across 23,000 enterprise GPU clusters revealed average hardware utilisation of merely 5%. PointFive observed committed provisioned throughput running at seven times actual consumption. These fixed monthly platform fees do not appear as token surges, making them particularly difficult for finance teams to spot on line-item reviews.

Unit price per token is the smallest commercial lever available to an enterprise. The decisive levers are aggregate token volume per completed task, and governance over which systems and individuals possess unmetered API credentials.

A worked example: how an AED 8,800 pilot became an AED 132,000 line item

The following model reflects real architectural patterns encountered across GCC financial institutions. The scenario examines a 3,000-employee banking group operating in the UAE that pilot-tested a customer-operations assistant as a single-turn query engine before converting it into an autonomous operational agent.

Phase 1: The pilot baseline (one call per query)

The initial prototype accepted a customer enquiry, performed a retrieval pass, and generated a resolution.

Input: 12,000 tokens (system policy rules, customer metadata, and retrieved knowledge context).
Output: 800 completion tokens.
Platform: Claude Sonnet 4.6 priced at standard published rates of $3.00 per million input tokens and $15.00 per million output tokens.

The unit calculation:

Input cost: (12,000 / 1,000,000) * $3.00 = $0.036
Output cost: (800 / 1,000,000) * $15.00 = $0.012
Combined cost: $0.048 (roughly five US cents per customer interaction)

At a volume of 50,000 customer queries per month, the monthly invoice totals $2,400. Converted at the UAE dirham peg of 3.6725, this equals roughly AED 8,814 per month. The CFO approved the business case.

Phase 2: Production agent deployment (15 calls per task)

To automate ticket execution, the engineering team deployed an autonomous agent architecture. Instead of returning text instructions, the agent queries the core banking ledger, checks identity verification databases, executes fraud verification routines, and reconciles system balances.

Across these actions, the agent averages 15 model calls per completed task. Each call sends approximately 12,000 input tokens (6,000 tokens of immutable system rules and API schemas, plus 6,000 tokens of dynamic state, tool outputs, and conversational context) alongside 800 output tokens.

The new unit calculation:

Task input volume: 15 calls * 12,000 tokens = 180,000 input tokens
Task input cost: (180,000 / 1,000,000) * $3.00 = $0.54
Task output volume: 15 calls * 800 tokens = 12,000 output tokens
Task output cost: (12,000 / 1,000,000) * $15.00 = $0.18
Combined cost per completed task: $0.72

At the same volume of 50,000 monthly transactions, total spend reaches:

Monthly expenditure: 50,000 * $0.72 = $36,000
Dirham equivalent: AED 132,210 per month (AED 1,586,520 annually)

The customer volume did not expand, the baseline questions did not alter, and provider rates remained static. The monthly invoice multiplied by 15 simply due to the agentic execution structure.

If integration faults cause five of those 15 steps to retry once due to transient database latencies, input consumption rises by an extra 60,000 tokens, adding $0.24 per task and lifting monthly invoices past AED 176,000.

Implementation stage	Invocations per task	Input per call	Cost per task	Monthly spend (50k tasks)	Multiple vs baseline
Pilot chatbot	1	12,000	$0.048	$2,400 (AED 8,814)	Baseline
Agent: Sonnet 4.6 (unoptimised)	15	12,000	$0.720	$36,000 (AED 132,210)	15.0x
Agent: prompt caching applied	15	12,000	$0.504	$25,200 (AED 92,547)	10.5x
Agent: caching + Haiku 4.5 routing	15 (8 Haiku, 7 Sonnet)	Dynamic	$0.292	$14,600 (AED 53,618)	6.1x

Applying structural engineering controls reduces monthly spend from $36,000 to $14,600: a 59.4% cost reduction. This confirms empirical benchmarks reported by Cockroach Labs, where systematic cost governance delivers 60% to 70% spend reductions on identical operational throughput.

How do you reduce high AI API bills?

Bringing runaway inference spend back under control requires three foundational engineering controls.

Control 1: Cache the static prompt prefix

Prompt caching delivers the highest immediate return on engineering time. As documented in Anthropic prompt caching specifications, reading a cached token on Claude Sonnet 4.6 costs $0.30 per million tokens, compared to the standard input rate of $3.00 per million: an immediate 90% discount on cache hits. Cache reads are priced at 0.1x of standard input costs across current generations, while a five-minute write costs 1.25x and an extended one-hour write costs 2.0x.

Because minimum cacheable thresholds sit at 1,024 tokens for Sonnet-class models, enterprise system prompts containing 6,000 tokens of workflow instructions and tool schemas qualify naturally.

In our financial services workflow, the fixed 6,000-token prefix is written to cache on call 1 at $3.75 per million tokens, and read from cache across the subsequent 14 execution steps at $0.30 per million tokens. The dynamic 6,000-token state payload continues to bill at the baseline $3.00 rate.

import os
from anthropic import Anthropic

client = Anthropic(api_key=os.environ.get("ANTHROPIC_API_KEY"))

# Fixed system instructions and comprehensive OpenAPI tool specifications
SYSTEM_PROMPT = os.environ.get("AGENT_BASE_PROMPT")

response = client.messages.create(
    model="claude-3-7-sonnet-20250219",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": SYSTEM_PROMPT,
            # Cache the static prefix: reads bill at 0.1x standard input rates
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[
        # Dynamic execution state sits after the cache breakpoint
        {"role": "user", "content": f"Task: {task_id}\nState: {current_state}"}
    ],
)

# Operational metrics to track in production monitoring
print(f"Cache write tokens: {response.usage.cache_creation_input_tokens}")
print(f"Cache read tokens:  {response.usage.cache_read_input_tokens}")

Engineering teams must prevent cache invalidation. Any non-static content placed ahead of the cache breakpoint invalidates the cryptographic hash, forcing a full input re-read. Placing timestamps, request IDs, customer identifiers, or non-deterministic schema ordering within the system prompt guarantees a near-zero cache hit rate. In one corporate implementation, an engineering team wondering why prompt caching delivered less than a 2% discount discovered that their orchestration framework inserted the current server microsecond timestamp into line two of the system prompt. Removing that single variable restored an 80% cache hit rate.

Control 2: Route cheap work to cheap models

Not every step in an agent pipeline requires flagship frontier reasoning. Complex tasks such as multi-variable planning, ambiguous policy reconciliation, and code synthesis demand frontier models. Simpler subtasks, including string extraction, classification, structural translation, and status formatting, execute reliably on compact models.

Current standard rates per million tokens illustrate the pricing tiers:

Model identifier	Standard input	Standard completion	Cached read	Optimal functional role
Claude Opus 5	$5.00	$25.00	$0.50	Complex legal reasoning, macro planning
Claude Sonnet 5	$2.00	$10.00	$0.20	Balanced enterprise execution
Claude Sonnet 4.6	$3.00	$15.00	$0.30	Deep agent loops, complex tool calling
Claude Haiku 4.5	$1.00	$5.00	$0.10	Extraction, schema translation, routing

In our 15-step agent scenario:

8 intermediate steps comprise deterministic extraction, parameter formatting, and simple checks. Routed to Claude Haiku 4.5 with pruned 4,000-token context windows and 300-token completions, these eight calls cost roughly $0.044 per task.
7 complex planning and ledger modification steps remain on Claude Sonnet 4.6 with prompt caching enabled, costing $0.248 per task.
Total per-task expenditure falls to approximately $0.292, saving over AED 78,000 every month on a 50,000-task baseline.

Model routing logic should live entirely within infrastructure configurations rather than hardcoded inside software repositories. Modern vendors are following this pattern: Factory introduced automated routing modules to match code generation tasks to cost-appropriate backends, and major hosted bills frequently show flagship requests routed dynamically to smaller tier endpoints where semantic complexity allows.

Regional governance and data residency

For organisations across the GCC, model routing addresses statutory compliance alongside financial waste. Under the UAE Federal Decree-Law No. 45 of 2021 regarding Personal Data Protection, as well as DIFC and ADGM data regulations, customer identifying information, sensitive banking records, and sovereign data assets must remain within controlled jurisdictions.

Intelligent gateways allow engineering teams to establish strict policy routes:

Workflows touching personally identifiable customer records, HR databases, or financial ledgers route strictly to self-hosted open-weights models running within local private data centres.
Sanitised, non-sensitive operational reasoning tasks route outward to international cloud endpoints.

This achieves stringent data sovereignty alongside optimal token spend.

Control 3: Token budgets, gateway enforcement, and chargeback

While prompt caching and model routing optimise unit costs, hard gateway guardrails cap total expenditure. Relying on end-of-month cloud provider statements is an expensive mistake:

Dedicated API keys per service principal: Never permit multiple microservices, engineering pods, or departments to share master provider keys. Issue dedicated, revocable credentials to each isolated workload.
Enforced token and spend quotas: Configure pre-flight budget allocations on hourly, daily, and monthly intervals. When a service hits 80% of its allocation, notify engineering teams automatically. When it hits 100%, block non-critical operations or degrade traffic gracefully.
Graceful degradation protocols: Instead of hard-failing mission-critical production systems when budgets expire, intelligent gateways should automatically down-route requests to economical fallback models or local open-source endpoints. The business continues operating, and finance avoids invoice shock.
Loop and retry circuit breakers: Implement hard ceiling thresholds on agent iterations. If an autonomous agent fails to converge on a valid tool output within six consecutive cycles, the gateway trips the circuit, halts inference, and alerts human operators.
Automated departmental chargeback: Departmental consumption metrics must map directly back to respective business unit cost centres. When operations, engineering, and compliance foot their own token bills, organizational consumption practices adjust quickly.

The architectural solution: FastLLM Proxy

Managing token governance, routing rules, data sovereignty, and consumption quotas directly within application code leads to brittle architectures. Every code update risks breaking financial guardrails or leaking API keys.

The reliable architectural solution is a centralized gateway deployed between internal applications and inference providers. This is the exact role fulfilled by FastLLM Proxy, our OpenAI-compatible gateway built to govern enterprise AI workloads.

FastLLM Proxy sits within your infrastructure, providing a unified access plane for on-premise inference engines such as vLLM, SGLang, and llama.cpp alongside 80 external model providers. It enforces the exact operational controls enterprise controllers require:

Fine-grained access controls: Issue keys with strict role-based access, rate limits, and hard daily or monthly dirham spend limits.
Intelligent traffic routing: Route requests based on caller identity, context size, remaining budget, latency, or semantic topic, without changing upstream client code.
KV cache affinity: Route repeated prefixes to the specific GPU instances that already hold cached KV states, maximizing hardware efficiency and minimizing latency.
Auditability without privacy risk: Log comprehensive token counts, response latencies, and financial costs per department while strictly omitting raw prompt and completion payloads.
High-throughput performance: On benchmarked infrastructure with GPU latency isolated, FastLLM Proxy delivers approximately 15 times the requests per second of standard Python proxies, ensuring zero bottlenecks.

Organisations deploying on-premise clusters or hybrid cloud topologies through our AI infrastructure practice use this pattern to establish self-hosted inference alongside audited commercial fallbacks. Sensitive corporate data never departs onshore environments, and financial executives retain full visibility over enterprise inference commitments.

Five questions for your IT lead this week

You do not need to read model weight matrices or debug CUDA kernels to establish rigorous AI financial governance. Pose these five questions to your technology leadership this week:

"Can you produce last month's AI inference spend broken down by individual business team, application name, and token volume?" If IT returns a consolidated cloud bill or indicates they must request details from vendors, your enterprise belongs to the 64% of organisations lacking direct token observability.
"What is our realized cost per completed business outcome, and is that unit cost declining over time?" Gross token volume is a measure of compute consumption, not enterprise value. If engineering cannot report cost per ticket resolved, claim processed, or report generated, teams are optimizing for vendor revenue rather than operational efficiency.
"What percentage of our production requests hit prompt caches, and what percentage route to lightweight models?" If both metrics sit near zero, your software architecture is running basic parsing tasks through premium reasoning tiers, overpaying by up to 70% on standard operations.
"What programmatic circuit breakers exist for runaway agent loops, and what happens when an application reaches 80% of its monthly budget?" If the answer is that budgets are reviewed after invoices arrive, your infrastructure has no defense against automated retry storms.
"Where do prompts containing corporate, financial, and customer data physically terminate, and how do we enforce UAE data residency regulations?" Routing rules must guarantee that proprietary records and regulated data classes remain inside designated geographic or on-premise boundaries.

What to do next

Remediating runaway AI expenses requires a clear four-step implementation sequence:

Conduct a comprehensive footprint inventory. Catalog every enterprise entry point consuming model APIs. Include developer coding assistants, department-level SaaS subscriptions, custom internal agents, vector database clusters, and corporate card subscriptions. Most leadership teams discover two to three times more active AI consumption points than originally budgeted.
Deploy an infrastructure gateway. Insert a unified gateway proxy between your internal microservices and model providers. Channeling traffic through a single ingress point immediately establishes visibility, cost attribution, and access enforcement.
Implement caching and dynamic routing. Restructure system prompts to separate static guidance from dynamic variables, enabling prompt caching across all supported providers. Establish routing configurations that direct basic extraction and formatting workloads to economical models.
Transition from token metrics to task metrics. Review AI financial performance on a cost-per-completed-task basis every month. Tie consumption allocations directly to departmental budgets through automated internal chargeback.

If structuring, deploying, and governing this infrastructure exceeds internal capacity, Azrty is a UAE company based in Dubai. Our own team in the UAE does the consulting, the deployment and the support for our clients.

Tokens are an accounting abstraction of compute. Completed business tasks are what generate economic enterprise value. Structure your software and financial governance around task delivery, and token costs will take care of themselves.

Originally published on Azrty.

AI demand forecasting for restaurants: a four-week test you can run on one outlet

Azrty — Sun, 11 Oct 2026 05:00:59 +0000

Most of the value in restaurant demand forecasting comes from data you already hold in your POS, your delivery channels, and an event calendar, not from another vendor dashboard. Here is how to test forecasting on one outlet and five high-waste items in four weeks, and judge it by waste and stock-outs rather than vendor demos.

The case is waste and stock-outs, not dashboards

AI demand forecasting for restaurants means predicting how much of each menu item, and therefore each ingredient, a given outlet will sell on a given day. That means moving beyond a generic sales trend chart to an operational statement: this branch will sell approximately 140 portions of salmon fillet on Friday evening, of which 45 will come through aggregators. Done properly, that output feeds prep lists, thawing schedules, and purchase orders directly. Done badly, it becomes another SaaS dashboard that kitchen teams ignore before morning service.

In the UAE and across the GCC, the commercial prize is unusually large. The Ministry of Climate Change and Environment estimates 3.27 million tonnes of food are wasted annually in the country, costing the economy roughly $3.5 billion each year. In Dubai, an estimated 38% of prepared food is thrown away, rising to 60% during Ramadan. Demand for meat, vegetables, fruit, and dairy rises by close to 50% during Ramadan, and much of the extra prepped inventory is discarded when service patterns shift. The national ne'ma initiative targets a 50% cut in food loss and waste across the UAE by 2030. For hospitality and F&B groups operating in the Emirates, measurement and reporting on waste will soon move from voluntary corporate sustainability targets to expected operational audits.

The commercial evidence shows that disciplined forecasting pays for itself rapidly. Hotels in the UAE using automated tracking and forecasting systems cut waste measurably: Hilton reduced food waste by 62% across 13 UAE properties in 2023, while Jumeirah Zabeel Saray avoided 20 tonnes of waste, equivalent to roughly 50,000 meals, in 2022 (Farrelly Mitchell).

The strategic question for F&B leaders is therefore not whether forecasting works in theory. It is how to establish whether it pays for your specific brand, at what infrastructure and subscription cost, before committing your teams to an expensive enterprise contract. This guide sets out a disciplined four-week test on a single outlet, using data your business already generates.

You already have the data: the four signals that matter

A 2026 review of Dubai restaurant technology found that the average mid-range restaurant runs 5–9 separate software tools and spends over $800 a month on technology, yet most operators cannot answer basic questions about sales by hour and channel because the data sits in disconnected systems (Nexara). Foodics alone holds roughly 40% of the UAE POS market, with competing systems such as Oracle Micros, Revel, and NCR Aloha carving up enterprise hotel dining. The good news for technical teams is that daily item-level forecasting does not require a complex multi-year data lake. You need only four primary signals.

Signal	Where it lives	What it gives the forecast
Item-level sales by day and hour	POS (dine-in, takeaway, and integrated delivery)	The base series: weekly seasonality, underlying trend, and item mix shifts
Delivery orders	Talabat, Deliveroo, Noon Food, via Deliverect or Otter	Channel-specific demand, which in Dubai Marina, JLT, and Business Bay can represent 60% of daily revenue
Reservations and covers	SevenRooms, Eat App, OpenTable	Known future baseline demand that can be banked days in advance
Events and calendar shifts	Group calendar, venue listings, national gazettes	Spikes that break naive moving averages: GITEX, Ramadan, Eid, DSF, and arena concerts

Two operational realities from the UAE context dictate how you clean and structure these signals:

Include delivery aggregators, or your forecast will fail. If aggregator orders flow directly into your POS through an integration bridge such as Deliverect or Otter, your central POS export will capture the full volume. If aggregator orders are received on dedicated tablets and rung in manually, or kept separate, you must export them independently. Deliverect allows operators to export order data to CSV with item-level line items, modifiers, timestamps, and channel tags. Splitting dine-in from delivery in your model is vital because delivery items have different prep lead times, packaging constraints, and margin profiles. Furthermore, sudden shifts in delivery promotions or aggregator commission structures alter item mix without warning.

The regional event calendar dictates variance. In Europe or North America, baseline weekly seasonality explains most restaurant variance outside statutory holidays. In the Gulf, macro events drive demand shifts of 30% to 100% on specific dates. A tech summit at Dubai World Trade Centre or Expo City floods nearby casual dining outlets while emptying suburban branches. School term dates, the shift from outdoor terrace dining in January to indoor dining in July, and the lunar calendar shifting Ramadan by 10 to 11 days every year make naive rolling averages useless. A model that only looks at the previous four Tuesdays will be confidently wrong on the dates that carry the highest revenue risk.

Week 0: measure the baseline before buying software

Before evaluating software vendors or building internal models, log physical waste for one full week across candidate items. This is not administrative overhead: it provides the baseline against which any algorithmic improvement must be judged. Choosing software before establishing your baseline waste profile is the primary reason inventory projects underdeliver (Focus HiDubai).

Keep the collection mechanism intentionally lightweight during this baseline week: a single laminated sheet or a shared digital spreadsheet with five columns placed directly at the prep and plating stations.

Item	Prep date	Qty wasted	Reason (spoilage, over-prep, customer return)	Est. ingredient cost (AED)

Select five items using two strict criteria:

High unit purchase cost (prime proteins, imported seafood, specialty cheeses).
Preparation that must occur before daily demand is known (items that require multi-hour marination, butchering, thawing, or par-cooking during the morning shift).

Omit non-perishable goods and long shelf-life dry stock from this initial pilot. Over-prepping frozen french fries or dry pasta forgives an erroneous forecast; misjudging fresh sea bass or wagyu tenderloin produces immediate financial write-offs.

From Week 0, calculate three foundational numbers for each of the five items:

Baseline waste percentage: total quantity wasted divided by total quantity prepped.
Stock-out incidence: count of services where the item had to be marked 86 (unavailable) on the POS and aggregator menus before the shift concluded.
Day-to-day demand variance: the standard deviation (σ) of daily portion sales, which establishes the required safety buffer.

Choose the approach: POS-native, standalone, or your own pipeline

In Dubai, AI forecasting primarily arrives through modern cloud POS providers rather than standalone tools (Nexara), while specialized F&B inventory vendors pitch direct POS-integrated forecasting engines to regional hospitality groups. The architectural choice comes down to organizational scale, technical capability, and the cost of forecast errors.

Approach	What you get	Underlying data inputs	Commercial model	When it is enough
POS-native (e.g. Foodics AI)	Predicted daily consumption per item, days of stock remaining, forecast vs actual analytics	POS sales and inventory ledger; no external integration plumbing	Included or sold as an add-on tier to existing POS contract	Single-brand operators or 1–5 outlets with standard menus and waste below 5%
Standalone F&B inventory platform	Item forecasting linked to automated supplier purchasing, invoice digitisation, recipe yield tracking	Real-time POS bridge, supplier catalogs, digital invoices, and physical waste logs	Monthly SaaS subscription per outlet (typically several hundred dirhams per site)	Multi-outlet hospitality groups, central production units, complex supplier networks
Custom data pipeline (Prophet, LightGBM)	Fully customizable item-level models incorporating custom event calendars, weather, and custom scrapers	Unified POS exports, aggregator APIs, table reservation engines, foot-traffic feeds	Open-source libraries on cloud compute plus internal or fractional engineering time	Groups with dedicated data engineering, unique menu concepts, or custom procurement backends

For operators running on regional POS leaders such as Foodics, testing the native route is straightforward. Foodics AI includes an inventory forecast dashboard that calculates anticipated daily consumption per item for the coming 30 days, projects stock-out dates, and provides forecast-versus-actual consumption variance filters by branch and SKU. Its forecasting engine, Foodics calls it Newton, runs directly against transaction history inside the POS database. For single-site operators or small regional chains, verifying Newton's projections against your Week 0 baseline is the fastest starting point because it requires zero API integration.

To structure this decision logically, apply this clear rule:

If food waste is under 3% of total food purchases and stock-outs occur less than once a month, turn on your native POS features, review them weekly, and avoid introducing new software.
If food waste sits between 3% and 8%, or core high-margin items run out weekly, execute the four-week pilot below using either POS-native tools or a lightweight custom script.
If your business runs multiple concepts, a central production facility, or experiences waste above 8%, a dedicated forecasting and inventory platform justifies its cost, but should still be piloted on a single high-volume outlet before group rollout.

A minimal forecast you can build yourself

If your POS lacks native predictive features, or if you want to benchmark vendor promises against a standard model before signing an annual contract, you can build an effective baseline using open-source tooling. Facebook's Prophet is well-suited for restaurant demand forecasting because it explicitly accommodates weekly seasonality, annual seasonality, and irregular regional holidays.

The official Prophet documentation highlights a critical requirement for time-series modeling in our region: every calendar event must be specified historically across your full training window as well as forward into the forecast horizon, with custom lower and upper windows to capture lead-up and spillover effects.

Below is an operational Python script configured for daily restaurant item sales, complete with UAE-specific holiday windows.

import pandas as pd
from prophet import Prophet

# Load daily item portions sold, extracted from POS reporting
# Required format: ds (date string YYYY-MM-DD), y (integer portion count)
sales = pd.read_csv("salmon_portions_daily.csv")
sales["ds"] = pd.to_datetime(sales["ds"])

# Define explicit regional calendar events with adjustment windows.
# Ramadan shifts annually on the Gregorian calendar; windows account for altered dining habits.
# GITEX represents a major multi-day corporate dining surge in central Dubai.
events = pd.DataFrame({
    "holiday": [
        "ramadan_period", "ramadan_period",
        "gitex_week", "gitex_week",
        "uae_national_day", "uae_national_day"
    ],
    "ds": pd.to_datetime([
        "2025-02-28", "2026-02-18",
        "2025-10-13", "2026-10-12",
        "2025-12-02", "2026-12-02"
    ]),
    "lower_window": [0, 0, -1, -1, 0, 0],
    "upper_window": [29, 29, 4, 4, 1, 1],
})

# Instantiate Prophet with multiplicative seasonality for growing concepts
model = Prophet(
    holidays=events,
    yearly_seasonality=True,
    weekly_seasonality=True,
    daily_seasonality=False,
    seasonality_mode="multiplicative"
)

model.fit(sales)

# Generate predictions for the upcoming 14-day service cycle
future = model.make_future_dataframe(periods=14)
forecast = model.predict(future)

# Output point predictions alongside 80% confidence boundaries
output_columns = ["ds", "yhat", "yhat_lower", "yhat_upper"]
print(forecast[output_columns].tail(14).to_string(index=False))

Two architectural insights apply when running models of this nature on restaurant POS datasets:

One full year of clean transaction history is the minimum threshold. Time-series models require at least two complete cycles to distinguish structural trend from seasonal patterns. Because Ramadan migrates through the solar calendar by roughly eleven days each year, historical data spanning 18 to 24 months is ideal to separate summer heat troughs from fasting periods.
Never optimize purely on the point forecast. If the model predicts a point estimate of 120 salmon mains with an 80% uncertainty interval spanning 102 to 138 portions, setting prep strictly to 120 guarantees that on half of the high-demand evenings your kitchen will run out of stock. Operational safety stock must be derived from the distribution interval, not left to kitchen guesswork.

From forecast to a prep sheet the kitchen will follow

A demand forecast that does not convert directly into morning kitchen tasks produces zero financial return. The translation from statistical output to line execution is an inventory safety-stock formula:

$$\text{Prep Target} = \text{Forecast} + (z \times \sigma)$$

Where $\sigma$ is the standard deviation of historical daily sales (or forecast residuals once the pilot is running), and $z$ represents the standard normal score corresponding to your desired service level.

For a 90% service level (1 stock-out in 10 services): $z = 1.28$
For a 95% service level (1 stock-out in 20 services): $z = 1.64$

Consider a worked operational example for Friday evening service:

Model point forecast: 120 portions of salmon fillet
Standard deviation of historical Friday sales ($\sigma$): 15 portions
Management target service level: 90% ($z = 1.28$)
Calculated safety buffer: $1.28 \times 15 = 19.2 \approx 20\text{ portions}$
Total prep target: $120 + 20 = 140\text{ portions}$

To prevent the common problem of over-prepping delicate proteins during the morning shift, divide the target into a two-stage prep sequence:

Stage 1 (Morning Prep, completed by 11:30): 110 portions prepped, portioned, and staged in the low-boy chillers.
Stage 2 (Service Par Adjustment, reviewed at 16:30): Kitchen display or head chef reviews mid-day sales and active evening reservations. If covers track at or above projection, prep the remaining 30 portions. If lunch service was sluggish or weather deters foot traffic, hold the remaining raw inventory sealed under temperature control.

Format the output as a physical single-page clipboard document or a static station screen. Kitchen staff under pressure will not log into analytical dashboards. The document must display only four columns: item name, morning batch target, afternoon checkpoint par, and actual prepped quantity signed by the station chef.

The four-week test schedule and evaluation framework

Execute the pilot across a single outlet and five tracked high-waste items using this four-week progression:

Week 0 (Baseline): Maintain standard operational routines. Log all spoilage, prep discard, and stock-outs on the manual tracking sheet. Calculate baseline waste mass, financial loss, and sales variance.
Weeks 1 and 2 (Assisted Prep): Generate daily prep sheets using your forecast model (whether POS-native or the Prophet pipeline). The head chef reviews the numbers each morning, retains authority to override any figure, but must record the reason for every manual adjustment (such as private catering booked off-system).
Weeks 3 and 4 (Closed-Loop Execution): Retain the prep sheet workflow while incorporating actual usage data from the prior fortnight back into the model parameters. Overrides should decline as kitchen confidence in the baseline figures stabilizes.

Evaluate the four-week pilot strictly against four objective metrics:

Metric	Measurement methodology	Success criteria after 4 weeks
Tracked ingredient waste	Physical scale logs and discarded portion tallies calculated against total prepped mass	Sustained reduction of at least 25% relative to Week 0 baseline
Stock-out frequency	Incident count of tracked items 86'd across POS and aggregator channels	1 or fewer stock-out incidents per tracked item per month
Forecast accuracy	Mean Absolute Percentage Error (MAPE) at a 24-hour horizon on target items	Consistent MAPE under 20% on stable core items
Protocol adherence	Proportion of shifts where prep sheets were completed, signed, and logged	Greater than 85% verified kitchen compliance

The final metric is the most critical. Industry operational benchmarks emphasize that even a 1% to 2% reduction in overall food cost compounds substantially across multi-unit operations, but only if forecasting, ordering, and waste logging are treated as a unified, followed discipline (Supy). A sophisticated predictive model exhibiting 12% MAPE that line cooks circumvent is useless. Conversely, a simple moving average with 22% MAPE that kitchen teams respect and execute daily will reliably eliminate over-prepping.

To understand the financial return on one mid-sized casual dining venue in Dubai:

Average daily revenue: AED 35,000 across dine-in and aggregators.
Monthly revenue: approximately AED 1,050,000.
Food cost at 30%: AED 315,000 monthly food expenditure.
Five tracked hero items represent roughly 20% of total food spend: AED 63,000.
If Week 0 shows prep discard on these items running at 15% (AED 9,450 monthly waste), reducing that discard by a modest one-third through structured prep targets saves over AED 3,100 per month on just five ingredients. Scaled across an entire kitchen or a group of five branches, the annual net savings exceed AED 180,000, completely outstripping the cost of the underlying technology.

What to do next

Select the candidate outlet: Pick the location with the highest inventory volatility or weakest food-cost margins, not your flagship showcase branch. High-volume, high-variance outlets expose operational friction fastest.
Execute Week 0 immediately: Print five-column waste logs and place them at prep stations on Monday. Measure baseline waste mass and 86 occurrences for seven consecutive days without altering kitchen habits.
Audit existing POS capability: Check whether your current POS subscription includes dormant forecasting modules. If you run Foodics, review the built-in inventory forecast views and verify whether historical recipe yields are mapped correctly.
Benchmark open-source performance: If your POS lacks native item forecasting, extract 12 months of daily item CSVs and run the lightweight Prophet script detailed above. Establish whether the open-source model delivers sub-20% MAPE on your five test items.
Connect the operating loop: When expanding beyond five test items to automated supplier purchase orders, eliminate manual CSV juggling. Connecting POS sales exports, time-series forecasting, and ERP purchasing into a reliable automated workflow requires robust platform engineering.

At Azrty, our team based in Dubai designs and builds these integrated AI pipelines and custom data architectures directly on your private infrastructure through our AI engineering services, ensuring your proprietary sales data remains secure. If you are evaluating where AI tools, automation, and predictive analytics will generate the highest financial return across your F&B portfolio before investing in custom software, our AI strategy and readiness engagement provides a clear, practical roadmap.

Run the test for four weeks on one outlet. Judge the outcome on kilograms saved and revenue protected, not on vendor slide decks. Your kitchen's own numbers will tell you exactly whether to scale the system.

Originally published on Azrty.

WhatsApp AI for real estate leads in Dubai: what to automate and what a broker must take over

Azrty — Sun, 11 Oct 2026 05:00:58 +0000

Dubai brokerages are being sold WhatsApp bots that answer every enquiry in seconds but hand agents half-qualified leads and annoy serious buyers with robotic follow-ups. The channel is right, the market is active, and the tooling is accessible. What most implementations get wrong is the boundary: what the AI may do alone, and the exact moment a human broker must take the conversation. This is a working rule for that boundary, with the numbers to hold it to.

WhatsApp is where Dubai property leads actually live. Buyers browsing from London, Mumbai or Riyadh at midnight tap the green button on Bayut, Property Finder and Instagram, and WhatsApp is the most used platform among internet users in the UAE (Statista). The market they are shopping in is the biggest on record: the Dubai Land Department counted over 270,000 transactions worth AED 917 billion in 2025, up 20% year on year, with an investor base of about 193,100 people, 129,600 of them new (Dubai Media Office).

So the pitch writes itself, and 2026 has produced plenty of vendors to deliver it: instant replies, bilingual qualification, viewings booked while the team sleeps. Some of it is true. What brokerages keep discovering after the demo is the failure mode: the bot answers instantly but hands over leads with half the qualifying fields missing, loops buyers through questions they already answered, or fires follow-up sequences that make serious buyers block the number. This article is the independent version: where AI earns its place, the hand-off rule that fixes the half-qualified problem, the compliance constraints specific to Dubai and to Meta's platform, a worked example with the numbers, and the metrics to hold any vendor to. It is written so you can brief an in-house team or a supplier with it directly.

What the vendors claim, and why pilots disappoint

The vendor content dominating this space is written by the people selling the tools, so treat its numbers with care. One vendor promises brokers "close 3x faster" (Groovy Web). Another claims chatbot leads convert "12% higher", that teams report "three times higher conversion rates", and that an agent saves "an average of 30 minutes per lead" in manual qualification (Korvax AI). A third claims "more than 85% of Dubai property enquiries start or continue on WhatsApp" (Propphy).

Check cited market figures to spot quality issues. That same roundup states Dubai recorded "around AED 761B in real estate transactions and 97,000+ deals in 2025". The official figure from the Dubai Media Office quoting DLD is over 270,000 transactions worth AED 917 billion. If vendor arithmetic on verifiable facts misses by that margin, treat conversion claims with caution until your pilot proves otherwise.

The disappointment usually has three causes, none of them about the underlying foundation model:

No hand-off rule. The bot has no explicit condition for when a conversation becomes a human's job, so it either hands over everything (creating spam for agents) or nothing (creating frustrated buyers).
No memory discipline. The bot re-asks budget or preferred area because qualification state is not persisted per lead across messages. Buyers read this as an incompetent agency.
Follow-up sequences tuned for volume, not judgement. Daily automated reminders and multi-step "still interested?" chains get the number blocked, and a blocked number is a dead channel for that buyer forever.

All three are design and process failures. They are cheap to fix relative to the build cost, which is why they belong in the brief before signing anything.

Where AI earns its place

Five tasks suit WhatsApp AI far better than an agent rota, all occurring within the first ten minutes:

First reply at 2am. A large share of Dubai enquiries arrive outside working hours, because the buyer is overseas or browsing after work. Vendors claim roughly half (Groovy Web); measure your own inbound timestamps before assuming. The first useful reply in under 60 seconds, at any hour, is the single biggest operational lever.
Arabic and English, natively. Language detection on the first incoming message, then staying in that language consistently, including for numbers and dates. A vendor that needs separate Arabic and English bots has not architected the system properly.
Qualification questions. Buy or rent, budget, target areas, off-plan or ready, cash or mortgage, resident or overseas, and timeline. Four to five questions maximum, asked conversationally, never repeated.
Answers from your listing data only. Service charges, payment plans, handover dates: answered exclusively from the fields in your CRM or portal feed, and "let me have a colleague confirm that" for anything not in the data payload. This single rule prevents most hallucinated-listing disasters.
Booking and logging. Offer viewing slots directly into an agent's calendar, and write every captured field into the CRM against the contact, with the source attribution (Bayut, Property Finder, Instagram, click-to-WhatsApp ad) attached cleanly.

One high-value special case: the AED 2 million property threshold for the UAE Golden Visa. A buyer whose stated budget clears AED 2 million should be flagged and routed to a senior broker immediately rather than sitting in the general qualification queue.

Meta's rules shape the design more than the AI does

Before arguing about system prompts, understand platform mechanics, as they dictate software architecture:

The 24-hour customer service window. When a user messages you, a 24-hour window opens for free-form replies, resetting with each user message. Outside this window, you can only send approved templates. The bot operates inside windows the buyer opens, matching the first-contact scenario.
Per-message pricing since 1 July 2025. Meta moved from conversation-based to per-message pricing; only delivered template messages incur charges, with rates varying by category and recipient country (Meta for Developers).
The free entry point window. If a lead arrives via a click-to-WhatsApp ad or Facebook page button and you reply within 24 hours, messages sent over the next 72 hours are free (WhatsApp Business Platform pricing). This makes running Instagram and Meta lead ads straight into the bot highly economical.
From 1 October 2026, budget for service messages. Meta charges per service message (bot replies), with 1,000 free per business number monthly, and charges utility templates sent inside an open window. UAE marketing message rates rose on 1 October 2025, with another increase scheduled for 1 October 2026. Meta introduced AED billing on 1 April 2026, allowing UAE businesses to track costs locally.

The design implication is clear: build a responder, not a broadcaster. An AI answering inbound enquiries inside open windows costs almost nothing in Meta fees today and modest amounts after October 2026. Running outbound follow-up blasts on marketing templates costs real money, converts worse, and risks phone number blocking.

The hand-off rule: brief anyone with this

Write this down before talking to a vendor. The hand-off rule has three parts: qualification conditions, immediate-escalation triggers, and guardrails.

1. The qualification condition

The bot hands the conversation to a human agent only when all four core fields are confirmed:

Budget range
At least one target area or community
Financing type (cash versus mortgage)
Purchasing or rental timeline

This is the anti-"half-qualified" rule. A lead lacking those four fields is not ready for an agent's active selling time.

2. Immediate escalation triggers

Regardless of qualification stage, the bot stops generating replies and alerts an on-duty human broker immediately when conversation touches:

An offer or price negotiation
A request for discounts or payment-plan flexibility
Mortgage specifics beyond pre-approval status
A customer complaint or expression of frustration
A legal, title deed or contract query
Anything about unit status or availability unverified in listing data
An explicit request to speak with a human broker

3. Guardrails

Maximum five qualifying questions per conversation. Never re-ask a field already captured in session state. The phrase "I want to talk to an agent" is always honoured immediately, with thread transfer and notification. The bot identifies itself as an assistant in its opening message: buyers tolerate transparent automated assistants far better than bots pretending to be human brokers.

Here is a configuration schema your engineering team or vendor can implement directly:

{
  \"language\": \"auto-detect, locked after first message (ar|en)\",
  \"identify_as_assistant\": true,
  \"max_qualifying_questions\": 5,
  \"qualifying_fields\": [
    \"intent\",
    \"budget\",
    \"areas\",
    \"finance\",
    \"timeline\"
  ],
  \"data_sources\": [
    \"crm_listings\",
    \"portal_feed\"
  ],
  \"answer_policy\": \"listing_data_only, else defer to agent\",
  \"handoff_condition\": \"budget && areas.length > 0 && finance && timeline\",
  \"escalate_immediately\": [
    \"offer\",
    \"discount\",
    \"payment_plan_flexibility\",
    \"mortgage_detail\",
    \"complaint\",
    \"legal\",
    \"agent_request\"
  ],
  \"golden_visa_flag\": \"budget >= 2000000\",
  \"routes\": {
    \"hot\": \"notify agent within 5 minutes, full thread attached\",
    \"warm\": \"morning digest, sorted by qualification score\",
    \"nurture\": \"weekly template update, opt-out link in every message\"
  }
}

What stays with the broker

Be explicit about what never belongs to AI, because this is where reputations and commissions are decided:

Negotiation and offers. Any proposed price, counter-offer, or request for bottom pricing routes to a human broker immediately.
Developer payment plans and off-plan specifics. Only verified listing data is quoted. Improvised details on escrow accounts, completion dates, or post-handover plans lead to formal disputes.
Mortgage advice. The bot asks if the buyer plans to use cash or mortgage financing and whether they hold pre-approval. It must never advise on banking products, margins, or eligibility.
Compliance-sensitive content. Every property advertised in Dubai requires a permit through the Trakheesi system. Since 24 April 2023, advertisements must carry the Madmoun QR code enabling buyers to verify authenticity. The Dubai Land Department states companies must comply to avoid legal violations (Dubai Land Department). A bot quoting listings in chat delivers marketing material: only listings with active Trakheesi permits belong in responses. Listing feeds must enforce this at data integration rather than prompt layers (Trakheesi user manual).
Anything emotional. A buyer unhappy about a missed viewing, confused by a Form B agreement, or anxious over deed transfers requires human attention from the first message.

Technical architecture: connecting WhatsApp to your stack

To make this reliable, avoid black-box tools that store customer leads in isolated silos. A production architecture built on the WhatsApp Cloud API requires four connected components:

[ Buyer on WhatsApp ]
         │
         ▼
[ Meta WhatsApp Cloud API ]
         │ (Webhook with HMAC-SHA256 signature)
         ▼
[ Ingress Gateway & Idempotency Store ]
         │ (Session state in Redis / PostgreSQL)
         ▼
[ Lead Qualification Engine & LLM Orchestrator ]
    ├── Listing Feed (Trakheesi-validated units only)
    ├── CRM Connector (Salesforce / HubSpot / Proptech CRM)
    └── Calendar Integration (Viewing slots)
         │ (Handoff trigger or Escalation)
         ▼
[ Agent Notification & Live Chat Console ]

Webhook ingress and idempotency. Meta webhooks deliver incoming messages via HTTP POST with X-Hub-Signature-256 headers. Meta frequently retries webhooks when latency spikes; your ingress service must store message IDs (wamid) in an idempotency cache (such as Redis) to ensure duplicate events do not trigger double replies.
State and session management. Maintain the lead's conversation stage, language preference, and captured parameters (budget, areas, finance, timeline) in persistent storage. If a lead takes four hours to answer the second question, the orchestrator retrieves their exact state rather than restarting the greeting sequence.
Data integration boundary. The LLM receives listing details through a retrieval tool that only queries active listings bearing valid Trakheesi permit numbers. If a buyer asks about a building where you have no verified inventory, the system executes an automated fallback: it records the enquiry, confirms interest, and routes the ticket to the listing acquisition desk.
CRM and calendar sync. When qualification completes, the orchestrator posts a structured payload to your CRM via REST API, creating or updating the contact record, logging the conversation transcript, and creating an assigned task for the broker with calendar viewing links generated through tools such as Microsoft Graph or Google Calendar.

Worked example: the night shift at a 12-agent brokerage

Take a 12-agent brokerage in Business Bay with a mix of portal and Instagram leads. These figures illustrate the model; you can substitute your own in an afternoon to evaluate whether automation pays for itself before contracting.

Baseline inputs:

400 total enquiries monthly across WhatsApp and portal-to-WhatsApp flows.
Inbox timestamps show 30% (120 enquiries) arrive between 10pm and 8am or on weekends, when unstaffed.
The sales team chases those 120 leads next morning, reaching roughly 60% (72 leads); agents report half have booked viewings with rival brokerages or stopped responding.
Average gross commission on a secondary sale is AED 80,000.

With the automated agent:

All 120 after-hours enquiries receive a relevant reply under 60 seconds, in the buyer's language, with qualifying criteria evaluated against live listing records.
45% complete initial qualification (54 leads).
Agents review and accept 70% as qualified (38 completed hand-offs) with budget, target area, finance method, and timeline logged in the CRM.
25% of accepted hand-offs convert into attended viewings (9 to 10 viewings monthly scheduled from after-hours traffic).
Over a six-month cycle, 15% of those viewings close (around 1.5 sales closings monthly, or 4 to 5 deals per quarter).

Financial return:
Four to five incremental sales closings quarterly generate AED 320,000 to AED 400,000 in gross commission from inbound leads that previously leaked to competing brokers.

Against that upside:

Custom builds range from AED 45,000 to AED 150,000, as quoted by Dubai market vendors (Korvax AI), typically requiring a 2 to 3 week WhatsApp Business API verification period, followed by a 6 to 10 week integration cycle.
Meta message fees: minimal today for in-window customer service replies, and from October 2026 modest monthly costs at this volume once the initial 1,000 free service messages per business number are consumed.
LLM inference costs per conversation: a fraction of the AED 0.15 to AED 0.40 per conversation fee typical of regional WhatsApp providers.

Even at the upper build estimate, the deployment achieves capital payback within the first operating quarter. The point is not buying software impulsively, but testing these unit economics against your brokerage's actual conversion numbers before allocating budget.

Build it, buy it, or bolt it on

Approach	What it looks like	Cost shape	Control over script and hand-off	Best suited for
Off-the-shelf tool on WhatsApp Business API	Third-party chatbot platform connected directly to your messaging inbox	Monthly recurring SaaS fee, plus template fees	Limited: vendor decides routing logic and dialogue trees	Solo brokers and small teams under 5 agents testing initial channel demand
Dubai real estate CRM with native WhatsApp	Built-in connector and workflow automation supplied by your existing CRM	CRM seat licences plus add-on messaging fees	Moderate: configurable workflows, but bound to the CRM vendor's roadmap	Brokerages already deeply embedded in that specific CRM platform
Custom agent on WhatsApp Cloud API with private LLM pipeline	Tailored integration connecting your listing feed, CRM and calendars to a dedicated AI backend	One-off implementation (AED 45,000 to 150,000) plus direct API consumption	Complete: full control over prompts, hand-off criteria, data privacy and custom models	Brokerages with 5 or more agents and significant monthly lead volume

If your brokerage already runs a real estate CRM with functional WhatsApp integration, test your script there first. Replacing a functioning core system solely for chatbot features adds unnecessary risk.

If your lead volume and brand standards justify owning the pipeline, build directly on the WhatsApp Cloud API. This ensures qualification scripts, customer records, and hand-off rules remain proprietary assets rather than data locked in third-party platforms. This is the integration architecture we design at Azrty, where our team in Dubai manages consulting, deployment, and ongoing technical support for clients across the UAE.

The numbers to track

An automation deployment is only as effective as its operational reporting. Track these seven metrics weekly from day one:

First response time (median and p95), segmented by hour. Maintain a target under 60 seconds at 2am as strictly as at 2pm.
Qualification completion rate. The percentage of inbound conversations capturing all four required qualifying fields. Rates below 40% suggest conversational friction.
Hand-off acceptance rate. The proportion of AI-transferred leads accepted by brokers as qualified. This monitors the half-qualified lead problem: if it drops below 60%, refine qualifying logic before investing more marketing budget.
Viewing booking conversion rate. Viewings booked per 100 inbound enquiries, comparing automated flows against manual handling, alongside attendance rates.
Escalation and block rate. Frequency of buyer agent requests or number blocks. Increasing drop-offs indicate poor tone or aggressive cadence.
WhatsApp cost per qualified hand-off. Monitor monthly to track Meta's October 2026 pricing updates and refine campaign sourcing.
Gross commission attributed to after-hours leads. The commercial metric justifying operational spend at annual review.

What to do next

Audit your last 100 unconverted enquiries. Review your previous 100 portal and social enquiries: record arrival timestamps, first-response latencies, and documented qualifying details. This afternoon spreadsheet exercise reveals whether your revenue bottleneck is after-hours responsiveness, follow-up discipline, or unqualified noise.
Formalise hand-off specifications before contracting. Use the configuration schema above. Any vendor unable to implement your exact hand-off rules and escalation triggers produces a generic chatbot rather than a lead engine.
Verify regulatory compliance at the database level. Ensure every listing in the agent's knowledge feed contains an active Trakheesi permit number, format listing cards with Madmoun QR verification codes, pre-approve outbound message templates with Meta, and ensure customer data complies with UAE data protection standards.
Run a four-week after-hours pilot. Restrict automated qualification strictly to WhatsApp enquiries arriving between 8pm and 8am and across weekends. Measure performance against the seven metrics above and compare directly with your 100-lead baseline.
Scale based on measured acceptance. Once broker hand-off acceptance sustains above 60% and confirmed after-hours viewings create pipeline value, expand integration across portal webhooks, Instagram direct messages, and click-to-WhatsApp paid campaigns, taking full economic advantage of Meta's 72-hour free messaging window.

An automated agent should never be tasked with closing complex real estate transactions. Its true objective is winning the first ten minutes of prospective buyer conversations, in Arabic and English, at any hour, delivering qualified leads directly to your brokers. When setting that boundary clearly, the underlying software delivers genuine commercial value.

Originally published on Azrty.