Yes, often. A single AI workstation usually runs on an ordinary office circuit. A shared departmental server needs a lockable room, a dedicated power circuit, cooling that copes with sustained load, and a location where fan noise will not disturb staff. We install on a single workstation, in the organisation's server room, or on a fully air-gapped site.
Does an on-premise AI server need a data centre?
No, not in most cases. The phrase "on-premise AI" makes people picture a raised floor and rows of racks, and that picture stops sensible projects before they start. We install in one of three shapes: on a single workstation, in the organisation's server room, or on a fully air-gapped site. The first of those is a desk-side machine in an ordinary room.
The question to settle first is not the building. It is how many people will use the system at once, and for what. One analyst running long jobs against internal documents is a workstation problem. (Our local OCR runtime has read scanned PDFs in controlled tests, and the extraction and ingestion integration into SIOS is still being completed, so treat document-heavy workloads as a live area rather than a finished one.) Sixty staff across two departments, all expecting an answer in seconds, is a shared server problem, and a shared server is what pulls in power, cooling, noise and locks.
Decide the concurrency, then decide the room, because concurrency is also what decides what the hardware has to be. Doing it the other way round is how organisations pay for a server room they did not need, or put a loud machine next to the finance team.
What power does an AI server draw in an office?
Enough that you should ask an electrician, and not enough to need anything unusual in most offices. Draw is dominated by the graphics hardware: one card in a tower is a different proposition from several cards in a rack chassis. MICKAI® SIOS is hardware-gated, and what a machine can run is decided by its memory and its graphics capability, so the specification conversation and the power conversation are the same conversation.
Two practical points. Sustained load matters more than peak. An office circuit copes happily with a kettle for three minutes. A machine working steadily through an afternoon is a different kind of demand, and the socket, the cable run and the protective device all need to suit it. Your electrician works to BS 7671, the wiring regulations, and will tell you quickly whether the room's existing circuit has headroom or whether you want a dedicated one.
Then, avoid sharing a circuit with a printer, a kitchen, or anything with a heating element in it. For a shared server, ask for a dedicated circuit. A single workstation can usually stay on the general office supply, if the electrician agrees.
How much heat does an AI server produce, and what cooling does a small room need?
Nearly all the electricity the machine draws leaves it again as heat, so heat output tracks power draw closely. A single workstation in a normally ventilated room is usually a non-event. The problem is the small sealed room: a converted store cupboard, or an internal comms room with the door shut and no extract. Temperature climbs, the machine protects itself by slowing down, and users report that the system got worse although nobody changed anything.
Test this rather than model it. Put the machine under sustained load for an hour, with the door closed as it would be in daily use, and watch both the room temperature and the machine's own thermal readings. If the temperature is still rising at the end of the hour, the room needs air movement: an extract, a vent into a larger space, or a cooling unit.
Be careful with existing comms rooms. Many have a small cooling unit specified years ago for network switches, which are a light thermal load by comparison. Sized for switches is not sized for this.
How loud is an AI server, and where should it sit?
Loud enough that placement is a real decision, and the form factor decides how loud. Rack-mounted chassis use small fans spinning fast, which is the least pleasant option, and it sounds worse under sustained load than in the short burst you hear at boot. Tower workstations use larger, slower fans and are noticeably quieter, which is one reason the workstation shape suits a first installation.
Do not put a shared server in open-plan space, in a meeting room, or against the wall of someone's office. If the only room available sits next to occupied desks, you are choosing between a cooling problem and a noise problem, because both are solved by opening the door. A rack in a lockable room away from desks answers both at once, and that is usually why organisations end up there.
What physical security does the room need?
Treat the machine as the data, because it is. NCSC's Cyber Assessment Framework covers this under principle B3, data security, which expects stored data to be protected against unauthorised access, physical access to the equipment included. In plain terms: a lockable room, a short and named list of people holding keys or card access, and a record of who went in. Where personal data is involved, UK GDPR Article 32 requires appropriate technical and organisational measures for the security of processing, and a room anyone can walk into is hard to argue as appropriate.
It is worth naming what does not cover this. The five controls in Cyber Essentials are technical: firewalls, secure configuration, security update management, user access control and malware protection. None of them locks a door. If your assurance story rests on Cyber Essentials alone, the room is a gap in it.
Two measures limit the damage on a bad day. Full-disk encryption at rest, so a stolen machine is an equipment loss rather than a data breach. And the Open Audit Record, which seals every consequential action under ML-DSA-65, the post-quantum signature scheme NIST published as FIPS 204 in 2024. That record is tamper-evident, not tamper-proof, and the difference matters in a room with a door: nothing physically prevents someone interfering with a file, but interference makes verification fail. An auditor exports the record and checks it offline with a public key, using tools that are not ours.
What happens to the AI during a power cut?
It stops, and you are the one who restarts it. That is the honest trade for keeping the system in your building. It is also a different kind of failure from a cloud outage, where the restart is someone else's to make. Cloud stays the right answer for plenty of non-regulated work. The difference here is who holds the controls when something breaks.
Work in progress at the moment of the cut is lost, and the job is run again. Actions already completed are sealed on disk and still verify afterwards. Because consequential actions wait for a named person to approve them, a cut during an approval leaves the action unapproved rather than half-done, which is the behaviour you want.
An uninterruptible supply sized for a graceful shutdown is the sensible baseline for a shared server: enough time to close down in order instead of dropping. Generators are a separate conversation. If the work truly cannot pause for the length of a power cut, that is a resilience requirement for the whole site, not an AI requirement.
What network isolation should the room have?
More than most installations are given. Put the machine on its own segment, deny outbound traffic by default, and allow only the internal services it genuinely needs: identity, the document stores you chose, logging. The system does not need the public internet to run models, so outbound internet access should be absent rather than merely unused.
Then be clear with yourself about which property you are relying on. No data egress is a design property of the software and a configuration property of your network, and both have to hold. If you would rather see it than be told it, a fully air-gapped installation removes the argument: nothing leaves because there is no route out, and updates arrive by a controlled, reviewed transfer. DSIT's AI Cyber Security Code of Practice sets out voluntary principles for the security of AI systems across their lifecycle, including securing the infrastructure they run on, and it is a reasonable checklist to hold your own installation against.
When does a proper server room or data centre become necessary?
When concurrency, resilience or headcount outgrow the room you have. Three triggers in practice: more than one department depending on the system daily, a service expectation that makes an afternoon of downtime a real problem, or a second and third machine, at which point rack space, structured cooling and proper power distribution are cheaper to do once than to retrofit twice.
The order I recommend is plain. Install on a workstation. Put real work through it. Learn what your own load actually looks like. Then size the room against your own measurements rather than anyone's assumption, including ours. Hardware-gated licensing means the machine you buy is the machine the system is bound to, so that decision deserves evidence behind it.
We are in closed beta, with one regulated company onboarding as a design partner, and in the conversations we have had so far the room has been easier to settle than people expected. None of this is exotic. It is power, cooling, a lock and a plan for the day the power goes. A data centre is where this ends for some organisations. It is rarely where it should begin.
Frequently asked questions
Can we put an AI server in a cupboard or a comms room?
Sometimes, and only after a heat test. A sealed cupboard with no ventilation warms up under sustained load until the machine slows itself down to protect its own hardware. A comms room is often better because it already locks, but check its cooling: units fitted for network switches were specified for a much lighter thermal load than this.
Do we need three-phase power for an on-premise AI server?
Usually not. A single workstation or a single server runs on the ordinary single-phase supply an office already has. Three-phase becomes relevant when you are filling a rack with several machines and want proper power distribution behind them. Have an electrician assess the actual circuit under sustained load rather than assuming: that assessment is quick and it settles the question.
Delete this FAQ item (question and answer) to return to five. Its content is already covered by "Can we put an AI server in a cupboard or a comms room?" and by the cooling H2.
That depends on the room more than the machine. A workstation in a normally ventilated office usually needs nothing at all. A shared server behind a closed door in a small internal room often does. Run it under sustained load for an hour with the door shut and watch the temperature: if it is still climbing at the end, you need cooling.
Who is responsible for physically securing the machine?
You are, and it should be written down with a named owner. NCSC's Cyber Assessment Framework covers data security under principle B3, which expects data at rest to be protected against unauthorised access, physical access to the equipment included. In practice: a lockable room, a short key or card list, and a record of entry. We advise on it, but the estate is yours.
Do we need uninterruptible power for an on-premise AI server?
For a shared server, yes, sized for a graceful shutdown rather than for continued running. Work in progress when the power drops is lost and the job is simply run again, while actions already completed are sealed on disk and still verify afterwards. A single workstation can often carry the same risk any office desktop carries.
What changes if we want a fully air-gapped installation?
The route out disappears and your process changes. Updates, models and data arrive by a controlled, reviewed transfer instead of over a network link, and somebody has to own that routine. The machine itself is no different: same power, same cooling, same noise, same lock on the door. The gain is that no data egress stops being a claim to check and becomes a physical fact.
Related briefings
Deploying private AI
- Private AI for Business: What It Is and How It Works
- Offline AI Assistant: What It Can Do for Your Staff
- Private AI Deployment Options: Workstation to Air Gap
- How Sovereign AI Integrates With Your Existing Systems
- From Data to Answers, Without the Data Leaving
Documents and retrieval
- What Is RAG? Retrieval-Augmented Generation Explained
- How to Prepare Documents for a Private AI Knowledge Base
Part of a series of 60 briefings on deploying and governing AI in UK regulated organisations, archived with a DOI at 10.5281/zenodo.22975756.
Evaluating AI for a regulated organisation? Mickai runs on hardware you own, offline. Consequential actions wait for a named person to approve them, and what the AI did is sealed into a signed record an auditor can check without us. Applications for the invitation-only closed beta are open. Apply for the closed beta.
Written by Micky Irons, founder and chief executive of Mickai LTD.
Top comments (0)