Start with a written AI policy owned by the executive, then attach it to controls the college already runs: data protection, safeguarding, assessment integrity and supplier due diligence. Jisc and the Association of Colleges publish sector principles a college can adopt as its statement of intent. Keep a dated log of which tools are approved, and why.
Who owns AI governance in a college?
The executive owns it: the principal or a named deputy, with the corporation holding assurance. AI governance goes wrong when it is handed to IT, because most of the decisions are not technical. They are decisions about learner data, assessment, safeguarding and staff workload.
I would name three seats in the policy itself. A senior owner with the authority to refuse a tool. The data protection officer, who has to be consulted before any personal data goes near a new system. And a curriculum or quality lead, because the people who set assessment rules should be the ones deciding what help a learner may use. IT takes the fourth seat, not the first. Its job is to tell you what a tool actually connects to and what leaves the building.
One thing matters more than the structure: a single approvals log. One list of tools, who requested each, what data it touches, who approved it and when. A college with that list can answer a governor's question in a minute. A college without it is relying on memory, and on nobody asking.
What should a college AI policy actually say?
It should say what staff and learners may use, for what, with which data, and who to ask when the answer is not obvious. A policy made only of prohibitions gets ignored, because staff and learners are already using these tools on their own phones.
Keep it short enough that people read it. Cover the approved tool list and the route to request something new. Cover what must never be typed into a general-purpose tool: learner names, safeguarding notes, support plan or EHCP detail, staff HR matters, unreleased assessment material. Cover the rule that a person checks any output before it reaches a learner, a parent, an employer or a funding return. Cover how AI use is declared in assessed work. And state what happens when the policy is ignored, for staff as well as learners.
One line saves most of the later arguments: AI may draft, a person decides. Date the policy, name its owner, and set a review date you will keep. Unapproved tools are a governance problem before they are a security problem, which is the argument I have made elsewhere.
How do the sector principles from Jisc and the Association of Colleges help?
They give you a credible starting position you did not have to draft yourself. Jisc published principles for the use of AI in FE colleges with the Association of Colleges technology reference group, written for colleges to adopt as a statement of intent and adapt locally.
Adopting them gives the board something to point at and staff something to recognise. It is not compliance. Principles say what you intend. Controls say what you do, and only controls survive an incident. Read them alongside the Department for Education's guidance on generative AI in education, which places responsibility for protecting personal data and for checking outputs on the setting rather than the supplier. Adopting sector principles is not an endorsement of any tool, including ours.
What does data protection require before a tool is approved?
A lawful basis, an entry in your record of processing, and a data protection impact assessment where the processing is likely to be high risk. The ICO requires a DPIA before high-risk processing and its screening factors include the use of innovative technology and data about vulnerable people.
A general further education cohort crosses that threshold without difficulty. You hold records on 16 to 18 year olds, some learners under 16, adults on return-to-work programmes, and learners with support needs. Settle these points in writing before approval: whether your data is used to train the supplier's models, where it is processed, how long it is kept, who the subprocessors are, whether you can obtain an export and a deletion, and who at the supplier can read it. If a supplier will not answer in writing, you have your answer. There is more detail in do you need a DPIA before deploying AI.
How do we protect learner data and meet safeguarding duties?
Keep safeguarding records out of general-purpose tools completely, and treat a new AI feature inside a system that already holds them as a change requiring review. A supplier switching on an assistant is not a minor release when the data behind it is child protection material.
Two practical risks are worth training on by name. First, a member of staff pasting a referral into a public tool to improve the wording: the name and the detail have then left the college, and no policy retrieves them. Second, filtering and monitoring. A chat feature inside an approved platform is a route to content your filters were never pointed at, so tell the people who run monitoring before you approve it, not after.
Where work genuinely needs the college's own records, the processing should happen on infrastructure the college controls. That is what we build. The Mickai Sovereign Intelligence Operating System (SIOS) runs on hardware the customer owns, is capable of running offline, and moves no data offsite, with private knowledge bases built from the organisation's own documents. Cloud services stay useful for work that is not sensitive: prospectus copy, timetable scenarios, general drafting. The line is the data, not the technology.
How do we handle assessment integrity without banning AI?
Change what you assess and how you verify it, and do not lean on detection software. Jisc's FE principles warn that AI detection tools cannot conclusively prove that a piece of text was written by AI, that they produce false positives and that they are straightforward to defeat. A false accusation against a learner is a serious matter, so no malpractice case should rest on a detector score.
What holds up is duller and works better: a declaration at submission stating what help was used; a per-unit statement of what is permitted, because awarding organisations set their own rules and one college-wide line will contradict some of them; verification through supervised, practical or oral components; retained drafts and version history; and staff trained on what evidence a malpractice process actually needs. An outright ban pushes use underground and leaves you with no record of it at all.
What should supplier due diligence cover?
Where data goes, what is done with it there, and what you can prove afterwards. The first two are on most procurement checklists. The third is usually missing, and it is the one you need when something goes wrong.
Ask for an exportable, verifiable record of what the system did: which user, which action, which data, when, and who approved it. Under SIOS, the Open Audit Record seals every consequential action with ML-DSA-65, the post-quantum signature scheme NIST published as FIPS 204 in 2024. An auditor verifies an exported record offline with a public key, using tools that are not ours. That is tamper-evident rather than tamper-proof, and the distinction is the point: nothing stops a person altering a record, but altering it makes verification fail, so the alteration shows. Consequential actions also wait for a named person to approve them, which is the control a college already applies to a payment run.
One caveat, since colleges hold a great deal of scanned paper. Our local OCR runtime has read scanned PDFs in controlled tests, and the extraction and ingestion path into SIOS is still being completed. I would rather say that now than have it found in a pilot. SIOS is in closed beta, with one regulated organisation onboarding as a design partner.
How do we evidence AI governance to governors and funding bodies?
Three documents, kept current: the dated policy with a named owner, the approvals log, and the DPIAs. Take those to the board rather than a presentation about AI.
Then add a standing item with numbers a governor can question: tools approved and declined this term, DPIAs completed, staff trained, incidents involving data entered into unapproved tools, and malpractice cases where AI was alleged. Put AI on the risk register with a named owner and a stated mitigation. Where AI output touches a funding return or the ILR, record that a person checked and signed the figure, because accountability for the number does not transfer to the software.
None of this is an argument against using AI in colleges, or against the companies building the compute and cloud layer underneath it. It is an argument against the assumption that a college must ship learner data offsite and take a supplier's word for what happened to it. The architecture we build on is covered by 104 filed UK patent applications carrying 2,340 claims, and the design goal in all of them is the same: whatever the system did, you can show it afterwards. Sovereign AI is the short name for that.
Frequently asked questions
Does a college need a separate AI policy?
Not necessarily a standalone document, but the decisions have to be written down somewhere. Most colleges find one short AI policy easier to maintain than amendments scattered across data protection, safeguarding, acceptable use and assessment policies. Whichever route you take, name an owner, date it, list the approved tools and state who approves new ones.
Can learner data be put into an AI tool?
Only into a tool you have approved for that purpose, with a lawful basis, an entry in your record of processing and a DPIA where the risk is high. Never into a general-purpose consumer tool. Safeguarding notes, support plan detail and anything about a learner's health or home circumstances should stay out of shared tools entirely.
Who should approve new AI tools in a college?
A named senior owner, advised by the data protection officer and IT, with the curriculum or quality lead consulted wherever assessment is affected. One person holds the decision and signs the approvals log. IT should say what the tool connects to and what leaves the college, but it should not be the function deciding whether the college accepts the risk.
Do we need a DPIA for AI in further education?
Usually yes. The ICO requires a DPIA before processing likely to result in high risk, and its screening factors include the use of innovative technology and data about vulnerable people. A college running records on young and vulnerable learners through a new AI system crosses that line easily. Do the DPIA before approval, not after the pilot.
How do we show governors that AI use is controlled?
Give them the dated policy, the approvals log and the DPIAs, then a termly report with numbers: tools approved and declined, DPIAs completed, staff trained, incidents involving unapproved tools, and malpractice cases where AI was alleged. Add AI to the risk register with a named owner. Evidence carries a board meeting better than assurance language does.
Related briefings
Education
- Private AI for University Staff: Keeping Data In House
- Student Work and AI Training: Copyright and Consent
Data protection and UK GDPR
- UK GDPR and AI: Does Your Data Have to Stay in the UK?
- UK Data Storage vs AI Processing: What Is the Difference
Part of a series of 60 briefings on deploying and governing AI in UK regulated organisations, archived with a DOI at 10.5281/zenodo.22975756.
Evaluating AI for a regulated organisation? Mickai runs on hardware you own, offline. Consequential actions wait for a named person to approve them, and what the AI did is sealed into a signed record an auditor can check without us. Applications for the invitation-only closed beta are open. Apply for the closed beta.
Written by Micky Irons, founder and chief executive of Mickai LTD.
Top comments (0)