AI privacy concerns in schools, and what to do about them

Where the risk sits

The risk people imagine is a dramatic leak. The risk that happens is a teacher at half past nine on a Sunday, pasting a paragraph about a child into a free chatbot to get help phrasing a report. No malice, real time pressure, and pupil data has left the school.

The second risk is quieter and larger: AI features appearing inside systems the school already pays for. An MIS adds a summarising feature. A reading platform adds an analytics model. A safeguarding tool adds automated flagging. Nobody made a decision; an update was applied, and personal data is now being processed in a way the school has not assessed.

What goes wrong, specifically

• Identifiable detail in a prompt — a name, a class, a diagnosis, a home situation. Once sent to a general-purpose service, the school no longer controls where it is stored or whether it trains a model.
• Re-identification from detail that looked harmless — "the boy in 3rd class whose father died in March" names nobody and identifies one child.
• Output taken at face value. A model that invents a plausible citation or a confident wrong answer about entitlement does real damage when it reaches a parent unchecked.
• Bias in anything that sorts or scores pupils, which lands hardest on the pupils with least margin.
• More data collected than the purpose needs — the data minimisation principle, broken by default in tools designed to ingest everything.

Controls that work in a real school

The controls that hold are the ones a busy teacher can follow at speed:

• A short, specific list of what may never go into a general-purpose tool — names, SEN and IEP detail, medical and behavioural information, anything from a child-protection record.
• A stated alternative, because a prohibition with no alternative gets routed around. If stripped-down planning use is fine, say so.
• Human review before any AI-assisted text reaches a pupil, a parent or a record.
• A named person who assesses new AI features in existing systems before they are switched on.
• A school-provided tool that is safe to use, which removes the incentive to reach for whatever is free.

That last one is the difference between a policy that is obeyed and a policy that is circumvented. Staff do not want to break rules; they want the report finished.

The policy half of this

Everything above has to live somewhere a school can point at: an AI policy adopted by the Board of Management, with a record of which version is in force and which staff have acknowledged it. Guidance circulated by email is advice, not a control.

That document — drafting it, consulting on it, getting it approved, tracking acknowledgement and reviewing it often enough to keep up — is Giraft’s job rather than AskArk’s. The two halves are different problems and it is worth not confusing them.