Ofsted and Ofqual publish their approach to AI

Ofsted

On 24 April 2024, Ofsted published its response setting out how it will use AI responsibly and fairly and its position on the use of AI by education providers.

In relation to the use of AI by the education providers falling under Ofsted’s regulation, it will assess the impact of AI on children and learners as part of its inspection and regulatory processes - and recognising that AI tools can help providers make better-informed decisions, reduce workload and lead to innovative ways of working, Ofsted will support use of AI by providers where it improves the care and education of children and learners. It will not, however, directly inspect the quality of AI tools used by providers as it is the responsibility of the leaders at institutions to ensure that their use of AI does not have a detrimental effect on outcomes (such as safeguarding), the quality of their provision or the decisions they take.

Ofsted then sets out what it expects from education providers in relation to their use of AI under the 5 principles set out in the AI regulation white paper:

• Safety, security and robustness - to assure themselves that AI solutions are secure and safe for users and protect users’ data and ensure they can identify and rectify bias or error
• Appropriate transparency and explainability – providers being transparent about their use of AI, and make sure they understand the suggestions it makes
• Fairness – to only use AI solutions that are ethically appropriate – in particular, Ofsted expects providers to consider bias relating to small groups and protected characteristics before using AI, monitor bias closely and correct problems where appropriate
• Accountability and governance- to ensure that providers and their staff have clear roles and responsibilities in relation to the monitoring, evaluation, maintenance and use of AI
• Contestability and redress – to make sure that staff are empowered to correct and overrule AI suggestions and allow and respond appropriately to concerns and complaints where AI may have caused error resulting in adverse consequences or unfair treatment

In terms of its own use of AI, Ofsted says that is currently using AI in its risk assessment of good schools to help decide whether a school that was judged good at its last inspection will receive a graded or ungraded inspection and it is also exploring how AI can help it make better decisions based on the information it holds, and to work more efficiently.

The areas where it believes AI could have the biggest benefits for its work include:

• assessing risk
• working more efficiently through automation
• making best use of data (particularly text) to generate new insights

In recognising the risks of AI it will follow the 5 principles set out in the AI regulation white paper by taking the following actions:

• Safety, security and robustness - to make sure AI solutions are secure and safe for users and that they protect users’ data and to continually test AI solutions to identify and rectify bias and error
• Appropriate transparency and explainability - being transparent about its use of AI, and test solutions sufficiently to understand the decisions it makes
• Fairness – to only use AI solutions that are ethically appropriate – in particular, to fully consider any bias relating to small groups and protected characteristics at the development stage and then monitor this closely and correct it where appropriate
• Accountability and governance- to provide clear guidance and rules for developers and users of AI within Ofsted about their responsibilities
• Contestability and redress – to make sure that staff are empowered to correct and overrule AI suggestions with decisions being made by the user (not the technology) and to manage concerns and complaints through its existing complaints procedure

Ofsted recognises the pace at which new technologies are developing and will keep up to date with published research and communicate with providers to learn from their experience of AI in their settings. It will also will develop inspectors’ knowledge about AI so that they have the knowledge and skills to consider AI and its different uses and will work with other regulators and central government departments, including the DfE and DSIT to ensure that its approach remains consistent with the latest guidance.

Ofqual

Also on 24 April 2024, Ofqual published its approach to regulating the use of AI in the qualifications sector. It comments that the emergence of widely accessible generative AI has raised several new considerations for the regulation of qualifications and assessments, including new opportunities for AI to support the design, development and delivery of high-quality assessment, alongside new risks around AI’s use in non-exam assessments.

As a result it has established 5 key objectives to shape its AI regulatory work:

• Ensuring fairness for students - to ensure the use of AI does not lead to unfair outcomes for students, loss of currency of their achievements, and/or lack of clarity over what constitutes malpractice
• Maintaining validity of qualifications - recognising and managing potential threats to validity through varying applications of AI, and identifying and acting on activities more susceptible to being adversely affected by use of AI
• Protecting security – being alert to malpractice including assessing and addressing vulnerabilities to assessment security from AI and protecting student data and questions and paper security
• Maintaining public confidence - ensuring that public confidence around the use and effects of AI, and that steps are taken to maintain confidence
• Enabling innovation - within design, development and delivery of qualifications, or ways to generate efficiencies in awarding organisation operations

Ofqual’s approach to regulating AI is to ensure that where AI is used by awarding organisations, it is applied in a safe and appropriate way that does not threaten the fairness and standards of, or public confidence in, qualifications and sets out what it is doing in the following areas:

• Managing malpractice risks
• Using AI to mark students’ work
• Using AI in remote invigilation

In terms of future steps Ofqual says it will:

• Evaluate evidence about how awarding organisations handle AI related malpractice, alongside following up operational developments and incidents where necessary
• Use evidence to introduce further guidance, advice and/or clarifications identifying where interventions are required to mitigate the harms that could come from inappropriate use of AI.
• Research into AI marking to assess the abilities and performance of such platforms
• Research into perceptions about the use of AI in the design, delivery and marking of qualifications, and the implications of malpractice to understand the views among students, teachers, parents and the wider public on the role of AI in assessment
• Liaise with regulated awarding organisations to build collective knowledge about the risks and opportunities from AI, and support introduction of guidance and advice that supports awarding organisation activities
• Implement AI specific categories to be used by awarding organisations when reporting malpractice to Ofqual

Comment

The DSIT’s engagement with regulators is just one of a range of measures it is taking to develop a pro-innovation regulatory framework for AI, in response to the recent consultation. This includes developing its domestic policy position on AI regulation, taking actions to promote AI opportunities and tackle AI risks, building out a central function to support regulators, and supporting international collaboration on AI governance. 

This builds on work already commenced in the education sector following the DfE’s Call for Evidence on Generative AI in education in November 2023. This identified a range of use cases for AI in education as well as identifying benefits, challenges, opportunities and potential legal risks in this area. This followed a ‘hackathon’ project in October 2023 to further understand possible use cases in the sector, the findings of which will be published in the Spring.

We can certainly expect more developments in relation to the domestic regulatory framework governing AI,  both in the education sector and beyond, in the months ahead. This was signalled by the DSIT in its consultation response, which noted that “…some mandatory measures will ultimately be required across all jurisdictions to address potential AI-related harms, ensure public safety, and let us realise the transformative opportunities that the technology offers”.

In the longer-term, with an estimated 10-30% of jobs being vulnerable to automation as a result of developments in AI, the DfE anticipates that the UK education system will need to adapt its offering to ensure that individuals in the workforce have the skills they need to make the most of the potential benefits that advances in AI will bring in the years ahead.

It is clear therefore that the development of AI is having, and will continue to have, profound implications for the sector.