The ICO calls for views as part of its consultation series

Why should I read this?

Generative AI (or GenAI) refers to the use of AI to create new content, like text, images, music, audio, and videos. GenAI has already become a fast-growing tool for people and businesses – and the legal landscape is changing quickly to adapt to it.

The ICO’s consultation series outlines the ICO’s preliminary thinking on the interpretation of data protection laws in relation to GenAI models.

Stakeholders with an interest in GenAI are invited to share their thoughts on this topic and help inform the ICO’s position (and, ultimately, its enforcement stance and strategy on regulating GenAI from a data protection perspective). In addition, it is likely the ICO will also collaborate and work closely with its sister regulators who form part of the DRCF (or Digital Regulation Cooperation Forum), namely, the FCA, Ofcom and CMA. The DRCF recently released its Workplan for 2004/5 which indicates AI as a one of the focal areas of the DRCF’s collaborative regulation strategy. Separately, it should be borne in mind that the ICO has also recently issued a separate and specific response to the UK Governments Pro-Innovation, White Paper on AI Regulation (UK White Paper) (along with other regulators).

The ICO has now released its fourth chapter in the series. This chapter is called: ‘Engineering individual rights into generative AI models’.

What do I need to know about this chapter of the consultation?

Under data protection law, individuals have a fundamental right (subject to certain exceptions and derogations) to access and understand how, and for what purpose(s), their personal data is used (and to ensure it is accurate and up to date). In addition, in certain circumstances, individuals have a right to object to how their personal information is used and ‘to be forgotten’ (i.e. for certain data to be deleted where its relevance has lapsed). In addition, in the context of solely automated decision making, individuals have a right to know how an automated decision has been made (i.e. what logic and rationale or rules have been used in making that decision) and even to have that decision taken manually. The latter is of particular relevance in the context of GenAI, profiling and the use of algorithms in data analytics and automation of processes (e.g., in the context of recruitment, facial recognition, performance assessment, monitoring, target advertising or credit scoring).

Organisations must enable the exercise of these rights, and that includes organisations that offer AI.

The ICO has highlighted the following rights:

• The right to be informed – AI developers will often collect personal data from sources other than directly from individuals – for example, web-scraping.

The ICO stresses that the right to be informed still applies. The ICO states that the processing of personal data via web-scraping is likely to be ‘beyond people’s reasonable expectations at the time they provided data to a website’. The ICO provides an example of someone who writes a blog post in 2020 whose data is then scraped by a GenAI model four years later. That individual would not even be aware that the data has been used.

Data protection law provides exemptions to the right to be informed, notably where it would be a disproportionate effort to provide privacy information to each individual (which may be the case given that web-scraped datasets often relate to millions of individuals). However, the ICO opines that developers will still need to publish specific and accessible information on the sources, types and categories of the personal data used, as well as providing prominent mechanisms for individuals to exercise their rights.

• The right of access - the ICO states that developers must have easy-to-use and evidenced methods to facilitate response to these requests. If developers cannot identify individuals, e.g. because of high numbers due to web-scraping, they should still explain this to individuals and offer the opportunity for them to provide additional information for their identification.

• Right to erasure, restriction and objection – The ICO explains that ‘input’ and ‘output’ filters should be utilised to enact these rights. Input filters are techniques to detect and amend user prompts (i.e. so the system realises if personal data is being inputted and then e.g. deletes or restricts that). Output filters are techniques that are used to detect and amend model outputs (e.g. to prevent personal data being published on the web e.g. if someone objects). The ICO seeks views on whether these mechanisms are sufficient.

What should I do?

Those in the industry who are interested in informing the ICO’s position on GenAI and data protection on this topic can respond to the ICO’s call for feedback by 10 June 2024 using the survey or by emailing generative.ai@ico.org.uk.

If you are considering using AI or GenAI tools (or procuring an application or software that may use Gen/AI), it may be necessary to carry out a data privacy and/or an AI risk assessment to identify the potential risks and compliance issues arising from its use and how personal data is processed in the context of such applications and software. As part of this, a key component of any such assessment will be how users or employees individual rights may be exercised and how any risks may be mitigated (as part of a privacy by design and default methodology).

This may include anonymisation or pseudonymisation of any personal data, restricting excessive and disproportionate and onward use, tagging any personal data to track compliance and assist independent third party auditors and/or incorporating automated processes to respond to data subject access requests and/or requests from individuals around automated decision making processes. As part of this, incorporating a manual human process into operational procedures can often help minimise risk to individuals and monitor potential errors or discrepancies (or hallucinations) in decision making.

You can read the ICO’s first three chapters of the consultation here (although please note – for these the deadlines have passed):

• Chapter one: The lawful basis for web scraping to train generative AI models
• Chapter two: Purpose limitation in the generative AI lifecycle
• Chapter three: Accuracy of training data and model outputs

There is a wider context with GenAI. There are other issues to be aware of, including but not limited to copyright and database rights concerns under intellectual property law, as well as website terms and conditions.

Further reading on GenAI

• The ICO calls for views on generative AI and data protection
• All AI-Board: Eversheds Sutherland’s legal guide to navigating artificial intelligence
• Artificial Intelligence: UK and EU regulatory round-up
• Commercially Connected - April 2024