Who's liable? Legal accountability in the age of AI: Part 2

As AI continues to advance and integrate into nearly every aspect of modern life, predicting its risks and potential harms has become an increasingly difficult challenge for developers and businesses. AI systems often operate in complex and unpredictable environments, and their decision-making processes can be opaque, making it hard to foresee how they will behave in real-world situations.

AI can also act in unexpected ways due to their scale, autonomy, and reliance on massive datasets. AI models, especially those based on deep learning, function as 'black boxes', making it hard to anticipate unintended outcomes, particularly where these emerge indirectly or over time.

This makes it incredibly difficult and arguably impossible to forecast all potential consequences before deployment. This poses a number of legal questions when it comes to establishing who should be liable for harm caused by AI models.

What's the legal issue? 

The ability to bring a claim, whether in contract or negligence, turns on the claimant having suffered loss and being able to establish that the loss was caused by the defendant. The traditional ‘but for’ test poses the question, would the loss have been suffered but for the defendant’s breach?

Not every breach of contract gives rise to a claim in damages. In fact, commercial contracts are breached all the time without there ever being serious risk of liability or dispute.

Under the law of negligence, even where of a duty of care has been established, damages cannot be claimed unless the harm was caused by the defendant and was a reasonably foreseeable consequence of their act or omission.

AI complicates these tests because its adaptive and autonomous behaviour can make causal chains hard to trace, even for its developers. As leading AI developers Anthropic put it: 'a surprising fact about modern large language models is that nobody really knows how they work internally'.

Autonomous AI is therefore challenging our existing legal frameworks because:

• learning systems evolve over time, generating outputs or behaviours not directly programmed by humans.
• their operation can depend on opaque decision-making models ('black boxes').
• even developers may not fully understand why an AI behaves in a certain way.

In such cases, can a developer or business deploying AI technology reasonably foresee a specific harm? Perhaps more philosophically, does the unpredictability or autonomy of a non-legal entity in the chain of causation absolve liability?

Practical scenarios

In Part 1, we considered the scenario of an AI recruitment tool designed to screen job applications. We questioned whether it is fair to impose a duty of care on the developer of that tool who has little connection with rejected candidates affected by discriminatory outcomes. If a duty can be established, does the fact that an AI tool operates in a way which is inconsistent with how it was designed and trained, mean the outcomes were foreseeable?

If the duty properly lies with the Company deploying the AI tool, to what extent could a claimant show that the Company acted negligently in its oversight of the outcomes where the decision making process of the AI tool is unclear and it positively affirms its compliance with legislation? Arguments could also arise as to who was responsible for the bias in the historic dataset. Should the Company have foreseen the potential bias and reviewed the data, or should the developer have programmed the AI tool to look for and avoid discriminatory outcomes?

Dealing with foreseeability

One of the ways in which contract law optimises for certainty and fairness is to protect defendants from liability where the loss could not have been reasonably foreseen when the contract was entered into. In other words, even if the claimant can show losses that would not have occurred ‘but for’ the defendant’s breach of contract, the claimant will only be able to claim compensation for losses naturally arising from the breach (direct losses), or for indirect losses that the parties could have reasonably foreseen as a possible result of the breach when the contract was made.

The principle of remoteness of damage (i.e. unforeseeable losses being unrecoverable) was established in the classic contract case of Hadley v Baxendale in 1854. In that case, Hadley, a mill owner, hired Baxendale, a carrier, to transport a broken crankshaft to the location it would be repaired and then back again to the mill. Baxendale delivered the repaired crankshaft to Hadley’s mill a week late which kept the mill closed and led to lost profits. The court held that Hadley could only be compensated for losses that were reasonably foreseeable when the contract was made. Because Baxendale could not have known how important the crankshaft was to the operation of the mill and that the delay would force Hadley’s mill to shut down, the lost profits were deemed too remote and not recoverable.

AI challenges this long-established framework borne of a different era of human innovation and progress. Contracts operate on the human-centric assumption that the harms remediable by law should always be reasonably foreseeable by a human. The doctrine of legal causation, particularly in relation to indirect loss, evolved on the human-centric assumption that, when entering into a contract, individuals would be able to foresee all the risks that the law should protect against. How can this principle be applied to commercial dealings in the age of algorithmic error?

The same issue arises in claims for negligence where damages cannot be claimed unless the harm caused was a reasonably foreseeable consequence of the defendant’s act or omission.

Addressing the problem

In a dispute context, the following points would need to be explored to assess the liability exposure presented by our scenario:

1. Are there clear metrics, testing and acceptance criteria within the contractual agreement and have these been met?
2. Does the contract address who is responsible for maintaining the system?
3. Under contract, have the parties foreseen this risk and pre-allocated risk accordingly?
4. Who is responsible for the data used to train the AI model?
5. Has the developer given any indemnities in relation to compliance or third party claims?
6. Is the developer in breach of any warranties given around quality or compliance?
7. What limitations of liability have been agreed between the parties that may cap the developer’s liability, or are the losses suffered by the Company caught by an exclusion clause for indirect or unforeseeable losses?
8. In negligence, were the discriminatory outcomes foreseeable? Autonomous and adaptive AI tools are, by their nature, unpredictable, and it may be difficult for the developer, supplier or the Company deploying the tool to argue that the harm was not reasonably foreseeable.

Arguably the autonomous and adaptive nature of AI systems means that new principles are required to deal with them. For example, it may be appropriate to adopt a reasonable algorithmic foreseeability test[1] to replace the traditional test. This would involve courts considering whether a reasonably trained AI system would have anticipated the harm based on its training data and operational parameters.

Whether the courts will adapt such a test remains to be seen. In the meantime, businesses developing and deploying highly autonomous and adaptive systems should carefully consider how AI’s distinct features will impact their contracting arrangements.

Those who develop and deploy these systems should remain mindful of the fact that risk will be harder to manage in negligence, when anyone to whom a duty of care is owed may well be able to recover broad and even unexpected losses that arise from malfunctioning AI systems.

In the absence of legislation addressing these questions it will be down to the courts to continue the tradition of the common law to adapt to changing circumstances. How that evolves is difficult to predict. The only certainty may be uncertainty.

Conclusion

Whether businesses will be able to predict the risk of potential harm caused by autonomous, adaptive AI, is unclear. The reality is, we don't know. Some harms will be predictable, but others will not, and the question of predictability may only arise once harm has been suffered and the question of causation is under the spotlight.

Those developing and supplying products involving AI will likely need to conduct risk assessments on a ‘worst case scenario’ basis and tread a careful balance between contractual limitations of liability, customer disclaimers, and insurance. Where the sweet spot lies is likely to depend on customer and supplier appetite for risk, and the cost and availability of insurance cover. We may find ourselves testing and refining approaches for some time before any clear answers emerge.

'Who's liable? Legal accountability in the age of AI' articles

• Part 1: The AI Liability Gap: Are existing laws capable of responding to AI-related liability?
• Part 3: What evidence will be available to navigate these liability issues and prove fault?
• Part 4: Which sectors and industries are currently under the spotlight and why?
• Part 5: Contracting considerations – what is the best approach to take?

[1] https://arxiv.org/html/2406.04671v1