The Dutch Data Protection Authority has imposed a fine of almost €825 million on Uber over its use of automated decision-making to deactivate drivers.
The decision is an important warning for business in the EU but also for those in the UK or which may be caught by the extra-territorial rules of the GDPR (for instance businesses employing people in the EU or UK, or providing services into the UK or EU.
Automating a decision that has a significant effect on an individual can create serious data protection risks if there is not sufficient human involvement.
This applies whether or not the automated system uses AI (but AI makes the automation of decisions more likely).
If your organisation automates decisions involving recruitment, employment, finance, access to services or other significant outcomes, there are seven questions worth asking.
What happened in the Uber case?
Between 2018 and 2022, Uber used software to monitor driver behaviour and customer ratings.
Where the system identified suspected fraud or persistently low ratings, drivers could have their accounts deactivated automatically. This could be temporary or permanent.
The consequence was significant: affected drivers could immediately lose their ability to earn income through the platform.
Crucially, the Dutch Data Protection Authority, the Autoriteit Persoonsgegevens (AP), found that there was no meaningful human review before these decisions were made.
The AP identified two key GDPR problems.
First, Article 22 GDPR gives individuals protections against certain decisions based solely on automated processing where those decisions produce legal or similarly significant effects. The AP found that Uber’s automated deactivation decisions fell within these rules.
Second, the AP found that Uber had not adequately informed drivers about the automated decision-making, including the logic involved and its potential consequences.
The investigation followed complaints from 171 French drivers. Because Uber’s European headquarters are in the Netherlands, the Dutch regulator took the lead on the investigation.
Uber has appealed the decision.
Why does this matter to UK businesses?
The decision was made by an EU regulator, but UK businesses should not dismiss it as an EU-only issue.
The UK GDPR also regulates significant automated decisions involving individuals. Although UK law has changed following the Data (Use and Access) Act 2025 (DUAA), businesses still need to consider the data protection requirements that apply when important decisions are automated.
Serious breaches of UK data protection law can also result in substantial regulatory penalties.
More importantly, the underlying issue is relevant to any organisation using software or AI to make important decisions about people.
What sorts of automated decisions create the greatest risk?
Recruitment and employment
Automated recruitment screening, performance management, disciplinary processes and other employment decisions require particular care.
For example, a business should examine a system that automatically rejects a candidate, recommends dismissal or triggers disciplinary action based on personal data.
The more significant the effect on the individual, the more important it is to understand how the decision is made and what role a human actually plays.
Gig economy and workforce platforms
Businesses using platforms to manage workers should examine automated account suspensions, performance scoring, task allocation and removal decisions.
Simply having a person somewhere in the process does not necessarily amount to meaningful human involvement.
Credit, insurance and financial services
Automated credit scoring, underwriting, insurance and other financial decisions have long been an important data protection issue.
Businesses should understand when automated systems produce significant outcomes and what safeguards apply.
Customer accounts and access to services
Automated fraud detection, account closures and access restrictions can also create data protection risks.
The relevant question is not simply whether software was involved. Businesses need to consider the significance of the decision for the individual and the extent of genuine human involvement.
Can businesses still automate important decisions?
Yes. The lesson from the Uber case is not that businesses must stop automating decisions.
Instead, organisations need to understand which legal requirements apply and build appropriate safeguards into their processes.
One of the most important distinctions is between genuine human involvement and a process where a person merely approves an outcome already determined by software.
7 Questions to Ask Before Automating an Important Decision
1. Do the automated decision-making rules apply?
Start by identifying exactly what the system does.
Consider whether personal data is being used to make an automated decision that has a legal or similarly significant effect on an individual.
Also establish what role, if any, a human plays in reaching the final decision.
2. Do we have a lawful basis for what we are doing?
Identify and document the legal basis for processing the personal data involved.
Where the specific rules governing significant automated decisions apply, check that the organisation can satisfy the relevant conditions and safeguards.
Do not assume that using AI or other software changes the underlying data protection requirements.
3. Have we properly explained the automation to individuals?
People should receive clear information about how their personal data is being used.
Where required, this includes appropriate information about automated decision-making and the consequences it may have for them.
Avoid vague statements that do not help individuals understand what is actually happening.
4. Can an individual genuinely challenge the decision?
Where applicable, there should be a practical way for an individual to challenge an automated decision and obtain appropriate human involvement.
A right that exists only in lengthy terms and conditions, or that is extremely difficult to exercise in practice, is unlikely to provide much protection.
5. Have we completed an appropriate DPIA?
A Data Protection Impact Assessment (DPIA) may be required where automated processing is likely to create a high risk to individuals.
The DPIA should reflect how the system actually operates in practice, rather than simply describing how it was originally intended to work.
It should also be reviewed when the system or its use changes.
6. Is the human involvement actually meaningful?
This is one of the most important questions.
Having a human somewhere in the process does not automatically make a decision a human decision.
The person reviewing the outcome should have sufficient information, authority and opportunity to assess the case properly and, where appropriate, reach a different conclusion.
Human involvement should not simply be a rubber stamp.
7. Are we checking whether the system changes over time?
Automated systems do not necessarily remain static.
Software updates, new data, changes in processes and, in some cases, AI model updates can alter how a system operates.
Businesses should therefore periodically review their automated decision-making processes, DPIAs and internal records to make sure the legal assessment still reflects what is happening in practice.
What should AI Directors do?
Businesses using automated systems to make significant decisions about individuals should identify those systems and examine the role humans actually play in the decision-making process.
Particular attention should be given to decisions affecting employment, income, recruitment, financial services and access to important services.
The key question is not simply:
“Do we have a human involved?”
It is:
“Does that person genuinely review the decision and have a meaningful opportunity to change the outcome?”
The Uber decision demonstrates the potential consequences when the answer is no.
Optional Additions
Suggested FAQ:
Q: Does Article 22 only apply to AI decisions?
A: No. Automated decision-making rules can apply regardless of whether the technology involved would normally be described as AI.
Q: Does having a human reviewer automatically solve the problem?
A: No. Human involvement needs to be genuine and meaningful. Simply approving an automated recommendation without properly considering it may not be sufficient.
Q: Can UK businesses still automate recruitment and HR decisions?
A: Automation is possible, but businesses need to understand the data protection requirements and safeguards that apply to the particular decision and system.
Q: What is a significant automated decision?
A: Broadly, it is an automated decision that has a legal or similarly significant effect on an individual. Decisions affecting employment, income, credit or access to important services are obvious areas to examine carefully.
Q: Should businesses carry out a DPIA before using automated decision-making?
A: If the decision is made with AI then almost certainly yes. A DPIA will be required where the processing is likely to result in a high risk to individuals. The UK regulator (the ICO ) has indicated that a DPIA is likely to be needed for decisions made by AI. It should reflect how the system operates in practice and be reviewed when circumstances change.
Q. Who could I speak to if I have any questions about this area?
A: Glad you asked - Richard Nicholas is an AI governance lawyer whose details are available are richardnicholas.ai. He is a lawyer who has worked on AI projects since 2015, has been Head of AI at a top 50 UK law firm and is the founder of Skill Diligence Ltd. He also writes this newsletter so if you subscribe you can contact him directly!