AI Licensing Marketplace Dominance Concerns

1. Introduction

AI performance scoring refers to the use of artificial intelligence, machine learning, automated analytics, productivity software, or algorithmic management systems to evaluate an employee's performance. Such systems may calculate scores based on productivity, attendance, sales, response time, customer ratings, keystrokes, targets, task completion, working hours, or other behavioural indicators.

AI performance scoring can improve consistency and assist employers in identifying performance trends. However, its legality depends upon whether the system complies with employment law, equality and non-discrimination principles, privacy/data-protection law, procedural fairness, contractual obligations, and applicable AI regulation.

A particularly important legal issue arises when an AI-generated score is used to determine pay, promotion, disciplinary action, performance warnings, or termination.

2. Meaning of AI Performance Scoring

AI performance scoring is an automated or semi-automated process through which an employee receives a numerical or categorical assessment.

For example:

Employee Data → AI Analysis → Performance Score → HR Decision

The system may analyse:

productivity;

sales performance;

attendance;

customer reviews;

response time;

work output;

computer activity;

task completion;

communication patterns;

targets achieved; and

other workplace data.

The important legal distinction is between:

AI as a decision-support tool and AI as the actual decision-maker.

Where a manager independently evaluates the AI recommendation, the legal position may be different from a situation where an employee is automatically dismissed or denied a bonus solely because an algorithm assigned a low score.

3. Is AI Performance Scoring Legal?

AI performance scoring is not automatically unlawful. Its legality depends upon the purpose, data used, methodology, degree of automation, impact on employees, and safeguards adopted by the employer.

A lawful system should generally satisfy the following requirements:

Legitimate employment purpose

Relevant and accurate data

Non-discrimination

Transparency

Data protection

Human oversight

Opportunity to challenge inaccurate results

Proportionality

Regular auditing

Procedural fairness

An employer cannot necessarily avoid legal responsibility merely because an adverse decision was produced by software.

4. Data Protection and Privacy

AI performance scoring frequently involves extensive employee monitoring. Information may include computer activity, attendance, productivity, communications, location, or behavioural information.

Therefore, employers must consider:

lawful basis for processing;

purpose limitation;

data minimisation;

accuracy;

retention;

security;

transparency; and

employee rights.

The UK Information Commissioner's Office specifically identifies productivity software, keystroke monitoring, internet activity monitoring and similar technologies as workplace monitoring activities requiring compliance with data-protection principles.

The ICO also states that automated workplace decisions can create significant effects where, for example, automated monitoring affects a worker's pay or employment opportunities.

5. Automated Decision-Making

One of the most important legal questions is whether the AI score merely assists a manager or effectively determines the outcome.

For example:

Lower AI Score → Manager investigates → Employee allowed to respond → Final human decision

is different from:

Lower AI Score → Automatic dismissal

The second situation creates substantially greater legal concerns.

Under the EU/UK data-protection framework, automated decision-making producing legal or similarly significant effects is subject to special safeguards. These can include meaningful human intervention, information about the logic involved, and an opportunity to challenge the decision.

Human oversight must also be genuine. A manager who simply approves every AI recommendation may not provide meaningful human involvement.

6. Discrimination and Algorithmic Bias

AI performance scoring may produce discriminatory outcomes even when the employer does not intentionally programme discriminatory criteria.

For example, an algorithm might give excessive importance to:

uninterrupted computer activity;

constant availability;

overtime;

customer ratings;

communication style; or

historical performance data.

Such criteria may indirectly disadvantage particular groups of workers.

Therefore, employers should conduct regular bias and adverse-impact testing.

The regulatory concern is not limited to intentional discrimination. Algorithmic systems can reproduce discrimination contained in historical datasets or generated by seemingly neutral criteria.

7. CASE LAWS

Case Law 1: OQ v Land Hessen (SCHUFA Holding), CJEU, Case C-634/21 (2023)

This is one of the most important modern cases concerning algorithmic scoring.

The Court of Justice of the European Union considered an automated scoring system under Article 22 of the GDPR. The system generated a probability value concerning an individual's ability to meet financial obligations.

The Court held that an automatically generated probability value can fall within the concept of automated decision-making where the score plays a determining role in a subsequent decision.

Relevance to employment

Although SCHUFA was a credit-scoring case rather than an employment case, its reasoning has significant relevance to workplace AI scoring.

An employer should therefore not assume that an AI performance score is legally irrelevant simply because a human manager formally makes the final decision.

Principle: An algorithmically generated score may itself become legally significant when it effectively determines the subsequent decision.

Case Law 2: Mobley v. Workday, Inc., 740 F. Supp. 3d 796 (N.D. Cal. 2024)

In Mobley v. Workday, the plaintiff alleged that Workday's AI-based employment tools discriminated against applicants based on race, age and disability.

The court allowed significant parts of the amended claims to proceed at the pleading stage. The allegations included that Workday's algorithmic tools participated in employment decision-making rather than merely performing a neutral administrative function.

The case primarily concerns AI recruitment, not employee performance scoring.

Relevance

The case demonstrates an important legal principle:

Where an algorithm performs a substantive employment decision-making function, the use of technology does not necessarily remove the underlying employment decision from anti-discrimination law.

This reasoning can be relevant where AI is used to determine promotions, retention, discipline, or termination through employee performance scores.

Case Law 3: EEOC v. iTutorGroup, Inc. (2022–2023)

The U.S. Equal Employment Opportunity Commission sued iTutorGroup alleging that its online recruitment software automatically rejected applicants based on age.

According to the EEOC, the software automatically rejected more than 200 qualified applicants, including women aged 55 or older and men aged 60 or older.

The case was subsequently settled, with iTutorGroup agreeing to pay $365,000 and provide other relief.

Relevance

Although this was an AI-assisted hiring case rather than performance scoring, it establishes an important principle:

Automating an employment decision does not immunise an employer from discrimination law.

The same principle may become applicable where an AI performance system produces discriminatory promotion, bonus, disciplinary or termination outcomes.

Case Law 4: Uber BV v Aslam [2021] UKSC 5

In Uber BV v Aslam, the UK Supreme Court examined the employment status of Uber drivers and considered the practical reality of the relationship between the platform and workers.

The case is important for algorithmic management because digital platforms can exercise control through technological systems rather than traditional supervisors.

Relevance

The case supports a broader employment-law principle that courts may examine the substance and practical operation of workplace arrangements, rather than simply accepting the formal description used by the technology provider.

Accordingly, an employer cannot necessarily avoid employment-law responsibilities merely by describing an AI system as a neutral technological tool.

Case Law 5: International Brotherhood of Teamsters v United States, 431 U.S. 324 (1977)

This United States Supreme Court case concerned employment discrimination and statistical evidence.

The Court recognised the importance of statistical patterns in demonstrating systemic discrimination.

Relevance to AI

AI performance systems generate large quantities of employee data. This data can be analysed statistically to determine whether particular groups receive systematically lower scores or suffer greater adverse consequences.

Thus, statistical evidence can become important in an AI-discrimination claim.

For example:

AI scoring → lower average scores for protected group → adverse employment outcomes

may raise questions concerning disparate impact, depending upon the applicable jurisdiction and legal test.

Case Law 6: Griggs v Duke Power Co., 401 U.S. 424 (1971)

In Griggs v Duke Power Co., the U.S. Supreme Court established an important principle concerning apparently neutral employment practices that disproportionately disadvantage protected groups.

The Court examined whether employment requirements that appeared neutral were sufficiently related to job performance.

Relevance to AI Performance Scoring

The case provides a useful analogy for AI scoring.

An employer might argue:

"The algorithm treats everyone according to the same formula."

However, equal application of a formula does not necessarily resolve whether the underlying criterion produces unlawful discriminatory effects.

Therefore, employers should ask:

Is the scoring criterion genuinely related to the job?

Does it measure actual performance?

Does it disproportionately disadvantage a protected group?

Is there a less discriminatory method of measuring the same performance objective?

8. Accuracy of AI Scores

Another major legal issue is algorithmic accuracy.

An AI system may incorrectly interpret:

temporary illness;

disability-related limitations;

caregiving responsibilities;

technical failures;

internet problems;

approved leave;

reasonable workplace accommodations;

different working styles; or

legitimate non-work-related interruptions.

A false low score could consequently lead to:

Low Score → Performance Warning → Disciplinary Action → Termination

If the initial score is inaccurate, every subsequent decision may be affected.

Therefore, employees should have mechanisms to challenge incorrect data and erroneous AI outputs.

9. Human Oversight

Meaningful human oversight is one of the strongest safeguards.

A proper process should involve:

AI generates performance assessment.

HR or manager examines underlying data.

Employee receives relevant information.

Employee is allowed to explain unusual circumstances.

Manager considers evidence independently.

Final decision is documented.

The human decision-maker should have genuine authority to disagree with the algorithm.

The ICO similarly emphasises that meaningful human involvement requires the decision-maker to critically assess the recommendation rather than simply applying the automated output.

10. Transparency and Right to Challenge

Employees should ordinarily be informed that AI is being used to evaluate them where applicable law requires such transparency.

Important information may include:

what information is collected;

why it is collected;

how it contributes to scoring;

the consequences of a low score;

whether third-party software is involved; and

how an employee can challenge an inaccurate result.

The ICO identifies transparency, bias testing and routes for human review as important safeguards in automated employment decision-making.

11. AI Performance Scoring and Termination

Termination based substantially on an AI score creates a particularly serious legal issue.

For example:

AI Score = 42%

Employer Rule = below 50% means termination

Such a system may be challenged where:

the score is inaccurate;

the data is discriminatory;

the employee was not informed;

relevant circumstances were ignored;

the algorithm was improperly designed;

no human review occurred; or

the termination procedure violated employment law.

The employer should therefore avoid treating an algorithmic score as conclusive proof of misconduct or poor performance.

12. AI Performance Scoring in Pakistan

In Pakistan, the legality of AI performance scoring must be considered through the existing framework of constitutional equality, labour and employment protections, contractual principles, privacy/data-related obligations, and applicable provincial and federal employment legislation.

Pakistan does not have a single comprehensive statute specifically regulating AI-based employee performance scoring comparable to a dedicated AI employment code.

Consequently, conventional employment-law principles remain important.

An employer using AI should ensure that:

employment contracts are respected;

disciplinary procedures are followed;

discriminatory treatment is avoided;

performance criteria are connected with legitimate job requirements;

employees are not punished solely because of unreliable automated information; and

applicable workplace laws and institutional policies are complied with.

The absence of a specific AI statute does not necessarily mean that an employer has unrestricted freedom to use automated performance scoring.

13. Employer's Legal Compliance Framework

An employer introducing AI performance scoring should ideally adopt the following framework:

A. Before Deployment

conduct an impact assessment;

identify the purpose of the system;

determine what employee information will be collected;

verify the reliability of the algorithm;

test for discriminatory effects.

B. During Deployment

provide appropriate transparency;

maintain accurate records;

monitor system performance;

conduct periodic bias testing;

restrict unnecessary data collection.

C. Before Adverse Action

verify the AI score;

review the underlying evidence;

allow the employee to respond;

consider disability or other relevant circumstances;

obtain independent human review.

D. After the Decision

preserve relevant records;

provide appropriate reasons;

establish an appeal or review mechanism;

correct demonstrably inaccurate data;

audit recurring patterns.

14. Major Legal Risks

AI performance scoring can create several categories of legal risk:

15. Conclusion

AI performance scoring can be a legitimate management tool, but its use is subject to existing employment, equality, privacy and procedural principles. The central legal issue is not simply whether an employer uses AI, but how the AI system collects information, calculates performance, affects employees, and contributes to employment decisions.

The emerging case law demonstrates that courts and regulators are increasingly willing to examine algorithmic systems as part of ordinary employment decision-making. SCHUFA is particularly important for understanding the legal significance of algorithmic scoring, while Mobley and iTutorGroup illustrate how employment discrimination principles can apply when automated systems influence employment decisions.

Therefore, a legally safer model is:

AI Assessment + Accurate Data + Bias Testing + Transparency + Meaningful Human Review + Right to Challenge = Responsible AI Performance Management.

AI should ordinarily function as an assistive mechanism rather than an unquestionable substitute for fair human employment decision-making.