Explainability of AI-Based Decisions — Detailed Legal Explanation

1. Meaning of Explainability of AI-Based Decisions

Explainability of AI-based decisions means the ability of a person affected by an artificial-intelligence or algorithmic decision to understand why the decision was made, what information influenced it, what criteria were applied, and how the person can challenge or correct the decision.

An AI system may be used to:

reject a loan application;

shortlist or reject a job applicant;

determine employee performance or promotion;

detect alleged employee misconduct;

calculate insurance premiums;

determine creditworthiness;

identify suspected fraud;

allocate public benefits;

determine welfare eligibility;

assess immigration or visa applications;

conduct facial recognition;

predict recidivism or risk;

moderate or remove online content; or

make recommendations affecting employment, education, healthcare or public services.

The central legal problem is the “black box” problem: an affected person may know the outcome but not understand the reasoning that produced it.

For example:

“Your application was rejected because our AI system classified you as high risk.”

That statement is generally not a meaningful explanation. A legally meaningful explanation may need to identify the relevant factors, the material data relied upon, the role of the automated system, and the opportunity available to challenge the result.

2. Why Explainability Matters Legally

Explainability is connected with several established legal principles.

A. Natural justice

A person affected by a decision ordinarily needs a meaningful opportunity to know and challenge the case against them.

If an employer says:

“The AI identified you as unsuitable.”

but refuses to disclose the relevant basis, the employee may be unable to challenge:

incorrect data;

discriminatory variables;

erroneous assumptions;

biased training data;

incorrect classification;

technical errors; or

inappropriate criteria.

Thus, AI opacity can undermine audi alteram partem—the right to be heard.

B. Duty to give reasons

Reasoned decision-making is an important component of administrative law.

Where a public authority uses AI, it cannot necessarily avoid its obligation to give reasons merely by saying:

“The computer generated the result.”

The legal decision remains attributable to the human authority or institution responsible for it.

The use of technology does not automatically eliminate the duty to explain.

C. Article 14 of the Indian Constitution

In India, arbitrary state action can violate Article 14.

An AI system can create Article 14 concerns where:

irrelevant factors influence decisions;

similarly situated persons are treated differently;

the algorithm produces unexplained classifications;

discriminatory proxies are used;

there is no meaningful review mechanism; or

the authority cannot demonstrate a rational connection between the criteria and the decision.

Explainability therefore becomes a mechanism for testing whether an AI-assisted decision is non-arbitrary and rational.

D. Article 21 and procedural fairness

Where an AI decision affects life, liberty, dignity, privacy or other important interests, Article 21 concerns may arise.

The more serious the consequence, the stronger the argument for:

disclosure;

reasons;

human review;

correction mechanisms;

procedural safeguards; and

an opportunity to contest the decision.

E. Right to privacy and informational autonomy

AI systems frequently process enormous amounts of personal information.

Explainability can therefore help an individual determine:

What data was used?

Where did the data come from?

Why was it relevant?

Was inaccurate data used?

Was sensitive information used?

Was a proxy used for a protected characteristic?

Was automated profiling performed?

This connects explainability with privacy and informational autonomy.

3. Different Levels of Explainability

Explainability is not one single obligation.

3.1 Outcome explanation

The person should know what happened.

Example:

“Your application was rejected.”

This is the lowest level.

3.2 Reason explanation

The person should know why the outcome occurred.

Example:

“The application was rejected because the system assessed the applicant's repayment risk above the applicable threshold.”

3.3 Factor explanation

The person should know which significant factors contributed.

Example:

“The principal factors were repayment history, outstanding debt and income-to-debt ratio.”

3.4 Data explanation

The person should be able to identify important information used by the system.

For example:

employment history;

salary;

credit history;

attendance;

performance records;

application information.

3.5 Logic explanation

The person should understand, at an appropriate level, how the factors affected the outcome.

This does not necessarily mean disclosing source code.

A useful distinction is:

Explainability does not always require publication of the entire algorithm.

The law may require meaningful information about the decision-making process without requiring disclosure of every line of code or trade secret.

This distinction is particularly important in commercial AI.

4. Six Important Case Laws and Their Relevance

There is no single universally applicable common-law rule saying that every AI decision must be fully explainable. Much of the present law is developing through existing principles of natural justice, reasons, proportionality, privacy and procedural fairness, together with cases directly involving algorithmic systems.

The following cases are particularly important.

Case 1: State v. Loomis, 881 N.W.2d 749 (Wis. 2016)

Facts

Eric Loomis was sentenced in Wisconsin. The court considered a proprietary algorithmic risk-assessment system called COMPAS, which generated an assessment concerning the likelihood of recidivism.

Loomis challenged the use of the system because the methodology was proprietary and therefore could not be fully examined by him.

Issue

Could a court rely on a proprietary algorithmic risk assessment in sentencing without providing complete access to its methodology?

Decision

The Wisconsin Supreme Court permitted consideration of COMPAS, but imposed significant limitations and cautions.

The algorithm could not be treated as the sole or determinative basis for sentencing, and the court had to independently consider the appropriate sentencing factors.

The court specifically recognized concerns associated with:

proprietary methodology;

accuracy;

individualized sentencing;

gender-related considerations; and

the limitations of algorithmic predictions.

Importance for explainability

Loomis is one of the foundational cases concerning algorithmic decision-making and due process.

It demonstrates an important principle:

An algorithm may assist a decision-maker, but the decision-maker cannot simply surrender independent judgment to an opaque algorithm.

It also demonstrates that human oversight and limitations on reliance can become safeguards where complete algorithmic transparency is unavailable.

The case is especially relevant to:

AI sentencing;

employee risk scoring;

automated disciplinary systems;

recruitment algorithms; and

predictive HR systems.

Case 2: R (Bridges) v Chief Constable of South Wales Police, [2020] EWCA Civ 1058

Facts

South Wales Police used Automated Facial Recognition (AFR) technology.

Edward Bridges challenged its use, raising issues concerning:

privacy;

data protection;

discrimination;

legal safeguards; and

the legality of automated facial recognition.

The Court of Appeal considered whether the police had sufficient legal controls governing the deployment of the technology.

Decision

The Court of Appeal found the use of AFR unlawful at that time, including because the legal framework did not sufficiently determine where the technology could be deployed and who could be placed on watchlists.

Importance for explainability

The case illustrates that AI-related legality is not merely a question of whether the technology works.

There must also be:

legally defined limits;

appropriate safeguards;

accountability;

meaningful controls; and

protection against arbitrary use.

In other words:

An automated system affecting individuals cannot operate in an uncontrolled legal vacuum.

For employers, the analogy is significant. If AI is used for employee monitoring or facial recognition, an employer should be able to explain:

why the technology is being used;

what it measures;

where it is deployed;

what decisions it influences;

who reviews its output; and

what safeguards prevent discriminatory or erroneous results.

Case 3: R (Bridges) v Chief Constable of South Wales Police, [2019] EWHC 2341 (Admin)

The High Court judgment in the same litigation is also important.

Importance

The case examined the legal framework surrounding automated facial recognition and the relationship between technological discretion and public-law requirements.

It demonstrates an important principle:

The fact that an authority possesses sophisticated technology does not itself establish that the authority has exercised its legal discretion properly.

For AI-based public decisions, authorities should be able to identify:

the legal basis for using the system;

the parameters governing its use;

the safeguards;

the human role;

the possibility of error; and

the method by which affected individuals can challenge the result.

Thus, algorithmic explainability is closely connected to lawful exercise of administrative discretion.

Case 4: SCHUFA — Case C-634/21, OQ v Land Hessen, Court of Justice of the European Union (2023)

This is one of the most important modern cases on automated decision-making and explainability.

Facts

SCHUFA generated credit scores used in decisions concerning individuals' creditworthiness.

The case concerned the relationship between automated scoring and Article 22 of the GDPR, which concerns decisions based solely on automated processing.

Decision

The CJEU treated the generation of a credit score, in circumstances where it substantially determines another person's decision about the individual, as potentially falling within the GDPR's framework concerning automated decision-making.

The Court also considered the right to obtain meaningful information about the logic involved.

Explainability principle

The Court emphasized that meaningful information about logic involves an explanation of the procedure and principles actually applied.

Importantly, merely handing over:

a complicated mathematical formula; or

an incomprehensible technical description

would not necessarily constitute a meaningful explanation.

The explanation should be:

relevant;

concise;

transparent;

intelligible; and

accessible.

Importance

This case provides an excellent distinction between:

Technical transparency
and
Legal explainability.

A company cannot necessarily satisfy an explainability obligation simply by saying:

“Here is our machine-learning model.”

The affected person must receive information that allows them to understand the material logic behind the decision.

Case 5: Maneka Gandhi v Union of India, (1978) 1 SCC 248

Although this is not an AI case, it is extremely important when analysing AI decision-making in India.

Principle

The Supreme Court substantially expanded the understanding of procedural fairness under Article 21.

The Court emphasized that a procedure affecting fundamental rights must not be arbitrary, unfair or unreasonable.

Relevance to AI

Suppose a public authority uses an AI system to determine whether an individual receives:

a licence;

a benefit;

permission;

employment;

immigration status; or

some other legally significant entitlement.

If the person receives only:

“Rejected by automated assessment”

without an effective opportunity to understand or challenge the decision, serious procedural fairness questions may arise.

Maneka Gandhi supports the broader proposition that:

Procedure affecting fundamental rights must be fair, just and reasonable—not merely technically efficient.

Therefore, AI cannot become a mechanism for bypassing procedural fairness.

Case 6: Kranti Associates Pvt. Ltd. v Masood Ahmed Khan, (2010) 9 SCC 496

Principle

The Supreme Court strongly emphasized the importance of reasoned decisions.

Reasons serve several functions:

They demonstrate application of mind.

They reduce arbitrariness.

They facilitate judicial review.

They allow affected persons to understand the decision.

They increase confidence in the decision-making process.

Application to AI

This principle becomes highly significant when an AI system assists decision-making.

A public authority should not simply write:

“AI assessment indicates rejection.”

That is not necessarily an adequate legal reason.

The authority should be capable of identifying the humanly intelligible reasons supporting the final decision.

Thus:

AI output ≠ legal reason.

The human decision-maker must be able to translate the AI's relevant output into legally sufficient reasons.

Case 7: Mohinder Singh Gill v Chief Election Commissioner, (1978) 1 SCC 405

Principle

The Supreme Court emphasized the importance of reasons and the principle that an administrative decision must stand on the reasons contained in the decision itself rather than being subsequently supplemented by new reasons.

Relevance to AI

This becomes particularly important with algorithmic decision-making.

Imagine:

Original decision: “Rejected based on automated scoring.”

Later, during litigation, the authority says:

“Actually, the AI considered 37 additional factors.”

This raises serious questions about procedural fairness.

The affected individual should ordinarily be able to know the material basis of the decision at the relevant time, rather than discovering the reasoning only after challenging it.

AI lesson

An organization should preserve:

the relevant inputs;

the applicable model/version;

material criteria;

human review;

decision rationale; and

relevant audit records.

This creates an audit trail.

Case 8: Puttaswamy v Union of India, (2017) 10 SCC 1

Importance

The Supreme Court recognized privacy as a fundamental right under Article 21 and other constitutional guarantees.

The judgment is highly relevant to AI because AI systems frequently depend on:

personal data;

behavioural data;

profiling;

inference;

prediction; and

automated classification.

Explainability connection

An individual affected by AI profiling may reasonably need to understand:

what personal information was processed;

why it was processed;

what inference was drawn;

how the inference affected the decision; and

whether the processing was proportionate to the legitimate objective.

Thus, privacy protection is not merely about preventing disclosure of information.

It also concerns how information is transformed into decisions about people.

5. What These Cases Collectively Establish

The cases do not establish that every AI system must disclose its source code.

Instead, they collectively support several broader legal principles:

6. Explainability Does Not Mean Source-Code Disclosure

This is one of the most important distinctions.

Suppose a company develops a proprietary recruitment algorithm.

An employee/applicant asks:

“Show me the source code.”

The legal answer will not necessarily be:

“The company must disclose everything.”

A more proportionate approach may require disclosure of:

categories of data used;

significant factors;

general decision logic;

relevant thresholds;

reasons for the particular outcome;

whether human review occurred;

relevant safeguards;

process for correcting inaccurate information.

The company may still have legitimate interests in protecting:

source code;

trade secrets;

cybersecurity information;

proprietary model architecture.

Therefore, explainability and complete technical transparency are different concepts.

7. AI Explainability in Employment Decisions

This is particularly important for employers.

Imagine an organization uses AI to rank employees for promotion.

The system evaluates:

productivity;

attendance;

project completion;

communication;

peer reviews;

manager ratings;

employee activity data.

The AI produces:

Employee A — 91% promotion suitability
Employee B — 64% promotion suitability

The organization should not assume that the numerical output itself constitutes a lawful reason.

Questions that should be answered

1. What data was used?

Was the data accurate?

2. What factors were material?

Was attendance more important than performance?

3. Were proxy variables used?

Could variables indirectly reflect:

age;

disability;

sex;

caste;

socioeconomic status;

location;

other protected characteristics?

4. Was the model validated?

Was its predictive accuracy tested?

5. Was human review conducted?

Did a manager independently assess the result?

6. Can the employee challenge the result?

Is there an appeal or correction mechanism?

7. Can the employer explain the outcome?

Can the employer articulate why the employee was ranked lower?

8. AI in Recruitment

Consider an AI recruitment system.

Candidate A is rejected automatically.

The employer says:

“The candidate's AI score was below the hiring threshold.”

This creates potential legal questions.

The candidate may ask:

What information was evaluated?

Was employment history considered?

Did the system consider educational institution?

Did the model use facial or voice analysis?

Was there a personality assessment?

Was the training data representative?

Did historical hiring patterns create bias?

Was the decision exclusively automated?

Was a human able to override the decision?

A defensible system should have a documented explanation pathway.

9. AI in Employee Discipline

Suppose an AI monitoring system determines:

“Employee likely engaged in misconduct.”

The employer immediately terminates the employee.

This creates serious procedural concerns.

An AI prediction is not necessarily proof of misconduct.

The employer should investigate:

underlying evidence;

data accuracy;

algorithmic reliability;

alternative explanations;

employee's response;

contextual circumstances;

human decision-making.

This follows the broader logic of natural justice and reasoned decision-making.

10. AI Bias and Explainability

Explainability is also essential for detecting discrimination.

An AI system may not explicitly use:

“sex”

but could use variables that correlate strongly with sex.

Similarly, it may not expressly use:

“caste”

but may use other variables that function as proxies.

Therefore, merely stating:

“The algorithm does not contain a discrimination variable”

is insufficient.

The relevant question is:

Does the system produce discriminatory outcomes, and can the organization identify and justify the factors producing those outcomes?

11. Black-Box AI and Legal Accountability

A particularly dangerous argument is:

“We cannot explain the decision because the AI is too complex.”

Complexity is not necessarily a legal defence.

If an organization voluntarily uses an AI system for consequential decisions, it may still have responsibility for ensuring:

governance;

validation;

documentation;

monitoring;

human review;

auditability;

error correction;

legal compliance.

The legal responsibility does not automatically disappear because the technology is complicated.

12. Human-in-the-Loop Is Not Enough by Itself

Another misconception is:

“A human approved the AI decision, therefore the decision is human.”

Not necessarily.

A human review must be meaningful, not merely ceremonial.

For example:

Weak human review:

AI: Reject candidate.
Manager: Clicks “Approve rejection.”

Meaningful human review:

AI: Candidate ranked low because of specified factors.
Manager examines those factors, verifies the underlying information, considers contrary evidence, and independently decides whether rejection is justified.

The second approach provides substantially stronger procedural safeguards.

13. Right to Contest an AI Decision

A good AI governance framework should provide an affected individual with the ability to:

1. Know that AI was used

The person should not necessarily be left unaware that automation materially influenced the decision.

2. Understand the material reasons

The person should receive meaningful information about the decision.

3. Correct inaccurate data

There should be a mechanism to challenge factual errors.

4. Provide additional information

The person should be able to present relevant circumstances that the model did not consider.

5. Obtain human review

A significant adverse decision should have meaningful human reconsideration where legally required or appropriate.

6. Challenge the final decision

There should be an appeal, grievance, administrative or judicial mechanism where applicable.

14. Proportionality of Explainability

The amount of explanation required should generally correspond to the importance and consequences of the decision.

Low-impact decision

Example:

Recommendation for a movie.

Limited explanation may be sufficient.

Medium-impact decision

Example:

Product recommendation affecting price.

Greater transparency may be appropriate.

High-impact decision

Example:

Termination of employment.

Much stronger safeguards are appropriate.

Extremely high-impact decision

Example:

Criminal sentencing or deprivation of liberty.

The demand for transparency, human judgment, procedural safeguards and review becomes particularly strong.

This distinction is illustrated particularly well by Loomis.

15. Explainability vs Accuracy

These are different concepts.

An AI model can be:

highly accurate but poorly explainable.

Or:

highly explainable but inaccurate.

Both create legal problems.

For example, a model may correctly predict employee turnover 90% of the time but rely on an impermissible discriminatory proxy.

Conversely, a transparent model may use legally appropriate factors but produce unreliable results.

Therefore:

Accuracy + fairness + explainability + accountability are separate governance requirements.

16. Explainability vs Transparency

Transparency

Means making information about the system available.

Explainability

Means enabling the affected person to understand why the particular decision occurred.

For example:

“Our model has 500 variables.”

This is transparency of a sort, but not necessarily meaningful explainability.

A better explanation is:

“Your application was rejected primarily because of X, Y and Z. The relevant information came from A and B. The decision was generated using model version 4.2, and a human reviewer confirmed the result.”

That is substantially more meaningful.

17. Explainability and Trade Secrets

A major legal conflict exists between:

Individual's right to understand the decision

and

company's right to protect proprietary technology.

The solution need not always be complete disclosure.

Possible safeguards include:

disclosure of decision factors;

confidential expert inspection;

regulator access;

judicial inspection;

independent audit;

controlled disclosure;

explanation of material logic without source code;

disclosure of model limitations.

Thus, trade-secret protection should not automatically become a justification for completely unreviewable decision-making.

18. Practical AI Explainability Framework for Employers

An organization using AI for employment decisions should ideally maintain the following.

A. AI system documentation

Record:

purpose;

model type;

vendor;

version;

date of deployment;

intended use;

prohibited uses.

B. Data documentation

Record:

data sources;

categories of data;

retention;

accuracy controls;

sensitive-data considerations.

C. Decision documentation

Record:

inputs;

significant factors;

output;

human reviewer;

final decision;

reasons.

D. Bias testing

Test for:

disparate outcomes;

proxy discrimination;

unequal error rates;

demographic disparities;

accessibility problems.

E. Human review

Define:

who reviews;

when review is mandatory;

what the reviewer must consider;

whether override is possible.

F. Appeal mechanism

Provide:

employee/applicant complaint process;

correction mechanism;

human reconsideration;

escalation process.

19. Important Legal Formula

A useful way of understanding the issue is:

AI may assist the decision, but it should not become a substitute for legal accountability.

Or:

Automation does not automatically eliminate the duty to give reasons.

And, particularly in high-impact decisions:

The greater the impact of the AI decision on an individual's rights or interests, the stronger the justification for meaningful explanation, human oversight and review.

20. Case-Law Comparison

21. Position Under Indian Law

India does not presently have a single comprehensive judicial doctrine stating:

“Every AI decision must be explainable.”

Instead, explainability can be derived from several existing legal principles.

For State action, particularly strong arguments can arise from:

Article 14;

Article 21;

natural justice;

proportionality;

reasoned decision-making;

non-arbitrariness;

privacy;

judicial review.

For private-sector AI, the analysis additionally depends on:

employment law;

contractual obligations;

applicable data-protection law;

anti-discrimination requirements;

sector-specific regulation;

principles governing disciplinary proceedings.

The precise legal obligation therefore depends heavily on who is making the decision, what decision is being made, what data is used, and how seriously the decision affects the individual.

22. A Significant Recent Indian Development

An important development in 2026 is the Supreme Court's approach to the use of AI-generated material in judicial proceedings. In Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd., 2026 INSC 668, the Supreme Court dealt with judgments that had relied upon AI-generated citations that were subsequently found to be nonexistent or incorrectly attributed. The Court addressed the consequences for judicial decision-making. (Science and Technology Ministry)

Although this is not itself a conventional “right to explanation” case, it demonstrates a broader judicial principle highly relevant to AI governance:

Human legal decision-makers cannot treat AI-generated output as inherently authoritative; AI output must be independently verified and subjected to human judgment.

That principle is particularly important when AI is used in courts, tribunals, employment decisions and administrative decision-making.

23. Key Takeaways for Examination / Legal Research

First

Explainability is not the same as source-code disclosure.

Second

A person affected by an important AI decision should, where law requires, receive meaningful and intelligible reasons, rather than merely a technical description of the algorithm.

Third

AI cannot be used to circumvent:

natural justice;

reasoned decision-making;

equality;

privacy;

proportionality; or

judicial review.

Fourth

State v. Loomis establishes an important caution against allowing opaque algorithmic assessments to become determinative.

Fifth

SCHUFA is particularly important for the proposition that meaningful information about automated decision-making requires an intelligible explanation of the logic actually applied, rather than merely providing a complicated mathematical formula.

Sixth

Indian constitutional cases such as Maneka Gandhi, Kranti Associates, Mohinder Singh Gill and Puttaswamy provide the doctrinal foundations through which explainability of AI-based public decisions can be assessed.

Seventh

The ultimate legal responsibility normally remains with the human institution or decision-maker, not with the AI system itself.

Conclusion

Explainability of AI-based decisions is fundamentally about accountability.

An individual should not be placed in the position of saying:

“A machine rejected me, but nobody can tell me why.”

Modern AI can make decisions extremely quickly and can process information beyond human capacity. But where those decisions materially affect rights, employment, liberty, privacy, reputation or access to important services, technological complexity cannot by itself justify legal opacity.

The emerging legal approach is therefore not necessarily:

“Disclose every line of code.”

It is closer to:

“Provide sufficient meaningful information, reasons, safeguards and human review so that the affected person can understand, challenge and obtain review of the decision.”

That approach is consistent with Loomis, Bridges, SCHUFA, and with the Indian constitutional principles developed in Maneka Gandhi, Kranti Associates, Mohinder Singh Gill and Puttaswamy.