Emotion-Recognition & Financial Decision Bans - Spain & EU AI Act

The Core Rule - EU AI Act Regulation 2024/1689

EU AI Act is horizontal - applies to banks, fintech, contact centers. Risk pyramid:

• Unacceptable Risk - Art 5 - BANNED - from 2 Feb 2025 - no compliance makes it legal
• High-Risk - Art 6 + Annex III - allowed with conformity assessment - from 2 Aug 2026 phased, financial services from 2 Dec 2027

Two separate bans intersect for banks.

BAN 1: Emotion Recognition in Workplace & Education - Art 5(1)(f)

Text: "The placing on the market, the putting into service for this specific purpose or the use of AI systems to infer the emotions of a natural person in the workplace and educational institutions is prohibited, except where the use is intended to be put in place or marketed for medical or safety reasons."

Commission Guidelines Feb 2025 - Definition:

• System that identifies or infers emotions - happiness, sadness, anger, surprise, disgust, embarrassment, excitement, shame, contempt, satisfaction, amusement etc.
• Based on biometric data - facial expression, voice, heart rate, gait, keystroke
• In workplace - includes recruitment, performance management, contact center agent monitoring, trader floor surveillance

What is BANNED since 2 Feb 2025 in Spain & EU:

• Webcams tracking employee facial expressions during Zoom client calls to evaluate performance
• Voice analytics scoring trader stress level, customer service agent friendliness
• AI stress detection for burnout monitoring using biometric inputs
• Emotion recognition during recruitment interviews - even if candidate consents - consent cannot override ban
• Meeting room AI that says "Room is 70% disengaged"

What is ALLOWED with exception:

• Medical: Detecting fatigue in pilot, trader for safety
• Safety: Driver drowsiness detection - but must be safety purpose, not performance

Fine Tier: Art 99(3) - Up to €35M or 7% global annual turnover, whichever higher - harshest tier.

Spain Specific: Spanish presidency of EU Council pushed this ban. Spanish draft Organic Law adapting AI Act - approved by Council of Ministers 2025 - creates Agencia Española de Supervisión de Inteligencia Artificial (AESIA) in A Coruña as supervisor. Spain will enforce ban via AESIA + Bank of Spain for financial entities. Sanctions mirror EU.

Example: Retail chain fined ~€15M in 2026 reporting for real-time emotion recognition across 4 EU states - placed under Art 5.

BAN 2: Financial Decision - Credit Scoring & Emotion as Vulnerability Exploitation

Emotion recognition ban is narrow to workplace/education. But using emotion to make financial decision triggers two other bans + high-risk classification:

A. Art 5(1)(a) & (b) - Manipulative & Exploitation of Vulnerabilities:

• Banned: AI system that deploys subliminal techniques or exploits vulnerabilities due to age, disability, socio-economic situation to materially distort behaviour leading to financial decision
• Example: Detecting customer is anxious/sad via voice and pushing high-interest loan, or detecting trader is euphoric and pushing risky product - qualifies as exploitation of emotional vulnerability - BANNED

B. Art 5(1)(c) - Social Scoring Ban - cannot evaluate creditworthiness based on social behaviour or personality traits.

C. High-Risk - Annex III Point 5(b) - Creditworthiness:

"AI systems intended to be used to evaluate the creditworthiness of natural persons or establish their credit score, with the exception of AI systems used for the purpose of detecting financial fraud"

This becomes high-risk from 2 Dec 2027 (after Digital Omnibus Regulation 2026/1744). Means:

• Must have risk management, data governance, human oversight, logging, conformity assessment, CE marking
• Cannot use emotion as feature for credit scoring - would be discriminatory and violate data minimisation
• Must provide explainability under GDPR Art 22

If you use emotion recognition to infer creditworthiness - e.g. analysing facial micro-expressions during loan video KYC to assess honesty - this is double violation:

1. If employee is monitored -> Art 5(1)(f) banned
2. If customer emotion used for credit decision -> High-risk Annex III 5(b) + Art 5(1)(b) exploitation ban if emotional vulnerability exploited

Spain Banking Example - Prohibited vs Allowed

PROHIBITED in Spanish Bank:

• Call center uses AI to score agent empathy from voice - Art 5(1)(f) - must switch off since Feb 2025 - No consent form overrides
• Recruitment platform analyses candidate emotions in video interview for teller role - banned
• Loan officer uses Affectiva-style AI to detect if SME borrower looks nervous during application and denies loan - Art 5(1)(b) + high-risk 5(b) - banned as exploitative
• Contact center monitors customer emotion to dynamically increase overdraft fees when customer sounds desperate - Art 5(1)(b) exploitation of socio-economic vulnerability

ALLOWED but HIGH-RISK - Requires Compliance from Dec 2027:

• AI credit scoring using financial data (income, history) - not emotions - Annex III 5(b) - requires conformity assessment, human oversight, bias testing
• Fraud detection AI analysing voice for deepfake - exception under 5(b)
• Customer sentiment analysis for product improvement - if not used to infer emotions in workplace context and not for financial decision - allowed but triggers Art 50 transparency: must inform person they are subject to emotion recognition

Case Laws

1. CJEU SCHUFA Holding - Case C-634/21 - 7 Dec 2023 - The Credit Scoring Precedent

Facts: German credit bureau SCHUFA generated automated probability value about person's payment behaviour. Bank relied heavily on score to reject loan. Consumer argued Art 22 GDPR automated decision.

CJEU Held: Credit scoring by bureau = automated individual decision under Art 22 GDPR when lender draws strongly on that score to establish/implement/terminate contract. Even though human makes final decision, if score has determining role, it counts as solely automated.

Impact for AI Act financial ban: Banks cannot hide behind "human in loop" - if AI emotion/credit score determines decision, it is automated decision. Must have GDPR Art 22 safeguards + AI Act high-risk obligations. This case now used by Spanish AEPD to require banks to disclose logic.

2. Commission Draft Guidelines on High-Risk AI Systems - 19 May 2026 - Non-binding but authoritative

Guidelines confirm: Emotion recognition systems are high risk with potentially wide-ranging implications for commercial applications. Although many use cases are permissible (even if high risk), in workplace and educational contexts, prohibited.

Also clarifies: Art 5(1)(f) ban applies even if system marketed for other purpose but deployed to infer emotions at work.

3. First AI Act Fines 2026 - Trade Reporting:

• Credit scoring provider fined ~€14M for Annex III documentation failures - creditworthiness assessment without proper data governance
• Retail chain fined ~€15M for real-time emotion recognition across 4 EU states - placed under Art 5 prohibited

Shows enforcement started before high-risk deadline - for Art 5 bans already live.

4. OECD Evidence Cited in EU AI Act Research:

Facial emotion recognition misreads across ethnicities, ages, neurological profiles - cited as reason for workplace ban. Cannot reliably infer internal state from external expression - scientific invalidity argument used in Spanish draft law preamble.

Spain Compliance Roadmap for Bank

Since 2 Feb 2025 - STOP:

• Inventory all AI systems that infer emotions from biometrics in workplace - contact center QA tools, trader surveillance, hiring platforms - decommission or switch to non-biometric performance metrics (e.g. sales numbers, not facial analysis)

By 2 Aug 2025 - AI Literacy Art 4:

• Train staff handling AI credit scoring on AI Act

By 2 Dec 2027 - High-Risk Compliance:

• For AI credit scoring systems - Annex III 5(b):
  • Risk management system
  • Data governance - no emotion data as input
  • Technical documentation, logging
  • Human oversight - two-person rule
  • Conformity assessment, CE marking
  • Registration in EU database

Penalty: Up to €35M / 7% for Art 5 violations - Spanish AESIA can inspect.