AI Inference Latency as a Competitive Parameter

Introduction

AI inference latency is the time taken by an AI system to process an input and return an output. In conventional software markets, competition has often focused on price, functionality, reliability and output quality. In AI markets, latency itself can become a significant dimension of competition.

For example, two AI models may provide broadly comparable answers, but one may return the first token in 200 milliseconds while another takes 2 seconds. For real-time applications—voice assistants, autonomous systems, fraud detection, algorithmic trading, gaming, customer-service agents, medical decision-support and industrial control—this difference can materially affect the attractiveness of the product.

Competition law therefore may need to treat latency as a non-price competitive parameter, alongside accuracy, reliability, throughput, context window, availability and interoperability.

Importantly, there is presently no established body of reported antitrust judgments specifically holding that AI inference latency, by itself, constitutes an antitrust violation. The legal analysis must therefore be developed by applying established principles concerning quality competition, discriminatory access, interoperability, foreclosure, tying, essential facilities and exclusionary conduct to AI inference infrastructure.

Recent EU policy developments are particularly relevant: the European Commission has identified cloud computing as important to AI development and has been investigating whether cloud-market characteristics may restrict competition, including interoperability and access issues. In June 2026, it preliminarily considered AWS and Microsoft Azure important gateways despite their not meeting the ordinary quantitative DMA thresholds.

1. Meaning of AI Inference Latency

AI inference occurs when a trained model is used to generate an output.

A simplified chain is:

User input → network → API gateway → model routing → accelerator/GPU → model computation → token generation → response

Latency can arise at each stage.

Important latency measurements

1. Time to First Token (TTFT)
   Time between submitting a request and receiving the first generated token.
2. Time per Output Token
   Speed at which subsequent tokens are generated.
3. End-to-End Latency
   Complete time between request and usable response.
4. Tail Latency
   Performance during the slowest percentage of requests, such as P95 or P99 latency.
5. Routing Latency
   Additional delay caused by model selection, API gateways or multi-model orchestration.
6. Network Latency
   Delay resulting from geographic distance and data-centre architecture.

Thus:

Competitive AI performance = price + quality + accuracy + reliability + latency + availability + interoperability.

2. Why Latency Can Be a Competition Parameter

Latency matters because consumers do not purchase an AI model solely for its theoretical intelligence.

Consider:

A dominant AI provider could therefore theoretically obtain competitive advantages by controlling:

• GPU clusters;
• inference APIs;
• cloud infrastructure;
• model-routing systems;
• edge computing;
• specialised AI accelerators;
• data-centre locations;
• network connectivity;
• inference optimisation software.

The competitive concern becomes particularly significant where a company controls an upstream inference infrastructure and simultaneously competes with downstream AI application providers.

3. Latency as a Non-Price Parameter of Competition

Competition law does not necessarily require competition to occur through monetary price.

Digital markets demonstrate the importance of quality and innovation competition. Indian competition jurisprudence, for example, recognises that digital markets may involve competition through quality, innovation and data even where services are provided at zero monetary price.

Latency can therefore function similarly to:

• search-result quality;
• delivery speed;
• streaming quality;
• network speed;
• transaction execution time;
• platform responsiveness.

A degradation in latency can effectively operate like a quality reduction.

Conversely, preferentially giving a dominant firm's own AI services superior latency could constitute a potential competitive concern.

4. How Latency Can Create Market Power

Latency can generate competitive significance through several mechanisms.

A. Geographic infrastructure advantage

A provider with strategically located data centres can reduce network latency.

B. Hardware advantage

Access to advanced GPUs, TPUs or AI accelerators can increase inference speed.

C. Model optimisation

Quantisation, batching, caching, speculative decoding and other optimisation techniques can reduce latency.

D. Priority scheduling

A cloud provider may theoretically allocate more favourable computing resources to its own AI applications.

E. API throttling

A provider could provide competing applications with slower or more restricted access.

F. Routing discrimination

A platform controlling model-routing infrastructure could direct its own applications toward faster models or hardware.

G. Vertical integration

The greatest concern arises where the infrastructure provider is simultaneously an AI application competitor.

5. Potential Competition Theories

A. Abuse of Dominance

If a company possesses dominance in an upstream AI inference or cloud market, deliberate manipulation of latency could potentially constitute exclusionary conduct.

The relevant question would not simply be:

"Did the competitor experience slower latency?"

Instead, authorities would examine:

• whether the provider is dominant;
• whether the latency difference is attributable to the provider;
• whether similarly situated customers receive materially different treatment;
• whether the difference has exclusionary effects;
• whether legitimate technical explanations exist;
• whether consumers are harmed;
• whether competitors can realistically obtain equivalent infrastructure elsewhere.

6. Latency Discrimination

One important hypothetical is:

Dominant cloud provider → own AI model → 100 ms latency

while:

Dominant cloud provider → competing AI model → 800 ms latency

If the difference results from legitimate architectural differences, there may be no competition problem.

But if the dominant provider deliberately assigns inferior infrastructure, throttles competitors, imposes unnecessary routing delays or reserves high-performance capacity for its own downstream AI service, the conduct could raise exclusionary concerns.

This resembles established competition-law concerns surrounding discriminatory access to infrastructure and vertically integrated platforms.

7. Case Law

1. Microsoft Corp. v Commission — Case T-201/04

The Microsoft case is highly relevant by analogy because it concerned control over interoperability information and Microsoft's position across interconnected software markets.

The European Commission found that Microsoft had refused to provide interoperability information necessary for competing work-group server operating systems and had also engaged in tying involving Windows Media Player. The General Court substantially upheld the Commission's findings.

Relevance to AI latency

AI inference infrastructure may similarly involve technical interfaces that competitors need to compete effectively.

For example:

AI cloud infrastructure → inference API → downstream AI application

If a dominant infrastructure provider gives its own AI application superior technical access while making competing applications slower, the Microsoft reasoning provides an important analytical analogy.

The central concern is not simply ownership of technology, but whether control over an important technical interface can impair effective competition downstream.

2. Google and Alphabet v Commission — Google Android, Case T-604/18

The General Court examined Google's conduct involving Android, Google Search, Chrome, Play Store and agreements with device manufacturers and mobile network operators. The case concerned product bundling, exclusivity payments and anti-fragmentation obligations and their exclusionary effects.

Relevance to AI inference

AI ecosystems can similarly consist of:

Operating system → cloud → model → inference API → application → distribution channel.

If a dominant ecosystem operator conditions access to one layer on preferential use of its own AI inference service, the Android reasoning becomes relevant.

Latency could strengthen the competitive effect where the preferred AI service receives:

• faster execution;
• priority compute;
• better hardware;
• lower network delay;
• preferential routing.

The important legal issue would remain foreclosure and competitive effects, rather than latency alone.

3. Intel Corp. v Commission — Case C-413/14 P

The Intel litigation concerned loyalty rebates and exclusionary effects in the microprocessor market.

The Court of Justice held that where the Commission assesses whether a dominant undertaking's rebate scheme is capable of restricting competition, it may need to examine the circumstances of the conduct, including the possibility of foreclosure of an as-efficient competitor.

Relevance to AI inference

Latency could become part of an analogous foreclosure analysis.

Suppose an AI infrastructure provider offers:

• low-latency inference to customers using its own model;
• materially slower inference to customers using rival models.

The authority could examine whether the resulting technical disadvantage makes it substantially harder for an equally efficient rival to compete.

Thus:

Latency differential → customer migration → rival foreclosure

could potentially form part of an effects analysis.

4. Qualcomm Inc. v European Commission — Case T-235/18

In Qualcomm, the General Court examined exclusivity payments involving LTE chipsets and the possibility of foreclosure effects. The case concerned whether conduct by a dominant supplier could impair competitors' ability to compete in a technologically important market.

Relevance to AI infrastructure

The AI equivalent could involve incentives or contractual arrangements under which customers receive superior inference performance only if they commit substantial workloads to a dominant infrastructure provider.

For example:

Exclusive cloud commitment → preferential GPU allocation → lower inference latency.

The competition concern would not be the existence of faster service itself. Rather, the question would be whether the arrangement artificially prevents rivals from obtaining sufficient scale or access to compete.

5. Google Search (Shopping)

The Google Shopping litigation is important because it demonstrates how non-price treatment and algorithmic positioning can become competition-law issues.

The case concerned Google's preferential treatment of its own comparison-shopping service relative to rival services. The General Court upheld the essential finding that Google's conduct could constitute an abuse of dominance. Scholarship concerning the case specifically identifies discrimination, equal opportunities to compete and competition on the merits as important aspects of the reasoning.

Application to AI latency

Consider:

Platform-owned AI assistant:
20 ms routing overhead

Independent AI assistant:
250 ms routing overhead

If the platform controls the relevant technical infrastructure and intentionally imposes the disadvantage on competing AI services, latency can operate as a form of technical discrimination.

This is particularly significant where users select services partly according to speed.

Thus:

Search ranking discrimination in one generation of digital platforms could have an analogous technical form in AI through latency discrimination.

6. Bronner — Case C-7/97

In Oscar Bronner GmbH v Mediaprint, the Court of Justice established strict conditions under which refusal by a dominant undertaking to provide access to infrastructure can constitute abuse.

The doctrine concerns circumstances involving indispensable infrastructure and the possibility of eliminating effective competition.

Relevance to AI inference

An AI inference infrastructure could potentially become an essential input where:

1. the infrastructure is indispensable;
2. effective alternatives do not realistically exist;
3. duplication is technically or economically impracticable;
4. refusal or discriminatory access eliminates effective competition.

Latency becomes important where access technically exists but the quality of access is materially inferior.

For example:

Access at 50 requests/second with 100 ms latency

may be commercially different from:

Access at 5 requests/second with 1,500 ms latency.

Thus, "access" should not necessarily be analysed merely as binary physical access.

7. Servizio Elettrico Nazionale v AGCM — Case C-377/20

The Court of Justice examined exclusionary conduct by an incumbent electricity operator and emphasised that competition law distinguishes competition on the merits from conduct capable of exploiting advantages arising from an incumbent position.

AI relevance

AI infrastructure providers may possess structural advantages arising from:

• enormous capital expenditure;
• existing cloud customers;
• proprietary hardware;
• network infrastructure;
• data-centre locations;
• integrated software ecosystems.

The competition-law question is whether a latency advantage reflects competition on the merits or whether it results from strategic exploitation of an upstream bottleneck.

A technically superior model that legitimately produces lower latency is normal competition.

Deliberately degrading competitors' inference performance is a different question.

8. Microsoft's Interoperability Principle and AI APIs

The Microsoft case is particularly useful because AI systems increasingly operate through APIs.

Consider:

Cloud provider

↓

Inference API

↓

Third-party AI applications

If the cloud provider controls the API and simultaneously competes with downstream applications, several issues arise:

• API access;
• rate limits;
• priority queues;
• token quotas;
• geographic routing;
• GPU allocation;
• caching;
• batch scheduling;
• model availability;
• response-time guarantees.

Latency therefore becomes a measurable component of technical interoperability.

9. Latency as a Potential Essential-Facility Quality Parameter

Traditional essential-facility analysis sometimes asks whether competitors have access to an indispensable facility.

In AI markets, that question could evolve from:

"Can the rival access the infrastructure?"

to:

"Can the rival access the infrastructure on sufficiently competitive technical terms?"

This creates a distinction:

Formal access

The competitor technically has API access.

Effective access

The competitor receives:

• comparable compute;
• reasonable latency;
• sufficient throughput;
• reliable availability;
• appropriate geographic routing;
• non-discriminatory rate limits.

A dominant provider could potentially satisfy formal access while undermining effective competitive access through inferior performance.

10. Latency and Self-Preferencing

Self-preferencing could occur at the inference layer.

Hypothetical structure

A cloud company operates:

Cloud infrastructure

and

Its own AI model

and

Its own consumer AI assistant

The provider could theoretically give its own assistant:

• premium GPUs;
• priority queues;
• edge deployment;
• faster networking;
• larger concurrency limits;
• better caching.

Rivals could technically remain available but experience substantially worse latency.

This could create:

Technical advantage → better user experience → greater adoption → more data → better model → greater adoption

creating a feedback loop.

11. Latency and Network Effects

AI markets can contain powerful network effects.

A simplified cycle is:

Lower latency

↓

More users

↓

More usage data

↓

Better optimisation

↓

Higher infrastructure utilisation

↓

Lower average cost

↓

More investment in infrastructure

↓

Further latency advantage

The danger from a competition-law perspective is not that network effects exist. Network effects can be legitimate economic efficiencies.

The concern arises if a dominant firm artificially creates or protects the advantage through exclusionary conduct.

12. Latency and Switching Costs

AI customers may become dependent on particular inference infrastructure because switching can require:

• model conversion;
• API redesign;
• software optimisation;
• data migration;
• benchmarking;
• retraining;
• hardware adaptation;
• regulatory validation.

If switching is costly, a small latency advantage can become commercially significant.

The European Commission's current cloud investigations expressly examine interoperability, data access, tying/bundling and contractual conditions in cloud markets, reflecting the importance of cloud infrastructure to AI deployment.

13. Latency as a Quality Competition Variable

The traditional competition equation:

Competition = Price

is insufficient for AI.

A more appropriate model is:

AI Competition = Price + Quality + Accuracy + Latency + Reliability + Innovation + Interoperability

Latency therefore resembles other quality dimensions.

A zero-price AI service can still compete through:

• response speed;
• accuracy;
• context;
• safety;
• reliability;
• multimodality.

Academic competition-law literature similarly recognises that quality can be an important competitive dimension in digital markets where monetary prices are zero.

14. Possible Theories of Harm

A. Latency discrimination

Different latency for similarly situated competitors.

B. Selective throttling

Competitors receive lower computational priority.

C. Preferential GPU allocation

The dominant provider reserves high-performance hardware for its own AI products.

D. Strategic routing

The provider routes rival models through slower infrastructure.

E. API degradation

A rival's API access is technically available but materially inferior.

F. Tying

Customers receive superior inference performance only if they purchase another product.

G. Exclusive dealing

Customers receive latency or capacity benefits in exchange for exclusivity.

H. Refusal to upgrade

A dominant infrastructure provider upgrades its own AI system while withholding equivalent technical capabilities from downstream competitors.

15. Legitimate Reasons for Latency Differences

Competition law should not automatically treat different latency as unlawful discrimination.

There can be legitimate technical explanations.

For example:

• model size;
• number of parameters;
• context length;
• security checks;
• geographic location;
• congestion;
• workload characteristics;
• hardware compatibility;
• batch size;
• computational complexity;
• reliability requirements.

A larger model may legitimately take longer to respond.

Accordingly, an authority would need to distinguish:

legitimate technical differentiation

from

strategic exclusionary differentiation.

This distinction is essential.

16. Evidence Required in an AI Latency Investigation

A competition authority would likely need extensive technical evidence.

A. Benchmarking

Compare P50, P95 and P99 latency.

B. Controlled experiments

Test identical workloads across competing AI models.

C. Hardware allocation records

Determine which GPUs or accelerators were allocated.

D. API logs

Analyse:

• request routing;
• queue position;
• throttling;
• retries;
• geographic routing.

E. Internal communications

Look for evidence concerning:

• competitor degradation;
• preferential treatment;
• capacity allocation;
• strategic throttling.

F. Customer evidence

Determine whether latency differences materially affected customer choice.

G. Counterfactual analysis

Ask:

What would latency have been absent the challenged conduct?

17. Relevant Market Definition

Several possible markets could arise.

Market 1: AI inference APIs

Competition among providers offering model inference through APIs.

Market 2: Cloud AI infrastructure

Competition among cloud providers supplying computational resources for inference.

Market 3: AI accelerators

GPUs, TPUs and other specialised AI processors.

Market 4: Edge AI inference

Low-latency computing closer to end users.

Market 5: AI application services

Consumer or enterprise applications using underlying models.

The correct market depends upon substitutability and the factual circumstances.

18. Latency and Market Power

Latency becomes especially important where a firm possesses:

High market share + scarce infrastructure + high switching costs + network effects + vertical integration.

For example:

Dominant cloud provider
↓
Controls GPUs
↓
Controls inference API
↓
Owns foundation model
↓
Operates downstream AI assistant

This structure creates a potential incentive to disadvantage rivals through technical parameters rather than explicit pricing.

19. Efficiency Defence

A dominant provider may legitimately argue that superior latency results from efficiency.

Possible efficiencies include:

• better hardware;
• superior model architecture;
• optimised kernels;
• caching;
• speculative decoding;
• efficient batching;
• better data-centre architecture;
• superior networking.

These can constitute competition on the merits.

The critical distinction is therefore:

20. Remedies

If unlawful conduct were established, possible remedies could include:

Structural remedies

• divestiture;
• separation of infrastructure and downstream AI operations.

Behavioural remedies

• non-discriminatory API access;
• transparent latency commitments;
• equal technical treatment;
• interoperability requirements;
• prohibition on discriminatory throttling.

Monitoring

Authorities could require:

• latency audits;
• independent benchmarking;
• API transparency;
• reporting of infrastructure allocation;
• P95/P99 performance disclosures.

A particularly relevant concept would be:

Quality-of-Service non-discrimination.

This would mean similarly situated downstream AI providers should receive materially comparable infrastructure treatment unless objectively justified.

21. Emerging Importance of Cloud Infrastructure

The issue is becoming more significant because AI deployment increasingly depends on cloud infrastructure.

The European Commission's 2025–2026 cloud investigations expressly recognise cloud computing as important to AI development and are examining interoperability, data access, tying/bundling and contractual conditions.

The Commission's June 2026 preliminary position regarding AWS and Azure is particularly relevant because it treats cloud infrastructure as potentially functioning as an important gateway even where ordinary quantitative gatekeeper thresholds are not met.

This does not establish that latency discrimination is unlawful. It demonstrates, however, the increasing regulatory importance of infrastructure-level control in AI markets.

22. Indian Competition-Law Perspective

Under the Competition Act, 2002, latency could potentially become relevant under several concepts.

Section 4 — Abuse of dominant position

Potential theories include:

• discriminatory conditions;
• denial of market access;
• leveraging;
• exclusionary conduct.

Section 3

Where latency coordination is achieved through an agreement between competitors, questions concerning anti-competitive agreements may arise.

Section 5

AI/cloud acquisitions may require examination where the transaction substantially changes competitive conditions.

Section 19

The Competition Commission of India could examine:

• market structure;
• market share;
• consumer dependence;
• entry barriers;
• technological advantages;
• vertical integration;
• network effects.

Indian digital-market jurisprudence increasingly recognises that competition may occur through quality, innovation and other non-price dimensions, rather than only monetary prices.

23. Six Core Case-Law Principles

24. Hypothetical Example

Assume AI Cloud A controls 65% of a relevant inference-infrastructure market.

It also owns AI Model A.

Competitor AI Model B uses AI Cloud A's infrastructure.

The technical data shows:

The mere existence of these differences does not prove an infringement.

The investigation would ask:

1. Are the models technically comparable?
2. Does AI Cloud A control the relevant infrastructure?
3. Could Model B obtain equivalent treatment?
4. Are the differences objectively justified?
5. Did AI Cloud A intentionally create the latency gap?
6. Does the gap materially affect customers?
7. Does it foreclose competing AI providers?
8. Are there viable alternative cloud providers?
9. Does the conduct benefit AI Cloud A's downstream model?
10. Are there demonstrable efficiencies?

Only after this analysis could competition-law liability potentially be established.

25. Key Legal Principle

The most important distinction is:

Latency advantage is normally legitimate competition; strategically imposed latency disadvantage can potentially become an antitrust issue when exercised by a dominant infrastructure provider in a manner capable of foreclosing rivals.

Competition law should therefore not punish a provider merely because its AI system is faster.

Instead, the inquiry should focus on why it is faster, who controls the relevant bottleneck, whether competitors receive equivalent technical opportunities, and whether the difference is the product of competition on the merits or exclusionary conduct.

Conclusion

AI inference latency is increasingly capable of functioning as a competitive parameter comparable to price, quality, reliability and innovation. Its importance is greatest in real-time AI applications and where downstream firms depend upon infrastructure controlled by a vertically integrated competitor.

The existing case law does not yet provide a standalone doctrine called "AI inference latency discrimination." Instead, the strongest legal framework comes from established doctrines involving:

1. quality competition;
2. competition on the merits;
3. discriminatory treatment;
4. self-preferencing;
5. interoperability;
6. refusal to supply;
7. essential facilities;
8. tying and bundling;
9. exclusive dealing; and
10. foreclosure of equally efficient competitors.

The combination of AI models + cloud infrastructure + specialised compute + inference APIs + downstream applications makes latency particularly important. Current EU cloud investigations reinforce the broader regulatory focus on infrastructure, interoperability and AI deployment, although they do not themselves establish an infringement concerning latency.

Accordingly, in future AI competition cases, P95/P99 latency, API throttling, queue priority, geographic routing, accelerator allocation and quality-of-service parity may become important pieces of evidence in determining whether an apparent technical-performance difference represents genuine innovation or exclusionary conduct.