Sampling vs Full Production Decisions

Sampling vs full production is an important issue in document discovery and e-discovery, especially where a case involves thousands or millions of documents, emails, contracts, HR records, transaction records, or electronically stored information (ESI).

The basic question is whether a party should be required to produce/review the entire relevant data population or whether a statistically or otherwise properly selected sample can reasonably establish relevance, responsiveness, completeness, or the need for further production.

1. Meaning of Sampling

Sampling means selecting a smaller, representative portion of a larger document/data population and examining that portion to draw conclusions about the larger population.

For example:

• Total emails = 1,00,000
• Random sample = 2,000 emails
• Sample is reviewed for relevance.
• If the sample shows a very low proportion of relevant documents, the parties may argue that full review/production is disproportionate.
• If the sample reveals substantial relevant material, a broader search or production may be required.

Sampling can be:

1. Random sampling – documents are selected randomly.
2. Stratified sampling – the population is divided into categories and samples are taken from each category.
3. Judgmental sampling – documents are selected based on identified issues or professional judgment.
4. Validation sampling – used to test whether a search, keyword protocol, TAR system, or predictive-coding process is missing relevant documents.

2. Meaning of Full Production

Full production generally means producing all documents falling within the legally relevant and properly defined production scope, subject to applicable objections, privilege, confidentiality, proportionality and other protections.

Full production may be appropriate where:

• the dataset is relatively small;
• the documents are central to the dispute;
• each document may materially affect the outcome;
• the population cannot reliably be sampled;
• the requesting party demonstrates a particular need;
• sampling has revealed significant relevant documents;
• the cost of production is reasonable compared with the importance of the dispute.

3. Sampling Does Not Automatically Mean “Produce Only the Sample”

An important distinction is that sampling can be a method of testing or validating a discovery process, rather than necessarily being the final production itself.

For example:

1 million documents → statistical sample → evaluate relevance → validate search/TAR → determine appropriate production population.

Courts may permit sampling where full production would create disproportionate expense or burden. In Duffy v. Lawrence Memorial Hospital, for example, the court permitted a random sample of 252 records from a much larger population because of the time and expense involved in reviewing the entire set.

4. Factors Relevant to the Sampling vs Full Production Decision

Courts generally examine considerations such as:

A. Relevance

The first question is whether the documents sought are relevant to the dispute.

Indian courts have repeatedly emphasised that discovery should relate to matters in controversy and should not become a fishing or roving inquiry.

B. Necessity

The requesting party should demonstrate why the information is necessary for fairly deciding the case.

C. Volume

If millions of emails or electronic records are involved, full manual review may impose enormous costs.

D. Cost and burden

The court may consider whether the cost of reviewing every document is disproportionate to the likely evidentiary value.

E. Importance of the dispute

In a case involving very important or potentially dispositive documents, a court may require broader discovery even where the volume is substantial.

F. Reliability of the sample

A sample should be designed so that conclusions drawn from it have a reasonable statistical or methodological foundation.

G. Consequences of missing relevant documents

If failure to identify a document could materially prejudice the opposing party, a larger review or additional validation may be necessary.

H. Privilege and confidentiality

Even where sampling is permitted, privileged or confidential information must be appropriately protected.

5. Important Case Laws

1. Duffy v. Lawrence Memorial Hospital — 2017

The court considered a large collection of electronic patient records and permitted the producing party to use random sampling rather than full production/review.

The court recognised the substantial time and expense involved in reviewing the complete population and found the proposed sampling approach justified.

Principle:
Where the volume and expense of full discovery are substantial, properly designed random sampling may be an appropriate discovery solution.

2. Da Silva Moore v. Publicis Groupe — 287 F.R.D. 182 (S.D.N.Y. 2012)

This is a leading e-discovery decision concerning technology-assisted review (TAR).

The court accepted computer-assisted review for a very large electronic document collection. The process involved human review of a smaller set of documents, which was then used to train the technology to identify potentially relevant documents in the larger population.

The court stressed the importance of a defensible methodology and quality-control testing.

Principle:
Courts can accept technology-assisted methodologies instead of requiring exhaustive manual review of every document, particularly in large-data cases.

3. Rio Tinto PLC v. Vale S.A. — 306 F.R.D. 125 (S.D.N.Y. 2015)

The court dealt with predictive coding and statistical validation in large-scale discovery. The protocol contemplated statistically valid sampling, including a sample designed around a specified confidence level and margin of error.

The decision is significant because it illustrates how sampling can be incorporated into a defensible TAR/e-discovery protocol.

Principle:
Statistical sampling can be used to evaluate the effectiveness of a document-review methodology rather than automatically requiring exhaustive manual review.

4. Pyrrho Investments Ltd v MWB Property Ltd — [2016] EWHC 256 (Ch)

The English High Court approved the use of predictive coding in a large electronic disclosure exercise.

The court recognised that technology-assisted review could save time and costs and emphasised that the process should contain appropriate checks and safeguards capable of independent verification.

Principle:
The law does not necessarily require every document to be manually examined when an appropriately controlled technological methodology can reasonably identify relevant documents.

5. Rajesh Bhatia v. G. Parimala — 2005

The Indian court discussed the scope of discovery under Order XI CPC and emphasised that discovery is intended to provide relevant documentary material necessary for fair disposal of proceedings.

The court also recognised that the power of discovery is discretionary and that production should not become an unnecessary or oppressive exercise.

Principle:
Discovery must be connected with relevance and necessity; courts have discretion concerning the extent of production.

6. Delhi State Industrial & Infrastructure Development Corporation Ltd. v. Shiv Kumar — 2013

The Delhi High Court summarised important principles governing discovery and production of documents.

It stated that the documents should be relevant, within the possession or power of the party, and that production should be necessary for fair disposal of the proceedings or saving costs. It also recognised that discovery may be general or limited to particular classes of documents depending on the circumstances.

The court also cautioned against using discovery for a roving or fishing inquiry.

Principle:
The scope of production can be limited where full discovery is unnecessary, irrelevant, or disproportionate.

7. 20th Century-Fox Corporation (India) Pvt. Ltd. v. F.H. Lala — 1973

In the context of an Industrial Tribunal, the court considered the production and inspection of documents and emphasised the requirement of relevance before compelling production.

The party requesting documents must provide sufficient material showing their relevance and explain the nature and necessity of the requested documents.

Principle:
A demand for complete production cannot rest merely on a broad assertion; relevance and necessity must be demonstrated.

8. Lalit Pratap Singh v. Union of India — 2022

In a disciplinary inquiry context, the court considered requests for production of documents and held that questions concerning the relevance and necessity of documents are matters for the inquiry authority under the applicable rules.

The decision illustrates that where relevant documents are necessary for defending charges, restricting access cannot be justified merely by another authority independently deciding that the documents are irrelevant.

Principle:
Where documents are demonstrably relevant to a party's defence, procedural fairness may require their production.

6. Sampling vs Full Production — Comparison

7. Sampling and Technology-Assisted Review

Sampling is particularly important in TAR/predictive coding.

A simplified process is:

Large dataset → Random/seed sample → Human coding → Algorithm training → Ranking/classification → Validation sample → Production

For example:

• 5,00,000 documents collected.
• Lawyers review a statistically appropriate sample.
• Relevant and irrelevant documents are coded.
• TAR learns from those decisions.
• The remaining documents are ranked.
• A validation sample checks whether relevant documents are being missed.
• Responsive documents are then considered for production.

Courts have recognised that TAR can be an acceptable alternative to exhaustive manual review when the process is appropriately designed and validated.

8. When Full Production May Be Preferred

Full production is more likely to be appropriate where:

1. The total number of documents is relatively small.
2. Documents are individually important.
3. Each document could affect liability or damages.
4. The sample reveals a high concentration of relevant documents.
5. The sampling methodology is unreliable.
6. The opposing party identifies specific missing documents.
7. There is evidence suggesting that relevant material exists outside the sample.
8. The information concerns a central issue rather than peripheral material.

For example, if a dispute concerns 10 specific employment contracts, sampling 2 contracts would generally not provide the same evidentiary coverage as producing all 10.

9. When Sampling May Be Appropriate

Sampling becomes particularly useful where:

• there are hundreds of thousands or millions of records;
• full review would impose disproportionate costs;
• the dataset is sufficiently homogeneous for statistical sampling;
• the purpose is to estimate the prevalence of relevant documents;
• the sample can be independently validated;
• the parties agree on the methodology;
• the court considers the approach proportionate.

The important point is that sampling should be defensible, documented and capable of validation.

10. Key Legal Principle

There is no universal rule that “full production is always required” or that “sampling is always sufficient.”

The appropriate approach depends upon:

Relevance + Necessity + Proportionality + Volume + Cost + Reliability of Methodology + Risk of Missing Material Evidence.

Indian discovery jurisprudence particularly emphasises relevance, necessity, fairness and avoidance of fishing inquiries.

In large-scale e-discovery, courts have also recognised that statistically supported sampling and technology-assisted review can provide practical alternatives to exhaustive manual review, provided the methodology contains appropriate safeguards and validation.

Conclusion

Sampling and full production are not mutually exclusive concepts. Sampling can be used initially to understand a dataset, test search terms, validate TAR, estimate the proportion of relevant material, or determine whether broader discovery is necessary. Full production may then be ordered for the relevant population or for particular categories where the sample demonstrates a sufficient need.

The central consideration is not simply whether every document was reviewed, but whether the discovery methodology was reasonable, relevant, proportionate, reliable and capable of producing the material necessary for a fair determination of the dispute.