The Bradley-Terry Model in Materiality Assessments: Statistical Consensus for CSRD Compliance

As the regulatory pressure of the Corporate Sustainability Reporting Directive (CSRD) mounts, organizations are abandoning subjective survey methods in favor of rigorous mathematical frameworks. To comply with the European Sustainability Reporting Standards (ESRS), specifically ESRS 2 IRO-1, companies must document a robust, defensible methodology for determining which Environmental, Social, and Governance (ESG) topics are material.

At the forefront of this shift is the application of mathematical consensus models. While many are familiar with broad concepts like forced-choice surveys, understanding the underlying statistical engines is critical for defending your results to an auditor.

In this article, we delve into one of the most powerful statistical frameworks for pairwise ranking: the Bradley-Terry model.

By applying this time-tested probabilistic model to ESG, sustainability teams can handle complex, incomplete datasets and extract a definitive, mathematically pure hierarchy of material impacts and risks. Let's explore how the Bradley-Terry model transforms messy human opinions into statistical consensus.

1. What is the Bradley-Terry Model?

First introduced in 1952 by statisticians Ralph Allan Bradley and Milton E. Terry, the Bradley-Terry model is a probabilistic framework for analyzing paired comparisons. Long before it was used to analyze the financial materiality of climate risks, it was utilized in fields ranging from sensory analysis (e.g., taste testing different cola brands) to calculating the Elo ratings of chess grandmasters and predicting sports tournament outcomes.

The premise of the Bradley-Terry model is simple: If you want to rank a group of items, you compare them two at a time (pairwise comparison).

When applied to ESG materiality, instead of asking a stakeholder to rank 30 topics simultaneously—a cognitively overwhelming task—you present them with pairs. "Which has a greater impact: Water Usage or Employee Health & Safety?"

The model takes the results of thousands of these micro-decisions across various stakeholders and uses probability theory to infer the latent "true" importance (or "worth") of every single ESG topic. It calculates a statistical consensus that perfectly reflects the collective intelligence of the stakeholder group.

For a deeper understanding of the broader framework this fits into, see our pillar article on the MLE Consensus Model for ESG Materiality.

2. The Mathematics Explained Accessibly

To appreciate why regulators and auditors trust the Bradley-Terry model, we need to look under the hood. While the math relies on advanced statistics, the core concepts are highly intuitive.

The Core Probability Formula: P(i > j)

The model assumes that every ESG topic $i$ has an underlying, unobserved "worth" or parameter denoted by $\theta_i$ (where $\theta_i > 0$).

The probability that a stakeholder will choose Topic $i$ over Topic $j$ is calculated as the ratio of Topic $i$'s worth to the combined worth of both topics:

$$P(i > j) = \frac{\theta_i}{\theta_i + \theta_j}$$

What this means in plain English: If "Climate Mitigation" ($\theta_i$) has a true worth of 4, and "Office Recycling" ($\theta_j$) has a true worth of 1, the probability that a stakeholder will choose Climate Mitigation in a head-to-head comparison is $4 / (4 + 1) = 4/5$, or 80%.

Our goal during a materiality assessment is the reverse: we observe the voting percentages (e.g., Climate Mitigation wins 80% of the time) and use those to work backward to find the true $\theta$ values for all topics.

Maximum Likelihood Estimation (MLE)

To find these $\theta$ values, we use Maximum Likelihood Estimation. We want to find the set of topic weights that makes the actual voting data we collected the most mathematically probable outcome.

If $w_{ij}$ is the number of times Topic $i$ was chosen over Topic $j$, we construct a log-likelihood function $L(\theta)$:

$$L(\theta) = \sum_{i < j} \left[ w_{ij} \ln(\theta_i) + w_{ji} \ln(\theta_j) - (w_{ij} + w_{ji}) \ln(\theta_i + \theta_j) \right]$$

To maximize this function, algorithms use iterative optimization techniques—such as the Newton-Raphson method or Minorize-Maximization (MM) algorithms. The algorithm starts with a guess for all $\theta$ values (e.g., they are all equal). It calculates how well that guess fits the voting data, adjusts the values to fit slightly better, and repeats this process millions of times per second until the numbers stop changing. When they stop changing, the algorithm has "converged" on the mathematical truth.

A Worked Numerical Example

Let's look at a simplified example with 4 ESG topics being evaluated by stakeholders for financial materiality:

• Topic A: Carbon Pricing Risk
• Topic B: Supply Chain Labor Disruption
• Topic C: Data Privacy Breach
• Topic D: Packaging Waste Regulations

Stakeholders cast the following pairwise votes (wins):

• A beats B: 7 times (B beats A: 3 times)
• A beats C: 6 times (C beats A: 4 times)
• A beats D: 9 times (D beats A: 1 time)
• B beats C: 5 times (C beats B: 5 times)
• B beats D: 8 times (D beats B: 2 times)
• C beats D: 7 times (D beats C: 3 times)

We feed these 60 total comparisons into the Bradley-Terry optimization algorithm. After iterating, the algorithm converges on the following normalized $\theta$ values (summing to 1 or 100%):

1. Topic A (Carbon): $\theta_A = 0.42$ (42%)
2. Topic B (Labor): $\theta_B = 0.25$ (25%)
3. Topic C (Data): $\theta_C = 0.23$ (23%)
4. Topic D (Packaging): $\theta_D = 0.10$ (10%)

This gives you a definitive, defensible ranking. Carbon Pricing Risk is mathematically proven to be roughly four times more financially material than Packaging Waste Regulations ($\frac{0.42}{0.10} = 4.2$).

3. Why Bradley-Terry is Ideal for ESG Materiality

When executing a Complete Double Materiality in 5 Steps, sustainability teams encounter real-world logistical challenges. The Bradley-Terry model is uniquely equipped to handle them.

Handling Incomplete Comparison Graphs

In a standard assessment with 30 ESG topics, there are 435 possible pairwise combinations. You cannot ask a single stakeholder to make 435 comparisons; they will suffer severe survey fatigue.

Therefore, you must show each stakeholder only a small subset of pairs (e.g., 20 pairs). This creates an "incomplete graph"—meaning not every topic is directly compared to every other topic by every single person.

The Bradley-Terry model excels at resolving incomplete graphs. As long as the graph is mathematically "connected" (meaning there is a chain of comparisons linking every topic to every other topic indirectly), the MM algorithm will still reliably converge on the true consensus ranking. If A beats B, and B beats C, the algorithm infers that A is likely stronger than C, even if A and C were never directly compared.

Probabilistic Output and Confidence Intervals

Because Bradley-Terry is a probabilistic model rooted in logistic regression, it doesn't just give you a ranking; it gives you confidence intervals. If Topic $B$ and Topic $C$ have scores of 0.25 and 0.23 respectively, the model can tell you the statistical variance. It can tell an auditor, "We are 95% confident that the true materiality of these two topics is effectively tied." This nuance is vital for setting strategic thresholds in a DMA methodology and scoring framework.

Massive Scalability

The computational efficiency of modern Bradley-Terry optimization means it can scale infinitely. Whether you are a local enterprise querying 50 local community members, or a multinational corporation aggregating the pairwise votes of 10,000 global employees, the model computes the consensus instantly without degrading in accuracy.

4. Bradley-Terry vs AHP: Complementary, Not Competing

In the world of mathematical consensus, you will often hear the Bradley-Terry model mentioned alongside the Analytic Hierarchy Process (AHP). (For a definition, see our MLE Consensus Model glossary). It is a common misconception that they are competing methodologies; in fact, they are complementary approaches to the same goal.

AHP: Intensity and Consistency

AHP, developed by Thomas Saaty, utilizes linear algebra (eigenvectors). A key feature of standard AHP is the Saaty 1-9 scale, which asks not just "Which do you prefer?" but "By how much do you prefer it?" (e.g., "A is moderately more important than B"). AHP is also famous for its Consistency Ratio (CR), which mathematically flags if a stakeholder is voting illogically (A > B, B > C, but C > A).

Bradley-Terry: Binary Simplicity

The standard Bradley-Terry model, conversely, works with simple binary wins and losses. It just asks, "A or B?" This results in a lighter cognitive load for the stakeholder, making it incredibly effective for large, diverse populations (like external community members) who might struggle with the nuance of a 1-9 intensity scale.

The Convergence

In practice, whether you use the eigenvector method of AHP or the Maximum Likelihood Estimation of Bradley-Terry, the resulting materiality rankings for ESG topics are almost always nearly identical. At ExecutESG, our engines utilize the strengths of both frameworks: using AHP logic to monitor individual respondent consistency, and MLE optimization to aggregate massive, sparse datasets into a singular consensus.

5. CSRD Compliance Angle: Satisfying ESRS 2 IRO-1

The European Sustainability Reporting Standards leave no room for ambiguity. ESRS 2 IRO-1 dictates that an undertaking must provide a description of the methodology used to identify its material impacts, risks, and opportunities.

When your methodology is "we sent out a 1-to-5 scale survey and took the average," an auditor will quickly poke holes in the validity of your data due to central tendency bias and unweighted respondent aggregation.

When your methodology is "we utilized a forced-choice pairwise comparison framework aggregated via the Bradley-Terry Maximum Likelihood Estimation model," the conversation changes entirely.

By adopting a probabilistic materiality model, you provide the auditor with:

1. Mathematical Reproducibility: The raw voting logs can be run through the formula by any statistician to produce the exact same materiality matrix.
2. Elimination of Bias: The methodology mathematically proves that structural biases (like social desirability bias) were controlled for.
3. Traceable Thresholds: The resulting quantitative, ratio-scale scores allow for clear, mathematically justifiable cut-off points for what is deemed "material" versus "immaterial."

(Note: While the rigor of the Bradley-Terry model is ideal for large undertakings subject to the full ESRS, it also provides an incredibly stable foundation for SMEs reporting under the voluntary VS (VSME) framework, ensuring their simplified reporting is built on rock-solid data).

6. Elevate Your Materiality Assessment with ExecutESG

Understanding the statistical elegance of the Bradley-Terry model is one thing; building the software to execute it is another.

Fortunately, your sustainability team doesn't need to write Python scripts or calculate log-likelihood functions. ExecutESG has built this mathematical rigor into a seamless platform.

Our proprietary engine facilitates the Pairwise Comparison process. We automatically route intelligent A/B choices to your stakeholders, handle the sparse matrix optimization in real-time using advanced MLE algorithms, and output a pristine, audit-ready Double Materiality matrix.

Ready to bulletproof your CSRD compliance? Move beyond the subjective survey and embrace statistical consensus.

[Book a demo of ExecutESG today and see the mathematics in action.]