Bradley-Terry: Maximum Likelihood Ratings on an Elo Scale

Elote now has a Bradley-Terry competitor, which makes it the second global-fit rating system in the library alongside Colley. I’ve compared rating systems before, but Bradley-Terry deserves its own post, because it’s the one that makes the two families of rating system legible.

The model

Bradley-Terry assigns each competitor a positive latent strength and says the probability that i beats j is its share of the pair’s total strength:

P(i beats j) = p_i / (p_i + p_j)

Reparameterize with p_i = exp(beta_i) and that becomes

P(i beats j) = sigmoid(beta_i - beta_j)

which is, up to a constant, the same functional form as Elo’s expected score. That’s a genuinely interesting fact. Elo is a logistic model of pairwise comparison, and so is Bradley-Terry. They differ entirely in how the parameters are obtained.

Elo nudges: each result moves both ratings a fixed step scaled by surprise. Bradley-Terry fits: given the whole set of observed comparisons, find the strengths that maximize the likelihood of having observed exactly those results.

The fit

The likelihood is concave in the betas, so there’s a unique maximum and any reasonable ascent method finds it. The standard approach is the MM (minorize-maximize) iteration from Hunter (2004), which for Bradley-Terry has a pleasant closed-form update: each competitor’s new strength is its win count divided by a sum over opponents of games played divided by combined strength. Iterate to convergence, then normalize by the geometric mean to pin down the overall scale, since the likelihood only determines strengths up to a common factor.

elote runs that fit after each result, over the affected connected component, matching the pattern the Colley competitor already used.

Four choices that made it usable

The model is textbook. The design decisions around it are where the work was.

Report on an Elo scale

Bradley-Terry’s natural output is a log-strength centered wherever the normalization puts it. Those numbers are meaningless next to a 1500-based Elo rating, which is a real problem in a library whose whole point is comparing systems.

So elote reports

rating = 1500 + (400 / ln 10) * beta

That constant is not arbitrary. Elo’s expected score is 1 / (1 + 10^(-(r_i - r_j)/400)), and substituting the transform above turns it into sigmoid(beta_i - beta_j), which is precisely the Bradley-Terry probability. In other words, after this transform, expected_score is numerically identical between the two systems. A Bradley-Terry rating of 1700 means the same thing as an Elo rating of 1700, and ratings from both are directly comparable on a leaderboard.

This is my favorite kind of design decision: it costs one line and it converts an incomparable number into a comparable one.

Regularize, because the MLE runs off to infinity

The raw MLE has an existence problem. If a competitor has won every game within its component, the likelihood increases without bound as its strength goes to infinity, and there is no finite maximizer. Same in reverse for a competitor that has lost every game. Both cases are extremely common early on, when everyone has played one match.

elote adds a light phantom-opponent term: a virtual win and loss against an opponent of unit strength, weighted at 0.1. That makes the likelihood strictly concave with a finite maximum everywhere, so an undefeated newcomer gets a high finite rating instead of inf or a divide-by-zero. The weight is small enough to be irrelevant once real games accumulate.

This trick is worth knowing generally. Whenever you fit a model whose MLE diverges on separable data, a weak prior is usually the cheapest fix, and logistic regression’s L2 penalty is the same move.

Don’t warm-start

The instinct with an iterative fit inside a loop is to seed from the previous solution. I tried it and removed it.

The Bradley-Terry log-likelihood is concave with a unique maximum, so warm-starting cannot improve the answer, only the iteration count. And it costs something real: the previous solution was computed under a different normalization with one fewer game, so it can be a worse starting point than a flat initialization, and it makes the fit path history-dependent, which means two identical datasets fed in different orders can converge to slightly different numbers. For a concave problem, a flat seed each time is both simpler and more reproducible.

The general lesson: warm-starting is an optimization for expensive non-convex fits. On a concave problem with a cheap iteration it buys nothing and costs determinism.

Ties are half a win each

Bradley-Terry has no native draw concept. The standard extension (Davidson’s) adds a tie parameter, which is more model than this library needs. elote counts a draw as half a win for each side, which is the conventional simplification and behaves sensibly: two evenly matched competitors drawing repeatedly stay level.

Using it

It drops into the existing arena, since the competitor interface is unchanged:

from elote import LambdaArena
from elote.competitors import BradleyTerryCompetitor

arena = LambdaArena(compare_func, base_competitor=BradleyTerryCompetitor)
arena.tournament(matchups)
print(arena.leaderboard())

No new dependencies, no arena changes.

When to prefer a global fit

The reason to have both families available is that they answer slightly different questions.

Incremental systems (Elo, Glicko, TrueSkill) are online by construction. They handle a never-ending stream, they respond to recent form, and they cost nothing per update. Their weakness is order dependence: the same games in a different order give different ratings, and early results have outsized permanent influence.

Global fits (Bradley-Terry, Colley) are order-independent by construction. Feed the same set of results in any sequence and get the same answer, which makes them much better for retrospective questions like “who was actually the strongest team last season.” The costs are that a fit is more expensive than a nudge, disconnected pools have no relative meaning at all, and the model has no notion of form, since a win three years ago counts the same as one last week.

My rule of thumb: if you’re ranking a completed season, fit globally. If you’re maintaining a ladder that never ends, nudge incrementally. And if you want to know how much your ladder’s ordering owes to the order games happened to arrive in, run both and compare, which is a diagnostic elote now makes a three-line experiment.