Research note

Calibrating predictions differently for friendlies vs tournaments

Status: Shipped (Variant 4 — per-tier Platt temperature scaling). Production calibrator uses a hybrid strategy: Platt for the tournament tier (where isotonic collapses to identity at n~70), isotonic for friendlies/qualifiers (where it's more expressive at n~400+). Gate passed. See results belowTopline + full note · 3,433 words

Topline

The shipping ensemble calibrator (scripts/fit_ensemble_calibrator.py) fits per-class isotonic regression curves on the uniform-averaged three-component output (Elo bracket MC + Dixon-Coles + Hierarchical Poisson MAP). The first cut lifted holdout ECE on the 365-day common-subset training pool from 4.62pp uncalibrated → 2.70pp under the pooled-across-tiers fit (5-fold CV, n_train = 939, current artefact at data/wc2026/ensemble_calibrator.json).

A subsequent tier-aware refit (three sets…

Full note

Standard Pass

Read the full research note

Calibrating predictions differently for friendlies vs tournaments runs 3,433 words. The Standard Pass unlocks every research note in full, plus the complete forecast and per-team and per-player ratings — valid through the tournament.

Get the Pass — $15

24h self-service refund·No subscription, no auto-renewal·Access through 31 Dec 2026. See refund policy.