检验我们的工作
概率是否成真？
当模型说一支球队有 70% 的概率时, 该球队应当约七次赢十次。本页验证是否如此。每场 2026 世界杯比赛在结束后立即评分, 以下数字只问一个问题: 概率是否诚实, 而不仅仅是自信？（技术术语是校准。）
Track record
Proven on past tournaments
The short version: when the model says 70%, it happens about 70%. Across these 987 matches its stated chances landed within ~5.6 points of reality, and on average it rated the actual result about 35% more likely than a blind 1-in-3 guess would.
The 2026 tracker below stays empty until the first match kicks off. So to show the model has been tested, not just described, we ran it against tournaments whose results you already know. For each one, the model was rebuilt exactly as it stood the day before kickoff, then graded on every match — it never sees the result it is being marked on. That is what “graded out of sample” means: no peeking, no hindsight.
Each tournament is scored by the production model reconstructed as it stood the day before the tournament's first match: Dixon-Coles and Hierarchical Poisson refit on matches strictly beforehand, Elo rolled forward to each match, and the tournament-tier calibration layer refit on the 24 months of matches before the cutoff. No data from the tournament, or any later match, touches any layer of the fit.
Graded across 987 matches at 24 major tournaments (2014–2024)
- 0.572
- 1.000
- 5.6pp
How close the forecasts landed to reality. Lower is better; blind 1-in-3 guessing scores ≈ 0.667.
Like Brier, but overconfidence is punished harder. Lower is better; blind guessing scores ≈ 1.099.
Does “70%” really mean 70%? The average gap between the two. Lower is better.
Tournament by tournament
One row per tournament: the model rebuilt as it stood the day before it began, then graded on every match through the final (Brier — lower is better, blind 1-in-3 guessing is 0.667). A few thin, early editions sit above that line; the honest measure is all of them pooled, in the box above.
| Tournament | Host | Matches | Brier |
|---|---|---|---|
| Copa América 2024 | United States | 32 | 0.522 |
| Euro 2024 | Germany | 51 | 0.613 |
| AFCON 2024 | Ivory Coast | 52 | 0.651 |
| Asian Cup 2024 | Qatar | 51 | 0.515 |
| Gold Cup 2023 | United States | 31 | 0.566 |
| World Cup 2022 | Qatar | 64 | 0.611 |
| AFCON 2022 | Cameroon | 52 | 0.686 |
| Gold Cup 2021 | United States | 31 | 0.341 |
| Copa América 2021 | Brazil | 28 | 0.481 |
| Euro 2021 | England | 51 | 0.554 |
| AFCON 2019 | Egypt | 52 | 0.546 |
| Gold Cup 2019 | United States | 31 | 0.405 |
| Copa América 2019 | Brazil | 26 | 0.542 |
| Asian Cup 2019 | United Arab Emirates | 51 | 0.496 |
| World Cup 2018 | Russia | 64 | 0.569 |
| Gold Cup 2017 | United States | 25 | 0.456 |
| AFCON 2017 | Gabon | 32 | 0.642 |
| Euro 2016 | France | 51 | 0.668 |
| Copa América 2016 | United States | 32 | 0.502 |
| Gold Cup 2015 | United States | 26 | 0.755 |
| Copa América 2015 | Chile | 26 | 0.686 |
| AFCON 2015 | Equatorial Guinea | 32 | 0.795 |
| Asian Cup 2015 | Australia | 32 | 0.434 |
| World Cup 2014 | Brazil | 64 | 0.565 |
Reliability diagram
Read it like this: each dot is a group of similar forecasts — left-to-right is what the model predicted, bottom-to-top is how often it actually happened. When the model says 70% and that happens about 70% of the time, the dot sits on the dashed line: perfect calibration. The closer the dots hug the line, the more honest the probabilities; bigger dots mean more matches in that group.
Brier by favourite confidence
Matches grouped by how confident the model's favourite was (its biggest of the home / draw / away probabilities) — so you can see whether it is as reliable on toss-ups as on heavy favourites.
| Favourite confidence | Matches | Brier |
|---|---|---|
| P_fav < 40% | 81 | 0.649 |
| P_fav 40-60% | 476 | 0.633 |
| P_fav 60-80% | 318 | 0.512 |
| P_fav >= 80% | 112 | 0.428 |
- Out-of-sample: the calibration layer is refit per tournament on pre-tournament data, so these numbers do not reuse the live shipped calibrator (which has seen these results).
- The uniform 1/3 forecast scores a Brier of 0.667; lower is better. Major-tournament football is high-variance, so a strong model still sits well above a league-season Brier.
- Calibrated and uncalibrated metrics are reported on the same fixtures so the calibration layer's effect is visible.
Built 2026-05-30 · model 1.0.0 · calibration layer refit on the 24 months before each tournament.
2026 世界杯, 实时追踪
Live grading begins once matches are played. Until then, the scoreboard above already grades the model on past tournaments, and the full method is on the methodology page.
No scored matches yet
First scored match expected 2026-06-11. Once the first match is played, this page grades the model in real time: a running accuracy chart and day-by-day breakdown, updated after every game.
Nothing to show yet: the forecasts already exist on each match page, but no games have been played to test them against. For the record so far, the scoreboard above grades the model on past tournaments.
Reliability diagram
Not enough matches yet to draw this. None graded yet; it appears once at least 50 are in. Until then, the scoreboard above already shows this same chart for past tournaments, and the full backtest is on the methodology page.
Brier by competition
| Segment | Matches | Brier | Δ vs overall |
|---|---|---|---|
| World Cup 2026 | — | — | — |
Brier by tournament stage
| Segment | Matches | Brier | Δ vs overall |
|---|---|---|---|
| Group stage | — | — | — |
| Round of 32 | — | — | — |
| Round of 16 | — | — | — |
| Quarter-final | — | — | — |
| Semi-final | — | — | — |
| Third-place play-off | — | — | — |
| Final | — | — | — |
Brier by favourite confidence
| Segment | Matches | Brier | Δ vs overall |
|---|---|---|---|
| P_fav < 40% | — | — | — |
| P_fav 40-60% | — | — | — |
| P_fav 60-80% | — | — | — |
| P_fav >= 80% | — | — | — |
三个数字的含义
想象一个天气预报员。任何人都能说「70% 的降雨概率」。好的预报员在说出这个数字时, 确实约有 70% 的时间是正确的。这三个数字以同样的方式检验模型。
- Brier score: 概率是否接近现实？ 对每场比赛, 我们测量预测与实际结果的距离, 然后取平均值。完美预测得 0 分; 盲猜三选一约得 0.667 分。越低越好。
- Log loss: 同样的思路, 但对过度自信的惩罚更重。 如果将某事称为几乎确定却预测错误, 这个数字会急剧上升。它是保持模型谦逊的指标。盲猜约得 1.099 分。越低越好。
- ECE: 「70%」是否真的意味着 70%？ 我们收集所有标为「约 70%」的预测, 检查这些事件实际发生的频率。各置信水平之间的平均差距就是 ECE。几个百分点意味着所声明的概率可以直接参考。越低越好。
前两个指标奖励正确且大胆的预测; 最后一个是诚实度检验。一个模型可以看起来很出色, 却仍然高估了自己的信心, 同时测量三个指标才能发现这一点。
想了解底层机制, 即组件模型和这些数字背后的样本外测试？全部内容在方法论页面。