Brier score
Brier = average of (forecast − outcome)² // outcome is 1 or 0
lower is better; 0 is perfect
always forecasting 50% on binary outcomes → 0.25
- 70% and it happened: (0.7 − 1)² = 0.09.
- 20% and it did not: (0.2 − 0)² = 0.04.
- 90% and it did not: (0.9 − 0)² = 0.81.
Log loss
log loss = −average of [ y·ln(p) + (1 − y)·ln(1 − p) ] coin flip (50%) → ln 2 = 0.6931
| Forecast | Log loss if it happened | Log loss if it did not |
|---|---|---|
| 5% | 2.996 | 0.051 |
| 30% | 1.204 | 0.357 |
| 50% | 0.693 | 0.693 |
| 70% | 0.357 | 1.204 |
| 95% | 0.051 | 2.996 |
Log loss punishes confident misses much harder than Brier. A model that says 95% and is wrong pays about 3.0 on that forecast.
Why "proper" scoring rules lead
Both are proper scoring rules: a forecaster gets the best expected score by reporting their true belief. Win rate is not proper — it rewards pushing every probability toward the favourite. That is why PropBetEdge model cards lead with Brier and log loss rather than a hit rate.
Reading a real model card
Out of sample over 7,047 bouts (2013–2026): Brier 0.2363 against a coin-flip 0.2500, log loss 0.6651 against 0.6931, accuracy 59.58%. A historical win-rate baseline scores 0.2447 / 0.6826.
Read it as the card does: a modest, honest improvement over the baselines across thousands of fights — not a machine that is usually right about individual fights.
On the 2025–2026 holdout the WNBA model card reports a Brier of about 0.212 and log loss of about 0.611. It also reports that on games with a sportsbook price, the de-vigged market predicted outcomes better than the model. The card says so plainly, and its PBE Edge figure is described as disagreement with the market, not proven value.
Scores are only comparable on the same games. A Brier of 0.21 on basketball games and 0.24 on UFC fights says nothing about which model is "better" — the sports have different base rates and different irreducible randomness. Compare a model with its own baselines, and with the market on the same events.
Go deeper: PBE Fight Model card · WNBA model card