The definition
A model is calibrated if, across many forecasts, events it calls X% happen about X% of the time. Group forecasts into buckets (50–55%, 55–60%…), then compare each bucket's average forecast with how often those events actually happened.
Why calibration beats "accuracy"
A model that picks the favourite in every matchup can post a respectable win rate while its probabilities are useless. Calibration asks the question that matters when you compare a model with a price: can I use this number as a probability?
| Metric | Question it answers |
|---|---|
| Accuracy | Did the side above 50% win? |
| Calibration (ECE, slope) | Do stated probabilities match observed frequencies? |
| Brier / log loss | Both at once — see Brier & log loss |
Calibration on PropBetEdge model cards
PropBetEdge model cards report calibration alongside scoring rules. The PBE Fight Model card publishes an expected calibration error and a calibration slope for its out-of-sample period; a slope near 1.0 means probabilities are neither too confident nor too timid. The WNBA model card publishes a slope for its holdout seasons as well.
Calibration needs volume. With 14 graded calls, a calibration chart is mostly noise, which is why PropBetEdge surfaces show calibration only after a minimum number of graded picks. A calibrated model can still be uninformative — always predicting the base rate is perfectly calibrated and useless.