Can We Beat the Over/Under?

We found something real: bad weather knocks points off a game's final score, even though it doesn't change who wins (that's the first post, if you missed it). So the obvious next move was to try predicting the total score before kickoff. We built a model and ran it on real games every week on the dashboard. This post used to say the model was usually off by about 12 points and that beating the sportsbooks was an open question. Both of those came from a mistake in how we tested it. Here's the honest answer, and how we caught it.

The mistake: grading with hindsight

To know how good two teams are, the model used their SP+ ratings, a well-respected measure of team strength. The catch was which SP+ we used: the end-of-season rating, calculated after every game had been played. That rating already includes the result of the very game the model was trying to predict. It's like predicting a student's midterm score from their final course grade, which already counts the midterm. Of course you'll look smart.

When the model predicts this week's games for real, it never gets that advantage, because the season isn't over. So every accuracy number we'd published was graded with information the real model doesn't have. Tested that way, it looked like it beat the sportsbooks: a 12.3-point average miss against their 12.9, and 59% of over/unders called right. That would have been an incredible result. It was too good to be true, and it wasn't true.

Average miss is in points (lower is better). Every model was trained only on seasons before the one it predicted.

The fix: only what you could know before kickoff

The model now builds its own team ratings as of each kickoff. Before the season starts, a team's rating is mostly last year's SP+. As games get played, it shifts toward how the team is actually doing this year, adjusted for who it played. And the exact same code builds the ratings the model learns from and the ratings it uses for this week's games, so the two can never drift apart again.

Then we tested it fairly: train only on seasons before a given year, predict that year, and repeat for every season from 2010 to 2025. The cheating version's advantage was biggest early in the season (12.2 versus 13.4 points in weeks 1–3), which makes sense: that's exactly when an end-of-season rating knows the most that a real forecaster doesn't.

Does it know anything the sportsbooks don't?

Maybe the model is worse overall but still catches something the sportsbooks miss? We checked by letting a simple formula blend the two numbers however worked best. It put 94% of the weight on the sportsbooks' line and basically none (−7%) on our model. In other words: our model's number adds nothing the posted total doesn't already know. Ouch, but fair.

Every ingredient, re-tested honestly

An earlier version of this post listed a bunch of ingredients that "didn't make the cut." Those tests used the same flawed ratings, so we re-ran all of them the honest way:

Every ingredient moves the model by less than a tenth of a point, while the gap to the sportsbooks is nearly half a point. Weather's effects on scoring are real (see the temperature and rain and wind research), but truly awful-weather games are rare, so they barely move a whole season's average.

One thing this doesn't change: the site's findings about what weather did to past games. Those use end-of-season SP+ to account for how good each team was, after the fact, which is exactly what hindsight is good for. The mistake only affected predicting games before they're played.

What the real predictions say so far

Every week, the model's picks get locked before kickoff and checked afterward in a weekly recap. The first full week on record, week 3, agrees with everything above: 47% of over/unders called right, with an average miss of 14.5 points against the sportsbooks' 14.0. One week is a tiny sample, but it points the same way as 16 seasons of honest testing.

So what's the model for now?

We're keeping it running on the dashboard as an ongoing experiment. Its numbers show what team strength and weather alone would suggest, which is fun to watch, but it isn't a good way to predict totals: the posted line is the better number. When the two disagree, the honest guess is that the sportsbooks know something our model doesn't, like injuries, depth charts and news, not the other way around. The "vs. Line" column just shows how far apart they are. It's not an edge, and nothing here is betting advice.

Update, September 26: a soggy, windy Virginia Tech–Boston College game made us check how the model handles real storms. It wasn't reacting enough. In games with at least half an inch of rain and 15 mph of wind, scores had been coming in about 9.6 points under the model on average. Now weather is its own adjustment that can only lower a total, which shrinks that miss to about 2.5 points without hurting accuracy anywhere else. The dashboard shows it under each prediction, like “−2.6 weather.” It still doesn't beat the posted line, though. The sportsbooks move their totals a lot for weather, too: that game's total dropped from 54.5 to 45.5 before kickoff.

Update, September 27: when a head coach moves from one FBS school to another (like Eric Morris going from North Texas to Oklahoma State), the model now starts his new team partway toward how good his old team was, instead of only looking at the new team's last season. It's a small improvement, mostly in those teams' first few games, and it still doesn't beat the posted line.

Also new: the model's forecasts now come from Open-Meteo, which keeps an archive of its past forecasts. That archive showed that big rain forecasts usually bring less rain than promised, so the model was overreacting to them. In games with a rainy forecast, it was predicting about 6 points too low. Now it translates each forecast into the weather that usually shows up, which cuts that to about 4.

We also tested whether how teams play matters, not just how good they are: whether certain offensive and defensive styles do better or worse against each other than their ratings suggest. None of the six matchup ideas we tried held up on later seasons, so the model stays as it is.

Update, October 1: turns out one matchup does matter. When two service academies (Army, Navy, Air Force) play each other, it's option offense against a defense that practices against the option every single day, and the clock just keeps running. From 2017 to 2025, those games came in 17.7 points under the model on average, and its over/under calls went 6–20. Yikes. So the model now takes points off any academy-vs-academy game, learned from its own past misses in them, and the dashboard shows it like the weather cut (“−12.2 academy matchup”). Each pairing gets its own cut, leaning on the all-academy average because each has only about 16 games of history. Navy–Air Force gets the smallest, since those games haven't run as far under as the Army ones. Tested on 2017–2025, the miss shrinks to 2.5 points and the calls go 16–10. That's only 26 games, so don't get too excited about the record. The important part is the model isn't way too high anymore.

Same day, a confession: a server hiccup on September 28 kept week 4's results out of the model until October 1, and some lower-division games had been sneaking into its ratings since week 2. Both are fixed, and this week's predictions were rebuilt.