There are flaws to mathematical ranking systems, for sure. I am of the tribe that believes margin of victory is an essential component to any computer-based system, but the BCS doesn’t agree. However, I’ve mentioned that here before, so I won’t again. I’m here today to talk about flaws in the human polls using a real example and a nearly-real example. How can an example be nearly real, you might ask. A fair question. Read on!
Stanford (#17) is down 7 points in overtime on the road in South Bend. The Cardinal drives to the 1. Two attempts—on second and fourth down—are deemed to be just short, but video evidence shows them to be so close that really a touchdown could be (should be?) awarded. With that touchdown, Stanford will tie the game, or perhaps go for the win.
I’m not going to argue that it was a touchdown, but I think it’s fair to say that any other set of referees were just as likely to call it a touchdown as not. So let’s freeze the play at the moment Stepfan Taylor is about to score/not score. While this play is frozen, let’s ask the nation’s AP voters to put out the next week’s poll. How many of them are going to move Notre Dame (#7) up to #5 for being in dire straits at home against a lower-ranked opponent? A few, but only because of losses ahead of them. No ounce of logic says that Notre Dame is a better team now than before.
A better question, though, is this: how many of those AP voters would lower Stanford’s rank at all. Remember, we’ve frozen the game at a point where it looks like perhaps Taylor has scored. Are those AP voters going to say to themselves, “wow, the Cardinal has the #7 team in the country on the ropes—on the road—let’s lower their ranking”? The answer should be that no one is going to make that argument. In fact, most voters would probably rank Stanford higher if forced to vote during the freeze.
Yet after the referees came out and awarded the Fighting Irish the win, sure enough Notre Dame went up to #5, and Stanford dropped to #20. Nothing new happened after I froze the game for these AP voters except an arbitrary decision by the referees.
Voters that only rank teams higher when they win, and lower when they lose, aren’t considering how little would have had to happen for the other team to win. Had Stanford been awarded a touchdown in that instance—which many of those AP voters surely would have supported during that little freeze—instead of going from #17 to #20, Stanford would actually have had a 50%-ish chance to win. And with that potential win, the Cardinal probably would have moved up to the single-digit ranks. Notre Dame, on the other hand, would have had that 50%-ish chance to lose and fall from #7 to maybe as low as #15 for a loss to a lesser opponent at home. The voters allowed literally an inch, or perhaps a bad angle for the referee, to change both Notre Dame’s and Stanford’s rankings by as many as 10 positions. That is grossly illogical.
How about that nearly-real scenario. In the South Carolina-LSU game, SC had a final drive opportunity, down two points to LSU. The Gamecocks didn’t get into field goal range, and ended up throwing an interception. But let’s just say they for example’s sake that they got to within 45 yards, and their kicker went out for a chance to win the game. Let’s freeze the game for AP voters again.
If SC makes the field goal, they stay #3, and LSU (#9) likely drops to maybe behind ND’s hypothetical spot—so like #16 or #17. If SC misses and loses, they would drop presumably to #9 (to where they actually dropped after losing), and LSU would rise to #6 (to where they actually rose). If I again asked the AP to vote during this freeze, none would be very comfortable moving either team up or down more than one spot.
Ok, so you’re someone who thinks a kicker’s clutch ability should determine a team’s ranking by 6 positions–the difference between #3 and #9? Fair enough. But what about from LSU’s perspective. You wouldn’t honestly believe that a team is different by as many as 10 or 11 rank spots because of their inability to block one kick, would you? No, of course not.
Here, I’ll ask any one of those voters to watch every college special teams try to block kicks, and the ones that do block kicks (in their one chance to do so) will move up 10 slots. That would mean that like 5% of all teams move up 10 slots, and 95% of all teams move down 10 slots. That is obviously absurd and impossible, but it is effectively what the voters would say in that situation with their votes.
The tiniest fractions of inches, or the result of one play—a kick in my semi-hypothetical game—changed teams’ rankings by double-digits because, all of a sudden, the first 59.9 minutes of the game don’t matter anymore. I find that very illogical.
The fact is that no system is perfect for rating teams in a 12-game season. But the advantage a mathematical formula has (especially if it uses margin of victory) is that it doesn’t find itself affected by the “recency bias.” As humans we put far more weight on recent events, and thus clutch events, and we tend to forget what led to the clutch situation. This is what we see on a weekly basis in the human’s version of the college football polls, and it has always bothered me.