 |
| IMDB here. |
First, there is an interesting issue of statistically significant differences. Here is a table that includes the simple percentage difference from 1 to 2, 2 to 3, 3 to 4, and so on:
| RK | TEAM | BCS Final Pts. | %Diff |
| 1 | Notre Dame | 0.9978 | 0.0569 |
| 2 | Alabama | 0.9441 | 0.0509 |
| 3 | Florida | 0.8984 | 0.0421 |
| 4 | Oregon | 0.8621 | 0.0480 |
| 5 | Kansas State | 0.8226 | 0.0707 |
| 6 | Stanford | 0.7683 | 0.0132 |
| 7 | Georgia | 0.7583 | 0.0096 |
| 8 | LSU | 0.7511 | 0.1118 |
| 9 | Texas A&M | 0.6756 | 0.0230 |
| 10 | South Carolina | 0.6604 | 0.0157 |
| 11 | Oklahoma | 0.6502 | 0.2883 |
| 12 | Florida State | 0.5047 | 0.0702 |
| 13 | Oregon State | 0.4716 | 0.0049 |
| 14 | Clemson | 0.4693 | 0.4325 |
| 15 | Northern Illinois | 0.3276 | 0.0037 |
| 16 | Nebraska | 0.3264 | 0.1365 |
| 17 | UCLA | 0.2872 | 0.1320 |
| 18 | Michigan | 0.2537 | 0.0096 |
| 19 | Boise State | 0.2513 | 0.0799 |
| 20 | Northwestern | 0.2327 | 0.2871 |
| 21 | Louisville | 0.1808 | 0.0118 |
| 22 | Utah State | 0.1787 | 0.1764 |
| 23 | Texas | 0.1519 | 0.1285 |
| 24 | San Jose State | 0.1346 | 0.7435 |
| 25 | Kent State | 0.0772 |
Since we don't know the distribution of the final points assigned under the BCS formula, a "room full of reasonable people" criterion like 10% as a significant difference might fly. I count 9 of 25 at this 10% level (we can't know about Kent State because rankings past the 25th spot aren't ever listed). By and large, comparing team-by-team down the ranking, the BCS formula had a tough time actually telling very many teams apart. Even at 5%, I count 14 of the 25 at this level (just over half).
At best, the BCS formula picked up differences at obvious, discrete intervals. FSU fell far below Oklahoma at the 12th spot, Northern Illinois fell far below Clemson at the 15th spot, Louisville fell far below Northwestern at the 21st spot, and SJSU falls off the earth at the 24th spot. The BCS formula appears to be a very coarse instrument at best.
Second, the BCS formula produces relative strength of teams in individual bowls quit at odds with betting lines. If both teams in a bowl game were ranked in the BCS 25, the ratio of the BCS Final Points is a relative strength index. For example, the Florida/Louisville ratio for the Sugar Bowl was 4.9690. The opening betting lines could also be found; for the Sugar Bowl it was Florida by 14.5 points. Here's a table:
| Bowl | Higher BCS Rank/Opponent | BCS Ratio | Opening Line |
| Sugar Bowl | Florida/Louisville | 4.9690 | 14.5 |
| Orange Bowl | Florida State/Northern Illinois | 1.5406 | 13.5 |
| Capital One | Georgia/Nebraska | 2.3232 | 8.5 |
| Fiesta Bowl | Oregon/Kanssas State | 1.0480 | 8 |
| Outback Bowl | South Carolina/Michigan | 2.6031 | 4.5 |
| Chick-fil-A | LSU/Clemson | 1.6005 | 3 |
| Cotton Bowl | Texas A&M/Oklahoma | 1.0391 | 3 |
| Alamo Bowl | Oregon State/Texas | 3.1047 | 1 |
| BCS Championship | Notre Dame/Alabama | 1.0569 | -8 |
Almost by inspection, the BCS Ratio (Higher BCS Rank/Opponent) and the opening line don't match up very well. The simple correlation between the two is a meager 0.456 and if you throw out the only real discrepancy with Notre Dame, the correlation falls to 0.350! One could argue that opening lines evolve, but they were at least simultaneous to the final BCS points announcement. I wanted to check into this for other years, but I can't find opening lines by season.
In our upcoming book (Sports Myths, Stanford University Press), Jason Winfree and I also poke other fun at the "tournament" that is to replace the BCS next year. Almost makes one yearn for the good old days when the pairings were just left up to bowl committees and polls determined the final rankings after the bowl games were played.
4 comments:
Rodney, you have incomplete BCS rankings. CBSsports actually has all of them listed:
http://www.cbssports.com/collegefootball/rankings/bcs
I am confused about your statement of not knowing the distribution of BCS points. We have information on the Harris, Coaches, and Computer polls so we can find the BCS points (and the link I provided has done all the work). So I am just not sure where an idea of a statistically significant difference between two schools would come from with BCS rankings since the BCS final points is a constant and not a random variable.
From your relative strength index comparison with opening betting lines (and knowing some results), it seems that the BCS does a poor job in predictions compared to the betting markets. Might this be evidence that the BCS is not intended to serve as predicting who the best team is but possibly some other underlying purpose?
Hi Robert.
I don't understand why you think the BCS Rankings are a set of constants? (Perhaps you'll enlighten me).
Of interest is the underlying distribution of playing strength, a random variable. And as far as I know, the characteristics of this distribution are unknown (e.g., mean, variance, any higher moments). The BCS rankings, calculated from the BCS chosen formula, are an estimate of this underlying distribution. Since we don't know the characteristics of the underlying distribution, we can't use the sample statistics for comparisons in the usual way (choose a significance level and test differences, for example). Hence my "room full of reasonable people" criterion. You are, of course, free to reject it as reasonable (that's why I also offered the similar observation for a 5% difference).
I agree (and I think you are spot on) that the BCS Rankings do worse than the opening betting line. While it could just be that the BCS formula is a lousy estimate, it could also be that the motives of the BCS are being revealed by this weak correspondence--since you are interested in sports economics, it should be particularly glaring that Notre Dame is slightly favored by the BCS rankings but more than a full touchdown light according to the opening line.
Thanks so much for the CBS listing. I'll be able to compare more bowl match ups!
[Please say hi to Wally Thurman for me? We were graduate students together at Montana State for a brief time. I see that Rick Stroup is on your faculty at NC State as well (adjunct). Please tell him I said hi, too--he was one of my profs at Montana State years ago.]
Thanks for the reply Rodney! I certainly agree with you that this national championship game is a good example that the BCS may have motives other than predicting outcomes. With 15 years of data on BCS rankings there should be enough data to find a relationship. The big issue seems to be finding a way to quantify the variables that explain the motivations for the BCS.
My initial thought on the distribution had to do with BCS standing throughout the season. Your explanation of using BCS rankings as an estimate for playing strength clears up my confusion and I see what you mean by not being able to accurately capture some measure of statistical significance.
Next time I see the Montana State people around campus I will let them know you said hi.
Post a Comment