Individual: Stats | Heisman | Fantasy    Team: Rank | Rank2 | Summary | Picks | Pick All | Champs    Conf: Rank | Standings | VS. | [?]
Showing posts with label ratings. Show all posts
Showing posts with label ratings. Show all posts

Sunday, July 20, 2008

Big 12 South, 2007(8 P)review

I think its important, before we get lost in the hype about certain teams this year, to look back at what these same teams did with their talent last year because, on average, more than 75% of the team is the same as last year. I'm using two measures that I developed to measure performance and reputation over the course of a season. Today, I'm focusing on the Big XII South.

My big questions for the Big 12:
1) Is Tech ready for a breakout year?
2) Who will win the Red River Shootout?
3) Is Oklahoma State on the way up or down?
4) Who will win the Big 12 South?


1) Texas Tech spent most of the season down in the pack in the Big 12 South. After losing the shootout to Oklahoma State, Leach asked for more from the defense and, generally it delivered. With Graham and Crabtree putting up ridiculous numbers, Tech became a real force and arguably one of the better teams in the country. If Leach (the best offensive mind in college football?) can change the status quo in Lubbock on the defensive side of the ball, Tech can realize some pretty high expectations.



2) OU started the season in 2007, behind the ridiculous start of Bradford, in high form which it never again realized. If Mr. Melton is right and Sam is regressing toward a mean, OU could be in for a mediocre season. On the other hand, while UT finally showed some signs of life against Arizona State after a whipping by rival A&M, Colt is not getting any better and super recruiter Mack Brown has not been incredibly successful without the incredible hulk under center. For now, I'll go with the Sooners.


3) (Performance is the solid line and Reputation the dotted line). Oklahoma State? Over-Rated! Oklahoma State never achieved that high of a level last season despite touting "the best offense ever." I doubt expect anything more this next season. With money and flash, they are a popular dark horse, but popular opinion last year and this year continues to rank them in the middle of the pack where they belong. Don't get lost in the hype.

4) Who will win it--dare I say Texas Tech? No, I daren't. Tech will be very successful this season and win a lot of games, but I think Leach needs at least one more year to instill an attitude of expectations that produces the consistent play necessary to win a conference half-championship. This year, like every year, the Big 12 South is OU's to lose.

Tuesday, December 4, 2007

Historic Rankings: 2003 - 2006

2006 Ratings
2005 Ratings
2004 Ratings
2003 Ratings

With the season beginning to wrap up, I decided it was time to start adding historic polls to the site. These polls use the same methodology that I have developed this season, but, due to a lack of data, I cannot (yet) do the potential, offensive and defensive ratings that I post for the current season. I have added a strength of schedule rating and a hybrid ranking (a slightly altered version of the combined rating). The Elo rating is the win/loss rating, but Elo just makes for a shorter name and pays homage to the fellow that developed the basic methodology that I have borrowed.

I have finished preparing results for 2003 to 2006. The results are similar to the BCS and AP final rankings. Initially, it looks like the Matrix gives less credence to bowl game outcomes than the human polls and more credit to mid-majors than the computer polls. The Matrix even had the audacity to rank Utah #1 in the performance poll for 2004 - I watched that team live several times and would have given them even odds against anyone in the country outside of the Coliseum. In fact, 3 teams were rated over 50 in the performance rating in the 4 year period - Texas 2005, Utah 2004 and USC 2004.

I have done my best to identify errors, but let me know if you find any. I am indebted to James Howell for making this data available.

Footnotes: I stumbled on this little chart - the final coach's ballot - while perusing through The Wizard of Odds. Obviously, coaches tried to sneak their team in the championship game. Beamer puts the Hokies at #2, with LSU at #1 (he couldn't put his team ahead of LSU after the beat down they gave them). And Richt put Georgia at #2 - I'd like to hear his explanation of how Georgia is more deserving than their conference champions (LSU).

But Bob Stoops wins the "I did my best to manipulate the polls and get my team in the championship game" award. First, he had the guts to put his two-loss Sooners at #1 - no surprise there. And Ohio State was #2 in the Stoops poll. Where did LSU fall? #6! No other coach ranked LSU lower. Georgia came in 8th, again among the lowest in the country. It looks to me like Stoops identified and dropped his toughest competition - to no avail.

Sunday, November 11, 2007

A Methodology of the Matrix

I've described aspects of the Matrix as it has evolved, but I think its about time that I give it one coherent description for anyone interested.

The Matrix uses three ratings- a general performance rating based on margin of victory (which is used for rankings), a recent performance rating, and a win/loss rating. The general rating and win/loss rating are calculated with a progressive adjustment model derived from the Elo chess rating system. Ratings are adjusted according to the improbability that a given outcome would occur. The model simulates the season a few hundred times, allowing smaller adjustments with each round, until, through automated trial and error, it arrives at the ratings associated with the least improbability.

For both the general performance rating and win/loss rating, the model assumes that a team's performance will vary and the probability of a particular performance level will fall somewhere on the normal curve. The ratings, therefore, theoretically represent the mean. The larger the point margin, the less effect an additional point will have on ratings, so the effect of "running up the score" is minimal. When estimating the improbability of an event, the model barely differentiates an 18 point win and a 40 point win.

The win/loss rating, obviously, uses only wins and losses and ignore the margin of victory. The factor actually has very little effect on the outcome of model, but I have included it for the sake of comprehensiveness. For the most part, close games really are primarily by luck, and so it is best that the model does not overemphasize the winning of the game. Because the model uses a marginal progressive adjustment method, it is able to handle undefeated teams without the problems faced by MLE approaches.

After the general and win/loss ratings have been calculated, a recent performance ratings is calculated using the deviation of a teams margin of victory from the expected margin of victory. Obviously, greater weight is given to more recent games.

The final component of the Matrix are the Navy adjustment factors. Essentially, these factors compare a team's opponent against past opponent in terms of its relative dependency on the pass and run and then adjusts the expected outcome to match any advantages or disadvantages a team may experience in match-ups. For example, if a team has plays terrible pass defense and now has to play Texas Tech, it should be expected to under-perform relative to its general, recent and win/loss performance ratings.

The general performance rating, win/loss rating, recent performance rating, and Navy adjustment factors are then weighted and used to estimate the margin of victory (along with an adjustment for home field advantage). Finally, I use a consistency rating (how predictable a team's performance has been) to estimate the probability of a suggested outcome (of a team winning or covering the spread).

Results:

These results are only relevant for the results before week 11, 2007.

Top 5 overall:
1. Ohio State
2. Oregon
3. West Virginia
4. LSU
5. Missouri

(Note: After the OSU lost and WVU struggled against Louisville, Oregon has taken the top spot and Oklahoma and Kansas have moved into the top 5)

Oklahoma fans might see a problem that Missouri is ranked higher than their own Sooners. This is a good example, though, where the model has punished Oklahoma more for the greater improbability of their loss to Colorado. Because both teams have only one loss and Missouri loss to a better team than Colorado (who just happens to be Oklahoma), Missouri is ranked higher. Oklahoma is 6th and only 2/10's of a point behind the Tigers.

Top 5 Win/Loss
1. Ohio State
2. Kansas
3. Hawaii
4. LSU
4. Oklahoma
4. Arizona State

Obviously, a win/loss rating should give extra kudos to undefeated teams. The three-way tie for 4th is a bit of an anomaly, but here the Sooners have the advantage over Missouri.

Top 5 Consistency
1. Kansas
2. Florida International
3. Utah State
4. Arizona State
5. Ohio State

Two types of teams find themselves among the most consistent. The surprisingly successful teams that just seem to win every week and the really, really bad teams that will always play poorly against D1A competition. I thought it was interesting that Kansas has been the most consistent team this season and they are 9-0 against the spread this year.

The five most unpredictable teams -
1. UCLA
2. Utah
3. Central Michigan
4. Iowa State
5. UNLV

Fitting.

Navy adjustment factor:

You can't produce a ranking from the adjustment factor, but we can guess which teams are going to have a tough match-up this weekend. The team most likely to get unusually lit up through the air this week was, coincidentally, Navy who gave up almost 500 passing yards and 62 points in a winning effort against the 1-7 (now 1-8) Mean Green of North Texas.

Recent Performance:

Again, it doesn't make much sense to rank teams on their recent performances, because it is relative to their general performance, but the hottest team going into this weekend was Iowa State (relative to their performance all season). Unfortunately for Boston College, another very hot team is Clemson - and a cold team is, well, BC.

When dealing with all these factors, I think it is important to consider their relative importance. The Matrix has the power to explain about 65% of the variance of point margins for games involving D1A teams this season. About 61% is explained by the general performance rating alone and the other 4% by the other adjustment factors and ratings. The win/loss rating barely makes an appearance, and is really just included so the model can be comprehensive and "hybrid," which is such a popular term is sports rating these days.

The model is still somewhat fluid as I make minor adjustments to deal with problems as they arise, but these are the general principles on which it is based. I will continue to publish rankings and predictions, and I will add other stats - consistency, recent performance, match-up warnings, unexpected results, etc.

P.S. according to the Matrix, the most unlikely outcome involving two D1A teams was Notre Dame over UNLV and #2 was UNLV over Utah.

Tuesday, November 6, 2007

Week 11 Rankings

Week 11 Rankings

#1?
Ohio State.

You can't really argue with that, can you? And if the Matrix picked the participants for the national championship game today we would see the Buckeyes against the Ducks. And, in case you're wondering, it would give Oregon a 60% chance of winning.

SEC fans might gripe about this, but SEC teams don't win big enough. LSU really shouldn't need a brilliant combination of luck and guts to beat Alabama in a late comeback. Oregon didn't need late heroics to beat Arizona State, who also has a pretty good team. And hanging your hat on the Florida-Ohio State game here won't do you any good. USC put as solid a whipping on Michigan as Florida did on Ohio State.

Some things to keep in mind when you look at the rankings. First, it is based on margin of victory, not wins and losses, but with a very rapid diminishing returns for large margins of victory. In other words, it sees a 1 point win as little better than a one point loss, but a one point loss versus a 1 point win can change a teams ranking about as much as a 15 point win instead of a 35 point win.

I've set it side by side with a few other important and sophisticated polls. The scores under Matrix represent the teams rating and the number on the far left side is the ranking according to the Matrix. The mean is a number I've pulled from masseyratings.com which compares over a hundred rankings. I'm a big fan of this compilation and of the Massey ratings.

There are a few kinks with the html that I need to work out, but that will have to wait until another day.

Click here to see the rankings

Monday, October 29, 2007

Why Some Teams are Good, Part 2 - The Importance of Population

Obviously, a team has a better chance of landing a recruit if he lives nearby (or, in the case of Joe McKnight, they might be wishing they had stayed closer to home). In this blog I provide some evidence to support a claim I made in part 1 that increasing population increased the talent pool and, therefore, led to better football teams.

I picked 8 states more or less at random. I tried to include states from a variety of regions, with a variety of sizes and that have experienced a variety of population trends. I have included both Nebraska and Oklahoma, and, honestly, I don't know why.

Ratings come from Soren Sorenson, who you will find listed in the Statistics Hall of Fame. I have added 5000 to all scores so that they are all positive (Sorenson's system ranges from -4000 to +4000, +or- a thousand). Population data is drawn from the Census. Census data is collected every ten years and I have used my own estimates to fill in the gaps.

I have looked at states as a whole, adding together the ratings of all teams in that state, because teams in the same state recruit for players in the same talent pool. For now, I am ignoring population growth in the region (e.g. Georgia benefits from population growth in Florida), and characteristics of the population (e.g. old people in Arizona don't play football), but some day I will look at those issues in more detail.

So, first we begin in 1950.

The 8 states are Nebraska and Oklahoma, which I already mentioned, Florida, Arizona, New York, Indiana, North Carolina and Alabama. The graphic on the left shows the teams as they were ranked in 1950, color coded by state. Florida State, UCF, USF, and Buffalo did not have D 1 programs at the time (or, in some cases, did not have a football team, or just started admitting boys to the school).

This chart is important because, from here on, I will be focusing on indexed values for the state, so that indexed value will always reference back to this starting point. For example, 1950 was a good year for Oklahoma and Army (perhaps the two best teams in the country). This will be important to keep in mind.

This next chart demonstrates an important principle as well. This compares the percent of the total points held by a state (with their scores added together) of all the points available against the percent of the US population in that state. So, New York, despite Army's success, was under-performing. Anyone who has been to a high school football game in Dallas and in Rochester knows why this is happening. It shouldn't surprise anyone that Oklahoma performed the best giving their population size. Alabama was facing a unique challenge in segregation. It would be another 20 years before Sam Bam Cunningham would convince Bear Bryant to integrate, allowing Alabama to dip much deeper in its talent pool.

The population of most states would grew over the next 50 years, but some grew much faster than others. Florida and Arizona are good examples of states that blew up in terms of population, while New York stagnated.

In the following charts, I present data for each state in terms of their performance and their population over the 5 decades from 1950 to 2000. The black line is the team's performance. It is a running four year average which I use under the assumption that players from a cohort will play for a team for four years. The red line is the indexed population, where 100 is equal to the population in 1950. The blue line is based on the same principle, but represents the percent of the US population represented by that state, so that if a team's population is growing slower, but slower than the entire US, the blue line will fall but the red line will rise. The red line, therefore, represents the real talent pool and the blue line the relative talent pool, and because teams are good relative to each other, we should focus on the blue line. (You can click the charts to see a bigger version.)


Nebraska had some kicking teams in the 70's and the mid to late 90's, which shows up in their chart. The population as a percent of the US population was actually going down, but Nebraska kept spitting out world class teams. It makes me think that Osborne may have been a much better coach than we give him credit for. Arizona's performance isn't improving with its rapidly growing population. I think two things are at issue. First, Arizona doesn't have as strong of a football culture as the rest of the South and, second, Arizona's programs might be experiencing a bit of a lag.

I was a little surprised to see how well Indiana fits the pattern. Notre Dame has a unique advantage to recruit nationally and should be able to overcome general demographic shifts. Notre Dame claims their challenges are rooted in high academic standards, so I guess I'll have to look at that claim another day.

Alabama has been generally outplaying its population since the 50's but, like all the others, its performance is generally falling with the decline in its relative population size. The effect of integration on performance is still a little unclear, but something I will definitely look at more closely in the future.

But the overall results from this little experiment are clear--population trends in a region definitely effect the performance of that regions teams. The black lines tend to go where ever the blue lines are going. It also shows that we can't ignore culture, quality of coaches and the power of programs to attract players from long distances.