One of the main priorities of our manifesto is Accountability. We haven't put too much focus on accountability relative to the bread and butter of any respected fantasy football website, the player projections and rankings. At least, we haven't beyond closed doors.

That certainly isn't to say we are embarrassed by the projections and rankings. Far from it! We will happily pit our projections up against any web site big or small. I know the detailed process Mike Krueger goes through developing the projections which provide a foundation for the Cheatsheet Compiler/Draft Buddy. He brings to the table years of experience projecting player performance. The numbers are realistic, well reasoned, and more times than not the seemingly overrated gems helped me win games rather than become dredges of disappointment taking up space on my roster.

Measuring projections vs. actual stats is more a factor of time and resources. Would I rather look at how our player projections panned out, or develop cool features like the Crank Score Calculator or MFL Power? The cool features win out pretty much every time.

Then it hit me. I have all of Krueger's past player projections in my database. I have all the player stats in my database. I should be able to compare the two without an extreme amount of hassle. Let's see where this takes us...

Quarterbacks

Without any filtering of the data at all, I get a table indicating Tony Romo, Vince Young and Matt Leinart as the most under-projected QB of 2006. That is, the actual stats far exceeded the pre-season projections. On the other end Chris Simms, Drew Bledsoe and Aaron Brooks are the most over-projected QB of 2006. Table - 2006 Projections: Quarterback Review No surprise with any of these as the first 3 were not opening day starters, and the last 3 were opening day starters only to lose their starting jobs.

With the QB position, while we know numerous opening day starters will not start 16 games due to injury, ineffectiveness or other reasons, it is pretty difficult to not project out most starters for 16 games. If you don't, then be ready for a barrage of email questioning your mental capacity from every team's homer fans you can imagine. Yes, even the Cardinals have homer fans. Aside: Dear Tom, I told you Kurt Warner was not going to win league MVP for 2006. (Waiting for Tom's "Leinart in '07" emails to start arriving...)

First, I think we can all agree that injury is unpredictable. Second, while we know it is inevitable a Vince Young will replace a Kerry Collins in the Titans lineup, predicting whether it will be after 2 games or 12 games or the following season is a guessing game at best. The Titans' coaches themselves don't know until they see what unfolds before their eyes.

Okay, let me filter out this data a little to just look at opening day starters to see the top hits and misses comparing 2006 preseason projections to 2006 end of season stats, at the QB position.

Under-projected:

On The Mark:

Over-projected:

Interestingly enough, Rex Grossman and Alex Smith were two QB that, based on the projections, Krueger did not expect to last the season. Both started all 16 games and Grossman in particular did rack up some nice fantasy numbers, which unfortunately was only helpful if you could possibly predict when he was going to go off.

Jon Kitna, J.P. Losman, Chad Pennington, Drew Brees and Michael Vick were the next highest to exceed expectations. Were Losman and Pennington still startable though? As the 22nd and 23rd ranked QB in points-per-game ("PPG"), I'd say not really.

In the case of Kitna, I think this shows us not to underestimate Mike Martz. Case closed, Martz is an offensive genius. Krueger definitely wasn't alone in undershooting on Brees, and Vick's 1,000+ yards on the ground vaulted him above projections. All of the players from Vick on up I've labeled as being under-projected. Their actual results exceeded the projections.

On the other end of the spectrum, we can identify Simms, Daunte Culpepper, Trent Green, Matt Hasselbeck, Jake Delhomme and Donovan McNabb as being off-base primarily due to injury. Ineffective play plagued the rest from the bottom up. I consider Brooks, Kerry Collins and Kurt Warner to be ineffective with a small asterisk. Each of their team's expectations quickly hit rock bottom, and/or the coach was trying to save his job (right Art? Denny?), so "let's see what these other guys can do" was the overriding decision making there. From McNabb down I've labeled as being over-projected, as the actual results were below the projections.

At the end of the day, I'd consider anyone from Carson Palmer to David Carr as being on the mark. Palmer was off by 515 yards and 2 TD for the season, which translates to only 32.2 yards and 0.125 TD per game. Anything within 40 fantasy points on either side represents only 2.5 fantasy points per game, which would seem to be a reasonable delta to assess a projection as being fairly accurate.

Comparison to Other Prognosticators

The difficult thing with this project is assessing if this is overall very good or bad, without comparing to other websites. I don't have that data (or time or resources) to do that analysis justice.

Fortunately, someone else did at a site called RotoSource, running an independent (and unknowing to us at the time) Fantasy Football Player Projection Competition. In this, the 2006 FF Today projections ranked #2 overall out of 17 fantasy football information services, which included both free and pay subscriber content model websites in the sample. To quote RotoSource,

FF Today didn't have the best projections in any single player category, but they were pretty good everywhere. These guys clearly did their homework, and came up with the third best QB projections, the fourth best RB projections, and the second best TE projections in our contest.

This website was especially strong in the high-scoring positions of QB and RB, coming up with the single best combined score for these two categories. They just about nailed Favre, McNabb, and Larry Johnson, among many others.

That's funny, because I included McNabb in my over-projected category, which is more assurance that I've appropriately set the boundaries of differences I consider on the mark as +/- 40 fantasy points.

Certainly this is a great result for FF Today, especially the comment, "pretty good everywhere... clearly did their homework", which would indicate care is taken making sure the projections are cross-checked for consistency and reasonable. Krueger didn't just fluke a win here with a strong result at one or two positions, but was solid across the board.

All right, maybe I should just stop my planned analysis of Krueger's projections right here. We have all the evidence we need right? Well for fun, I had another idea to assess the projections without the benefit or ability to compare to other fantasy football websites.

Players That Matter

What really matters are the projections that vary the most from "average" projections (or draft rankings) as these are the ones that will influence FF Today users the most. Of course, these will be the highest risk from Krueger's perspective because these are the players he is going out on a limb on.

Regardless, here are the QBs that had the biggest difference between their projection rank from their Average Draft Position ("ADP") near the end of the preseason. I used a +/- 5 spot cutoff. These are considered my "players that matter."

The way I see it, if a player was projected higher than his ADP, then FF Today visitors were more likely to draft that player. In assessing Krueger's projection for that player, it would be considered a good projection if he under-projected or was on the mark vs. actual stats for that player.

Conversely, if a player was projected lower than his ADP, then FF Today visitors were less likely to draft that player. Krueger's projections would be considered good if he over-projected or was on the mark vs. actual stats for that player.

With that understanding, here is the table again showing where the player landed from our projections vs. actual comparison table, and an assessment of whether that was a good or a bad thing.

We have 9 players who were projected +/- 5 spots from their ADP ranking, and 6 were assessed as good and 3 bad. Note however that Carr and Palmer are on the edge of our on the mark range.

It was a good year for QB scoring. Palmer actually finished 5th amongst QB in total scoring and 6th in PPG, so I'd say that is more than fair to assess Krueger as being on the mark with Palmer (projection rank of 7). Otherwise, you're drafting Palmer 2nd QB off the board which ended up meaning you overpaid for him.

Best Projection

The best projection Krueger made at the QB position last year was probably, surprisingly, Daunte Culpepper. His ADP was the 7th QB. Krueger projected him 17th. Basically, NO ONE using FF Today projections should have drafted Daunte Culpepper, which turned out to be a very good thing. Yes, injury factored into Culpepper's numbers, but the primary cause of Culpepper's collapse was due to insufficient recovery from an injury we knew about prior to the season. Krueger interpreted that lack of recovery time correctly, by swaying FF Today users away from Culpepper in '06.

Also, the Favre and McNair projections were very good. Favre because he was projected 10th while he was being drafted 17th amongst QB, and his projections were pretty much spot on missing out by only 5.5 fantasy points. McNair was similar but turned around the other way with a projection 7 spots lower than his ADP, and missing the projection by only 2.2 fantasy points for the season. Its uncanny he missed the total passing yards for McNair by only 20.

Worst Projection

The worst projection I'm going to give a tie to Drew Bledsoe and Jake Plummer. The average drafter respected these players as steady performers but they weren't in love with them, drafting them 11th and 14th. Krueger had them projected to finish 5th and 6th. Of course, you probably weren't hurt too badly if you were paying attention to the tea leaves early in the season and backed them up with Tony Romo and Jay Cutler off waivers, but from a pure grading standpoint these ones jump out at me.

Quarterbacks Overall Grade

Overall, there are a lot of factors to consider whether you can judge projections to ultimately be good or bad. It isn't just a simple math exercise. We cannot expect that projections are going to be within +/- a dozen or so fantasy points across the board. That isn't realistic.

Projections are estimates based on information that existed at a point in time. Comparing to actual stats over 4 months and 17 regular season games later, many things happened that can result in big differences. In most cases, these things were unforeseeable at the time of preparing the projections. There is no reason to hold that against the forecaster.

When I'm judging whether projections are generally good or bad, I consider the time and effort that a person puts into their projections. I look to see if the numbers reflect known coaching, personnel and other relevant changes. I assess if the numbers are reasonable in terms of historical norms, and cross-check for consistency such as passing stats to the receiving stats.

In the case of Mike Krueger, from year-to-year he consistently gets a strong grade from me for the quality of his projections, because I know he meets all my criteria for preparing good projections. Plus the RotoSource study backs up the quality of the projections in spades.

As for the specific projections from 2006, and FF Today's accountability of its projections, I'm going to give Krueger a QB grade of B. I'm sure many of our FF Today patrons would have happily accepted a B average in school. I know Krueger would have...