Back in 2004 Mike Krueger and I published an article called FF
Today: A New Look And Attitude. This article represents
a key milestone towards developing FF Today into what it is today.
A review of the article allows for a nice assessment of whether
FF Today is accomplishing what it set out to do 3 years ago.
2006 Projections Review: Quarterbacks
|QBs | RBs
One of the main priorities of our manifesto is Accountability.
We haven't put too much focus on accountability relative to the
bread and butter of any respected fantasy football website, the
player projections and rankings. At least, we haven't beyond closed
That certainly isn't to say we are embarrassed by the projections
and rankings. Far from it! We will happily pit our projections
up against any web site big or small. I know the detailed process
Mike Krueger goes through developing the projections which provide
a foundation for the Cheatsheet
Compiler/Draft Buddy. He brings to the table years of experience
projecting player performance. The numbers are realistic, well
reasoned, and more times than not the seemingly overrated gems
helped me win games rather than become dredges of disappointment
taking up space on my roster.
Measuring projections vs. actual stats is more a factor of time
and resources. Would I rather look at how our player projections
panned out, or develop cool features like the Crank
Score Calculator or MFL
Power? The cool features win out pretty much every time.
Then it hit me. I have all of Krueger's past player projections
in my database. I have all the player stats in my database. I
should be able to compare the two without an extreme amount of
hassle. Let's see where this takes us...
Without any filtering of the data at all, I get a table indicating
Tony Romo, Vince Young and Matt Leinart as the most under-projected
QB of 2006. That is, the actual stats far exceeded the pre-season
projections. On the other end Chris Simms, Drew Bledsoe and Aaron
Brooks are the most over-projected QB of 2006. Table
- 2006 Projections: Quarterback Review No surprise with any
of these as the first 3 were not opening day starters, and the
last 3 were opening day starters only to lose their starting jobs.
With the QB position, while we know numerous opening day starters
will not start 16 games due to injury, ineffectiveness or other
reasons, it is pretty difficult to not project out most starters
for 16 games. If you don't, then be ready for a barrage of email
questioning your mental capacity from every team's homer fans
you can imagine. Yes, even the Cardinals have homer fans. Aside:
Dear Tom, I told you Kurt Warner was not going to win league MVP
for 2006. (Waiting for Tom's "Leinart in '07" emails
to start arriving...)
First, I think we can all agree that injury is unpredictable.
Second, while we know it is inevitable a Vince Young will replace
a Kerry Collins in the Titans lineup, predicting whether it will
be after 2 games or 12 games or the following season is a guessing
game at best. The Titans' coaches themselves don't know until
they see what unfolds before their eyes.
Okay, let me filter out this data a little to just look at opening
day starters to see the top hits and misses comparing 2006 preseason
projections to 2006 end of season stats, at the QB position.
||On The Mark:
Projections Review: Quarterbacks - Opening Day Starters
|| FF Pts
|| FF Pts
|| FF Pts
Interestingly enough, Rex Grossman and Alex Smith were two QB
that, based on the projections, Krueger did not expect to last
the season. Both started all 16 games and Grossman in particular
did rack up some nice fantasy numbers, which unfortunately was
only helpful if you could possibly predict when he was going to
Jon Kitna, J.P. Losman, Chad Pennington, Drew Brees and Michael
Vick were the next highest to exceed expectations. Were Losman
and Pennington still startable though? As the
22nd and 23rd ranked QB in points-per-game ("PPG"),
I'd say not really.
In the case of Kitna, I think this shows us not to underestimate
Mike Martz. Case closed, Martz is an offensive genius. Krueger
definitely wasn't alone in undershooting on Brees, and Vick's
1,000+ yards on the ground vaulted him above projections. All
of the players from Vick on up I've labeled as being under-projected.
Their actual results exceeded the projections.
On the other end of the spectrum, we can identify Simms, Daunte
Culpepper, Trent Green, Matt Hasselbeck, Jake Delhomme and Donovan
McNabb as being off-base primarily due to injury. Ineffective
play plagued the rest from the bottom up. I consider Brooks, Kerry
Collins and Kurt Warner to be ineffective with a small asterisk.
Each of their team's expectations quickly hit rock bottom, and/or
the coach was trying to save his job (right Art? Denny?), so "let's
see what these other guys can do" was the overriding decision
making there. From McNabb down I've labeled as being over-projected,
as the actual results were below the projections.
At the end of the day, I'd consider anyone from Carson Palmer
to David Carr as being on the mark. Palmer was off by 515 yards
and 2 TD for the season, which translates to only 32.2 yards and
0.125 TD per game. Anything within 40 fantasy points on either
side represents only 2.5 fantasy points per game, which would
seem to be a reasonable delta to assess a projection as being
Comparison to Other Prognosticators
The difficult thing with this project is assessing if this is
overall very good or bad, without comparing to other websites.
I don't have that data (or time or resources) to do that analysis
Fortunately, someone else did at a site called RotoSource, running
an independent (and unknowing to us at the time) Fantasy
Football Player Projection Competition. In this, the 2006
FF Today projections ranked #2 overall out of 17 fantasy football
information services, which included both free and pay subscriber
content model websites in the sample. To quote RotoSource,
FF Today didn't have the best projections in any single
player category, but they were pretty good everywhere. These
guys clearly did their homework, and came up with the third
best QB projections, the fourth best RB projections, and the
second best TE projections in our contest.
This website was especially strong in the high-scoring
positions of QB and RB, coming up with the single best combined
score for these two categories. They just about nailed Favre,
McNabb, and Larry Johnson, among many others.
That's funny, because I included McNabb in my over-projected
category, which is more assurance that I've appropriately set
the boundaries of differences I consider on the mark as +/- 40
Certainly this is a great result for FF Today, especially the
comment, "pretty good everywhere... clearly did their homework",
which would indicate care is taken making sure the projections
are cross-checked for consistency and reasonable. Krueger didn't
just fluke a win here with a strong result at one or two positions,
but was solid across the board.
All right, maybe I should just stop my planned analysis of Krueger's
projections right here. We have all the evidence we need right?
Well for fun, I had another idea to assess the projections without
the benefit or ability to compare to other fantasy football websites.
Players That Matter
What really matters are the projections that vary the most from
"average" projections (or draft rankings) as these are
the ones that will influence FF Today users the most. Of course,
these will be the highest risk from Krueger's perspective because
these are the players he is going out on a limb on.
Regardless, here are the QBs that had the biggest difference
between their projection rank from their Average Draft Position
("ADP") near the end of the preseason. I used a +/-
5 spot cutoff. These are considered my "players that matter."
The way I see it, if a player was projected higher than his ADP,
then FF Today visitors were more likely to draft that player.
In assessing Krueger's projection for that player, it would be
considered a good projection if he under-projected or was on the
mark vs. actual stats for that player.
Conversely, if a player was projected lower than his ADP, then
FF Today visitors were less likely to draft that player. Krueger's
projections would be considered good if he over-projected or was
on the mark vs. actual stats for that player.
With that understanding, here is the table again showing where
the player landed from our projections vs. actual comparison table,
and an assessment of whether that was a good or a bad thing.
||Proj. vs. Actual
||on the mark
||on the mark
||on the mark
||on the mark
We have 9 players who were projected +/- 5 spots from their ADP
ranking, and 6 were assessed as good and 3 bad. Note however that
Carr and Palmer are on the edge of our on the mark range.
It was a good year for QB scoring. Palmer actually finished 5th
amongst QB in total scoring and 6th in PPG, so I'd say that is
more than fair to assess Krueger as being on the mark with Palmer
(projection rank of 7). Otherwise, you're drafting Palmer 2nd
QB off the board which ended up meaning you overpaid for him.
The best projection Krueger made at the QB position last year
was probably, surprisingly, Daunte
Culpepper. His ADP was the 7th QB. Krueger projected him 17th.
Basically, NO ONE using FF Today projections should have drafted
Daunte Culpepper, which turned out to be a very good thing. Yes,
injury factored into Culpepper's numbers, but the primary cause
of Culpepper's collapse was due to insufficient recovery from
an injury we knew about prior to the season. Krueger interpreted
that lack of recovery time correctly, by swaying FF Today users
away from Culpepper in '06.
Also, the Favre and McNair projections were very good. Favre
because he was projected 10th while he was being drafted 17th
amongst QB, and his projections were pretty much spot on missing
out by only 5.5 fantasy points. McNair was similar but turned
around the other way with a projection 7 spots lower than his
ADP, and missing the projection by only 2.2 fantasy points for
the season. Its uncanny he missed the total passing yards for
McNair by only 20.
The worst projection I'm going to give a tie to Drew
Bledsoe and Jake
Plummer. The average drafter respected these players as steady
performers but they weren't in love with them, drafting them 11th
and 14th. Krueger had them projected to finish 5th and 6th. Of
course, you probably weren't hurt too badly if you were paying
attention to the tea leaves early in the season and backed them
up with Tony Romo and Jay Cutler off waivers, but from a pure
grading standpoint these ones jump out at me.
Quarterbacks Overall Grade
Overall, there are a lot of factors to consider whether you can
judge projections to ultimately be good or bad. It isn't just
a simple math exercise. We cannot expect that projections are
going to be within +/- a dozen or so fantasy points across the
board. That isn't realistic.
Projections are estimates based on information that existed at
a point in time. Comparing to actual stats over 4 months and 17
regular season games later, many things happened that can result
in big differences. In most cases, these things were unforeseeable
at the time of preparing the projections. There is no reason to
hold that against the forecaster.
When I'm judging whether projections are generally good or bad,
I consider the time and effort that a person puts into their projections.
I look to see if the numbers reflect known coaching, personnel
and other relevant changes. I assess if the numbers are reasonable
in terms of historical norms, and cross-check for consistency
such as passing stats to the receiving stats.
In the case of Mike Krueger, from year-to-year he consistently
gets a strong grade from me for the quality of his projections,
because I know he meets all my criteria for preparing good projections.
Plus the RotoSource study backs up the quality of the projections
As for the specific projections from 2006, and FF Today's accountability
of its projections, I'm going to give Krueger a QB grade of B.
I'm sure many of our FF Today patrons would have happily accepted
a B average in school. I know Krueger would have...
Next Up: RBs