7/20/09
So your fantasy football league has picked out of a hat, drawn straws,
or done something a bit more adventurous– and the draft order
is set. You’re not a big fan of your draft spot, and you’re
thinking of making a trade offer. Or someone has made you an offer
that is interesting. How do you evaluate the value of picks in a
trade?
We start with a quick overview of existing widely available tools.
This overview raises some questions, and we respond by building
a model to address them. We analyze some specific cases and conclude
with a generalized analysis of the value of each draft spot under
relevant sets of assumptions.
Overview of Existing Tools
There are a few tables / calculators out there to help evaluate
draft picks. I found two main types. The first type is based on
the draft pick value
chart NFL teams use to value draft positions. The second type
is a black-box calculator that allows you to input one or more draft
picks and compare the value to other draft picks.
The NFL chart is useful for getting some general perspective on
how the value of draft positions goes down as picks get later. That
said, a quick look at some specific picks on the NFL chart will
make it clear that it’s not well-suited for a fantasy calculation.
Details are outside the scope of this article, but the key point
is that an NFL team is interested in the marginal value a player
adds to their existing team. As a fantasy owner, you don’t
have an existing team, and you are more interested in the marginal
value a player will add over a waiver wire pick. Applying the NFL
chart directly to fantasy football tends to overvalue the number
1 pick, and undervalue pairs and triples of lower picks in a fantasy
context.
The black-box style calculators seem to account for the nuances
of fantasy better. For example, they have a better sense of the
relative value a number 1 pick has compared to lower pairs (and
triples) in a fantasy context. That said, what assumptions are embedded
in these? Do these tools assume that you know the future at the
time of the draft, and you pick the best available player at each
position based on the future performance in the season?
I decided to look into this some more, and build my own model to:
1) Help me better understand what some other tools are doing and
2) Potentially improve upon them.
Baseline Assumptions For this
initial set of analysis, the following assumptions are held constant:
- 12-team PPR league, with 15 rounds
- Focus on rounds 1, 2, 3, and 4
- Assume the following distribution of picks:
Draft Pick |
RB |
QB |
WR |
TE |
1 |
100.0% |
0.0% |
0.0% |
0.0% |
2 |
100.0% |
0.0% |
0.0% |
0.0% |
3 |
100.0% |
0.0% |
0.0% |
0.0% |
4 |
100.0% |
0.0% |
0.0% |
0.0% |
5 |
100.0% |
0.0% |
0.0% |
0.0% |
6-12 |
75.0% |
12.5% |
12.5% |
0.0% |
Round 2 |
50.0% |
25.0% |
25.0% |
0.0% |
Round 3 |
33.3% |
58.3% |
8.3% |
0.0% |
Round 4 |
33.3% |
50.0% |
8.3% |
8.3% |
|
In other words we assume the first 5 picks are always RB and
no TE is picked before round 4. We assume round 2 will on average
have 6 RB’s, 3 QB’s, and 3 WR’s picked but that
will vary by scenario of the simulation, etc.
Our conclusions are not unique to 12-team PPR leagues, and would
not change a whole lot for many changes to these baseline assumptions.
We stick with these as a starting point for simplicity.
Building And Back-Testing The Model
Our initial approach assumes each manager, after choosing the
position they will draft, drafts the best available player at
that position. In other words, we assume we are in a 2008 draft,
and each manager has magical powers to be able to predict exactly
how every player will perform in advance – and they draft
the best performing player for 2008 at the start of their draft.
We call this our “Optimal” results case. Note under
this set of assumptions Tom Brady would go undrafted in 2008.
We define the value of a player based on their total points scored
in excess of the last drafted player for that position, assuming
48 RB’s, 24 QB’s, 60 WR’s, and 12 TE’s
will be drafted. In other words, we focus on their value in excess
of a waiver wire pick. And then we run 1000 simulations of drafts,
letting the choice of position vary for each draft – and
we look at the average value of each pick over all the simulations.
Let’s start by looking at the value of the #1 pick, compared
to some lower picks. The following chart includes two points from
our results.
|
Value |
Compare To #1 Pick |
#1 pick |
205 |
|
Team 12 Round 2 and 3 pick (#13
and #36 overall) |
221 |
8% |
|
Interestingly, the relative value of these picks (along with many
other combos I tried) corresponds pretty closely with black-box
tools I tried. This tells us two things. The first thing is that
it would not be shocking if the corresponding black box tools tend
to assume that each manager is drafting the next best player (i.e.
drafts optimally knowing the future). The second and more important
observation is that our simulation back-tests to existing black-box
formulas under a specific set of assumptions. This helps establish
relevance of the tool.
But what about this assumption that managers magically pick the
player that will perform best during the season at each position
at each pick. Obviously that’s not the case in reality –
but does it matter? For example, could introducing that unknown
component swing a favorable trade to an unfavorable trade? The answer
is yes, and it has important implications on pre-draft trade evaluations.
We explore in more detail below.
So What If I’m Not That Great At Telling
The Future?
Welcome to the club. Let’s introduce a random variable built
for us non-omnipotent folks into our model which represents the
quality of draft choice. In other words, we no longer assume that
a manager draft choice will be whoever is going to be the best performing
player at the end of the season of those available. We instead assume
that although there’s a chance you’ll pick perfectly,
there’s also a chance your pick will underperform a bit, underperform
a lot or even end up completely worthless (e.g. Tom Brady in 2008).
We model this idea assuming that a player will either be the best
available, perform a half-round worse, a full round worse, three
rounds worse, or be worthless. There’s a lot of ways to think
of it, but hey, that’s one starting point.
Here are some numbers that help elaborate on how we do this –
where we quantify the idea of underperforming in terms of number
of picks.
|
RB |
WR |
QB |
TE |
Optimal |
0 |
0 |
0 |
0 |
Slightly underperform |
5 |
5 |
3 |
2 |
Moderately Underperform |
10 |
10 |
6 |
4 |
Severely Underperform |
20 |
20 |
12 |
8 |
Bust |
NA |
NA |
NA |
NA |
|
In other words, if a team picks a WR and their pick is categorized
as “slightly underperform” – then instead of
picking the best WR available, they pick the WR that will end
up being the 5th best available. If their TE pick is categorized
as “severely underperforms”, then instead of picking
the best TE available, they pick the 8th best TE available. And
if their pick busts, then their pick ends up being worthless.
This captures the idea of non-optimal draft choices in a simple
transparent way that is easy to model.
In that context we assign a distribution of the likelihood of
a pick being optimal to busting and we let that vary by round.
We reflect an assumption that the first 3 picks are more reliable
than the rest of the first round, and that each round thereafter
involves less reliability (and more likelihood of the pick to
underperform). With this in mind, we add to our “Optimal”
assumptions 3 more sets of assumptions. We call them Optimistic,
Realistic, and Pessimistic – in descending order of reliability
of picks. And we re-run our simulations.
Let’s go back to our original example, and look at how our
initial comparisons look now.
|
Pessimistic |
Realistic |
Optimistic |
Optimal |
#1 pick |
117 |
127 |
163 |
205 |
Team 12 Round 2 and 3 pick (#13
and #36 overall) |
90 |
119 |
144 |
221 |
|
-23% |
-6% |
-12% |
8% |
|
That’s pretty interesting. Under our set of baseline assumptions
– under the optimal case (remember, this involves everybody
drafting perfectly knowing the future), the #13 and #36 pick come
out a little more valuable than the #1 pick. But as we relax the
optimal draft behavior, the #1 tends to look more attractive.
In fact, when we’re pessimistic about our drafting ability,
the #1 pick is a lot more valuable!
In other words, from the perspective of a #1 pick, introducing
the lack of knowledge of the future has turned an attractive trade
into a stupid one.
Here’s another way to think of this. Let’s say you
have the #1 pick, and you have a large enough sample of fantasy
seasons to conclude you tend to make draft picks that look pretty
smart at the end of the season pretty consistently. Are you’re
not a big fan of any of the top RB’s available this year
and you’re thinking of trading down. With that in mind,
you might consider a trade like #1 for #13 and #36. But let’s
say your picks don’t tend to pan out that well at the end
of season - then you probably shouldn’t consider this particular
trade, and you should just keep your #1 pick.
A Quick Note On Handcuffing (No We’re
Not Calling The Cops)
We now have some background on the relative value of some draft
positions, and we have some perspective on how that varies depending
on our view of the likelihood of managers picking well. It’s
worth noting that handcuffing is directly related to this discussion.
If you handcuff your top players, you can think of this as a move
to an increasingly optimistic assumption in the context of our
discussion. In other words, if you handcuff your pick, you hedge
against some of the downside exposure of a bust, and you are less
likely to drastically underperform expectation with your picks
(e.g. if you had Matt Cassell in 2008 when Brady went down –
your pick did not bust, you ended up with a pretty good QB. If
you didn’t have Matt Cassell, your pick was officially a
bust).
If you don’t plan to handcuff your top players, then you
should probably tend to assume less optimistic assumptions.
OK – I’ve Gotten This Far –
Let’s See Some More Numbers In A Chart I Can Use
Below is a full chart of the results of my simulations, under
the baseline set of assumptions for the Optimal, Optimistic, Realistic,
and Pessimistic draft performance assumptions. I normalize all
the values so the highest pair of picks has value 100.
Draft pick |
Pessimistic |
Draft pick |
Realistic |
Draft pick |
Optimistic |
Draft pick |
Optimal |
1 |
100 |
1 |
100 |
1 |
100 |
1 |
100 |
2 |
99 |
2 |
98 |
2 |
96 |
2 |
90 |
3 |
97 |
3 |
96 |
3 |
93 |
3 |
89 |
4 |
86 |
4 |
86 |
4 |
86 |
4 |
81 |
5 |
84 |
5 |
84 |
5 |
84 |
6 |
80 |
6 |
79 |
6 |
80 |
6 |
79 |
7 |
79 |
7 |
79 |
7 |
79 |
7 |
79 |
8 |
78 |
8 |
79 |
8 |
79 |
8 |
78 |
5 |
78 |
9 |
78 |
9 |
78 |
9 |
76 |
9 |
78 |
10 |
77 |
10 |
76 |
10 |
75 |
10 |
76 |
11 |
76 |
11 |
75 |
11 |
74 |
11 |
75 |
12 |
75 |
12 |
75 |
12 |
73 |
13 |
72 |
13 |
49 |
13 |
56 |
13 |
59 |
12 |
72 |
14 |
48 |
14 |
55 |
14 |
58 |
14 |
71 |
15 |
48 |
15 |
55 |
15 |
58 |
15 |
69 |
16 |
47 |
16 |
54 |
16 |
57 |
16 |
67 |
17 |
46 |
17 |
54 |
17 |
57 |
17 |
65 |
18 |
46 |
18 |
53 |
18 |
56 |
18 |
65 |
22 |
46 |
21 |
53 |
19 |
55 |
19 |
64 |
21 |
46 |
19 |
53 |
20 |
54 |
20 |
62 |
20 |
45 |
22 |
53 |
21 |
54 |
21 |
62 |
19 |
45 |
20 |
53 |
22 |
54 |
22 |
61 |
23 |
45 |
23 |
52 |
23 |
53 |
23 |
60 |
24 |
45 |
24 |
52 |
24 |
52 |
24 |
58 |
25 |
32 |
25 |
42 |
25 |
32 |
25 |
56 |
26 |
31 |
26 |
41 |
26 |
32 |
26 |
54 |
27 |
31 |
27 |
41 |
27 |
32 |
27 |
51 |
28 |
30 |
28 |
41 |
28 |
32 |
28 |
50 |
29 |
30 |
29 |
40 |
29 |
31 |
29 |
48 |
30 |
30 |
30 |
40 |
30 |
31 |
30 |
46 |
33 |
29 |
31 |
40 |
32 |
31 |
31 |
43 |
32 |
29 |
32 |
39 |
33 |
31 |
32 |
41 |
31 |
29 |
33 |
39 |
31 |
30 |
33 |
39 |
34 |
29 |
34 |
39 |
34 |
30 |
34 |
38 |
35 |
29 |
35 |
39 |
35 |
30 |
37 |
38 |
36 |
29 |
36 |
39 |
36 |
29 |
35 |
37 |
39 |
24 |
38 |
35 |
37 |
29 |
38 |
36 |
40 |
24 |
39 |
34 |
38 |
29 |
39 |
36 |
38 |
24 |
40 |
34 |
39 |
29 |
36 |
35 |
37 |
24 |
41 |
34 |
40 |
28 |
40 |
35 |
41 |
24 |
42 |
34 |
42 |
28 |
41 |
34 |
42 |
23 |
43 |
33 |
41 |
28 |
42 |
33 |
43 |
23 |
44 |
33 |
43 |
28 |
43 |
32 |
44 |
23 |
45 |
33 |
44 |
27 |
44 |
31 |
48 |
23 |
46 |
32 |
45 |
27 |
45 |
30 |
45 |
23 |
47 |
32 |
46 |
26 |
46 |
29 |
47 |
23 |
48 |
32 |
47 |
26 |
47 |
29 |
46 |
23 |
37 |
31 |
48 |
26 |
48 |
28 |
|
Some Comments:
- Note that the value of picks is pretty much decreasing but not
strictly decreasing. As an example note in the pessimistic case
draft pick 42-48 have normalized values around 23 but they don’t
come out exactly in the order 42-48. This should be interpreted
as these picks being similar in value – not that pick 48
is more valuable than 46.
- Note that under the pessimistic assumptions, beyond the first
round there is not a lot of difference between the values of picks
within a round. This is in stark contrast to the illustrative
optimal assumptions, where the first pick of a round is clearly
worth more than later picks in the same round. The other assumptions
fall somewhere in between.
- Note the drop-off from the 1 pick to the 2 pick is less and
less significant, the more pessimistic you are about the ability
to predict the best player with certainty.
- Note that a number 1 pick is worth less than a #12 and #13 pick
in all cases. Under optimal assumptions, 2 second round picks
are worth more than a #1 pick, but as pessimism increases 2 second
round picks tend to be worth less – eventually falling below
the value of a #1 pick in the pessimistic case.
Next Steps
Next week we will investigate the impact of a “mistake”
in your fantasy draft in a similar framework. If you have any
questions or comments, please feel free to email
me, feedback is welcomed. See you next week –
|