Concocting a draft value chart based on actual draft pick trade history

Note from the editor: This is a guest post from the great EricT (@BSH_EricT), from Broad Street Hockey.

Dating back to Jimmy Johnson’s original draft pick value chart, a lot of people have looked at the performance of players selected to try to estimate the true value of any given pick. Today at broadstreethockey.com, I took a different angle: instead, I looked at what trades had been made to try to estimate the market value of each pick.

This approach won’t tell us whether teams consistently undervalue or overvalue high draft picks, but it gives a sense of what teams will actually have to pay if they want to move up in the draft.

I had a worry about the NFL data that I didn’t have for the NHL. The recent change in rookie compensation could result in a change in how draft picks are valued, which could mean that collecting several years of data gives us the wrong answer. So to see if things changed, I started off by building a model using the trades made from 2008-2011 and checked to see if it matched the 2012 results.

I wasn’t surprised to see it was a little bit off, but I was surprised by the direction. If we build a model looking at the trades made from 2008-2011, we come to the conclusion that in 2012, 20 of the 22 teams that traded up underpaid.

The common assumption is that the limitations on rookie salaries should increase the value of the top picks, but we actually saw teams paying less to move up than they did in past years. In one particularly clear example, in 2011 Kansas City got picks #27 and #70 in return for pick #21, but in 2012 New England trading away pick #21 was only able to get picks #27 and #93.

Of course, not all drafts are the same. It seems hard to believe that restricting the pay for top draft picks made them less valuable, so the most likely explanation is that scouts were telling their GMs that the drop-off in talent was not particularly steep last year, making trading down more attractive.

Either way, while a systematic skew is a bit of a concern, the differences weren’t very big at all. The model gave nearly identical answers regardless of which years I used, so I went with the full five years of data.

The result does quite well overall. 75 out of 98 trades in the last five years come within 10% of our market estimation, and 94 out of 98 come within 20%. Given the differences between draft classes and individual team needs, this seems very reasonable. So I am content to use the five-year data for now and expect minor revisions as we get more information about how teams value picks under the new CBA.

Without further ado, here is the table estimating market value for draft picks based on 2008-2012 trades.

NFL draft chart 2

Some final notes on the data:

I’ve used the 98 trades over this time period which met two criteria: a) the trade involved only draft picks, and b) all of the draft picks were in the current draft. Excluding trades which involve players is necessary because including them would require having a good estimate of the players’ value in comparison to the draft picks. Excluding trades which involve future years’ picks allows us to escape the question of how likely a team’s second round pick is to be #33 or #64.

Those 98 trades don’t include very many trades of the top 5 picks, so the sharp increase at the top of the draft is an extrapolation that probably isn’t very trustworthy — and even if it were, the value of picks that high is probably quite dependent on which specific players available in a given draft. Still, the chart seems to be a reasonable model overall and with the possible exception of the very highest picks, is probably a pretty decent indication of what teams will have to pay to move up.

Draft value chart

12 Comments

  1. buy mens suits…

    wonderful web website. tons of useful info right here. i’m sending it to some pals ans also sharing in tasty. and of course, thanks in your work!…

  2. Michael says:

    Quoting Broad Street Hockey? Come on Jimmy. They continuously fail in their attempt to apply advanced statistics to hockey. There are too many variables in hockey to quantify, and all of the stats that they look at have alternative explanations and potential post hoc ergo propter hoc problems. Not to mention, the writers lack knowledge of NHL system play and knowledge basic hockey fundamentals that go into making player evaluations. Weak sauce.

    1. Eric T. says:

      A few responses come to mind.

      1) What does any of what you say have to do with the work here? Are there too many variables in hockey to quantify draft pick value? Is there a post hoc ergo propter hoc problem in assuming that when teams trade pick #9 + pick #40 for pick #7, it means they think pick #9 + pick #40 has roughly the same value as pick #7?

      2) Its too bad NHL teams aren’t as savvy as you about the value of advanced stats in hockey. Multiple teams have asked for my input and made roster and lineup decisions based on the information I provided. It’s also too bad the MIT Sloan Sports Analytics Conference organizers aren’t as savvy about advanced stats as you; they invited me to present this year. I hate to brag about credentials, but since you’ve provided absolutely no specifics to debate, no examples of where I might have gone wrong, I’m not sure how else to address your ethereal claims.

      3) Thanks for reading!

    2. jeff says:

      YOU JUST GOT KNOCKED THE FUCK OUT!

  3. Null…

    你是怎么想的呢?…

  4. NYG_Slater says:

    pick 20-100 value is pretty consistent, this make sense.

    I’d argue that the first 10-20 picks in the 1st round are tied to the players on the board and not a random chart.

    EX. the 2nd pick in 2012 was/is worth more than the 2013 2nd pick because RG3 was available in 2012. The value of the pick is tied directly to the perceived value of the player, and players in that tier. If there is a large difference in perceived value between pick 2 and 3, trading for pick 2 will cost a premium based on the chart.

    On the flip side, If there is a perceived minor difference in talent between picks 1 through 20 the values of the picks go down and trade for a discount.

    1. NYG_Slater says:

      On a macro sense, we are seeing discounts compared to Johnson’s chart ( atleast for the first 20 picks). This could be explained within the post above. If the college game has gotten more competitive in the past 20 years and the quality of athletes across the board has improved, the perceived value differences between individual players may decrease (in general).

      1. Eric T. says:

        Yup, it could be that the player pool has gotten deeper, or it could be that modern systems limit the impact a single elite player can have, or it could be cap financials, or it could just be that Johnson’s chart was based on a pretty crude measure of value (Pro Bowl selections) and they’ve refined their understanding.

        But for whatever reason, it does look like they’re not putting as much weight on those top 20 picks nowadays.

  5. Eric T. says:

    I guess I should comment on the differences between this and the Jimmy Johnson chart, which seemed to be taken as gospel 20 years ago. It seems pretty clear that teams are valuing first round picks less now than they did then.

    Part of this may be due to the salary cap (which leaves teams with constraints on just trying to get the very best players they can at all times), but part of it may just be that teams have refined their understanding of how likely those players are to become great players as various things about the game have changed.

    1. Still, looking at that graph, it’s pretty close, except at the highest portion.

      Ultimately, when you’re trading up in that range, you’re trying to grab a specific player that you may be willing to “overpay” for.

      1. Eric T. says:

        Yup, and that very highest portion is the part I’m least confident in. I think the upshot is “teams still put roughly the JJ value on picks, but with a little less weight on top-20 picks than he had.”

  6. I like it. Would love to see how it compares proportionately with the old “Jimmy Johnson” chart.

%d bloggers like this: