Do NFL Draft Grades Matter?

In the wake of the NFL Draft each year, one of the most common practices is to hand out grades and report cards to each team.

While these grades are fun to take in and give you a wide perspective on the perception of a team’s draft class from across the football landscape, you have to wonder; do draft grades hold any sort of correlation to future performance?

Thanks to the site Football Outsiders, which puts together a yearly draft report card that agglomerates grades from a variety of writing sources across the gridiron universe, we can easily compare how well grades have correlated to performance.

To measure player performance, the best metric we can use to quickly get an relative value of a player’s impact is Pro Football Reference’s Approximate Value. It is far from a perfect stat, but it quickly gives us an approximation of a player’s career worth.

I looked at the 2012 and 2013 drafts and compared each team’s Football Outsiders report card grade ranking (linked directly above) to their ranking in cumulative AV from that draft class (includes only value accumulated by the players for the team that drafted them). Here are the results; the teams are ranked best to worst by draft grade from left to right (black bar) while the line graph represents their class’ ranking in career Approximate Value gained for the team that drafted them. The green line is the trend line.

You can see based on the green trend lines on both charts that there is a slight positive correlation here in terms of a higher draft grade resulting in a higher ranking in draft class production. While the middle range is extremely sporadic, the teams with very good and very bad grades tended to match their perceptions relatively well.

In 2012, of the top six teams in draft grading, each ranks in the top half of the league in cumulative AV, with four in the top ten. In 2013, of the top twelve teams in draft grading, ten rank in the top half of the league in cumulative AV, with six of them in the top ten.

The teams with negative grades have also matched their grim outlooks at a fairly strong rate. In 2012, of the bottom eleven teams in draft grading, nine ranked in the bottom half of the league in cumulative AV, including each of the bottom five ranked teams in AV. The major outlier was the Seattle Seahawks, who own the 2012 draft’s most impactful class in spite of a 30th-ranked report card.

In 2013, of the bottom nine teams in draft grading, eight ranked in the bottom half of cumulative AV. The prime outlier was the Dallas Cowboys, whose 2013 class ranks 4th in cumulative AV in spite of their 32nd-ranked report card.

Clearly, as demonstrated by 2012 Seattle and 2013 Dallas, a negative report card doesn’t bury you. However, if your team graded out extremely positively or extremely negatively, there seems to be a solid chance the graders are right about your team’s draft class. If you are somewhere between about 7th/8th and 21st/22nd, then you can pretty much throw the grades out the window in terms of their predictive value.

On average, the difference between a team’s ranking in grading and true value was 7.6 spots - a difference that is nearly equal to a quarter of the league. This margin was consistent across 2012 (7.5) and 2013 (7.7).

What are the odds your team’s draft grades are pretty much in the right ballpark? Let’s say a difference of 4 spots between a team’s grading rank and value rank is a close guess. What percentage of teams’ classes have been graded that closely to their eventual true value?

Based on the two years studied, the odds of a team’s draft class being close in value (up to 4 spots difference) to what their grades predicted is about 38% - in 2012, 11 of 32 teams were within that many; in 2013, 13 of 32 teams were.

Here is a breakdown of how close draft grades tend to come:

After collecting this data, I’ve come away convinced that draft grades actually hold more value than I thought they did. I was expecting that, being the arbitrary measures of quality that they are, the grades would have a completely sporadic relationship with true value, in turn rendering them meaningless as a predictor. That doesn’t seem to be the case. There certainly seems to be a small correlation between grades and output, specifically for teams on the extreme ends of the scale.

Hopefully for the Jets, who ranked 10th in Football Outsiders’ aggregate grading this year and received an “Elite” grade from Pro Football Focus, this correlation holds true.

What are your thoughts on the value of draft rankings?

Follow Michael on Twitter! @Michael_Nania