NY Jets: Playoff QBs Revisited

By now perhaps you've had enough of analyzing the clutch factor in playoff quarterbacks.  But it's a long offseason, and if there ever is a time to indulge a bit of analytical overkill, perhaps now is the time.

We started this conversation by looking at whether or not Tom Brady deserved his reputation for being an ultra clutch quarterback.  Then Jeremy raised an interesting question as to whether or not almost all quarterbacks had worse passer ratings in the playoffs than in the regular season.  From there we put together a list of the top 20 quarterbacks in terms of all time playoff passing attempts, and we saw that in fact ten of the twenty raised their passer ratings in the playoffs, and ten lowered their passer ratings in the playoffs.  The average decline in playoff passer rating was an insignificant 1.2%.  Here is how that chart looked:

But something kept nagging at me.  It didn't make sense that quarterbacks had virtually identical passer ratings in the playoffs. Defenses were tougher; weather conditions were less favorable; why wasn't a decline in passer ratings showing up in the numbers?  Then JetFanOverseas raised some interesting questions as to whether the chart should be ordered in terms of percentage increase in the playoffs rather than absolute points increase.  I ended up coming around to his point of view in thinking perhaps points increase made more sense.  And as often happens, in the process of re-examining my views, I discovered a tangential "eureka!" moment while pouring over Kurt Warner's numbers.  In short, I think perhaps I was comparing apples to oranges in the data set, and that was the reason the expected decline in playoff numbers failed to materialize.

Let me explain.  In using career passer ratings as the comparison number, I was lumping in all those years quarterbacks failed to make the playoffs with the years they made the playoffs, then comparing them to passer ratings in the playoffs which, by definition, were only playoff years.  By doing so I inadvertently lowered the career comparison number, making it appear that passer ratings held steady in the playoffs, when in fact they did not.  An easy way to conceptualize this is that when quarterbacks have bad years, in general they don't go to the playoffs.  Those bad years lower the quarterbacks' career passer ratings, but are not factored into the quarterbacks' career playoff passer ratings, which only take into account the years the quarterback was playing great and getting into the playoffs.  This oversimplifies things; quarterbacks can have great years and miss the playoffs, and they can have bad years and make the playoffs. But in general playoff years encompass higher regular season passer ratings than non playoff years, and that makes the career passer rating a misleadingly low number for comparison.

Perhaps a simplified example is in order.  Suppose Aaron Rodgers had only two years in his career, to keep things simple.  Suppose one year he had a nagging arm injury, and a porous offensive line due to injuries, and had his best receiver miss most of the year. All this misfortune leads to a poor year for Rodgers, say a 90 passer rating, and the Packers just miss the playoffs.  Now say in the other year Rodgers is healthy, as is his line and his best receiver, and Rodgers has a typical Rodgers year, say a 110 passer rating.  In this good year the Packers go to the playoffs, and Rodgers posts a 100 passer rating in the playoffs.  Now, if we compare that playoff passer rating to his 2 year career average of 100, it looks like Rodgers' playoff performance matches his regular season numbers.  But that is perhaps a misleading comparison.  His playoff performance came in his good year, when things went right for him in many ways.  The more accurate comparison is to his regular season numbers in the playoff year, not to the composite number including much easier comparisons from the year things went wrong, factors that were not present in his playoff numbers.  So if we just compare apples to apples, so to speak, Rodgers actually declined 10 points in his playoff performance compared to his regular season numbers. That's obviously a gross oversimplification of a more complex dynamic, but it perhaps serves to illustrate why the better comparison is to the playoff years numbers, not the aggregate career numbers.

So, putting all this together, we probably need a chart that ranks quarterbacks by their point increases, and compares career passer rating to the aggregate regular season passer rating for only those years the quarterbacks made the playoffs.  Here's that chart.  You may notice a few things that look more "right" about it.  Joe Montana rises in the ratings, as does Joe Flacco, two quarterbacks who most would agree raised their games in the playoffs.  Peyton Manning falls considerably further behind Tom Brady, something that more closely fits the popular narrative regarding their respective clutch factors.  And perhaps most gratifyingly from my standpoint, the mystery of the missing playoffs performance decline is solved.  When measured apples to apples, 13 of the 20 quarterbacks saw a decline in their playoff numbers compared to their regular season numbers, and playoff performances fell by an average of 4.1 points, or almost 5%, which seems much more likely to reflect the reality of poor January weather and good playoff defenses.

Thanks for indulging me while I tugged at this thread.  The questions and thoughts presented in the comments were really thought provoking and led to an interesting place, and a chart which perhaps better reflects the reality of playoff performances.  Here's the revised chart: