Should you grade on inputs or outputs? Interceptions and interceptable passes as an example

Newton’s third law of motion states “Every action has an equal and opposite reaction.” In much the same way, perhaps this is why the push back to analytics has coincided with its more regular use and discussion.

One such example of that is “interceptable passes.” The intent of interceptable passes is to account for the degree to which the quarterback put the ball at risk. The common refrain is something along the lines of “Why would I treat a ball that wasn’t intercepted as worse than one that was?”

To answer that question, because sometimes a QB does everything wrong and the defender simply drops it.

On that play Buffalo Bills quarterback Josh Allen does everything in his power to throw an interception to C.J. Mosley. The only thing he didn’t do is catch the ball for Mosley, which Mosley then failed to do. Does that mean that this play should be treated as a run of the mill incomplete pass simply because Mosley didn’t complete the play?

By comparison, on other plays, an interception happens even when the QB has done nothing wrong.

On this play Kansas City Chiefs quarterback Patrick Mahomes did nothing wrong. He hits his receiver perfectly in the hands, in stride, and leads him into the open field. The receiver simply missed it, and the deflection went straight into the defender’s hands, so it ends up as an interception on Mahomes’ box score. If this happens to Mahomes 10 times then is he suddenly an interception prone quarterback, or a quarterback that is falling victim to poor pass catcher play?

Regardless of those questions, if we use interceptions then we would grade Allen’s play as better than Mahomes’ play, even though we all know that Allen’s performance on his play was far worse than Mahomes’ on his. Interceptable passes is intended to bridge that gap such that Allen gets the demerit (an increase in his interceptable passes by 1) while Mahomes does not (his number of interceptable passes would not increase).

If we really think about what this does, it is moving the focus from player outputs to player inputs. Of note, this is exactly what the literature would suggest, because it has greater predictive validity long term.

Behaviorism puts forth that via valuing behaviors (which are in the control of the focal actor) we can better account for long-term outcomes (which are subject to random chance to some degree). This is because outcomes are going to be explained by both behavior and random chance, with behaviors being more consistent over time than is random chance. Using this framework is how many were able to predict that Ryan Fitzpatrick’s 2016 season would likely see an uptick in interceptions per pass after he had an interceptable pass every 18.7 passes on average but an actual interception every 37.4 passes in 2015.

Thinking about this through another lens, pretend two first time gamblers step up to identical slot machines, put the same amount of money in, and then pull their respective lever. In this example, assume Gambler A hits the jackpot while Gambler B does not. Would we proclaim Gambler A to be a gambling genius that is far superior to Gambler B, or would we just call Gambler A lucky? For me, I would tend towards the latter. In line with that, interceptions would focus on the jackpot outcome and say Gambler A was good while Gambler B was not. By comparison, interceptable passes would say that each would be equally likely to hit the jackpot moving forward if their behaviors remained unchanged.

With that said, interceptable passes is not a perfect statistic either. It requires a subjective component to determine what qualifies as “interceptable,” and that may vary based on the scorer. However, the focus on what the player can control rather than what the outcome achieved (that is a result of many factors beyond that of the player) is something worth considering.

As I wrote this article, I used interceptions and interceptable passes as the vessel within which to discuss the input/output divide. However, this focus on behavior over outcome is something that applies to not only interceptions but also other advanced analytics such as completion percentage over expected, any adjusted statistics, any of the next gen data that allows for player tracking, and many others.