Ever since David Romer's paper "Do Firms Maximize?", a paper that examines the fourth down decision of NFL coaches, there seems to be an acceptance that the expected points approach is the appropriate way to examine virtually all such situations. However, there may be situations where it is difficult to have a lot of confidence in the expected points approach. The situation in the Seattle - Washington game might be an example. Many commentators suggested that a key factor in the decision was the injury to the Washington kicker earlier in the game. How much data was available on the potential success of any replacement kicker (or an injured kicker)? How many attempts did they have in NFL games? How many from this distance? How many in the closing minutes of the fourth quarter? How many with the team 8 ahead? The number of observations on which to base any decision is likely to be very small.
That is not to say that a small amount of data on a kicker of a football does not add value. Goalkeepers in the English Premier League like to keep data on a small number of penalty kicks by opposing players. On the left hand side of the picture is the water bottle of one goalkeeper with data on West Ham United penalty kickers. There is only a small number of kicks for each kicker but there does seem to be an element of predictability. It is not exactly "big data" but it might be helpful. We might call it "small analytics".
It is possible that Romer's paper changed the convention. Brian Goff clearly suggests that Bill Belichick's play-calling was influenced by his knowledge of the paper (Goff has a chapter in Sports Economics Uncut called "Bill Belichick as economist"). Would the results be replicated with data from 20 years after the original paper?
Most would object to using Romer's data to guide play-calling in 2021. It is clearly from a different era. Different rules, different players. But players are constantly changing. How does Bill Belichick or Josh McDaniels use the data to guide decisions for the rookie quarterback? They may be willing to pay the expected points price in some situations in order to develop Mac Jones. Maybe the Washington Head Coach, (Riverboat) Ron Rivera, was not just betting on the current situation. Maybe he was gathering data on the ability of his quarterback to convert. Or he may have been investing in the development of his quarterback by providing him with the experience (passing up the chance to give any alternative kicker an opportunity). Even if the analytics did not support Rivera's decision, we should not automatically assume he was incorrect. I support Romer's approach but not to the exclusion of all other possibilities.