By John Considine
There is a recent paper in the Journal of Business Economics that further illustrates the benefits of technology in understanding referee bias. I will focus almost entirely on the methods used by Ulrike Holder, Thomas Ehrmann, and Arne Konig. (I will return to other elements of the paper in a later post.)
A decade ago, Tobias Moskowitz and Jon Wertheim published Scorecasting. It included the analysis of 1.15 million baseball pitches that umpires called as either balls or strikes. Technology allowed Moskowitz and Wertheim to identify umpire error. The error rate was 14%. However, that error rate changed systematically with some non-rule factors. These systematic changes are classed as a bias. For example, the player behind in the count got more favourable calls. Technology allowed Moskowitz and Wertheim to identify umpire errors directly. The Holder et al (2021) paper is a step in this direction for soccer.
The beauty of the Journal of Business Economics paper is that it uses VAR technology to focus on referee error directly. It uses four successive seasons of data (two pre-VAR and two post-VAR) from the German Bundesliga and Italian Serie A. Impressively, the authors also "validated all VAR decisions by examining video recordings via youtube.de of the respective situations". I want to highlight one of their findings. Home teams were awarded more penalty kicks than away teams but the "introduction of VAR did not change this distribution, and the VAR system intervened with comparable frequency for both the home and away teams". In other words, there was no referee bias in the errors.
For a variety of reasons, not all of the literature seeks to identify the referee bias directly. Much of the literature on referee bias attempts to identify the bias by purely statistical methods. The biggest difficult is separating referee behaviour from player behaviour. One prominent paper from 2012 compares the rate at which the referee decides the ball must be turned over (“mainly traveling violations and offensive fouls”) with the rate of turnover by players (“mainly bad passes and lost balls”). Tens of thousands of observations are then used to establish the statistical significance of the relationship between these turnovers and other variables, e.g. home team. The authors of this paper understand the limits of this approach.
It is encouraging that more papers like Holder, Ehrmann and Konig are starting to appear in the literature. Technology is facilitating a better approach to understanding referee bias.
A decade ago, Tobias Moskowitz and Jon Wertheim published Scorecasting. It included the analysis of 1.15 million baseball pitches that umpires called as either balls or strikes. Technology allowed Moskowitz and Wertheim to identify umpire error. The error rate was 14%. However, that error rate changed systematically with some non-rule factors. These systematic changes are classed as a bias. For example, the player behind in the count got more favourable calls. Technology allowed Moskowitz and Wertheim to identify umpire errors directly. The Holder et al (2021) paper is a step in this direction for soccer.
The beauty of the Journal of Business Economics paper is that it uses VAR technology to focus on referee error directly. It uses four successive seasons of data (two pre-VAR and two post-VAR) from the German Bundesliga and Italian Serie A. Impressively, the authors also "validated all VAR decisions by examining video recordings via youtube.de of the respective situations". I want to highlight one of their findings. Home teams were awarded more penalty kicks than away teams but the "introduction of VAR did not change this distribution, and the VAR system intervened with comparable frequency for both the home and away teams". In other words, there was no referee bias in the errors.
For a variety of reasons, not all of the literature seeks to identify the referee bias directly. Much of the literature on referee bias attempts to identify the bias by purely statistical methods. The biggest difficult is separating referee behaviour from player behaviour. One prominent paper from 2012 compares the rate at which the referee decides the ball must be turned over (“mainly traveling violations and offensive fouls”) with the rate of turnover by players (“mainly bad passes and lost balls”). Tens of thousands of observations are then used to establish the statistical significance of the relationship between these turnovers and other variables, e.g. home team. The authors of this paper understand the limits of this approach.
It is encouraging that more papers like Holder, Ehrmann and Konig are starting to appear in the literature. Technology is facilitating a better approach to understanding referee bias.
RSS Feed