VoxEU Column

Judging under public pressure

17 Oct 2021

In many settings, independent decision makers employed to make determinations in disputes or contests between parties can be subject to public pressure This column examines the extent to which crowd pressure affects referees’ decisions in Germany’s top football division, exploiting the introduction of video technology as well as the absence of crowds during the Covid-19 pandemic. It finds no evidence of bias in awarding goals or penalties, but a tendency for bias towards the home team when deciding whether or not to issue a yellow card, which disappears in the absence of a crowd.

Authors

Alma Cohen

Zvika Neeman

Florian Auferoth

In many settings, an independent decision maker is required to make one or more determinations in a dispute or contest between parties with competing interests. This type of ‘judging’ is very broad, and includes decisions by judges and jurors in court cases, decisions by arbitrators in arbitration proceedings, determinations by independent fact-finding commissions on contested issues, and decisions by referees in sports events.

Even when the individual who engages in judging is independent of the relevant parties, her determinations may be subject to and affected by public pressure. In the case of judges, this is especially relevant for elected state court judges who expect to run for re-election, as well as for other judges who have career concerns and expect the conformity of their decisions with those favoured by the general public to affect their career prospects. As for jurors, there is long-standing recognition and concern that pre-trial publicity, and the resulting public opinion on the ‘correct’ outcome, may have a significant effect on jurors’ decisions. Accordingly, in many common law jurisdictions, there are limitations on public commentary for cases that are sub judice (Latin for ‘under a judge,’ that is, under trial or otherwise under consideration by a judge or court). In the US, the First Amendment guarantee of free speech has prevented tight restrictions on comments regarding matters that are sub judice, but state rules of professional conduct governing attorneys often place restrictions on the out-of-court attorney statements regarding ongoing cases. Moreover, judges often seek to minimise as much as possible the exposure of jurors to outside public pressure (for an extensive discussion of these issues, see Phillipson 2008).

We study whether and when public pressure affects judging in the Bundesliga, the premier soccer league in Germany, from which it is possible to obtain rich and detailed data that facilitates such an investigation (Cohen et al. 2021). For a recent survey of the subject of referee bias in professional sports, see Dohmen and Sauermann (2016).

Our approach is distinguished by two things. First, we consider referees' errors in addition to referees' decisions. This is important because correct decisions do not imply a bias, even if they are made disproportionately more often in favour or against one of the teams. Errors, on the other hand, do indicate bias if they are disproportionately made in favour or against one of the teams. Second, we are fortunate to be able to compare our results across three different regimes:

a ‘pre-VAR regime,’ which covers the period before the introduction video assistant referee (VAR) technology; a ‘VAR regime,’ which is identical to the first, except that referees had access to VAR technology; and a ‘VAR/no-crowd regime,’ which is identical to the second regime in that referees had access to VAR technology, but due to the Covid-19 pandemic, all games included in this regime were played behind mostly closed doors, with small or no crowds.1

Our study produces two main insights. First, crowd pressure has less influence on referees’ decisions when errors are indisputable, presumably because in such situations the countervailing force of referees’ concern for their reputation is strong. By contrast, we expect crowd pressure to have more influence on referee decisions when errors are not indisputably observable, because such situations make it easier for referees to rationalise and defend their decisions without incurring large costs to their professional reputation.

Our data include two types of decisions: (1) referees’ decisions whether to validate goals and whether to award penalty kicks (which are likely to result a goal), and (2) referees’ decisions whether to sanction players with yellow cards (cautions). The correctness of referees’ decisions with respect to goals and penalty kicks is (relatively) indisputably observable. Live TV coverage provides an immediate replay of all player movements in the seconds before the actions that are the subject of the referee decisions, often from several different angles, and this generally enables an objective assessment of whether the referee made a correct or wrong call.

By contrast, referee decisions on whether to issue a yellow card are (relatively) more subjective. Such decisions typically involve a judgement call over which reasonable people may disagree. As a result, a referee can more easily defend a decision to issue (or refrain from issuing) a yellow card that benefits the home team without incurring a significant reputational, if any.

We find that errors made in referee decisions whether to validate goals or award penalty kicks do not exhibit a bias in favour of either team. By contrast, we find evidence that decisions to issue yellow cards are biased in favour of the home team. Notably, we observe this effect under both the pre-VAR and the VAR regimes, but not under the VAR/no-crowd regime. That is, this bias in favour of the home team disappears in the absence of crowds.

Our second insight concerns the circumstances that enhance the magnitude of the effect of crowd pressure on those decisions that are amenable to such pressure (in our setting, the issuance of yellow cards). We hypothesise that when referees make an error on a decision they have little discretion over, they will try, possibly subconsciously, to make up for their error in some way. The fact that, on average, 90% of the crowd in our sample is composed of fans of the home team suggests that referees are more likely to make up for their errors against the home team than for their errors against the away team.

We find that referees make up for their errors against the home team by issuing more yellow cards to the away team, but not when the game is played behind closed doors. This pattern is asymmetric. Referees do not make up for their errors against the away team by giving more yellow cards to the home team (or fewer yellow cards to the away team). Moreover, the increase in bias in favour of the home team is magnified when the referees’ erroneous decision occurred (i) during a game that is more important, or (ii) at a point in time in the game when the score is close and so the error is likely to be more consequential. However, we find that referees do not make up for their errors with respect to the validation or invalidation of goals and the awarding of penalty kicks, by making additional such errors.

The introduction of VAR technology in 2018 and the restrictions imposed by the Covid-19 pandemic, which caused games to be played without crowds for the second half of the 2019-20 season, reinforce our results. As expected, the availability of VAR reduces the number of referee errors. VAR does not completely eliminate referee errors because a referee can still make a mistake if no consultation takes place between the referee and the VAR technology when it should. We find that VAR has no effect on the number of yellow cards issued. This is to be expected because the rules for engaging the VAR do not allow for yellow cards to be checked by it. The pattern of no bias with respect to referee errors is preserved under VAR.

Notably, under the VAR regime, the compensation to the home team through yellow cards to the away team after an error against the home team increases in size. Presumably, this is due to the fact that the availability of VAR technology makes a referee's errors more glaring (because not only did the referee make an incorrect call, but the incorrect call could have been corrected). Thus, with VAR, referee errors are likely both to elicit a stronger reaction from the crowd and to strengthen the referee's subconscious desire to compensate the team he has wronged. Interestingly, with VAR, referees also make up for errors against the away team by giving more yellow cards to the home team, but the number of games played under the VAR regime is not large enough to make this effect statistically significant. This is not surprising, because the fact that referees' errors become more glaring as explained above, implies that the crowd's reaction to these errors is likely to be stronger, and the referees may feel a stronger compulsion to make up for their errors for both teams.

Under the third regime (VAR/no-crowd), the data show that once the crowd disappears, so does the home advantage in goals. This suggests that the crowd has a strong effect on the players. Referee errors are unaffected, but referees' tendency to make up for their errors against one team by giving more yellow cards to the other team, which was manifested under the second regime with VAR, disappears as well.