Making better decisions by reducing ‘noise’ – VAR and the Qatar World Cup

Nov 4, 2022 |5 min read

The FIFA World Cup of 2018 seems a lifetime ago. So much has changed since, with at least two years lost to a global pandemic. However, one thing hasn’t changed: Video Assistant Refereeing (VAR) keeps getting decisions wrong.

At the last World Cup, VAR was introduced on the world stage. Up until that point, football, as one of the world’s most established and popular sports, was well behind the times in its use of video technology to aid decision making. And its introduction was not without controversy, with many divisive incidents over the course of the competition.

 

Cognitive biases in VAR

Following the tournament, we put together a review of the technology’s implementation from the perspective of behavioural science. That is to say, how good VAR was at removing cognitive biases from verdicts. Needless to say, we weren’t particularly impressed with how FIFA had enacted the system, even if FIFA President Gianni Infantino has said himself that it was ”an undisputable success”. Rather than removing cognitive biases, the approaches in place to help decision making were often reinforcing them. We highlighted confirmation, framing and anchoring biases.

Furthermore, we showed a number of incidents, particularly around handballs, where ambiguous rules led to highly subjective interpretation and decision making. One of these handball infractions occurred on the biggest stage of all, the World Cup Final, where France was awarded a penalty for a less than clear cut handball by a Croatian player, after Blaise Matuidi had headed the ball onto his arm.

We outlined three recommendations for improving the system:

  1. Reduce information overload – the referee has plenty to process under pressure without adding more stimulus, e.g. replays on the pitch-side monitor
  2. Leave decisions to the 4th official in charge of VAR – this will help ensure decisions remain as objective as possible by their independence from the on-field referee
  3. Make the rules clearer – VAR works best resolving binary scenarios; ambiguity around some laws like ‘handball’ leave decisions too open to interpretation and vulnerable to biases

In subsequent years, and despite years of fans bemoaning the lack of technology in football, VAR hasn’t exactly covered itself in glory. Many of the issues are down to the above suggestions not being addressed. Or, as with amend to the handball law at the start of the 20/21 Premier League season, not being implemented in a way favourable with fans. In fact, from a poll of more than 33,000 supporters by the FSA, 95% of those who had experienced VAR in-stadium said that VAR had made watching football less enjoyable.

 

VAR changes for the Qatar World Cup

So, what can we expect from this year’s tournament? The good (or bad) news is that some changes are being implemented. FIFA has announced that semi-automated offside technology will be used in Qatar, “offering a support tool for the video match officials and the on-field officials to help them make faster, more accurate and more reproducible offside decisions”. The new technology uses 12 dedicated tracking cameras and an Inertial Measurement Unit (IMU) sensor in the ball to track its position and those of up to 29 data points of each individual player. 50 times per second, the exact position of all data points is known on the pitch, providing an automated offside alert to the video match officials.

This feels like a welcome addition; even if a fraction of a toenail rules someone offside, a least there will be greater objectivity than the crudely drawn on lines we’ve grown used to. The speed of offside decisions – and often whether we can celebrate a goal or not – will be much quicker. But is there anything else that the footballing powers should be considering beyond our 2018 suggestions?

 

Noise: A flaw in human judgement

In the last year, the ‘Godfather’ of behavioural science, Daniel Kahneman added to his magnum opus: Thinking, Fast and Slow  - the bible on cognitive bias - with a book on Noise: A Flaw in Human Judgement. Luckily for you, I’ve read it so you don’t have to (it is a rather dry tome after all), but there are further lessons in it for improving judgements.

In this new book, Kahneman, with collaborators Olivier Sibony and Cass R Sunstein, starts by using a series of targets to explain the difference between bias and noise. He says: ”Put simply, bias is the average error in judgments. If you look at many judgments, and errors in those judgments all follow in the same direction, that is bias. By contrast, noise is the variability of error. If you look at many judgments, and the errors in those judgments follow in many different directions, that is noise.“

 In other words, we would hope decision makers in the same role (e.g. football referees) to be interchangeable; that is, faced with the same case they would reach the same or very similar judgments. In reality, not only do different referees make different judgements, the same referee (as a flawed human) can be strongly influenced by irrelevant factors, such as their current mood, the time since their last meal, and the weather when judging an identical case in a different scenario. As established, we call this variability of judgment ‘noise’. Noise doesn’t just cause inconsistent judgements, but comes on top of biases that incline judgement towards a particular outcome already.

 

Reducing noise in VAR decisions

Experienced professionals tend to have high confidence in the accuracy of their own judgments, and they also have high regard for their colleagues’ intelligence. This combination inevitably leads to an overestimation of agreement and so will be the case with referees and VAR. We’ve highlighted possible ways that VAR can improve with a reduction in biases, but how can it potentially address noise? Following lessons learnt in the book, we’ve outlined below some thoughts for FIFA to consider.

 

In summary, with humans involved in judgements, there is always going to be error. And whether this is what makes sport unpredictable and exciting is also a point of debate. However, I think it’s hard to argue VAR could not be improved drastically by reducing both biases and noise. What the upcoming Qatar World Cup holds in store, we’ll have to wait and see, but I’d bet on there being one or two dubious refereeing decisions.