UDRS in its first 30 months: Consequences, Criticisms and Implications

Posted on December 30, 2010 by

54


The Umpire Decision Review System is a complex expert-technical apparatus designed for assisting on-field Umpires in International Cricket. It involves the use of a number of specific simulations – Hawkeye (or Virtual Eye), HotSpot, Super Slo-Motion and other video replay tools within a complex system of communications protocols involving On-field Umpires, Batsmen, Fielding Captains and TV Umpires. It sets up an economy of error by permitting players (batsman or fielding captain) to request a second opinion on a fixed number of occasions (originally this was set to 3, now it is 2) per innings. A referral is only used up if the decision on the field is not reversed. In a nutshell, it works as follows: A delivery is bowled on the field, an appeal occurs and a decision is made by the Umpire. If it is a Not Out decision, the fielding side has the right to ask for it to be referred (pitting their judgment against the Umpire’s), if it’s an Out decision, the batsman can do the same, provided that their respective teams still has referrals in balance. If the TV Umpire, upon examining the evidence offered by pre-approved tools finds that the Umpire on the field has ruled in error, he recommends that the decision be reversed. The standard required here is that it be conclusively clear that the Umpire on the field has made a mistake.

Like all such complicated apparatuses, it has consequences, both for the specific issue it seeks to solve – that of obvious, glaring umpiring errors, and for the larger game of cricket. Not all of these consequences are immediately apparent. Neither do the complications in the implementation of UDRS seem to be clear to many of its die-hard supporters. It is considered cool to be a technophile in this Information Age, but technophilia tends to give short shrift to the consequences of technologies (a largely unquestioned assumption is that technology is inherently good).

Let’s consider the technologies used in UDRS. Hot-spot is a moving image of the action that relies on heat signatures. It is able to capture things like the impact of bat on ball, bat on pad, bat on ground, ball on glove, ball on pad etc. etc. It is not foolproof. There have been occasions when batsmen have walked, and nothing has been registered on Hot-Spot. There have been other occasions where random hotspots have appeared at places where there is no evidence of any contact being made. These anomalies could occur due to any number of technical reasons.

Hawkeye is graphical 3D simulation of the trajectory of a ball in flight. It also has a predictive element which is based on a database of earlier trajectories which are used (though apparently not exclusively) in the statistical calculation which predicts the path of the ball as it might have been had the ball not hit the pad or the bat. The authors of the two major versions of Hawkeye (the term is used generally for this tool), Hawkeye and Virtual Eye differ when it comes to claims about accuracy. Paul Hawkins of Hawkeye Innovations Ltd. (based in London) has no hesitation in recommending Hawkeye as a decision making aid, while Ian Taylor of Virtual Eye (the name of the company is Animation Research Ltd. and it is based in New Zealand) is circumspect. Mr. Taylor would prefer Virtual Eye to remain a broadcasting aid, and not a decision making aid.

This raises one of the most important issues in the whole UDRS business – the involvement of the broadcaster. Hawkeye and Virtual Eye, as well as Hotspot and other technologies, typically enter into contract with the broadcaster. The broadcaster for any series is hired by the host Cricket Board and not by the ICC (unless it is an ICC event such as a World Cup). Hence, the ICC, which would like UDRS to be used in all international cricket as a cricketing matter, is unwilling or unable to get involved in actually doing so, because it has no real say in the contract between the broadcaster and the host cricket board. This also means that the ICC and its representatives (Umpires, Match Referee) at a Test Match do not control the technology that is installed. The TV replays that they see are controlled by the director of the broadcast.

The fact that the technology is purchased by the broadcaster, primarily as a broadcasting aid for the commentators (ostensibly to add value to the entertainment), is also geared towards doing this. This means, that crucial information about uncertainty (margin of error) in a technology like Hawkeye is not made available, because, in the words of Paul Hawkins, “Almost all commentators are ex cricketers, and generally not that scientifically bright. They would have no ability to explain what an uncertainty ellipse is”. He further told me that “Broadcasters prefer a “definitive” where was the ball going – some commentators are then good at interpreting that information saying something like: “Hawk-Eye shows it just clipping leg stump, so a good decision to give the benefit to the batsman”. An uncertainty ellipse is region around the path of the ball which shows the range of paths a ball might have traversed. According to Hawkeye’s own documentation (I will focus on Hawkeye here, since this technology is being touted as a decision making aid, unlike Virtual Eye), shows that the margin of error depends on a number of factors – how far away from the stumps the batsman was struck and how close to the pad the ball pitched, to name just two. There must be a number of other factors which are not explicitly considered in the Hawkeye manual, such as the number of deliveries already in the Hawkeye database for a given venue and a given pitch on that venue (different wickets on the same square behave differently) and the number of deliveries from a particular bowler in the database. All of these factors should non-trivially affect the prediction. Nevertheless, Mr. Hawkins has argued that either there is enough information for a prediction, or there isn’t, which is to say that given the method used by Hawkeye, there is a good chance that for the vast majority of situations, there is enough information available to make a prediction.

This is a circular argument, because if certain factors are not explicitly considered in the model, then they will never be required in the “enough information” set. The real question is whether Hawkeye considers enough factors.

The old dictum that “all models are wrong but some are useful” is worth bringing up here. The question is, useful for what? As a broadcasting aid, Hawkeye is wonderful. But even if the commentators are not “scientifically bright” enough to intelligently interpret Hawkeye (Mr. Hawkins concedes that the good ones will do so), what of the Umpires? Can they not be trusted with the more complicated, data rich evaluation, instead of the candy-coated colorful moving picture that we as viewers are saddled with? Apparently not.

Be that as it may, Hawkeye and UDRS have now been used off and on for about 30 months in Test Cricket. Several issues arise on account of its implementation.

First, players are now allowed to question the Umpire’s decision. Especially in the case of the fielding side, they do not necessarily have any extra information on the basis of which to do so. Batsmen can know about obvious inside edges and cases where they have missed the ball. Fielding sides do not have this privilege. In many instances, this has resulted in the tactical use of the review.

Second, the tools used in UDRS, not surprisingly, have proved to be fallible. That there are other tools, such as snickometer, which are available to broadcasters, but not considered reliable enough to be used in UDRS, do not make things any better. Further, the single most unaccountable agents in the whole process – Commentators, seem to have no training in using UDRS. They are paid to be partisans in a gentlemanly way in most cases, and using Hawkeye fairly tends to come in the way of doing this. As an example, consider Robin Jackman on Day 4 of the Durban Test (December 29, 2010) after the deVilliers LBW. Yet another peculiarity about commentators is that in the pre-Hawkeye era, they would offer opinions about whether or not a ball was going on to hit the stumps without compunction. These same commentators are unwilling to offer an opinion on this without consulting Hawkeye, at which point they merely describe what Hawkeye is showing without any caveats. In doing so, they have become redundant as experts, because everybody can see what Hawkeye shows – it doesn’t need interpretation, the broadcasters have deliberately required it to be this way, as Mr. Hawkins told me.

Third, the limitation of two reviews has proved to be problematic, because many times obvious errors have gone uncorrected because a team had already exhausted its reviews. This undermines the stated goal of UDRS – which is to correct obvious umpiring errors.

Fourth, it has implicitly changed the interpretation of many laws. For example, the marginal LBW decision is a thing of the past. Marginality in the LBW, is now limited to the margin of error in Hawkeye and nothing else. This is extremely narrow, and fails to take into account issues like whether or not the batsman offered a shot. It is likely to change batting techniques if things like these don’t matter any more.

We could consider this at greater length, but for now, I will conclude by suggesting that UDRS in its first 30 months has, if not betrayed, then at least failed its stated goal of correcting obvious, glaring Umpiring errors, and strayed into the dubious realm of attempting to erasing subjectivity in Umpiring decisions altogether. Technophile defenders of UDRS could accuse me of being postmodernist about this, but it is incumbent upon them – who tend to discuss UDRS devoid of its many complications, competing interests and implications when they consider it, to answer to the specific issues I have raised.

As for me, I would offer the following modification to UDRS, one which can be implemented immediately, and one which would explicitly reinstate subjectivity into the process, and temper pipedreams about objective conjectures about LBWs. In the case of any decision which is referred, the third umpire would first have to make a subjective decision after watching one or more replays as to whether or not the decision on the field is (1) obviously correct, (2) marginal, and (3) obviously wrong. In the case of an on-field decision deemed to be marginal, the on-field decision should stand, but the team requesting the referral should not be docked a referral. In the case of an on-field decision deemed to be obviously wrong, it should be corrected. In the case of an on-field decision deemed to be obviously right, it should obviously be allowed to stand, and the team requesting the referral should be docked a referral. The decision as to whether or not a decision on the field is marginal should be the third umpire’s and not Hawkeye’s. Doing so would make UDRS palatable and also curb its tactical use. It would be a conservative use of tools which, it has to be accepted, are currently quite poor.

————————————–

Article by Kartikeya Date