How sharp are Hawk-Eyes really? I'm talking about commercial systems, such as Hawk-Eye and GoalControl, that use cameras to record whether tennis, soccer, cricket and other balls fall in or out of bounds/the goal box/whatever.
Novak Djokovic relied on Hawk-Eye extensively this weekend to challenge calls during his Wimbledon match with Andy Murray, which Murray won. Those Hawk-Eye replays they show on TV look pretty authoritative, but actually, not all of the system's calls are accurate. Even robot eyes can fail sometimes. The Guardian posted an opinion piece today outlining some of Hawk-Eye's potential weaknesses.
Because of flaws in cameras, Hawk-Eye can't always be right, but publicly, there's not a lot of information about how big its errors could be. Acting as many companies would, Hawk-Eye doesn't publish the math behind its system in detail. Officials at sports federations probably have an inside scoop, but in a 2008 interview, the head of the International Tennis Federation's technical center, Stuart Miller, didn't seem inclined to let watchers know more about Hawk-Eye's errors. "I'm not sure that the role of Hawk-Eye is public education," he told Nature News.
Some researchers—and mathematically inclined sports fans—argue that there's a better way. At least one sport does it right, they say. Cricket is transparent about when Hawk-Eye is uncertain and umpires have to use their own judgment. The way that tennis uses Hawk-Eye, on the other hand, hides the technology's uncertainty.
For a few years now, both researchers and science writers have called for Hawk-Eye replays to show the range of error in the system's reconstruction of plays. That is, they want to see, in that replay cartoon, how far off a Hawk-Eye call could be.
Why is there error in the first place? As The Guardian explains, cameras can't catch every moment of a ball's flight and fast-flying tennis balls could move pretty far in between frames. Systems like Hawk-Eye depend on mathematical formulas to estimate what actually happened in between.
Companies' reluctance to publish their formulas helps them protect their intellectual property, but makes it difficult to know what to think about public debates about their errors.
In 2008, two social scientists from Cardiff University in the U.K. published a paper in which they guessed at how many millimeters off a Hawk-Eye call could be. At the time, Hawk-Eye's inventors disputed the Cardiff analysis, saying that it assumed a pattern of error that the actual Hawk-Eye doesn't follow. The Cardiff folks said (I'm summarizing here; this is not a real quote): "Well, we used a normal distribution and we can't use anything else unless you tell us more about your math." Many things in nature have a normal distribution, which describes the likelihood of any particular outcome in a continuum.
Hawk-Eye didn't release more information about its math. Instead, its inventors just said its error doesn't fall in a normal distribution. Inventor Paul Hawkins also said that in tests, the system has made the right call every single time.
For now, the statistical sausage is a secret. Whether it stays that way perhaps depends on how many more fans learn about ball-callers' flaws and say they want to know more.
Well they don't have too publish their math. The math to find trajectories is not difficult. Even after a call has been made, checking speed and trajectory afterwards would give probability to the call as well. Simply putting another camera above the court and comparing the two images would be even more accurate. The fact that Hawkeye doesn't do this for redundancy is a little alarming. It could easily be "full proof". Why not just make it that way?
"Do not try and bend the spoon. That is impossible. Only try and realize the truth - there is no spoon."
Show me a real close-up high-def image! Enough of this fake cartoon stuff. We still don't have the technology for solid camera views??
If a technology has an AVERAGE error of 3.6mm, it should not be used to make in or out calls of 1mm. That is just a guess!
my buddy's sister-in-law makes $70 every hour on the laptop. She has been out of a job for 9 months but last month her paycheck was $16749 just working on the laptop for a few hours...... www.bay95.com