Controlling this robotic excavator is like playing a video game
This machine can be operated remotely through augmented reality.
Digging in the dirt is tough business. An excavator will speed up the work, but operating one is tricky and requires years of experience. One that is operable remotely, and comes with an augmented reality system, could make the process easier, more convenient, safer—and, perhaps, more fun.
“We have made a system that can take someone with no real skills of excavation and make them into an expert operator, way sooner than would normally be possible,” says Reuben Brewer, a roboticist who led the project. His team at SRI International, a non-profit research institute headquartered in California, has transformed an unwieldy excavator into a smart one with a gamified feel.
Traditional excavators have non-intuitive controllers. To move the scooper up and down, the operator needs to execute a right-left motion on the joysticks and levers from within the vehicle. “There’re so many joints, and they don’t really map up in the right direction,” says Brewer. “It gets so confusing.” Operators also need intensive training to learn how to dodge buried gas lines, water lines, and internet cables to make precise holes in the ground.
So, the researchers at SRI International have given excavators a robotification upgrade. Their smart digger can now be operated with more intuitive controls, and the operator does not even need to be in the driver’s seat. Users can perform maneuvers on these excavators right next to the vehicle, if they wish, or they can manipulate the machinery from the comfort of home or anywhere around the world, as long as they are connected to the internet.
This “robotification kit,” as Brewer calls it, can be tacked onto any existing manual excavator to make it operable as if straight out of a video game. The researchers motorized all the levers and pedals in the machine, all which would be synced up to a handheld controller over the internet. Six overhead cameras atop the hood of the excavator provide remote users a 360-degree view of the field site. An Oculus headset reconstructs the excavator’s surroundings for the wearer from the live camera feed, imbuing them with depth perception so they can toggle the arm or burrow accurately from a distance. The connected controller lets remote users execute the desired digging motions, and an advanced software tracks the controller’s position in real time and mimics the user’s motions with the excavator arm. For experienced users who prefer the more familiar controls, there’s the option of piloting the excavator with a wireless joystick harness. “Augmented reality helps with [making] you really feel like you’re sitting in the cab driving the excavator,” says Brewer.
He adds this isn’t the first excavator that can be operated from afar. In 2015, Volvo showed off a similar concept. But his company’s version has added safety controls that make it more practical. These safety features can freeze up toggling motions or force the excavator to slow down when its overhead cameras detect a human nearby. These commands automatically kick in even when a remote user is behind the virtual wheel. These design elements are important for carrying out construction work productively and safely in crowded urban areas bustling with human traffic.
Unlike some previous efforts to automate excavation work, the vision for the project isn’t to get rid of the human driver, says Brewer (although the company has also programmed excavators to be able to park themselves). After all, the consequences of smashing a pipe or damaging infrastructure while digging can be costly, so some level of human judgment during excavation is still required. Instead, the researchers hope to give users “superpowers,” says Brewer, such as the ability to see all around and behind them, ideally eliminating common blind spots.
The researchers are currently partnering with construction companies to get user feedback on their robotic excavator. In the future, depending on interest, the researchers say they can extend their robotification kit to other lumbering machines.
“It’s very broadly generalizable to basically any manually operated construction equipment,” says Brewer. “Tweak the specific geometry of where we’re putting the motor … and suddenly, instead of just doing excavation, we’re doing forklifts or front loaders or steamrollers. It’s very easily adaptable.”
Watch a video of smart excavator here:
