From Darpa Grand Challenge 2004DARPA’s Debacle in the Desert
Behind the scenes at the DARPA Grand Challenge, the 142-mile robot race that died at mile 7
by Michael Darter
When last we visited with the men and women, the boys and girls, the Red Teams and Blue Teams and Road Warriors of the DARPA Grand Challenge off-road robotics race, back in March, we signed off on a note of authentic ambivalence. The teams themselves were all over the map, from rehearsing victory speeches to praying they would pass the qualifying round and be allowed on to what was anticipated to be a 210-mile course from outside Los Angeles through the Mojave Desert to somewhere just west of Vegas. The race’s organizers, for their part, couldn’t quite muster a consensus on how to handicap the event. Race manager and resident sunny optimist Col. Jose Negron unblinkingly predicted that a team would cross the finish line in under 10 hours to claim DARPA’s million-dollar prize in the race’s inaugural run-yet course designer Sal Fish couldn’t bring himself to share this official vision. “Its still hard to get it in my brain,” Fish said, “that this is all going to happen with robots.”Chalk one up for Mr. Fish.
Here, to spare you the suspense, is how things looked once the dust had cleared on race day, March 13: Carnegie Mellon University’s Red Team, the presumptive race favorite-in the minds of many race insiders, the only team with a realistic shot at the million-dollar prize-had ended the race at mile 7.4, its Humvee’s belly straddling the outer edge of a drop-off, front wheels spinning freely, on fire. SciAutonics II dropped out of the running at mile 6.7, its Israeli dune buggy stuck in an embankment. Digital Auto Drive quit at mile 6.0, its Toyota Tundra stymied by a football-size rock. The Golem Group stopped at mile 5.2, its pickup stuck on a hill with insufficient throttle to move forward. Team Caltech, another race favorite, dropped out at mile 1.3, its Chevy Tahoe SUV having careened off course and through a fence. Team TerraMax, a heavyweight collaboration between Ohio State University and the Oshkosh Trucking Corporation, was out at mile 1.2, stopped of its own accord, a 32,000-pound six-wheel military truck flummoxed by some bushes. These, it should be noted, were the Grand Challenge success stories. The rest of the field went haywire at or just beyond the starting chute in full view of the press who packed the grandstands erected for the event.
The two teams that had become media darlings and unofficial DARPA pets had suffered particularly inglorious flameouts. At the last minute, Anthony Levandowski, the UC Berkeley grad student-cum-visionary behind the Blue Team’s autonomous motorcycle, scratched from the race proper, his navigation systems nowhere near race-ready. But, as he proved on an earlier qualifying attempt, Levandowski had successfully realized an ingenious software system that could keep the bike moving forward (or in circles) through constant steering and countersteering corrections. At the Grand Challenge, DARPA agreed to let him stage a remote-control demonstration for the by now autonomous-bike-crazed media. Alas, when the checkered flag went down, so did the bike, without a whimper. The first attempt of the high school team from Los Angeles, the Palos Verdes High School Road Warriors, aborted when their vehicle, a modified Honda Acura MDX, lurched right immediately after start-up and headed for the grandstands until DARPA hit the disabling “E-Stop” button. Race organizers granted the team the luxury of a second try and time to make some quick software fixes, and for Take 2, the Acura came roaring out of the gate hard left and knocked over a two-and-half-foot-high concrete guard before DARPA could hit the E-Stop, a don’t-try-this-at-home moment of autonomous mayhem that proved popular with the evening TV news broadcasts.
What went wrong? In a sense, nothing. These results were-or should have been-entirely unsurprising. Unlike the computer whirring on your desk, mobile robots have to thrash around in the real world, which makes the entire enterprise finicky and unpredictable. The 15 machines that took the starting line in Barstow, California, were attempting a quantum jump in performance over the robots that putter around university artificial-intelligence labs, avoiding table legs at one or two miles an hour. To have a shot at the deadline and the big prize on what was ultimately a 142-mile course (prudently downsized to make a 10-hour, one-day race a feasible goal) from Barstow to Primm, Nevada, the Grand Challenge bots would have had to average nearly 15 mph, and in the flat stretches reach speeds of up to 50 mph. They would stay in one piece by tracking via GPS technology the latitude-longitude waypoints that defined the course and avoiding obstacles with their own internal sensors: video cameras, laser scanners, radar and the like. Good luck. By comparison, the winning vehicle in the last Baja 1000 off-road race, with an actual human behind the wheel, averaged just over 50 mph, though on much stiffer terrain.
Traversing less than six percent of the course may not sound like a grand result, but it should be noted that the four lead bots did get through the first section of the course, a flat looping dirt road that passed through four fence-gate openings, each only about 12 feet wide. After mile 4, the narrow, rocky road begins its snake-line ascent of a vertiginous ridge, a series of tight switchbacks with cliff drop-off on one side, hillside on the other, and heartbreak written all over it. Making it only partway up Daggett Ridge is no shame.
After the race, some of the robotics teams grumbled that DARPA-the Defense Department’s R&D wing, which staged the race in an effort to tap the deep well of American amateur ingenuity-had guaranteed that its Grand Challenge would be a short and unhappy one with Daggett Ridge so close to the start, instead of reversing course and running the race from Primm to Barstow instead. It´s a debatable point. The Primm area has its own minefields, notably silt beds and a steep ridgeline about 12 miles from the finish.
If DARPA was plainly guilty of anything, it was not managing inflated expectations. Instead of billing this inaugural Grand Challenge as a not-ready-for-prime-time field test to calibrate what was needed for future efforts, race manager Negron, in the months leading up to the checkered flag, continued to predict a victor. (At a press conference before the race’s start, Negron´s second-in-command, Tom Strat, wisely if belatedly redefined victory as a matter of winning young techie hearts and minds, not miles traveled: “I can’t tell you if the vehicles will go 1 mile or 20 miles, but I can say the Grand Challenge has already been a great success.”)
DARPA had gone so far as to remake a 6,500-seat arena in Buffalo Bill´s Resort & Casino in Primm (a cheap Vegas knockoff a half hour southwest of the city) into the Challenge Operations Center. Once the bots were well launched, the media were to leave the spartan charms of the starting-gate bleachers by Barstow’s Slash X Ranch Cafe roadhouse and hightail it over to Buffalo Bill’s to nosh, listen to a smooth-rock cover band, and follow the race progress on two huge screens running bits of video footage shot from DARPA helicopters as the bots made their way to the finish line just outside the Buffalo Bill´s parking lot. But what to do when the race was effectively over by 9 a.m. and officially over by 11?
At noon, the Buffalo Bill’s operations center had the sad, pretentious look of an overproduced birthday party or bar mitzvah for which the guests had all declined to show up. The eye couldn’t fail to notice one tall, powerfully built 60-year-old man slumped back in his chair. William “Red” Whittaker, CMU robotics professor and eponymous leader of the Red Team, had made sure he was in Primm well ahead of the 10-hour deadline, even as his team’s vehicle, Sandstorm, was being dug out of the Daggett Ridge dirt. “We’ve come back from worse,” he said, looking utterly poleaxed.
Whittaker and his team were arguably the one group that might have been capable of writing a different ending (and, for that matter, middle) to the Grand Challenge. The Red Team had a fanaticism more typical of an underdog, enviable material resources (it had become one of corporate America’s more assiduous shakedown artists), and a culture of discipline that emanated from the big man himself. The result was a battle-tested Humvee that was churning through impressive automonous test runs in the Mojave in February, when most of the other competitors had yet to plug in all of their circuit boards.
Technically speaking, Whittaker figured that Sandstorm’s edge derived from its elaborate mapping system, which electronically stores information (roads, topography, notable features) about every square meter of the course. During the Grand Challenge qualifying trials held at the California Speedway in Fontana the week before the race, the Red mapping team would repair after each day at the track to a hillside trailer adjacent to the Divine Word Mission. The gospel according to Carnegie Mellon involved punching up contiguous sections of the Mojave Desert on 15 separate computers, each Red Team “editor” working away at optimizing Sandstorm’s routes and speeds as the vehicle passed, in virtual computer reality, through his or her respective block of the master map. Two nights before the Barstow start, the team’s technical director, Ph.D. candidate Chris Urmson, tried to dispel any excess of editorial caution. “We have a battering ram of a car,” he said. “At 22 mph, Sandstorm is just a beast on a roll.”
The Palos Verdes High School team had its own technical issues to contend with at the qualifying trials, knottier in their way than any computerized map. The Road Warriors, almost from their inception, had been riven by two distinct parental factions. One group, consisting of Alice Parker, a USC electrical engineering professor, and her husband Don Bebel, an engineer at Northrop Grumman, had mentored a group of technically sophisticated students-first among them Joe Bebel,
the couple’s wunderkind 16-year-old son-who had worked up their own Linux-based operating system to drive the team’s autonomous Acura. An opposing group of engineer parents emerged who resented what they perceived as Parker’s and Bebel’s proprietary attitude toward the project and who jumped at any chance to jettison the homegrown Linux system for a commercial Windows-based one that they regarded as technically superior and far more feasible for the non-nerd students-in many cases, their own kids-to work with. By the time the team arrived at the California Speedway, the Road Warriors had ruptured at the seams, with Parker and Bebel banished to the sidelines, and Joe and his high-tech peers conscripted by the regnant adult mentors to work with the prefab Windows-based system they despised.
The results on the test track were equivocal. The new adult team committed the fatal error of accidentally erasing its operating system code, which was by some accounts working very well. Late-night or all-night programming sessions at Fontana became routine (one 15-year-old student, Dan Jacobowitz, briefly wound up in the hospital for dehydration), as did a series of qualifying attempts that were embarrassing in their cumulative awfulness. Finally, Joe and the Linux warriors were brought in for one glorious autonomous run using the original Linux OS. The Doom Buggy navigated around more than half of the course, through cones, sand traps and fences, before crashing into a planted obstacle, a car parked on the track. “It was our one moment of glory,” said Graham Robertson, Palos Verdes science teacher and the team´s faculty leader. “I was thrilled that the students had done it and the adult mentors celebrated along with them even though they say that that system is not the way to go.” Added Joe, with characteristic diplomacy, “It was a nice moment to have one team again.”
At around 3:30 a.m. on the day of the race, the 15 Grand Challenge competitors received from DARPA a CD containing the 2,000 or so GPS waypoints that described the Barstow-to-Primm course. The Red Team fed the information into the Sandstorm’s mapping software; in 10 minutes’ time it had mapped out the exact route it intended to follow. Projected race time: 13 hours-3 hours over the limit. From his aprs-battle station at Buffalo Bill’s, Whittaker seemed almost to savor the moment. “The easy thing to do would have been to relax and show up at the parking lot in Primm at 7 p.m., in 13 hours,” he said. “But we entered this challenge that was declared a year ago, and for us 10 hours was sacred.” The Red Team went back to the software, tweaking up vehicle speed and slicing the margin of error. “I was clear,” Whittaker said. “Let it run. Victory or death.”
And so death it was. That Sandstorm expired near the top of Daggett Ridge, just a switchback away from 15 miles of clear sailing, was not, Whittaker feels, a reflection on route or racing strategy, which he regards as perfection itself. “By the time we finished,” he said, “we were tuning the vehicle to mud puddles.” The exact explanation for why Sandstorm carved a turn too sharply and nearly flipped over the side of an embankment awaits exhaustive analysis of the onboard data. But the team is agreed that the vehicle’s sensing systems had not fully recovered from a rollover crash during an overly ambitious test run 10 days before the race. A quarter million dollars’ worth of electronics was crushed in an instant. The team rose to the occasion-“that galvanizing moment that levels a team to its knees so that it rises to its own greatness,” in Whittaker’s Churchillian formulation. The parts were replaced, but the vehicle´s ability to reliably avoid obstacles was never quite the same. Sandstorm whacked fences and poles on that perfect route even before it entered its fatal hairpin turn.
At the evening reception at Buffalo Bill´s, the DARPA officials couldn´t congratulate themselves enough for a job well done. Even the disappointing matter of a 142-mile race that lasted only 7.4 miles was jocularly laid at the feet of course designer Sal Fish, the president of the SCORE off-road racing organization. “The first part (of the course) was definitely the hard part, and I definitely blame Sal Fish,” DARPA director Anthony Tether declared in celebrity-roast high spirits. He was on firmer ground when he advanced an oft-repeated yet nonetheless plausible argument that the race had succeeded admirably in its primary mission of galvanizing engineers (even those still in high school) to get to work on a new generation of autonomous support and supply vehicles.
Clearly, the Grand Challenge strange brew of technical audacity and Johnny-get-your-laptop populism had touched some kind of national nerve. “I haven’t seen this much interest in something related to national security since the days of the Apollo space program,” Tether declared, before announcing that Grand Challenge 2005 would go forward-this time with a $2 million prize. According to Red Whittaker’s e-mail “Race Log,” a kind of computer diary of Captain Ahab-ian obsession-filed in late March, “The Grand Challenge will be completed in September or October of 2005, hence about 550 days remain to race day.”
The media might be a little wary about a full-court coverage of next year’s race, but evidently the people who build and sponsor robots can´t wait. (“Next year we’ll have seniors on the team,” Palos Verdes student leader Chris Seide exulted to the assembled multitudes.) DARPA’s Tom Strat anticipates some 500 aspiring entrants, five times the number this year. And while much of this year’s field was unable to cope even with preliminary trials-the first day at Fontana, only two of eight bots attempting to qualify made it out of the gate-the consensus is that, with the benefit of experience, next year´s second-time contenders ought to be ready to rumble. From DARPA’s perspective, what’s not to like? This just gives them another round of technological innovation paid for mostly by someone else.
While Grand Challenge 2004 had its moments of winging-it improvisation, who’s to say it wasn’t the right hot-house atmosphere to grow the next generation of American roboticists? The Blue Team’s Anthony Levandowski, for one, learned something about grace under pressure. “Right before the demonstration, the crowd was cheering and we were so excited that we forgot to switch the bike over from autonomous to drive-by-wire,” he said. “Next time, I’ll have it tattooed to my arm.”