Google’s self-driving fleet of robo-Priuses have been cruising around the San Francisco area for months now, logging over 190,000 miles. But until recently, the technology behind the autonomous cars had been kept secret. Last month, Sebastian Thrun, a Stanford professor and head of the project, and Google engineer Chris Urmson, delivered a keynote speech at the IEEE International Conference on Intelligent Robots and Systems in San Francisco, explaining how the car works. Their presentation included a video of the car’s tech, which also showed what the car “sees” as it drives, a trippy neon image of the surrounding area, with roughly rendered cars and people moving around it.

The “heart of the system” is its Velodyne 64-beam laser that sits on the roof of the Prius, creating the 3-D map of the surrounding environment. This 3-D image is combined with existing high resolution maps programmed into the car. Four radars (one for the front, back, left and right) are used to give the car far-sighted vision for handling high speeds on freeways. There’s also a camera near the rear view mirror for monitoring stop lights. GPS, an inertial measurement unit, and wheel encoder keep track of where the car goes.

GPS alone is not nearly as accurate as the combination of GPS and the high resolution maps. According to Urmson, a GPS-determined location alone could be off by several meters–fine for directions, but disastrous for a car negotiating traffic.

The car is remarkable in its restraint, yielding to pedestrians during a left turn at an intersection. But what’s even more remarkable, is its ability to be more “aggressive” when it needs to be. For instance, it’s able to compensate when other motorists don’t exactly adhere to right of way rules. In response, the car will inch forward into the intersection, signaling the other drivers its intention.

As an ongoing demonstration of their autonomous vehicle program, Google has automated its golf carts, which ferry visitors and workers around its campus. The carts can deliver themselves to a person needing a ride, giving the passenger the option of driving themselves, or being driven to their destination.

IEEE Spectrum