I'm riding in an Audi SQ5, heavily modified by auto electronics supplier Delphi so it can drive itself, in Mountain View, California. Approaching an intersection with a traffic light, the car begins braking to a stop, despite the fact that the light is green. A Delphi employee, trained and licensed to operate self-driving cars, grabs the wheel and accelerates through the intersection.
Delphi calls this a "takeover" moment, one of two that I experience on this half hour drive.
Rather than letting a failure like this stymie efforts to develop self-driving technologies, Delphi Vice President Glen De Vos, riding in the car with me, points out how important it is to run into these types of issues in real-world scenarios. In this case, the sun was visible right next to the traffic light, which confused the car's computer when it tried to process the camera imagery.
A red hot area of research in the automotive industry, self-driving cars show potential to greatly reduce or eliminate the tens of thousands of deaths that occur on US roads every year. This technology may also reduce traffic jams, a major fuel and time waster in US cities. Along with automakers, equipment suppliers like Delphi, start-ups and big tech companies like Google are all developing self-driving car technology.
Delphi maintains a research facility in Mountain View, and is licensed to test its cars on California public roads. The San Francisco Bay Area is unique in that self-driving cars from companies such as GM-owned Cruise Automation and Google regularly roam the streets. This conglomeration is partly due to the local high-tech industry, but also to the generally good weather, allowing year-round testing.
The SQ5 used as a development platform by Delphi includes LIDAR, a laser-based sensor, Radar and cameras. De Vos says that Delphi believes all of these sensors will be necessary to realize the self-driving car of the future. Inside the car, its computer, what Delphi calls a Multi-Domain Controller, runs software that compiles all of the sensor data in real time to map out the car's immediate environment. This computer also makes driving decisions using software from a company called Ottomatika, which Delphi acquired last year.
For the takeover moment I experienced, engineers will go over the data logged by the car, examining it to understand what the car saw with its sensors and why it made its decision to stop. Based on what it uncovers, Delphi could make adjustments to its image processing code to better identify traffic lights, potentially even including sun position information, so that the car will not confuse the sun and traffic light.
As another means of solving this issue, Delphi's SQ5 is also equipped with vehicle-to-infrastructure (V2X) technology. If that traffic light were also equipped for V2X, it would have broadcast a signal to its current state: red, yellow or green. It could also broadcast how soon it will change. V2X receivers in the SQ5 would then have precise information as to the state of the light, and would not need to rely on camera imagery.
Similarly, Audi demonstrated a system in Las Vegas where the local traffic control agency makes the real-time status of all its traffic lights available over the internet. Audi's data center takes that information and sends the relevant light status to all its cars based on their locations.
During my ride in Delphi's car, the other takeover moment occurred where the inside lane on a three-lane road was closed for construction. The car was in the middle lane, but when it detected the traffic cones to the left it began to brake, and the operator had to take over again. In this scenario, it was much harder to figure out at a glance what caused the car to stop, something the data logs will surely reveal.
This type of real-world testing is what it will take to train self-driving cars to deal with every potential situation they might come across in the real world. And most importantly, as each situation is successfully solved, it will enter the programming for every other self-driving car, which is why developers like Delphi don't fear these takeover moments, but use them to advance the research, and inform the production cars that will be driving the roads in the near future.