As part of its licensing process to allow the testing of autonomous vehicles on public roads, the State of California requires AV developers to submit what is called a disengagement report. Despite sounding like a Philip K. Dick-based sci-fi movie or an early aughts indie rock band, the disengagement report shows exactly how many times an autonomous vehicle test driver had to intervene in the operation of a self-driving car during testing on public roads.
The disengagement report is an interesting measure of how effective a given company's -- in this case Waymo's -- platform is in the real world. The report sums up the number of disengagements that occurred during the period between December 2016 and November 2017, and the number of incidents for the number of miles traveled is pretty impressive.
"We're building the most experienced driver on the road with 4 million miles of testing in 25 cities, as well as driving on our private test track and in simulation. Our low disengage rate shows continued improvement year over year," a Waymo spokesperson said in a statement to Roadshow.
Waymo reported that during 2017, its test vehicles had 63 disengagement events during the 352,544.6 miles that they traveled in California. That works out to roughly 0.18 disengagements per 1000 miles traveled. Not bad, right? Well, just to give some data for comparison, GM's Bolt-based Cruise autonomous test program had 105 disengagements in approximately 125,000 miles traveled.
Waymo's disengagement report also breaks down the incidents further, going on to specify what caused each disengagement as well as what type of road it occurred on and how long it took for the test driver to take manual control of the vehicle. In 2017, the bulk of Waymo's disengagements occurred due to an unwanted maneuver of the vehicle followed by perception discrepancy and hardware discrepancy.
What might surprise you is that there was only one recorded disengagement for the year due to other drivers behaving recklessly. What isn't surprising though is that the vast majority of disengagements took place on surface streets, with just six happening on highways. In nearly all of the disengagements, the test driver took control in less than one second.
What does all of this mean in big-picture terms? It means that while autonomous vehicles have come a very long way in the last few years, we still have a long way to go before we can truly tune out and let our car do all the driving.