Google Now tops Siri at accuracy, says analyst
Both voice assistants have improved since last year, says analyst Gene Munster. But in a new contest, Google Now scores slightly higher than its rival from Apple.
Google Now and Apple's Siri both do their best to answer your queries. But Google's voice assistant fared slightly better in a new competition conducted by Piper Jaffray analyst Gene Munster.
Voice assistants have slowly become more of a key feature of mobile devices with Siri, Google Now, and Microsoft's Cortana for Windows Phone 8.1. But they still face several challenges. They can easily misinterpret your question by not understanding your words. They can have trouble finding the right information. And at times they can be downright frustrating in their failure to even come close to the right answer. So a voice assistant's level of accuracy and performance can be a selling point for a mobile phone and its operating system.
As described in two investors notes released Tuesday, Munster challenged Google Now and Siri by throwing a series of 800 questions at them and then measuring their ability to understand and respond to each query. Among the 800 questions, half were asked indoors, the other half outdoors. The questions themselves centered on five categories -- local information, commerce, navigation, general information, and OS commands.
Munster's team found that Siri used Google as a source for only 3 percent of all questions, an ongoing drop from a 27 percent reliance on Google in December 2012. Instead, Siri used Bing as the default search engine and Apple Maps for navigation questions. Siri was also able to answer 4 percent of all the queries on its own, an improvement from less than 1 percent in December 2013.
"We believe Siri will continue increasing the number of queries it can answer without consulting outside sources," Munster said in one of the investors notes. "This is important because if Siri consistently directs users to other search engines, they will be more likely to simply use Google/another search engine instead of going through Siri."
Siri also fared well in the competition by being able to cull data from more than one source and sorting certain results based on distance or rating. In the end, Siri received a grade of a B-, an improvement from the C+ that Munster doled out last December.
Google Now slightly one-upped its competitor by answering more questions accurately. Google Now accurately answered 86 percent of all questions it heard correctly, while Siri's score in that area was 84 percent. Back in December, Google Now had accurately answered 81 percent of all questions it heard correctly, while Siri had scored 83 percent in that category.
Google Now's strongest skill was finding navigation data, as well as local and general information. It was weaker than Siri at interpreting OS commands, such as "Play a song by the Beatles," though it has shown improvement in this area over time.
"We believe that Google Now will continue to improve its OS Command performance as it continues to release more voice-compatible devices (i.e. Glass, Moto X, and Moto G)," Munster said.
For the final tally, Google Now took home a grade of B, up from the C+ given last December.
Siri is due for updates via iOS 8, which is now in beta mode and is expected to launch in September.