Apple TV's Siri-ous challenge: Roku, Fire TV and Nexus Player take Apple's voice control test
We compared to see how Google's Nexus Player, an Amazon Fire TV and a Roku 3 could handle the voice commands Apple highlighted with the new Apple TV.
- Self-taught tinkerer, informal IT and gadget consultant to friends and family (with several self-built gaming PCs under his belt)
Siri is coming to your TV -- and she's gunning for Roku, Amazon and Google.
Earlier this week, Apple unveiled a new version of its Apple TV video streaming box, adding long-awaited features such as an iPhone-like app store, a new touch-enabled remote and the ability to simultaneously search for content across multiple video providers like HBO, Netflix and Hulu. But it's the new box's inclusion of the same Siri voice control found in Apple's iPhones and iPads that may be the new model's killer app.
The new Apple TV, which starts at $149 USD (which converts to £95 and AU$210), won't be available until late October. But rather than wait until then, we decided to see how well the forthcoming product's advertised voice features -- as demonstrated by Apple's Eddy Cue and Jennifer Folds during the presentation -- compared to the current crop of Apple TV competitors: the Google Nexus Player, Roku 3 and Amazon Fire TV box, each of which offer some degree of voice recognition. Those models retail for $100 apiece.
The main takeaway: If the new Apple TV's Siri-powered voice recognition capabilities are as good as advertised, Apple's streaming video rivals will need to up their game.
Before we start
Let's establish some important caveats before getting to the results of our tests.
We haven't tested the new Apple TV yet. Aside from a very brief hands-on experience right after the Apple event, CNET has not tested the new Apple TV. In this case, we're relying on Apple's canned on-stage demo, which was specifically optimized to show the best aspects of the company's new product. When we eventually review the product, we'll rerun these tests ourselves, as well as many more.
The competing products are older and less expensive. Amazon's Fire TV box was released in April 2014. The new Roku 3 hit a year later. And Google's Nexus Player dates from October of last year. And the Apple TV has a starting price that's 50 percent higher (a $200 version, which doubles capacity from 32GB to 64GB, will also be available). So, to a certain degree, the newer and pricier Apple TV comes to the table with some advantages straight-off.
We deliberately omitted the Xbox One. While the new Apple TV is at least in the same price universe as the competing $100 boxes, the $500 Xbox One -- the high-end bundle that includes the Kinect sensor and microphone -- is clearly in another class altogether. So, while the Xbox's voice controls may well be competitive with Apple's Siri, we didn't include it in this test.
Putting Nexus Player, Roku and Fire TV to the Siri test
We transcribed the exact verbal commands used by Apple's Eddy Cue and Jennifer Folds from the video stream of Apple's press event, and issued them to the Google Nexus Player, Roku 3 (2015 version) and Amazon Fire TV box. Where indicated, we tried different iterations of the same phrases as well.
We skipped a few of the follow-up commands from the demo (such as "What did she say?" and "The James Bond ones") because they were contextual or proprietary to the new Apple TV.
These are our results:
"Show me funny TV shows"
- Apple TV demo: A list of TV comedies including "New Girl" and "Arrested Development" is displayed.
- Nexus Player: A Google Now card for the TV show "Show Me the Funny" is displayed on the screen with cast information. It also displays YouTube videos with the keyword "funny" and "show", along with a selection of what appear to be random apps from the Play store, and a Hulu listing for episodes of the TV show "Show Me Yours."
- Roku 3: The command was unable to find results.
- Amazon Fire TV: Displays a list of TV comedies that you can buy and watch through Amazon Instant Video.
"Show that Modern Family episode with Edward Norton"
- Apple TV demo: Jumps to the "Modern Family" homepage.
- Nexus Player: Displays results from YouTube for both Edward Norton and "Modern Family," clips of Norton from Hulu and a Google Now card for Modern Family, which gives you the option to buy on Google Play.
- Roku 3: The natural language command was unable to find results, but saying either "Modern Family" or "Ed Norton" separately will give you listings for both ("Edward Norton" did not work).
- Amazon Fire TV: Same as the Roku 3. The natural language command was unable to find results, but a search for "Ed Norton" and "Modern Family" separately resulted in a list of content for purchase from Amazon Instant Video.
(Said when in a movie or TV show): "Skip ahead 7 minutes"
- Apple TV demo: The video jumps ahead as requested.
- Nexus Player: The voice search feature didn't work while inside of Netflix, Hulu or the Google Play Movies app. Pressing the microphone button while in Google Play Movies only gave us the option to search for movies and TV shows on the Play store.
- Roku 3: Tapping the search button will exit the app and bring you back to the Roku home screen.
- Amazon Fire TV: Pressing the microphone button removed us from Amazon Instant Video and opened a new search on the previous screen.
"Show me some action movies"
- Apple TV demo: Shows a selection of top recent action movies, such as "Furious 7" and "Mad Max: Fury Road."
- Nexus Player: Displayed a list of action movies that could be bought through Google Play. It also showed YouTube videos with the keyword "action" and "movie" in them, along with Hulu results for the TV show "Show Me Yours."
- Roku 3: It was unable to search for genres. Both the "show me some action movies" and simply "action movies" commands turned up no results.
- Amazon Fire TV: The command "show me some action movies" turn up no results, but saying "show me action movies" provided a list of random movies and TV shows, such as "Curious George" and "Sex Sent Me to the ER," that could be purchased from Amazon Instant Video. The solo command "action movies" provided more accurate results.
"What are some new movies that are good to watch with kids?"
- Apple TV demo: Shows a selection of top recent kids movies, such as "Paddington" and "Big Hero 6."
- Nexus Player: Displayed movies for kids that can be bought from Google Play. It also showed YouTube videos with the keyword "kids" and "movies" in them, and for some reason episodes of "The Simpsons" on Hulu.
- Roku 3: The natural language command provided no results.
- Amazon Fire TV: The natural language command provided no results, although the command "new movies for kids" provided a list of kid-friendly films from Amazon Instant Video.
(Said when in a movie or TV show): "Who stars in this?"
- Apple TV demo: Shows cast list for "Paddington" (the movie showing in the background).
- Nexus Player: It was unable to search while inside an app. Pressing the microphone button takes you out of the app and will give you the option to search for movies and TV shows on the Play store.
- Roku 3: Pressing the search button takes you back to the Roku home screen.
- Amazon Fire TV: Pressing the microphone button would take you out of apps and launch a search on the previous screen. Amazon instead has their so-called X-Ray feature that provides information on the cast and crew for TV shows and movies on Amazon Instant Video. X-Ray is accessed by pressing the up arrow on the remote.
"How did the San Francisco Giants do yesterday?"
- Apple TV demo: Displayed the score of the most recent San Francisco Giants game.
- Nexus Player: Displayed a Google card with the score of the most recent San Francisco Giants game.
- Roku 3: The natural language command provided no results.
- Amazon Fire TV: The natural language command provided no results.
"What's the weather like in Juneau, Alaska?"
- Apple TV demo: Displayed an iPhone-like weather card with the weather in Juneau, Alaska.
- Nexus Player: Displayed a Google card with the weather in Juneau, Alaska.
- Roku 3: The natural language command provided no results.
- Amazon Fire TV: The natural language command provided no results.
"Open Music"
- Apple TV demo: Opened the Apple Music app.
- Nexus Player: Displayed a Google card that gives you the option to open or install Google Play Music. It won't actually open the app unless you use the "Open Google Play Music" command.
- Roku 3: There was no command to open programs. Saying "Netflix" or "Hulu" will bring you to the service page and give you the option to open it.
- Amazon Fire TV: Could not open a program with the command. It instead showed a list of TV shows and movies with the keyword "music" in them.
The takeaways
Let's again stipulate that the deck was totally stacked in Apple's favor here. In other words, it's no surprise that the Apple TV came off shining in what was essentially an advertisement for the product.
That said, the results still offer an interesting overview of how the battle of the streaming video boxes is going to shape up when it comes to voice control and ease of use.
The voice commands on the Apple TV seemingly (and unsurprisingly) leverage the high degree of sophistication that already exist for Siri on the iPhone and iPad. More than voice recognition, the Apple demo shows voice control, with specific action items that will be relevant to video playback and content search -- including context from within existing search results.
Likewise, it's unsurprising that the Nexus Player did well on some of the more challenging options like weather and sports scores -- because it's tapping the same "OK Google" functionality that works so well elsewhere on the Web. That said, the Nexus fell behind when searching for content.
The Roku 3 has an extensive search catalog, but it's unable to decipher natural voice commands. You are limited to searching for content based on an actor, director, movie/TV title or app name. Don't look for genres, let alone Apple's far more sophisticated video transport ("skip ahead") and smart rewind options ("what did he/she say" automatically skips back a few seconds and activates captioning on Apple TV).
The Amazon Fire TV had trouble understanding certain natural language commands and would either provide inaccurate results or no results at all. The company's X-Ray feature was able to display cast information in movies and TV shows, but only those on the company's Instant Video service.
So, the Apple TV got the edge -- from what we saw during the company's presentation -- when it came to natural voice commands, especially when used to search for new content. If it actually works, being able to find the exact episode of a TV show that a specific actor guest starred in is impressive to say the least.
Moreover, the Apple TV was quite adept at using the Siri voice control from any screen without interrupting the viewing. Videos continued to play in the background, and commands seemed available at any point. By contrast, hitting the voice search button on Roku will pop you out to a search menu and stop the action.
Streaming competition is ramping up
So, are Roku, Google and Amazon dead in the water? Hardly. The latter two have resources that rival Apple's, and strong voice control chops in other products -- adding the voice functionality in Google's Android phones and Amazon's Echo speaker to those companies next streaming products would go a long way to even the playing field. And with the Nexus Player and Fire TV box getting long in the tooth, it wouldn't be shocking to see new models of each before the end of 2015.
Roku, on the other hand, is likely to feel the squeeze a bit harder. We preferred its no-voice Roku 2 model because it delivered all the speed and power of the Roku 3 for an even lower price. But the Apple TV demo shows a next-level voice control feature set that far surpasses actor and title shortcuts. Roku may continue to thrive in the lower end of the market (the Roku 1 box and Roku Streaming Stick both start at just $50), and it derives licensing fees from companies like TCL, Hisense and Best Buy's Insignia line that build the Roku interface into their line of smart TVs. But the new Apple TV is finally co-opting some of Roku's best features, including a more open app store and searchability across multiple video providers, while upping the ante on usability through its Siri-powered voice control.
Stay tuned for our full review of the new Apple TV in the coming weeks, along with updated comparisons to Roku, Fire TV and more.
CNET's Iyaz Akhtar contributed to this story and its accompanying video.