Red alert! Voice commands make this Star Trek game even more fun
Thanks to IBM Watson, the Star Trek computer in this game can now recognize your voice.
On the right armrest of my captain's chair, on the bridge of the USS Aegis, a button glows tantalizingly red. One press, and my crew will go to battle stations, raising shields and arming photon torpedoes.
But what's the fun in that? Everyone knows real Starfleet captains use their lungs to issue commands -- and starting today, Ubisoft's VR game Star Trek Bridge Crew will let you do just that.
"Red alert!" I shout into the microphone. The crew obeys. A wide grin spreads across my face.
Bridge Crew is a VR game. Players can look around and interact with their stations in 360 degrees.
The coolest part? I don't need to memorize hyper-specific phrases -- because Ubisoft teamed up with IBM Watson, the closest thing we have to a real-life Star Trek computer, to process my voice in near real-time.
Arriving today for the PlayStation VR, HTC Vive and Oculus Rift versions of the game -- but only in English for now -- the new voice commands aren't something you'd necessarily use every time you play.
They control your AI crew members -- but optimally, you'll be playing Star Trek: Bridge Crew with three other humans serving as your Helmsman, Tactical Officer, and Engineer. The game's at its best when the crew is bantering and bickering with each other, shouting famous Star Trek phrases like "I canna change the laws of physics, Captain!!" at the top of their lungs, and role-playing hilariously unreliable Starfleet officers who deserve to be sent to the brig.
But if one of your human friends is MIA (real-life responsibilities, perhaps?), the new IBM Watson integration should make for way smoother sailing.
Instead of having to teleport between crew-member stations (eh) or robotically issuing instructions to AI teammates from a menu, you can just hold down a button (Touchpad Up on Vive, Move Button on PS VR, or A on Oculus), then shout out your instructions.
How it works: When you let go of that button, the game transmits your voice to IBM Watson's servers in the cloud. A speech-to-text engine converts your voice into readable language, and then Watson's Conversation API takes over to figure out what you said. It filters out errors and checks your words (and their likely synonyms) against a database of likely commands.
Just say "Prepare for warp," and the crew will line up a destination and charge the warp coils. Then channel your best Jean-Luc Picard and Make it So.
In the case of Star Trek, the team had to build a custom recognition model to handle combinations like "Fire photons," because they don't sound like standard English. "That's not what people say, unless they're watching Star Trek," explains Michael Ludden, director of IBM's Watson Developer Labs.
Though the process sounds complicated, there's barely a delay between the time you speak and the time the game reacts. Plus, Watson did a stellar job of letting me add flavor to my voice commands: you can say complex phrases like "Ensign, give me full speed to the impulse engines" even if "full speed" is the only thing the game actually needs to listen for.
(Ubisoft added some must-have Trek phrases to the dictionary, too: you can also totally pull a Picard, saying "Make it so" or "Engage!" when you want to go to warp, or respond to a hail with a mere "On screen.")
Sadly, you can't quite play the entire game solo without lifting a finger just yet, because "Plot a course to the Vega system" and "Lock phasers on the Intrepid" don't work. Until or unless Ubisoft adds all the proper nouns, you'll have to manually pick waypoints from the Captain's console.
But even if Ubisoft doesn't go further with Star Trek voice commands (the game's creative director, David Votykpa, won't commit to more just yet), the IBM Watson team sees a promising future for voice control in virtual reality.
Using IBM's open-source VR speech toolkit, it only took a single Ubisoft engineer to integrate the feature into Bridge Crew -- and a week for the first proof-of-concept, says Ludden.
"I wanted [Star Trek] on board as a showpiece, to show the art of the possible, and I hope we can tie it back to the call to action: hey, you can do this too, AR/VR developers."
Ludden says the IBM Watson team is already working with a couple of game developers focusing on "Telltalle-like games for VR" with "story-driven narratives," and also sees voice playing a bit role in VR training simulations for surgeons, baseball players and more. Some anecdotes he brought up in our conversation:
- "Right now you have to point and you have to click and you have to highlight and you have to pick a menu option... If you're a site inspector, would you click on something? You'd say, 'Close that bandsaw.'"
- "What if you could say 'Can you play that pitch back in slow motion?' or 'How fast was that pitch?' How much better is that if you're actually training?"
- "With surgery, you actually have to talk to your assistants. 'Scalpel, please.' That's a skill you should learn. That's something that should come out of your mouth."
And for VR art, with applications like Tilt Brush: "What if I could literally say 'give me the brush I had two brushes ago, make it purple and two points larger' and have it pop into your hand?"
"My view is voice is potentially always useful in certain circumstances in VR," says Ludden. "It's almost never useful in every circumstance."
Disclosure: Senior Editor Sean Hollister's wife works for Facebook, owner of Oculus. Also, CBS, owner of Star Trek, is CNET's parent company.
Does the Mac still matter? Apple execs tell why the MacBook Pro was over four years in the making, and why we should care.
Tech Enabled: CNET chronicles tech's role in providing new kinds of accessibility.