Baseball pitches augmented reality to catch fans

Commentary: Apple and Major League Baseball’s At Bat app are adding AR so fans can watch a game within the game. Count me in.

Terry Collins Staff Reporter, CNET News
Terry writes about social networking giants and legal issues in Silicon Valley for CNET News. He joined CNET News from the Associated Press, where he spent the six years covering major breaking news in the San Francisco Bay Area. Before the AP, Terry worked at the Star Tribune in Minneapolis and the Kansas City Star. Terry's a native of Chicago.
Terry Collins
4 min read

Augmented reality is coming soon to a big league ball park near you. 


During Apple's iPhone launch two weeks ago, I spotted a 10-second augmented reality demo that could potentially change the way we watch baseball.

"What the [expletive] is that?" I said, as Apple marketing chief Phil Schiller teased a new AR feature in Major League Baseball's popular At Bat mobile app. He demonstrated how fans at a game could see special stats, including how hard a ball was hit, how far it was thrown or how fast a player runs in real time, simply by pointing their iPhone toward the action on the field.

Also watching the demo, my colleague Claudia Cruz yelled my name across the office to make sure we both saw the same thing. I emailed MLB spokesman Matthew Gould. "Let me get back to you," he replied.

A week later, Claudia and I found ourselves jockeying with other reporters in a crowded suite at AT&T Park in San Francisco, about a 45-minute drive north of Apple's spaceship campus in Silicon Valley. We were promised a peek at the new AR capabilities during an actual game between the San Francisco Giants and the Colorado Rockies. What we saw was an internal prototype. The goal is to debut the updated app in 2018. 

It's no secret Apple wants its augmented reality developer kit, announced in May, to be used to create apps for the iPhone and iPad that will prove more enduring than last year's Pokemon Go phenomenon. Now the tech giant is experimenting with At Bat, the most popular sports league app, to give diehard baseball fans like Claudia and me more data than we probably can consume. The AR feature may also get MLB teams looking to cheat -- er, um, gain an advantage -- to use their iPads during games.

Enlarge Image

Apple's Phil Schiller teases the company's plans to add an AR feature to Major League Baseball's At Bat app. 


"We want this to be fun for fans in the stands, to tell them something new about the game they can't see on the scoreboard or have to search hard for," said Chad Evans, a mobile product executive with MLB Advanced Media, baseball's interactive arm. "Everything is in play."

For non-sports fans, Claudia and I compare this AR feature to those Omnioculars used to watch Quidditch matches in the Harry Potter movies. "They will provide the user with a play-by-play breakdown if desired, showing the names of the manoeuvres performed by players," according to the Harry Potter Wiki page.

And that's sort of what the AR feature in the MLB At Bat app does. On Wednesday, Evans and Greg Cain, MLBAM's senior data director, pulled out iPad Pros and showed us how, by clicking on a player's augmented image, you can get his data from Statcast. The Statcast tool uses HD cameras, a Doppler radar and machine learning to track every move on the field.

Statcast gives teams, broadcasters and fans an array of metrics and trivia, including that New York Yankees rookie Aaron Judge's 495-foot home run is the longest this season. Or how his teammate, Aroldis Chapman, has thrown the season's fastest pitch at 105 miles per hour, with the ball spinning a dizzying 2,600 times after it left his hand. 

During the Giants-Rockies game, we grilled Evans and Cain on what stats are available. Could they tell us how hard Rockies star and National League MVP candidate Charlie Blackmon hits a baseball? It's around 86 mph (his exit velocity), compared with Judge's shots, which can reach around 117 mph. We also learned that Rockies catcher Jonathan Lucroy has a 28 "sprint speed," which means he's too slow to steal a base.

Too inside baseball? Developers are trying to strike a balance with the data:  They don't want to overwhelm casual fans, but they also don't want to bore stats freaks.

"We're going to try to create the right information at the right time. We just need to figure out how to do it," Evans said. "There's all of these data stories we are going to try to tell."

The demo left us hankering for more. Claudia wants to see instant replays in AR, as well as the weight, height and strength comparisons of players on the field. She also wants the app to anticipate when a record could be broken.

I'd like the feature to show me what a hitter's batting average is when he has two strikes and the bases are loaded. Maybe it could also tell me the probability of him hitting either a triple or a double in that same situation.

We could be asking too much, but they tell us being able to marry data with the world is the promise of AR.

"This is not a static product," Cain said. "We will continue to grow and build it out over time."

Solving for XX: The industry seeks to overcome outdated ideas about "women in tech."

Special Reports: All of CNET's most in-depth features in one easy spot.