Texas-size tech behind Super Bowl stadium
Cowboys stadium is home to the NFL title game, and it takes lots of gear to keep a Jerry Jones-size event like a Dallas Super Bowl running smoothly.
ARLINGTON, Texas--In my role as a reporter, I've had the good fortune to visit a number of control and/or command centers, such as those running a massive radio telescope, a nuclear submarine, a national laser fusion facility, and several others.
For anyone who remembers the Matthew Broderick vehicle "War Games," the bar for what a control room looks like is high: massive screens, dozens of workstations, long tables, and people moving around everywhere. But in the commodity PC era, that kind of room is mostly long gone. While I've seen
That's why I was so pleased Tuesday when I was ushered into the internal video control booth at Cowboys Stadium here, the gargantuan home of the Dallas Cowboys and, oh yes, this Sunday's Super Bowl XLV between the Pittsburgh Steelers and the Green Bay Packers. While not up to "War Games" standards, this dimly lit space was a jumble of video screens, control panels of all kinds, and dozens and dozens of different video feeds. It was a feast for the eyes.
This room, tucked away high above the Cowboys Stadium field but still well under its cavernous ceiling, is where technicians will coordinate all the video that fans, players, coaches, and others on the field during the Super Bowl will see. TV networks do their video production magic in trucks outside the stadium. Here, a crew will be working overtime to ensure that the stadium's mammoth HD video board--the world's largest, by the way--plus smaller screens, long LED message boards, and more--are always displaying the right thing.
And, according to Cowboys head of technology Pete Walsh, this is no easy task. Indeed, showing the game live on the $40 million, 72-foot-high, 162-foot-long Diamond Vision boards, which weigh 600 tons, and which include 30 million light bulbs and 25,000 square feet of total display space, requires two separate productions: one for one side of the board, and--to make sure live video playback of a great touchdown run or some other play doesn't appear to be going in the opposite direction of the actual players on the field--another for those watching the DiamondVision on the other side of the field.
105,000 people
Cowboys Stadium, the $1.2 billion baby of the team's famous owner, Jerry Jones, is pretty much a testament to everything big, including the latest technology that can be used and presented to help streamline the experience of everyone on-site for a game.
Yet most of this technology will never be seen by the vast majority of the 105,000 people who will be on hand for the Super Bowl, or who visit for Cowboys games or other events here.
It all starts with a state-of-the-art data center that still manages, with a large Cowboys star, to conjure up enthusiasm for Dallas' famous football club. At its heart, the data center features an architecture of 128 Hewlett-Packard blade servers, a 10TB HP storage area network, and a passel of Cisco routers. This infrastructure is responsible for keeping just about every system inside the giant building running smoothly, be it lighting, concessions, security, heat, or the famous retractable roof.
"Everything you touch, see, smell, taste, or hear" inside the stadium, said Lance Caserotti, the solution architect from CDW who led the technology integration in the 2-year-old stadium, "will either come from the data center, or has some impact on the data center."
The idea, Caserotti said, was to build an infrastructure that could help bring the best possible experience to each and every fan.
The project began, Caserotti said, when the Cowboys called CDW and asked the company to come and help design the technology setup for the team's new digs, which were set to replace the 40-year-old original Cowboys Stadium.
Over the next 18 months, CDW and the Cowboys worked on designing and installing the new system, and as Walsh explained during a meeting with tech press during Super Bowl Media Day festivities Tuesday, the technology has been tested and tested, at Cowboys games, at the recent Cotton Bowl, and at the NBA All Star Game. But the Super Bowl will be real the main event.
To begin with, the Super Bowl should put the biggest possible strain on one major component of the system, the wireless network that is intended to make it possible for each and every person on hand to send a picture, a text, or a video of, say, the championship game's kickoff, the halftime concert by the Black Eyed Peas, or the end of the game. With 885 wireless access points spread throughout the stadium, the idea is to make it possible to get online from any mobile device anywhere inside the facility.
"All the carriers realize [there will be a few specific spikes in demand on their networks]," Walsh said, "and they'll be monitoring that. But I think we've got enough bandwidth. They've pulled in more capacity just for the Super Bowl."
One of the problems that the Super Bowl presents team, stadium, and CDW officials with is that the demographics of attendees will be different than a normal Cowboys game, and that means demand on different cellular carriers is hard to predict. Walsh explained that the Dallas area is dominated by AT&T--and a majority of Cowboys ticket holders seem to be iPhone users--but the same is not true of Pittsburgh and of Green Bay, Wis.
That means, Walsh said, technicians inside the stadium will be keeping a close eye on which networks Super Bowl fans are using, and attempting to tweak the back-end infrastructure to ensure that carriers like Sprint, Verizon, and others can handle what might be very large demand. Walsh estimated that because so many people will be coming from Pittsburgh and Wisconsin, Verizon will likely be the most-used network during the Super Bowl. But it all "depends on the corporate guests," Walsh said.
Of course, because Wi-Fi at Cowboys Stadium is provided by AT&T--and is free to AT&T Wireless subscribers--the team and the service provider are hoping that a large number of people will choose to get online via Wi-Fi and reduce the demand on the carriers' mobile broadband systems, Walsh said.
No matter how much planning has been done, though, there's no way to know the network will perform until Super Bowl Sunday. "Until you get the bodies in here," Walsh said, "it's all theories."
But regardless of how the wireless system holds up, the first people to know will almost certainly be the ones in the network operations center, just adjacent to the main data center. Here, technicians can monitor a series of data screens that show, in real-time, the performance of the complex wireless network.
Among the data that can be seen is a map showing each access point on any level of the stadium, and for each, how many people are connected, and how the AP is performing. If it's green, it's doing fine. If it's yellow, it's worth watching. And if it's red, there's a problem, Walsh explained. It's also possible to see what kind of devices are being used to access the network at each point, as well as how much bandwidth is being used at any given moment. If something serious happens, the system--or any individual access point--can be shut down in seconds.
This map will be a crucial tool for the technology team on Sunday, Walsh said, because it will allow technicians to see, via the connectivity to each of the access points, the flood of ticket holders streaming into the stadium. That's because each ticket will be scanned with a wireless scanner. And because most ticket holders will be coming in on the east side of the stadium, it will be possible, Walsh said, to evaluate the system's performance right from the get-go by seeing how well the APs hold up to the assault by ticket holders.
Patent pending
Though the Super Bowl could well be the biggest event to hit Cowboys Stadium for years to come--no small statement given that this is Jerry Jones and the Cowboys' house--there is even more tech already on the map for the next year or so.
According to Walsh, the next big thing to hit the stadium--and all those 885 access points--is a series of mobile apps that the Cowboys hope will encourage all kinds of transactions at the 212 concession stands and 665 point of sale terminals located throughout the venue. The idea, Walsh said, is to release a series of mobile apps that will leverage the latest location-based services technology in order to push special deals on, say, beer or soda or hot dogs or team apparel to fans, based on where those fans are in the stadium.
The technology behind pushing out these kinds of offers is complex enough, Walsh said, that the Cowboys have applied for patents on the distribution system that would control them.
At the same time, the Cowboys are also hoping to release an art tour app that would lead ticket-holders on a self-guided tour of the many art pieces installed in the stadium by Jones' wife.
10TB of storage
A big part of the overall tech infrastructure, said Caserotti, is the HP storage area network. With 10TB of capacity, he explained, the system was built to be future-proof, at least for a few years.
Asked what the stadium data technicians needed that much capacity for, especially when he pointed out that a single terabyte is the equivalent of 1,000 Encyclopedia Britannicas, he explained that the many different operations inside the venue require massive amounts of data transfer, analysis, and storage.
In addition to having to process and analyze data coming in constantly during events from the 212 concession stands and 665 point of sale terminals--so that, for example, customers can order a hot dog wirelessly from their seat and pay for it electronically--the storage system also is responsible for all the team's data, as well as for that of 38 of Jerry Jones' other businesses.
The system is also designed, Caserotti said, to be able to provide the Cowboys with real-time statistical analysis of a game in progress. The idea? To give the coaches the best possible odds on what kind of offensive play the visiting team will run next based on what they've run so far and what they've done in past games. And while this is obviously not going to be infallible, the Cowboys clearly hope that this digital tool will give them a leg up on the competition.
As for Caserotti's view on opposing teams' efforts to figure out the Cowboys' next play? "They're going to have to use their brains, I guess."