Why companies test phones in the wild
Though it may sound unusual that a company like Apple would allow unreleased cell phones to leave its headquarters, there are good reasons for doing so.
As CNET reported yesterday, a second unreleased Apple iPhone has gone astray. The secret device was lost at a San Francisco bar in late July, CNET reporters Declan McCullagh and Greg Sandoval wrote, and Apple quickly conducted an aggressive campaign to find it.
If it sounds familiar, that's because it is. Remember that last year, an Apple computer engineer accidentally left a prototype
If you're surprised that it could happen again, we don't blame you. Nor do we blame you for wondering why an Apple employee would even have an unreleased device out in the wild at all. On the surface it does sound crazy, but Apple, not to mention wireless carriers and other handset manufacturers, have good reasons for doing so. Here's why.
Even with the risks involved, Apple can't reliably test a new cell phone without a few devices leaving the company's Cupertino, Calif., headquarters. Keeping the handsets under lock and key may sound safer, and it no doubt would be, but Apple can do only so much in one place. It can test the media player, it can capture photos with the camera, and it can evaluate the general performance of iOS in an office or lab, but it can't fully evaluate the phone's most critical features like call quality, the strength of the cellular connection (for both voice data data), and the GPS connection.
Inside
As I explained after I visited Sprint's testing labs in Overland Park, Kan., in July, testing a phone's antenna and cellular connection involves two parts. First, the device is tested inside a sophisticated RF (radiofrequency) chamber to ensure that its antenna can connect to a carrier's network, that it operates properly on the network, and that it doesn't interfere with a carrier's other handsets.
The chamber itself is designed to block out all external signals so that technicians can get a "pure" result. Like Sprint, Apple has such a chamber at its headquarters, which it showed to a select group of journalists last year. Later, additional tests are performed in other specially designed rooms to check the sound clarity and quality.
And out
All the data from the above tests is then analyzed and collected. And if problems arise, designers and engineers address them. But as useful as the information can be, tests that simulate "real-world" usage conditions are equally important. For example, engineers have to evaluate how the phone performs in environments where other signals--from radio signals, to Wi-Fi, to other cell phones--are present. Sure, you could find such a place in the middle of Apple's cafeteria at lunchtime, but it's a safe bet that Apple's entire campus is wired for great cell phone coverage. So the test wouldn't be very fair.
As a result, companies like Apple need to find those areas where coverage isn't fantastic and collect more data. Is the phone pinging the tower correctly? How is the "hand-off" between different towers and different networks (EDGE, 3G, etc.)? Is it putting an unusual strain on the network? Is it sending and receiving voice and data properly? What are the data speeds and voice quality? Is it suffering from any interference or static? All of these questions need answers.
What's more, even without other wireless equipment around, engineers need to evaluate the phone's performance in areas that just have a poor signal quality. Though engineers can mimic poor strength in a lab, they also collect performance data from real places where real people use the phone.
Finally, real-world locations are important to test features like Bluetooth, GPS signal strength and accuracy, and the hundreds of apps that interact with both features. Here again, that information isn't reliable if you only collect it in one place. So Apple has to gather a range of locations, from a conference room in Cupertino to a bar in San Francisco.