Testing the iPhone 3GS (or why the phone gods hate product testers)

Last year I had a bone to pick with Apple for not including a call duration feature on the iPhone. Not a big deal to most people I'm sure; however, when it comes to testing talk time here in CNET Labs, I can't think of a more useful feature.

With last year's iPhone 3G, I was able to circumvent this inconvenience by tracking the phone's "Time since last full charge" duration. This number, as you might have guessed, tells you the amount of time that's passed since the phone was last fully charged. So, all I had to do was charge the phone fully, start the test, wait until the battery died, plug it back in, and check that number.

This worked fine last year, but with the current round of iPhone 3G and 3GS testing, I'm getting extremely varied results--so varied, in fact, that I can no longer trust this method.

No matter, though, since the latest iPhone OS 3.0 added a call duration feature. I figured I'd just set up the talk time test--which involves taping earbuds from a MP3 player to the mic of the iPhone and a landline phone, calling the landline phone, and waiting for the battery to die. Then I'd come back several hours later and have delicious results waiting for me (kind of like an iPhone-testing-results Crock-Pot).

Once again, I'm slapped back to reality as, unfortunately, the couple of times I've tried using the call duration feature on the 3G and 3GS I've had little luck. Apparently, if the phone's battery dies in the middle of a call, the duration of that call isn't saved on the phone.--yet another roadblock.

Barring sitting around monitoring multiple phones (3G, 3GS, and Palm Pre), waiting 10 to 12 hours for them to die--and hoping I'm around when that actually happens--my only alternative was to log on to AT&T's wireless site and access the call durations from there. Of course, nothing can ever be that easy. As it turns out, we only have the account information for one of the phones. Apple has told us that it either doesn't have or can't access (they weren't very clear) the account information for the 3GS.

So, as suggested to me by our resident iPhone expert, Kent German, I decided to switch the SIM card of the 3G (which we do have the account information for) and use it for the 3GS. Although testing with this method will take twice the time, at least the results will be accurate.

None of this would not be so bad if, in addition to talk-time testing, I didn't have to test the phone's Web site loading performance (through 3G and Wi-Fi); battery duration during music; and battery duration during video--each in 3G and airplane modes.

Also, we run each test at least two times to ensure accurate results. In addition, the music tests can last up to 35 hours or more. And the video tests, which last from 8 to 12 hours, require me to be present the whole time thanks to the iPhone's lack of a video repeat feature. I'm required to come back and restart the movie every two hours. I can track the audio, no problem, thanks to the device mentioned here.

Here's hoping sometime next week, all testing will be complete and I'll be free from iPhone testing...at least for a year.