On call: Why we haven't lowered the iPhone 3G's rating

When you review an Apple product, you can be sure that readers will say one of two things: either you didn't rate it high enough or you rated it way too high. Rarely, if ever, is there any middle ground.

Just take the review of the original iPhone, for example. Soon after we posted it last June, letters from CNET readers started pouring in. As veteran Apple reviewer Donald Bell had predicted, the letters fell into the aforementioned camps. However, after some time, I noticed that most readers felt we were too hard on the iPhone, which received an "excellent" rating of 8.0 (or four stars out of five). From our perspective, the rating was justified: although the iPhone was a gorgeous device with a fantastic display, a great user-friendly interface, a top-notch media player, and a breakthrough Web browser, it suffered from variable call quality and a lack of basic cell phone features. That's why Donald and I withheld the CNET Editors' Choice award.

Now fast-forward a year to our review of the iPhone 3G. Since the second-generation iPhone brought a host of added features and better call quality, we assigned a higher rating--an "excellent" rating of 8.3, which also equals four stars under CNET's new rating system. This time, however, we left the possibility of an Editors' Choice award on the table and decided to wait for full battery testing results to make that decision--on the first day of use, I had noticed the battery depleting too quickly while using the 3G network. So we posted our review and waited.

While we waited, I got a different set of letters. As concerns about the iPhone's dropped calls, battery life, and shaky 3G connections grew louder, I received many letters complaining that we had been too easy on the iPhone 3G. Readers asked me to reconsider the rating because of the problems that were affecting so many iPhone customers. Of course, the problems concerned me, but I wanted to see them for myself before I took action. (It would be irresponsible and a disservice to CNET readers if I adjusted the rating based solely on reports I had heard from other people.)

It took a while to put our review iPhone 3G though the battery drain paces and get it back from CNET Labs, but once I did, we performed more usability testing. And gradually, we did notice a few problems. Dropped calls on our phone remained rare, but the 3G connection issues were apparent. Specifically, I noticed the weak connection and the sloppy hand off between the 3G and EDGE networks. Also, while the official CNET Labs battery testing results fell within the promised times, our iPhone's battery came close to running out after a long day of heavy multitasking. I added the new observations to the iPhone 3G review and spent time questioning whether they warranted a rating change.

Ultimately, we felt the issues that we experienced on our review device weren't reason enough to reduce the iPhone's 3G's rating. We are, however, withholding the Editors' Choice award once again. We agree that these problems are significant, but the iPhone 3G remains a landmark product in many ways. It still offers a great design, the interface and display are no less lovely, the music player and Web browser remain top-notch, and we admire the App Store. Even the sometimes-frustrating Microsoft Exchange e-mail support merits a few points. But more importantly, we want to see whether Apple successfully fixes the problems, as the company said it is planning to do on Friday, September 12 with the 2.1 software update. If the problems continue to persist over the next few months, we'll revisit the issue.

Kent German, CNET's cell phones guru, answers your questions about cell phones, services, and accessories and reports on the state of the industry. Send him a question.