Like all good reviewers, CNET editors rate phones based on specific criteria, including its design, performance and camera quality, in addition to how well it compares to other devices in its field. Here, we walk you through our process for scrutinizing any phone we review. (And be sure to check out CNET's cell phone buying guide if you need more tips.)
Design and usability
First, we consider what type of device we're looking at, and what it's for. We mostly see smartphones these days -- and the occasional simple device like a flip phone -- but there are other specialized features we may need to consider, like resistance to water.
We take in the phone's physical attributes -- its size, weight, thickness, and the materials used to construct it. In general, handsets crafted from metal and glass feel more premium. Of course sophisticated fabrication techniques such as molding a phone from a seamless slab of sturdy plastic can lend an air of luxury as well.
Next, we examine the screen, accounting such things as screen size, resolution, brightness, contrast, and color accuracy. Button layout plays a big role in a phone's usability, as do any special software enhancements (or hindrances) that manufacturers add on top of its stock interface. These points may all sound like minor quibbles, but they add up to really affecting whether you form a true bond with your handset.
Speed and performance
For serious smartphone shoppers, nothing quite delivers purchase validation, or bragging rights, like a handset's processor performance. These days, octa-core chips are king, whether made by CPU specialists such as Qualcomm or MediaTek, or built in-house by handset manufacturers like Samsung.
To scientifically measure a phone's raw computing prowess, we run synthetic benchmark apps designed to test processor speed. Android smartphones make up majority we review. For those, we use the following diagnostics: Geekbench 3 gathers single-core and multi-core results that we often present alongside other phones in the contender's class. We're more interested in how the scores compare than we are in the test's raw result.
We also run 3DMark Ice Storm - Unlimited, a graphical and gaming benchmark we also use for tablets. Gaming is such a large stressor and performance indicator, this test lets us predict how the phones will respond to a range of graphically-intense and resource-heavy titles.
Another test we use is the tried-and-true Quadrant benchmark. We typically run the full form of Quadrant Standard Edition, which challenges a device on multiple levels, these being subsystems for memory, both 2D and 3D graphics, input and output, along with CPU.
Devices can be manipulated to yield fantastic diagnostic results, which is why real-world tests form an important part of the review. Editors play a graphically-demanding game of their choice that they know well, paying attention to frame rate and graphical quality. For example, a budget phone might have a terribly low score, but decent gameplay regardless.
For the iPhone, Windows, BlackBerry OS and others, we may need to adjust the diagnostic tests we use, depending on availability at the time of the review.
A cell phone's battery life is just as critical, or even more critical, than its processing power. To get a handle on a phone's longevity and how much staying power it offers, we take a few approaches. On devices, we measure how long a phone can loop an HD video before it dies. To eliminate as many variables as possible, all wireless radios are switched off and volume and screen brightness levels are set to 50 percent. The screen remains always-on.
The other option is to log a handset's talk time over its cellular connection. This test is handy for smartphones that can't run a video files, like a simple feature phone. Scores for both tests are reported in runtime by hours.
Editors test battery at least twice during the length of the review process, sometimes three or four times if results are varied.
Since phone owners don't exclusively watch video, editors also weigh in on anecdotal evidence, the battery life they witness during their review process. Typically, phones last a work day without needing a charge, but reviewers will point out if battery life is especially impressive or poor.
It's the common joke: "Who makes calls?" Yeah, yeah, text-based communication trumps good ole' voice conversations, but we're still going to make sure you and your caller can be heard when you do need to dial.
CNET is a global site, and our readers use hundreds of carriers all over the world. While it would be impossible to predict just how a phone will sound on your specific carrier in your city (call quality can even fluctuate depending on the time of day), we can and will point out if a call sounds especially good or terribly wrong.
To test a device's call quality, cell phone calls are made in various indoor and outdoor locations to a landline, if possible. Editors will take note of volume level, distortion and clarity.
As any smartphone user can tell you, fast data speeds are essential for a satisfying experience. So to give you an idea of what to expect, we run tests on the fastest available network (LTE, 3.5 HSPA+, or even 3G) that the phone uses.
As a diagnostic test, we use the SpeedTest.net app by Ookla, a free download for iOS, Android, and Windows platforms, to measure download and upload speeds. We list scores for the test in units of megabits per second (Mbps) and complete at least five test runs from the same location. And if time allows, we'll also conduct more tests in other locations.
Editors may also note real-world experience, especially if the phone tends to often run into problems throughout the testing period.
Photo and video quality
How well a handset snaps pictures has become a crucial factor in making a buying decision.
At a basic level, any camera consists of three main components: a lens, a sensor, and the electronics that coordinate the whole show. The best phones, and cameras in general, quickly capture correctly exposed images with natural and lifelike color. They can also deal with myriad lighting conditions, be it low light indoors, weak winter sun, or strong backlighting with silhouetted objects. A phone should also not be fazed by fast-moving subjects like athletes, children or pets.
With every phone, we take at least three photos on automatic mode: one of an indoor studio still life, one outdoors in daylight, and another indoors under everyday office or home conditions. If a handset boasts a full-featured camera app with plenty of extras such as a wide range of shooting and scene modes, plus special color filters, that's icing on the cake.
When a phone introduces a new hardware or software feature, especially one that comes with a particular claim, we test that addition as much as possible. For example, new mobile payment system or wireless charging addition -- so long as we have the tools (i.e., local vendors accept the mobile payments).
Star rating and scoring
To complement the written review, an algorithm calculates the final phone score based on these sub-ratings: design, features, camera, performance and battery life. Although it's infrequent (and occurs only after much discussion), editors have the ability to slightly adjust the calculated rating to take into account special features, or perhaps an extra credit bonus if, say, the handset is worth more than the sum of its parts.