It's a cloudy day in San Francisco, and I'm standing on the corner of 9th Avenue and Kirkham trying not to be blown over by the biting wind. A man named Randy hops out of the rented Chrysler and opens the minivan's trunk to reveal two large art portfolios, both containing four top-of-the-line Samsung Galaxy handsets.
No, this isn't a shady Craigslist deal, or some undercover operation with the cops. Instead, I'm joining RootMetrics -- specifically, a scouter (the aforementioned Randy Dosher) and the company's vice president Julie Dey -- as they ride around the city independently testing the country's wireless carrier networks. (Back in 2013, CNET's Jessica Dolcourt joined them on a similar ride-along, too).
Based in Washington state, RootMetrics sends scouts all around the US, UK and Canada to test the coverage of major phone networks. For the US, that means Verizon, AT&T, T-Mobile and Sprint. In addition to measuring download and upload data speeds, it tests calling performance (how stable the call is and if there are any dropped connections) and texting performance (the speed it takes to send a text) too.
The company conducts these tests many times throughout the year, in all 50 states, including 125 of the most densely populated areas and the 50 busiest airports. Its reports are also cited by carriers themselves (usually when the results are favorable of course).
In the latest round of testing for the second half of 2015, Verizon scored the best in overall performance nationwide. It also lead in network reliability, speed, data and call reliability.
But don't feel too bad if you're using the other carriers. AT&T often comes in as a close second in the same categories, and it tied with Verizon for first place in texting performance.
While T-Mobile doesn't usually win on a national or statewide scale, Dey says it does notably well in high-population metro areas. As for Sprint, it is third in overall performance, but it's making significant headway for its LTE coverage. It also came in second place, beating out AT&T, for call performance.
These rankings are calculated from the tests that Rootmetrics scouts conduct. On a regular day, a scout heads to one of the predetermined locations set on a GPS, and uses handsets that typically clock in some of the best test results with each carrier. During our ride-along, Dosher was equipped with a Galaxy S6 Edge for Verizon, two Galaxy Note 5s for AT&T and T-Mobile and a Galaxy S6 for Sprint.
Back on the corner of 9th Ave., Dosher is about to enter a local grocery to test the networks. He leaves one set of phones running in the van parked outside and heads inside the grocery with the other set. He walks around the aisles, mimicking the movements of a casual customer walking around the store. When I tell Dosh that seeing someone walk around with a suitcase looks a little suspicious, he says that for the most part, no one really gives him any grief. If they do, he simply explains he is carrying equipment for work.
Meanwhile, both the phones he is carrying and the phones outside in the van are making calls, sending texts, and measuring download and upload rates. The data is then beamed back to their servers in Washington where they crunch the numbers. At the end of this testing trial, Dosher buys a bottle of Gatorade. He says he often buys something small as a sort of gesture of gratitude for milling around these businesses and buildings.
I was curious to see what networks had the fastest data speeds in CNET's own building in downtown San Francisco. Based on the above info, I assumed Verizon would be the leader, and while it did score the fastest median upload speed (with T-Mobile and AT&T following behind), that wasn't the end of the story. T-Mobile dominated every other carrier by clocking the fastest download rates, with Verizon and AT&T coming in second and third. Sprint came in fourth in both tests.
Of course, our office is just one location and the tests were done at a single, random point in the day. The best network for our building, city, or even state may not be the same where you work or live, and these numbers just offer a tiny glimpse of each carriers' coverage as a whole. For those who live around the area, you can use these numbers as a guideline when determining the fastest carrier, but don't consider it absolute gospel.
Knowing what network to choose is difficult -- especially when you consider pricing and family plans and data buckets and throttling. But when it comes to performance, it's nice to know there's a third-party out there like RootMetrics who are sending out people, hitting the pavement, and doing the dirty work.