X

How We Test VPNs

Here's what it takes for a virtual private network to be recommended by CNET's VPN experts.

Attila Tomaschek
Attila is a Staff Writer for CNET, covering software, apps and services with a focus on virtual private networks. He is an advocate for digital privacy and has been quoted in online publications like Computer Weekly, The Guardian, BBC News, HuffPost, Wired and TechRepublic. When not tapping away on his laptop, Attila enjoys spending time with his family, reading and collecting guitars.
Expertise Attila has nearly a decade's worth of experience with VPNs and has been covering them for CNET since 2021. As CNET's VPN expert, Attila rigorously tests VPNs and offers readers advice on how they can use the technology to protect their privacy online and
Attila Tomaschek
9 min read
030-vpn-generic-logo-on-laptop-security-2021

Here's how we test VPNs at CNET.

Sarah Tew/CNET

Whether you're using a virtual private network to hide your online activity from your ISP, bypass firewalls, avoid censorship or simply stream your favorite sports and entertainment, CNET's expert-tested, hands-on VPN reviews will help you quickly choose the best service for you. 

A virtual private network routes your internet traffic through an encrypted tunnel to a secure VPN server in a location of your choosing, keeping your online activity private and giving you broader access to the internet. With a VPN connection, outside entities like your internet service provider, network administrators, government snoops and hackers are unable to see what you're doing online. Because the VPN routes your encrypted traffic through a different location and hides your IP address in the process, you can bypass firewalls to access blocked sites on your network and unblock geographically restricted content online.

How we test VPNs

Comparing VPN services can be confusing, given the range of options and each one's specialized features and pricing structures. Choosing the right VPN is especially important if you have heightened privacy needs. That's why CNET has developed a detailed and rigorous VPN testing and review process to help quickly guide you toward a confident decision. All of the VPNs we review are meticulously evaluated on everything from features to usability, speeds, server network, privacy policy, ownership, jurisdiction, price and everything in between. 

We always incorporate hands-on testing, in addition to research and interviews with VPN representatives and experts, into our reviews. While it's a lengthy process, we stand by the time we put into our VPN reviews because that allows us to deliver an authentic assessment that's thorough, cuts through the hype and helps you make an informed choice. CNET's team of VPN experts includes Attila Tomaschek and Moe Long. Together, we have nearly two decades' worth of experience using, testing, researching and reviewing dozens of VPNs. We've run thousands of speed tests, written hundreds of articles and had expert commentary on VPNs and digital privacy published in various online publications.   

Our testing and review process takes into consideration the many reasons to use a VPN and focuses on what you need to know about the VPNs we review. All the VPNs we review are selected by our team of experts and are assessed with complete editorial independence. To be considered by our experts for a spot on CNET's best VPNs list, a VPN must meet our criteria for privacy, speed and value. It must also be easy to use, unblock various streaming services and be available on most major platforms. To earn an Editors' Choice designation, a VPN must consistently demonstrate exceptional capacity in one or more of those areas. 

How we assign VPN ratings

Our VPN reviews include numerical ratings of the services we evaluate. We assign an overall rating on a 10-point scale based on the VPN's performance in five subcategories: privacy, speed, features, value and usability. Each subcategory is rated separately on its own 10-point scale, and we calculate the average of the subcategory scores to determine the overall rating. 

We assign a discrete list of standards to each numerical score within the 10-point scale for each subcategory. For example, our speed ratings are based on the percentage of speed loss measured, so a speed loss of 11% to 20% would earn a score of 9, while a speed loss of 10% or lower would earn a 10 for speed. Whenever we re-review a VPN provider, we take a fresh look at its performance and its ratings can change (up or down) based on that evaluation.    

We weigh each subcategory equally because of the variety of ways and reasons to use a VPN. Weighing subcategories differently would unfairly favor certain uses. A journalist or activist in a region with oppressive online censorship and surveillance will have vastly different priorities than someone on vacation who simply wants to watch their Netflix shows from abroad. Both users have valid reasons to use a VPN, but while the former example would put a heavier emphasis on privacy, the latter would probably be more interested in speed and usability.

Our VPN ratings aim to provide an easily digestible view of the overall quality of the VPN at a glance. And ultimately, we hope that our rating system will help drive VPN companies to keep improving their services for their users by seeing where there’s room to improve. To find the VPN that best fits your needs, we strongly encourage you to carefully consider the sections within our individual VPN reviews that reflect your biggest considerations.

How we test VPN speeds

Using a VPN will slow your internet speeds. This can be a result of various factors, but it's mostly because the VPN encrypts your traffic and routes it through a different server location, both of which take time and contribute to the speed loss. Slow speeds can negatively affect things like streaming, gaming, video conferencing and even general browsing. VPN speeds are important for many activities, so one of the main things you'd want to know about a potential VPN service is how much you can expect it to slow you down.

VPN speed testing is an inexact science, but when we evaluate a VPN's speeds, we do our best to replicate the conditions in which the typical VPN user would use their VPN. With the help of CNET's Labs team (a dedicated team of data and product testing experts) in Louisville, Kentucky, we first measure our internet speeds on a residential Wi-Fi network without connecting to the VPN. This is to get a baseline reading for comparing the speeds we get through various VPN server locations.

Although we report on the actual speed numbers we achieve with and without the VPN, we focus mainly on the percentage of speed lost through the VPN connection. For example, we'd report a VPN cutting our speeds by 17% rather than just saying we got 345.23 Mbps through the VPN's German servers. Network speeds vary wildly, and the network(s) we're testing on may be significantly faster or slower than the network you're using. Framing speed performance in terms of percentage loss helps you translate the exact speeds to your internet connections. 

We use the popular speed testing platform developed by Ookla, a network testing firm based in Seattle, to run three rounds of speed tests using OpenVPN and another three on WireGuard (or the equivalent proprietary protocol, if applicable) on both Windows and MacOS. We run tests using OpenVPN and WireGuard because they are two of the most popular VPN protocols. We recommend OpenVPN for optimal privacy and WireGuard for optimal speeds.

Each round of speed tests includes three tests without the VPN, then three tests each to six different server locations: New York, the UK, France, Germany, Singapore and Australia. Including the baseline internet speed tests, we run a minimum of 252-speed tests on each VPN we review. Once we've collected all of our speed test readings, we take the averages and calculate the average speed loss. 

In addition to reporting the average speed loss, we highlight which protocols, server locations and operating systems delivered the fastest speeds. We also call out any inconsistencies or anomalies we detect. For example, we'll note in our review if we saw any major speed fluctuations from one test to another, or if we noticed significantly faster speeds to a distant location than we did to a location closer to our physical testing location(s). We'll also recognize VPNs that deliver consistent speeds throughout the testing process.

Many VPNs can cut your speeds by 50% or more, but we look for VPNs that cut speeds by 20% or less -- those providers can qualify for placement on our list of the fastest VPNs.

How we assess VPN value

When we look at a VPN's value, we consider the price of the service itself and the overall value it delivers through its features and performance. We take into consideration all of the subscription plan options available, but we generally don't recommend signing on with a VPN provider for more than a year at a time. Things tend to shift rapidly in the VPN industry, and performance can fluctuate as VPN companies consolidate, server networks expand or contract, policies change and features come and go. We also check which payment options are available and whether the VPN offers a money-back guarantee or a free trial (or even a free version). If a money-back guarantee includes any stipulations, we make sure to call it out. 

As with any product or service, the overall quality and performance of the VPN should be commensurate with the price you pay for it. This is why we consider every aspect of the VPN when evaluating its value, including its feature set, server network, number of simultaneous connections and ease of use. We check the VPN's help section for tutorials and help articles and we actively engage with the customer support team to assess how responsive and effective it is.

We test the VPN's streaming capabilities through various server locations on several different streaming services, including Netflix, Disney Plus, Hulu and Amazon Prime Video. We test the service on all available platforms and operating systems to assess its compatibility and consistency across platforms. If applicable, we test any “quick connect” feature to ensure it works as intended and connects us to an optimal server. We also look out for and actively test any unique features that the VPN may offer.

To be considered for a spot on CNET's list of the best cheap VPNs, the VPN must deliver a premium level of performance at a budget-friendly price.

How we evaluate VPN privacy

Because privacy is the often biggest selling point of a VPN, we pay special attention to and scrutinize everything about a VPN's privacy posture. Evaluating a VPN's privacy protections involves deep research, frequent communication with VPN representatives (whether via email, video conference or both) and meticulous hands-on testing. 

We carefully examine the VPN's privacy policy, with special attention to things like logging practices and data collection and sharing. Even though VPN ownership is often murky territory, we carefully research the company's corporate structure and history, making sure to flag anything potentially dubious. We want to make sure VPNs are as transparent as possible and that you know the entity behind the VPN you're entrusting with your online privacy. If anything is unclear or if we uncover anything suspect in the VPN's corporate structure or policies, we make sure to reach out to the company for clarification.

We look for things like third-party security audit reports and transparency reports -- how many have been published and how often? We also look for any court cases the VPN company has been involved with -- with special attention to cases where authorities have requested activity logs. The country from which the VPN operates is also an important consideration, so we research the VPN's jurisdiction to see what privacy laws are in place and whether the country is part of the 14 Eyes data-sharing alliance. 

At a minimum, a VPN should have a no-logging policy and offer AES 256-bit encryption, DNS leak protection and a kill switch. We put each VPN's kill switch feature through its paces to make sure it's doing what it's supposed to be doing and we run multiple DNS leak tests on various VPN servers using leak testing sites like ipleak.net. We also check to see whether the VPN still offers outdated and vulnerable VPN protocols like PPTP, and if so, we make sure to flag it because it can give users a false sense of security when using the VPN. 

We look for VPNs that offer a diskless, RAM-only server infrastructure as well as advanced privacy features like obfuscation, multihop connections, split-tunneling, ad/malware blocking and Tor over VPN. Whenever possible, we thoroughly test these features, along with any additional privacy features the VPN may offer that go above and beyond.

A VPN must offer excellent privacy protections, deliver stable connections and go beyond the call of duty in terms of transparency in order to earn its stripes as one of CNET's top VPNs.