My day job and personal obsession is network and application performance for a very large global corporation. Key factors that affect your speed test performance:
1. Latency, or PING time. The higher the PING time between you and the test machine, the more important TCP tuning becomes. Speedtest.net does a good job configuring their test machines. If you see slower scores for higher PING times, it's time to improve your TCP tuning. See below.
2. Your broadband provider's peering arrangements. There are many carriers on the Internet, they pass off traffic at peering points, which are like highway intersections. Doesn't matter how many well-paved lanes both highways have if the cloverleaf intersection (peering point) is a rutted mud road. If traffic flow is uneven - more traffic flowing into Verizon's network from that other carrier, for example - then the peering point can become a p*ssing contest, where traffic is throttled while payments are debated. Found interesting results some years back, when Verizon wanted to punish customers for streaming off the reservation. There were Speedtest.net test points that gave good readings weekday mornings, but lousy results when the peering point was saturated that same evening. Try your test on a weekday morning to see if this is an issue with your carrier.
3. Your broadband carrier's network between that peering point and your house. Hotels and airlines overbook - sell 105 seats on a flight with 100 seats - since there will always be a few no-shows or last minute cancellations. Similarly, broadband carriers oversubscribe. They may have only 1 Gbit/sec capacity to your neighborhood, while total subscriber download capacity is far higher. They figure that very few people will use high bandwidth at the same time. However that link can get saturated during the evening. Continuing streaming growth, at ever-higher resolution, aggravates congestion.
4. Your home network. Most accurate reading is via Ethernet cable between a LAN port on your broadband router and your machine. Once you have that number, disconnect the Ethernet and test wireless. Then move further from the wireless access point and test again. If all readings are the same, you've got a fabulous home network. Otherwise, understand you may have "below rated" speed when you use wireless from your easy chair.
5. TCP tuning. Even Windows 10 has sub-optimal TCP tuning as installed. Macintosh is better. Only recently has iPhone TCP tuning allowed real high download speed. TCP auto-tuning is nice, but... your system has to be able to expand buffersize to 10 MBytes or larger. Bandwidth-Delay product isn't even table stakes, there's a lot more latency than just PING. Plus a few other knobs.
6. Oh, and not all speed test sites are created equal. Too-modest TCP tuning on some speed test machines means that remote users will never get good numbers. The speed test machine may have inadequate bandwidth, too-small memory, insufficient CPU power or use congested peering points. For example, CNET used one machine to cover the US, guaranteeing a lot of high PING time tests. Their machine wasn't tuned well enough to compensate for that high latency.
As another post noted, your broadband vendor's on-network machine will give best-possible results. Paying staff to handle bandwidth complaints costs more than giving the test machine(s) endless bandwidth, robust configuration and "turn it up to 11!" TCP tuning.
Sorry to run on, but there's a lot of slow between your keyboard and an Internet bandwidth test site.