Speed Tests Do Not Tell You How Well Your ISP is Doing

Does your smartphone have an Ethernet port? Do you own spare Ethernet cables? Do you own a port converter to connect Ethernet to your smartphone?

And if you do run speed tests on your PC, do you use Wi-Fi or direct connect using Ethernet? All those questions matter because they essentially invalidate all the home broadband speed test data we see so often. 

Testing your smartphone’s “speed” when connected to Wi-Fi only tells you the bandwidth you are getting from that device, at that location, for the moment, over the Wi-Fi connection. It does not tell you the actual speed delivered to your home broadband location by the internet service provider. And the home broadband speed enabled by the ISP can be as much as 10 times higher than the measured speed on your Wi-Fi device. 

Methodology matters. 

The Central Iowa Broadband Internet Study, for example, conducted in the first half of 2021,  illustrates many issues faced by rural households as well as the testing methodology issue. 

 In rural areas studied, some 27.5 percent of internet users had some form of non-cabled access--satellite, fixed wireless or mobile. 

Area wide, 42 percent of download speed tests failed to reach 25 Mbps, the study says. The number of town/city respondents failing to meet the threshold was about 32 percent. In rural areas the percentage of tests delivering less than 25 Mbps was about 64 percent. 

But the study also suggests a big methodological problem: speeds delivered by the internet service provider likely were not tested. Instead, respondents likely used their Wi-Fi connections. And that can mean underreporting the actual speed of the connection by 10 times. 

To be sure, that same problem happens with almost every consumer speed test data, as most such tests use Wi-Fi-connected devices. 

The point is that ISP delivered speeds quite often degraded by performance of the in-home Wi-Fi networks, older equipment or in-building obstructions. Actual speeds delivered by the internet service provider to a router are one matter. Actual speeds experienced by any Wi-Fi-connected device within the home are something else. 

source: CMIT Solutions 

One important caveat is that speed tests made by consumers using their Wi-Fi connections might not tell us too much that is useful about internet access speeds. In other words, consumers who say they do not get 25 Mbps on their Wi-Fi-connected devices could well be on access networks that actually are bringing speeds 10 times faster (250 Mbps) than reported. 

Of the respondents reporting they use a non-terrestrial (cabled network) for home broadband, 41 percent used a satellite provider. Some 30 percent used a fixed wireless provider and 29 percent reported using a mobile network. 

Only about 1.5 percent of survey respondents buying internet access reported they use a non-terrestrial provider for internet access. About 6.7 percent of survey respondents said “no internet service is available at their home.”

“The average download speed recorded was 80.7 Mbps, but the median download speed was just 34.0 Mbps,” the study reports. 

The median download speed for city/town respondents (101.6 Mbps) was three times higher than the median speed among rural respondents (34.0 Mbps), the study says. 

Keep in mind, however, that the speed tests likely were conducted over a local Wi-Fi connection, the study says. That matters, as speed actually delivered to the premises quite often is significantly higher--as much as an order of magnitude--than the Wi-Fi speed experienced by any single device within the home or business. 

Complain all you want about map inaccuracies. The amount of divergence from “reality” from that source of error arguably pales with testing error that only measures Wi-Fi device performance, not the actual speeds delivered to any location by an ISP.

In fact, virtually all user tests of speed are outside the margin of error by such a huge margin that the reported speeds are likely wrong--and undercounted--by as much as an order of magnitude.