X

Netflix viewership stats just got more meaningless

You now count as a viewer of The Witcher and 6 Underground even if you turn them off after the first scene.

Joan E. Solsman Former Senior Reporter
Joan E. Solsman was CNET's senior media reporter, covering the intersection of entertainment and technology. She's reported from locations spanning from Disneyland to Serbian refugee camps, and she previously wrote for Dow Jones Newswires and The Wall Street Journal. She bikes to get almost everywhere and has been doored only once.
Expertise Streaming video, film, television and music; virtual, augmented and mixed reality; deep fakes and synthetic media; content moderation and misinformation online Credentials
  • Three Folio Eddie award wins: 2018 science & technology writing (Cartoon bunnies are hacking your brain), 2021 analysis (Deepfakes' election threat isn't what you'd think) and 2022 culture article (Apple's CODA Takes You Into an Inner World of Sign)
Joan E. Solsman
5 min read
witcher-cavill-sword

Netflix said The Witcher is its most popular freshman series yet, according to its new metric that inflates popularity by a third.

Netflix

Netflix has dragged its fixation on flexing viewership stats into the new year, but it brought a surprise twist this time. On Tuesday, it touted how fantasy show The Witcher turned in its most popular first season of a series yet, with an estimated 76 million accounts watching it in the first four weeks. It bragged that even more people watched 6 Underground, an explosion-fest directed by Michael Bay, which hit 83 million household views in the first month. 

All you have to do is squint your eyes and pretend it matters, even if millions of people could have turned off both programs as soon as the Scene I ended.

Netflix 's surprise: The company switched to a new viewership metric that essentially inflates its numbers by more than a third compared with its previous standard. Netflix said it will now count a title as "watched" if you choose to watch it and let it play for two minutes. That's it. In the past, Netflix wouldn't start counting something as "watched" until you got through 70% of the first episode of a series or of a film's total runtime.

The company took this kind of measurement out for a spin last month, when it released popularity rankings for all of 2019. 

Netflix advocates for the new standard by saying the new two-minute threshold is more fair to all titles, regardless of their length. Under the old measurement, long films were at a disadvantage. A 3.5-hour epic like The Irishman had a much taller hurdle to clear to hit that 70% mark than a show like Special with 15-minute (or shorter) episodes. Netflix interactive titles, which produce an eye-popping amount of total watchable video to account for all possible choices, were also difficult to gauge with the 70% model.

Ryan Reynolds in 6 Underground

6 Underground with Ryan Reynolds was one of Netflix's most popular titles in the previous quarter, which included its crucial holiday period. 

Christian Black/Netflix

But In the case of The Witcher and 6 Underground, any viewer could make it to the main title sequences but then turn off the programs out of boredom or disgust -- and those accounts still cast their vote in Netflix's tally of what is hyper popular. Those weren't the only titles Netflix highlighted: The second season of psychological thriller You got a projected 54 million account views; holiday flick Klaus, which is nominated for a best-animated-feature Oscar, got 40 million account views; and the third season of big-budget historical drama The Crown got 21 million account views.

For years, Netflix was notoriously tight-lipped about viewership. The creator of House of Cards, which put Netflix's original content efforts on the map, once said the company wouldn't even share viewership metrics with him. But lately Netflix has loosened up to help recruit talent and stoke up buzz.

And even before this change inflating its numbers, Netflix's stats needed big disclaimers. For one, they aren't independently verified, nor are they backed up by detailed data from the company. Netflix is in the unique position that it can cherry-pick highlights, and we don't have much independent data to verify it. Traditional media companies, on the other hand, have their box office performance independently monitored, and they're at the mercy of Nielsen ratings as the barometer for TV shows.

Netflix, Nielsen and the counting game

Speaking of which, some advice: Don't compare Netflix's numbers to metrics like Nielsen ratings or box office figures. It's tempting to compare how many people watched a Netflix show versus one on regular TV, or to estimate how much money a big movie on Netflix would have made at the box office. But these metrics aren't even close to comparable because the methods behind them differ wildly.

People will inevitably compare the viewership of The Witcher to Nielsen ratings for Game of Thrones. But Nielsen uses a metric that is dramatically more difficult. These aren't apples-to-apples comparisons. They aren't even apples-to-oranges comparisons. Think more like apples-to-peppercorns, or ice-cream sprinkles, or grains of salt. Pick your flavor, but it's almost always wildly inaccurate. 

Why? Traditional TV ratings by Nielsen are what's known as "average minute audience." Roughly speaking, this is an approximation of audience who tuned into a program if you counted them during any random minute of the show. This bar is way, way higher than the one tech companies -- including Netflix now -- clear when they report a number of views for an online video.

Rarely do we get an example of just how inaccurate it is to compare online viewer counts to Nielsen viewer counts. But in 2017, ex-FBI Director James Comey's congressional testimony provided a great example: 

Nielsen reported 19.4 million viewers for live broadcasts of Comey's hearing on TV -- that's the total number of people watching all the networks airing it live, about nine channels. (Reminder: This is "average minute audience.") Twitter and Bloomberg said their Twitter livestream of Comey's testimony reached 2.7 million unique viewers. The typical instinct here is to compare the 19.4 million over nine TV channels to the 2.7 million on Twitter. But what's unusual here is that Twitter/Bloomberg also provided an "average minute audience" rating for their online livestream, which is rare. That figure? Only 129,000 people out of the 2.7 million. 

In other words, less than 5% of Twitter's total unique viewers would have counted under an "average minute audience" Nielsen measure.  

Nowadays, Twitter, YouTube, Facebook and other internet companies set an even lower bar than Netflix does. Twitter counts a video view after just two seconds. Facebook counts after three seconds, including any autoplaying video you happen to barely glance at in your news feed. YouTube waits roughly 30 seconds before counting a view. 

By that comparison, Netflix is being exceedingly rigorous by claiming a view at two minutes. But nobody would argue that Netflix's programming is more like that on Twitter than it is more like stuff on regular TV. 

Choosing to watch something and then sticking with it for at least two minutes is "long enough to indicate the choice was intentional," Netflix says. But by Netflix's definition, a movie that makes audiences walk out of the theater in the first five minutes is still popular. (Hat tip: this tweeter.) 

And that's the problem. Intention doesn't signal popularity. It signals fleeting interest. But at least tons of people watched Ryan Reynolds' plane tumbling out of the sky in that one scene in 6 Underground...

New Movies Coming in 2023 From Marvel, Netflix, DC and More

See all photos