Facial recognition is getting better at making matches around face masks

A study of 65 face recognition algorithms on masked photos found they were as good as the top performers in 2017 with no masks.

Alfred Ng Senior Reporter / CNET News
Alfred Ng was a senior reporter for CNET News. He was raised in Brooklyn and previously worked on the New York Daily News's social media and breaking news teams.
Alfred Ng
3 min read
Person wearing a face mask while walking on a city sidewalk.

Some facial recognition algorithms have gotten 10 times better at matching people in masked photos, researchers found.

James Martin/CNET

Scientists agree that face masks are here to stay, and research finds that facial recognition technology is starting to catch up. Since the start of the pandemic, facial recognition providers have been working to get around the coverings, and they've gotten marginally better, results from a US government study shows. 

The US National Institute of Standards and Technology, or NIST, is considered the leading authority on facial recognition accuracy rates, and it's been conducting a series of studies on how face masks affect the technology since May 1. 

The initial results, published in July and August, showed that masks were thwarting facial recognition algorithms and increasing error rates by up to 99% in some cases. The error rates increased for every algorithm once researchers added masks to the test photos, even for facial recognition that was designed specifically for the coverings. 

The latest results, published Tuesday, show that facial recognition has gotten significantly better at making one-to-one matches, even when people are wearing masks. The study looked at 65 algorithms submitted after face masks became required in several countries and tested on 6.2 million photos. 

Watch this: See the new face mask that might solve shortages

The error rates are still higher once a mask is factored in -- jumping from 0.3% without masks to about 5% with masks. Still, the NIST study said there was a "notable reduction in error rates" compared to algorithms submitted before the pandemic. 

In some cases, face recognition algorithms became 10 times better at making matches than their pre-pandemic versions, the study found.

"The current performance of face recognition with face masks is comparable to the state-of-the-art on unmasked images in mid-2017," the study found. 

Face recognition providers have been training their algorithms to detect identities despite masks using social media photos of people in masks, digitally adding masks to photos, asking their own staffers to send in masked images and buying photo sets. 

They're able to make matches even when 70% of your face is covered by picking up recognizable points on your nose and eyes. In September, NEC, one of the world's largest face recognition providers, said it developed an algorithm specifically for face masks by focusing on the position, shape and size of a person's eyes and nose. 

NIST's findings don't mean that the facial recognition industry has completely figured out face coverings. For starters, the agency only tested for one-to-one matches, where they already have a photo of a person and looked to see if it matched the same photo but with a mask digitally added. One-to-many algorithms would test facial recognition's capabilities to match people against a group of images rather than the same photo. 

Because the masks are also digitally added, it gives perfect conditions that real masks with different texture, colors and shapes wouldn't have. The study used the exact same color for its masks in each test, but noted that masks in black and red are better at thwarting facial recognition than masks in blue and white.

In the real world, images likely would be less than perfect because of issues with lighting and angle and image quality. 

The study also didn't factor in race and gender in its test photos. Facial recognition is known to have higher error rates for women of color, but NIST's tests haven't separated its results by demographics. 

"We deferred tabulating accuracy for different demographic groups until more capable mask-enabled algorithms have been submitted to [the Facial Recognition Vendor Test]," NIST said.

See also: The best facial recognition security cameras of 2020