When facial recognition is everywhere, anything you do is fair game for public shaming and punishment.
Facial recognition supporters in the US often argue that the surveillance technology is reserved for the greatest risks -- to help deal with violent crimes, terrorist threats and human trafficking. And while it's still often used for petty crimes like shoplifting, stealing $12 worth of goods or selling $50 worth of drugs, its use in the US still looks tame compared with how widely deployed facial recognition has been in China.
China's facial recognition system logs nearly every single citizen in the country, with a vast network of cameras across the country. A database leak in 2019 gave a glimpse of how pervasive China's surveillance tools are -- with more than 6.8 million records from a single day, taken from cameras positioned around hotels, parks, tourism spots and mosques, logging details on people as young as 9 days old.
The Chinese government is accused of using facial recognition to commit atrocities against Uyghur Muslims, relying on the technology to carry out "the largest mass incarceration of a minority population in the world today."
"China uses facial recognition to profile Uyghur individuals, classify them on the basis of their ethnicity, and single them out for tracking, mistreatment, and detention," a bipartisan group of 17 senators said in a letter to Secretary of State Mike Pompeo on March 11. "And these technologies are deployed in service of a dystopian vision for technology governance, that harnesses the economic benefits of the internet in the absence of political freedom and sees technology companies as instruments of state power."
China's aggressive development and use of facial recognition offers a window into how a technology that can be both benign and beneficial -- think your iPhone's Face ID -- can also be twisted to enable a crackdown on actions that the average person may not even consider a crime. Chinese officials have used surveillance tools to publicly shame people wearing sleepwear in public, calling it "uncivilized behavior."
The punishing of these minor offenses is by design, surveillance experts said. The threat of public humiliation through facial recognition helps Chinese officials direct over a billion people toward what it considers acceptable behavior, from what you wear to how you cross the street.
"The idea is that the authorities are trying to put in place comprehensive surveillance and behavioral engineering on a mass scale," said Maya Wang, a senior researcher on China at the Human Rights Watch. "The authorities want to create a kind of society that would be very easy for them to manage."
The idea that China's facial recognition would automatically put your name and photo on a billboard for jaywalking and privately text you a fine instills fear in people to behave a certain way, Wang said.
Technology has often had that ability to affect behavior, even outside of China.
Behavioral engineering is the concept of combining technology and psychology to nudge people to act a certain way, and it's something we see every day. It's evident in "dark patterns," interface designs like hidden opt-out buttons that trick people into giving up their personal data, for example.
But there's a key difference in how behavioral engineering is carried out in the US compared with China and its facial recognition.
"It's expressed in a commercial manner in the US, whereas in China it's a state effort," Wang said. "The dynamics are different, and the amount of power is different, but it has striking similarities."
In the US, behavioral engineering can be done through amassing data on people, and pushing or excluding content to them based on predicted personality traits.
Facebook's 2018 scandal with Cambridge Analytica stemmed from the fact that the now-defunct UK data analytics firm used data from millions of people to target advertising that would sway people to vote a certain way.
Russia's 2016 US election meddling efforts involved using Facebook to create divisive groups and stage events where people would show up to protest. These posts would be targeted to police groups or specifically to Spanish speakers, for example.
While that type of behavioral engineering is mostly geared for selling products and profits, China's push is more about instilling fear and control over its population -- and facial recognition plays a key part in that.
"It conditions the idea of 'this is how society works,' that you're always being watched," said Jake Laperruque, a senior counsel at the Constitution Project. "It seems like trying to reinforce to people that 'we're watching all the time, if you ever do anything to piss us off, we will see and we will find a way to embarrass you.'"
In China, no one is safe from facial recognition, or the public shaming that comes with it.
Cameras set up at crosswalks to identify and post photos of jaywalkers are commonplace, and a report from Abacus in May 2019 showed photos of children jaywalking on a digital billboard. The local traffic police said "children should be treated the same as adults," according to the outlet.
A Chinese park has also relied on facial recognition to prevent people from taking too much toilet paper, scanning people's faces before dispensing out the rolls.
"It's kind of an 'anything goes' system there, no matter how minor," the Constitution Project's Laperruque said. "Any sort of offense goes for this kind of tech."
And while in the US, facial recognition is going through a reckoning over its racial bias and human rights concerns, in China, the surveillance technology's providers boast its abilities to single out people of different ethnicities.
Last November, IPVM found that Chinese surveillance company Hikvision marketed that its cameras could automatically identify Uyghur Muslims with its facial recognition.
There's pushback on facial recognition in the US because researchers are able to bring to light their concerns about the technology's racial bias, but there isn't similar scrutiny in China, researchers said.
China's facial recognition accuracy rates aren't challenged, even when it accidentally picks up a person on a bus ad and considers them a jaywalker.
"Americans can talk about racial inequalities without fear," the Human Rights Watch's Wang said. "These systems are not discussed or pushed back in China. There's no allowed space at all, both online and offline, to discuss racial bias and the persecution of minorities in China."
China's model of facial recognition, from its oppressive surveillance of Uyghur Muslims to its every day grip over a population of 1.4 billion people, also poses a concern for the international community, US lawmakers said.
Last September, the Carnegie Endowment for International Peace found that China was a major supplier of AI surveillance, providing the technology to 63 countries.The next closest provider for AI surveillance tech was Japan's NEC Corporation, which sells facial recognition to 14 countries.
In April, Reuters reported that Amazon bought cameras from Dahua, a Chinese surveillance company blacklisted by the US over allegations that it helps China detain and monitor Uyghur Muslims.
The $10 million deal was for 1,500 cameras to help monitor the spread of COVID-19, according to the report.
In the senators' letter on March 11, they raised issues about China's influence on how facial recognition should be used when it's the technology's top exporter.
Wang noted that while facial recognition is often a subsidiary for US tech giants, like Amazon's Rekognition or Microsoft, in China, there are multiple companies already dominating the industry. The country's acceptance of a surveillance state allows facial recognition providers to push forward with the technology, even for the most trivial uses.
With that level of influence, there're concerns that China's model of how to use facial recognition -- a widespread network designed for public shaming and control -- could spread to the rest of the world.
"Unfortunately, China has indicated a willingness to use standard setting bodies in perverse ways to normalize global opinions about Orwellian surveillance technology," the group of lawmakers said. "By shaping the debate about the legitimate uses of artificial intelligence and facial recognition, China can expand opportunities for countries, particularly those in the developing world, to utilize Chinese surveillance technology."