Facebook removes bogus accounts that used AI to create fake profile pictures
Don't trust every image you see on social media.
- 2022 Eddie award for consumer analysis
- I've been covering technology and mobile for 12 years, first as a telecommunications reporter and assistant editor at ZDNet in Australia, then as CNET's West Coast head of breaking news, and now in the Thought Leadership team.
Facebook has been battling divisive content on its platforms.
Facebook said Friday that it shut down hundreds of fake accounts, pages and groups that misled users, including some that used artificial intelligence to generate fake profile pictures. Researchers who studied the group of accounts said it's the first time they've seen AI-generated images used on such a large scale, raising concerns about how the social network will tackle this problem ahead of the 2020 US presidential election.
The company also took down fake accounts from the photo-sharing service Instagram , which it owns. It pulled down 610 Facebook accounts, 89 pages, 156 groups and 72 Instagram accounts that were from the US and Vietnam. About 55 million accounts followed at least one of the pages and most of these followers were from outside the US.
Profile pictures for these fake Facebook users were generated using artificial intelligence. The neck is distorted on the left and there's something off about the background in both images.
"We're constantly working to detect and stop this type of activity," Facebook said in a blog post. "We don't want our services to be used to manipulate people."
The company said it removed the accounts for engaging in foreign and government interference or "coordinated inauthentic behavior," which means that whoever was running the accounts misled others about their identity and what they were doing.
Targeting US citizens as well as Vietnamese, Spanish and Chinese-speaking audiences, the fake accounts "typically posted memes and other content about US political news and issues including impeachment, conservative ideology, political candidates, elections, trade, family values and freedom of religion," Facebook said.
Many of the fake accounts promoted pro-Trump content by a US media outlet called The BL, short for The Beauty of Life, according to Facebook. Fact-checking organization Snopes alleges that The BL is linked to Epoch Media Group, which owns The Epoch Times. Former Epoch Times employees run The BL or work for the media outlet, according to Snopes.
Stephen Gregory, the publisher for the US editions of The Epoch Times, denied in an op-ed that the companies are affiliated with each other. He acknowledged that former Epoch Times employees work for The BL but said that isn't evidence the two companies are connected. The BL is also a publication of Epoch Times Vietnam, but as of 2018 it's no longer listed as part of Epoch Media Group.
Facebook, which already bars Epoch Media Group from advertising, said it also banned The BL. The social network said The BL violated its rules including ones against spam and misrepresentation. Facebook said it was able to link the fake account activity to Epoch Media Group despite efforts to evade detection.
The BL didn't immediately respond to a request for comment. Epoch Media Group pointed to Gregory's op-ed.
Graphika, a social media analysis company, and the Atlantic Council's Digital Forensics Lab analyzed the network of fake accounts and found that a lot of the English language content was pro-Trump or attacked the president's rivals.
The accounts used fake profile pictures and pretended to be Americans to join groups such as "America Needs President Trump" and "Trump for America's President." Using AI, these fake accounts included profile pictures of people who don't actually exist.
Researchers noticed subtle details in the profile pictures that signaled they were fake. For example, in some photos accessories such as glasses and earrings weren't symmetrical. The dimensions of someone's neck might've been slightly distorted or the background of the photo may've looked off.
"The ease with which the operation managed to generate so many synthetic pictures, in order to give its fake accounts (mostly) convincing faces, is a concern," according to the report by Graphika and the Atlantic Council.
In other cases, fake accounts used photos from stock images or glamor shots found online.
Another network of fake accounts originated from the country of Georgia. Facebook removed 39 Facebook accounts, 344 pages, 13 groups and 22 Instagram accounts, which collectively had more than 450,000 followers and members. The network pretended to be media, political parties and activists, posting about news, political issues and political criticism.
Earlier Friday, Twitter shared data of 5,929 accounts originating from Saudi Arabia that it removed for manipulating information.
"These accounts represent the core portion of a larger network of more than 88,000 accounts engaged in spammy behavior across a wide range of topics," Twitter said. "We have permanently suspended all of these accounts."
Originally published Dec 20, 10:52 a.m. PT.
Updates, 1:30 p.m.: Adds more background about AI-generated profile pictures; Dec. 21: Includes more information about the content promoted by the accounts.