Amid George Floyd protests, weaponized misinformation floods social media

Confusion runs rampant on Twitter and Facebook.

Dara Kerr Former senior reporter
Dara Kerr was a senior reporter for CNET covering the on-demand economy and tech culture. She grew up in Colorado, went to school in New York City and can never remember how to pronounce gif.
Shara Tibken Former managing editor
Shara Tibken was a managing editor at CNET News, overseeing a team covering tech policy, EU tech, mobile and the digital divide. She previously covered mobile as a senior reporter at CNET and also wrote for Dow Jones Newswires and The Wall Street Journal. Shara is a native Midwesterner who still prefers "pop" over "soda."
Dara Kerr
Shara Tibken
9 min read

A makeshift memorial has been set up where Patrick Underwood was killed in a drive-by shooting.

Dara Kerr/CNET

Bouquets of brightly colored flowers and prayer candles rest at the door of a small windowed guard booth in downtown Oakland, California. A ribbon with an American flag motif waves from the door handle. Two bullet holes pockmark the booth's windows.

This is where Dave Patrick Underwood, 53, was fatally shot around 9:45 p.m. on May 29 while working as contract security officer for the Department of Homeland Security, according to the FBI.

The shooting, which also seriously injured another DHS contract security guard, took place a few blocks from where thousands of protesters had gathered to mark the killing of George Floyd. The 46-year-old unarmed Black man died after a white police officer jammed a knee onto his neck for nearly 9 minutes. The officer was fired and has been charged with murder. Like Floyd, Underwood was African American.

Initial news reports of Underwood's shooting linked the incident to the protests. As is often the case, the conjecture on Twitter and other social media sites ran away from the changing facts on the ground. As early as the night of May 29, the Oakland Police Department sent an alert to reporters saying the shooting didn't appear to be related to that night's demonstrations. And the FBI has never linked the two in its statements to the press. It confirmed to CNET in early June that Underwood died in a drive-by shooting but declined to comment further at the time because its investigation was ongoing.  

On Tuesday, the FBI said it would charge a 32-year-old Air Force sergeant, named Steven Carrillo, and his alleged accomplice, Robert Justus, in the fatal shooting. Officers linked Carrillo to the right-wing Boogaloo Movement that mistrusts law enforcement and anticipates a second American Civil War, referred to as the "boogaloo."

Officials on Tuesday reiterated that the shooting had nothing to do with the protesters. "We believe Carrillo and Justus chose this date because of the planned protest in Oakland," Jack Bennett, the FBI special agent in charge, said Tuesday during a press conference. "It provided them to target multiple law enforcement personnel and to avoid apprehension due to the large crowds attending the demonstrations."

Carrillo last week also was charged in the June 6 fatal shooting of Damon Gutzwiller, a Santa Cruz County sheriff's sergeant.

For Twitter, the FBI's initial report -- let alone the actual arrests -- appeared to be too late. Hashtags like #PatrickUnderwood and #JusticeForPatrickUnderwood had already started trending, buoyed by accounts that declared Underwood was "murdered" by protesters and "rioters." The association with Oakland's protesters, later proven to be untrue, had already spread -- going as far as the White House. 

Misinformation on social media is nothing new. Russian agents tried to sway the 2016 US presidential election with divisive tweets and Facebook posts. Message board chatter about "Pizzagate," a conspiracy theory that falsely accused Hillary Clinton and others of operating a child sex ring out of a restaurant, led to gunfire in Washington, DC. Hoaxes, often disguised as legitimate news, have spread far and fast, thanks to social media sites like Facebook and Twitter, as well as 4chan, 8kun and other anonymous message boards. 

But social media's inability to contain the explosion of misinformation takes on new urgency as peaceful protesters battle the perception that all of the demonstrations have devolved into looting and violence. Twitter's role in spreading news in real time without any checks makes it particularly vulnerable to manipulation. Over the past few weeks, along with tweets about protestors being responsible for Underwood's death, other false theories have made the rounds, including an internet blackout in Washington and the far-left militant group antifa sending protesters to cause unrest in cities across the US. And once something hits Facebook, with its over 2.6 billion monthly active users, it's hard to counteract.

"It's very important for both of these platforms to get it right, not just one or the other," said Jennifer Grygiel, an assistant professor of communications at Syracuse University. "We need both."

In a speech on June 1, President Donald Trump said the nation "has been gripped by professional anarchists, violent mobs, arsonists, looters, criminals, rioters, antifa and others," painting a grim picture of the mostly peaceful protests that have swelled to more than 140 cities across the country. Trump brought up Underwood.

"A federal officer in California, an African American enforcement hero, was shot and killed," he said. "These are not acts of peaceful protest. These are acts of domestic terror."


Around 15,000 protesters peacefully march through Oakland demanding justice for George Floyd.

Dara Kerr/CNET

Even as misinformation has flourished on social media, allowing critics of the protests to craft their own narrative of the demonstrations, the platforms continue to serve another purpose. Organizers also have relied on the sites -- especially Twitter -- to coordinate demonstrations and share important updates on the ground. Journalists use it to report developments, and citizens follow along to learn what's happening in their cities. But some people have taken advantage of the widespread use of social media to sow confusion among the protesters. 

Twitter says it's trying to address this problem.

"We're taking action proactively on any coordinated attempts to disrupt the public conversation around this issue," a Twitter spokeswoman said. "For example, we are actively investigating the hashtag #dcblackout and during this process have already suspended hundreds of spammy accounts that Tweeted using the hashtag." 

The spokeswoman also said Twitter "may" prevent certain content -- like topics that incite hate on the basis of race -- from trending.

Facebook, for its part, said in a statement that its teams "have been working to find and remove violating activity since the protests started. We're using automated detection systems, fact-checking, reducing content distribution and removing content that violates our policies."

Facebook pays an army of fact-checkers to verify potentially problematic posts, but Twitter doesn't have that sort of manpower. It tends to lean on machines, though humans step in for big decisions. And while it will sometimes slap a warning label on a tweet, as it recently did to several posts from Trump, lots of misinformation simply flows through the site untouched. 

"It doesn't really matter who or where it's coming from," said Maddy Webb, an investigative researcher at First Draft, an organization that tracks and combats online misinformation. "There is such an erosion of trust that we have really decimated this platform that is super valuable for organizing."

Misinformation's spread 

Early on the morning of June 1, the hashtag #dcblackout had begun circulating throughout Twitter. It referenced a fake story that said no social media would be accessible in the city due to the civil unrest. As the hashtag spread, the claims behind it became wilder -- that phones and other means of communication were being blocked and police were replacing rubber bullets with gunpowder ones.

Enlarge Image

The #dcblackout hashtag got nearly 1 million mentions on June 1.

Screenshot by CNET

By the afternoon, the hashtag had nearly 1 million mentions, according to NPR.

Netblocks, a group that tracks internet disruptions and shutdowns, followed the tweets about the #dcblackout and monitored network traffic in real time. It never saw instability in the connections, indicating the rumor wasn't true. 

"Fact check," NetBlocks tweeted on June 1. "Washington, D.C. did not have a city-wide blackout."

In a fast-paced, constantly evolving environment, like the protests around Floyd's death, false information can overwhelm Twitter. On May 31 alone, Black Lives Matter, Floyd and the protests about his death were mentioned 21.2 million times across all forms of media, including Twitter, according to Zignal Labs. From May 25 to June 2, there were over 101 million mentions, the media intelligence company said. Some of that information has been weaponized to fit people's political agendas.

One hoax spreading widely is that antifa is behind the country's more violent protests and that it has been busing protesters into predominantly white areas of cities to loot homes and businesses. 

Kathleen Carley, a computer science professor at Carnegie Mellon University, said a study of tweets around the protests show that essentially no humans have been using the antifa hashtag. Instead it's largely bots, according to the data crunched by Carley and her team from May 25 through May 31. She estimates that 30% to 49% of users posting about the protests likely are machines, not humans. It doesn't mean all of them are posting inaccurate information, but they're helping sow confusion, she said. (Twitter hasn't said how many bots are involved in recent tweets, but some social media experts have disputed the tallies.)

Just shutting down all bots isn't the answer, she said. Many spread helpful news, like earthquake alerts.

"Disinformation is really hard to discern sometimes," Carley said. "Based on the hashtags, [bots are] more [often] making the protests a political issue about things other than Black Lives Matter or other than George Floyd."

While Trump hasn't been using antifa hashtags, he has been tweeting about the activists, labeling them a terrorist organization and blaming them for the country's protests. Over one week, people posted similar sentiments on Facebook more than 6,000 times, according to The New York Times. And those posts tallied more than 1.3 million likes and shares, the paper said June 1.

The widespread scrutiny and chaos on social media is causing both Twitter and Facebook to react. 

Social media's response

On May 26, Twitter did something never before seen on its site: It put a label on two tweets from President Trump, warning they contained "potentially misleading information about voting processes." A couple of days later, Twitter also hid an overnight tweet from Trump behind a label, saying it violated the company's rules about "glorifying violence." 

"When the looting starts, the shooting starts," Trump had tweeted, using a phrase from a Miami police chief in the 1960s that's widely seen as a violent threat against protesters.

Facebook, meanwhile, took a more hands-off approach. The social networking giant, which makes a lot of money from political ads (something Twitter bans on its site), left up a similar post from Trump. In response, Facebook employees staged a rare protest, and several said they would quit working for the company. Online therapy company Talkspace also ended its partnership discussion with Facebook because the startup "will not support a platform that incites violence, racism and lies," in the words of its founder Oren Frank. 

Watch this: Trump issues order to stop alleged 'unchecked power' of Twitter

Trump struck back. On May 29, he signed an executive order that called for a reinterpretation of Section 230 of the Communications Decency Act of 1996. The law, which ensures free speech on tech platforms, designates online companies such as Facebook and Twitter as "distributors" rather than "publishers" of content. That prevents them from being sued for every negative review or comment posted on their sites. Many experts say Trump's order is largely symbolic and unenforceable.

"When Section 230 was crafted, no one thought about having to try to moderate the president of the United States," Syracuse's Grygiel said. "They're still grappling with this."

Typically, fact-checkers call on social media to take a few steps when it comes to combating misinformation. They suggest moderating the content, as well as getting rid of the retweet and share buttons, which would make it harder to disseminate fake information quickly.

Enlarge Image

The guard booth where Patrick Underwood was killed.

Dara Kerr/CNET

In the case of the protests, those strategies likely won't work, says First Draft's Webb. While retweeting and sharing information can spread misinformation quickly, those tools also are vital for protestors to coordinate. And there's too much information flowing during protests for the social media sites to moderate them without hurting free speech and potentially labeling accurate information as false. Setting fair removal policies could be tricky, she said.

"We know that those kinds of policies target people who already don't have a lot of opportunity to speak," Webb said. 

If Twitter and Facebook can't find ways to scrutinize themselves, it may be up to their users to be more careful about what information they share and believe. That means doing things like reverse image searches and waiting before immediately retweeting something, she added. And eventually, the companies could face more government regulation.

As Twitter tries to walk the line between allowing free speech and curbing misinformation, Underwood's name was still used to spread a false narrative. Meanwhile, his friends and family have been trying to remember the man who lost his life on May 29.

The week after Underwood was killed, the federal building where he worked was still cordoned off with police tape. On the makeshift memorial at the guard booth where he died was a piece of paper with his picture and a link to a GoFundMe site for donations for his family. Underwood's sister and a cousin didn't respond to requests for comment. 

"Two of our Federal Contract Officers were ambushed in an act of violent cowardice," Jennifer Tong, the supervisor for Underwood and his colleague, wrote on the fundraising site. "I have worked side by side with both men and cannot describe the pain we as their brothers in arms feel."

CNET's Alfred Ng, Andrew Morse and Queenie Wong contributed to this report.

Two weeks after this story was originally published, the FBI made an arrest in the case and charges were filed. The story has been updated to reflect that new information.