X

An IBM computer debates humans, and wins, in a new, nuanced competition

AI isn't just winning at board games. Now it's learning the art of persuasion.

Stephen Shankland Former Principal Writer
Stephen Shankland worked at CNET from 1998 to 2024 and wrote about processors, digital photography, AI, quantum computing, computer science, materials science, supercomputers, drones, browsers, 3D printing, USB, and new computing technology in general. He has a soft spot in his heart for standards groups and I/O interfaces. His first big scoop was about radioactive cat poop.
Expertise Processors, semiconductors, web browsers, quantum computing, supercomputers, AI, 3D printing, drones, computer science, physics, programming, materials science, USB, UWB, Android, digital photography, science. Credentials
  • Shankland covered the tech industry for more than 25 years and was a science writer for five years before that. He has deep expertise in microprocessors, digital photography, computer hardware and software, internet standards, web technology, and more.
Stephen Shankland
5 min read
IBM presents its Project Debater on stage as black obelisk with a screen for a face. A passive blue oval shows when it's idling, but animated blue circles move around as it's processing speeches or listening to a debate opponent.

IBM presents its Project Debater on stage as a black obelisk with a screen for a face. A passive blue oval shows when it's idling, but animated blue circles move around as it's processing speeches or listening to a debate opponent.

Stephen Shankland/CNET

We saw computers beat humans at chess in 1997, beat humans at Jeopardy in 2011 and vanquish the world's best human players of the ancient game of Go in 2017. On Monday, a computer edged out a victory over people in a far more nuanced competition: debate.

Watch this: IBM's new AI can debate you

IBM created a system called Project Debater that competes in what the company calls computational argumentation -- knowing a subject, presenting a position and defending it against opposition. At a press event, IBM pitted the system against two humans with a track record of winning debates.

In one debate, Noa Ovadia overall nudged two people among a few dozen in a human audience toward her perspective that governments shouldn't subsidize space exploration. But in the second, Project Debater soundly defeated Dan Zafrir, pulling nine audience members toward its stance that we should increase the use of telemedicine.

You're not going to lose your job yet to Project Debater's commercial spin-off right away as Big Blue tries to profit from the IBM Research project. Project Debater betrayed its inhuman nature several times over the course of its 20 minutes of off-the-cuff speech. But Debater did demonstrate that artificial intelligence can handle some complexities of human interaction, not just the clear-cut rules and victories of a board game or game show.

"Our life is not black or white. It's ambiguous, it's subjective," said Ranit Aharonov, director of Project Debater. "AI will have to navigate in that territory."

IBM Project Debater researchers Noam Slonim and Ranit Aharonov

IBM Project Debater researchers Noam Slonim and Ranit Aharonov.

IBM Research

Project Debater was trained in advance on debating methods, but not the details of the debate itself, which it found out about only moments before the debate started. To formulate its argument, it had at its disposal a collection of 300 million news articles and scholarly papers, previously indexed for quick search results. But it had to find the information, package it persuasively, listen to its opponents' arguments and formulate a rebuttal.

"It's amazing to see this technology pull from 300 million sources and distill it into what sounds like a conversational narrative in debate," said Clea Conner Chang, chief operating officer of Intelligence Squared Debates, an organization that runs its own debates with a similar argue-and-respond style.

Echoes of 2001: A Space Odyssey

Project Debater's brain is a group of computers in a distant data center, but IBM chose to represent it as a black pillar on the stage. Sci-fi fans might have been reminded of the alien obelisk in Arthur C. Clarke's 2001: A Space Odyssey, the book and movie that also made famous HAL 9000, a well-spoken but malevolent AI. Debater paid homage to the author, too, at one point quoting his famous maxim, "Any sufficiently advanced technology is indistinguishable from magic."

Watch this: IBM creates AI to destroy humans... in debate

Although IBM didn't try to pretend Debater is human, it speaks in a well-modulated female voice, and researchers call it "she." At the top of the pillar was a display showing its state of mind with curvaceous shapes in a soothing blue.

Three circles swapped places when Project Debater was thinking. They merged into an oval with a waving sinusoidal line when the machine was speaking. And when it was listening to its human opponent, the three circles hopped up and down like the animations you see when your computer is waiting for a software update to install.

Debater showed a grasp of debating's nuts and bolts. It marshaled evidence, told you its stance, explained how its argument would proceed and made its case. It quoted authorities, like German ministers touting the economic benefits of space exploration or scientific studies showing better health outcomes for diabetes patients monitored from afar.

Plenty of its words were lifted directly from its corpora of data, IBM said. But it does construct its own sentences, too, as when presenting its summary of the arguments it plans to make. And it does plenty of other language work, too -- assessing arguments, deciding which points to raise specifically or broadly, and evaluating the counterarguments made by its opponent.

Project Debater, run out of IBM's labs in Haifa, Israel, began in 2011 when IBM was looking for the next challenge after its Jeopardy victory. Researcher Noam Slonim suggested a human versus computer debate, and Aya Soffer, who runs IBM Research's global AI team, liked it as well. It was one among 30 projects IBM evaluated, she said.

"From our perspective, the debate format is the means and not the end. It's a way to push the technology forward and part of our bigger strategy of mastering language," Soffer said. "In general, computers are lagging significantly in understanding and being able to express themselves. If we expect AI to be useful, being able to communicate with people is critical."

IBM Project Debater, represented by a black obelisk with animated blue circles, takes on human debater Dan Zafrir.

IBM's Project Debater, represented by a black obelisk with animated blue circles, takes on human debater Dan Zafrir.

IBM Research

Comic relief

It throws in jokes, too -- human-written jokes, but the machine has to master the timing.

"I can't say it makes my blood boil, because I have no blood, but it seems some people naturally suspect technology because it's new," Debater said in its closing argument in favor of telemedicine.

And arguing for space exploration subsidies, Debater said it supports technology, "being as I am a prime example of its power." The audience chuckled, though an audience outside IBM's Watson West offices in San Francisco might react differently to a computer that can figure out the right moment to inject that comment into a discussion.

There were gaffes, like the moment Debater said subsidizing space exploration "is more important than good roads, improved schools or better health care." That statement would torpedo most would-be politicians' chances at election.

Debate 'defines what it is to be human'

But Chris Reed, a professor at the University of Dundee, said we shouldn't take the achievements for granted. He was impressed that Debater successfully figured out what raw material supported the argument it was making and that it got verb tenses correct and mastered other basics that have tripped up computer debaters.

Debater even demonstrated procatalepsis -- the ability to anticipate its opponents' arguments and pre-emptively attack them, Reed said. He was clearly impressed.

"Argumentation is one of the fundamental things that defines what it is to be human," Reed said. "Argument and debate is the engine that drives the process of science, that characterizes what happens in political fora, that frames our conception of religion. if we can go any way toward building computer models that tackle these issues, we're doing something really important."

'Hello, humans': Google's Duplex could make Assistant the most lifelike AI yet.

Blockchain Decoded: CNET looks at the tech powering bitcoin -- and soon, too, a myriad of services that will change your life.