Brain implants let paralyzed man write on a screen using thoughts alone
Researchers combine neural implants with AI to develop a "mindwriting" system that converts imagined writing to text on a screen.
Eric MackContributing Editor
Eric Mack has been a CNET contributor since 2011. Eric and his family live 100% energy and water independent on his off-grid compound in the New Mexico desert. Eric uses his passion for writing about energy, renewables, science and climate to bring educational content to life on topics around the solar panel and deregulated energy industries. Eric helps consumers by demystifying solar, battery, renewable energy, energy choice concepts, and also reviews solar installers. Previously, Eric covered space, science, climate change and all things futuristic. His encrypted email for tips is firstname.lastname@example.org.
ExpertiseSolar, solar storage, space, science, climate change, deregulated energy, DIY solar panels, DIY off-grid life projects. CNET's "Living off the Grid" series. https://www.cnet.com/feature/home/energy-and-utilities/living-off-the-grid/Credentials
Finalist for the Nesta Tipping Point prize and a degree in broadcast journalism from the University of Missouri-Columbia.
Humans have long been fascinated with the idea of someone or something that can read our minds -- be it a telepath, a computer or Santa Claus. Now scientists say they've developed a system that combines machine learning and a brain-computer interface, or BCI, to read handwriting that takes place in the brain rather than on paper.
A team of scientists worked with a 65-year-old man paralyzed from the neck down, using sensors implanted in his brain to detect neural activity linked to writing by hand. As the volunteer imagined writing letters, that activity was fed to an algorithm that translated it, in real time, to text displayed on a screen.
"Right now, other investigators can achieve about a 50-word dictionary using machine learning methods when decoding speech," Shenoy said in a statement. "By using handwriting to record from hundreds of individual neurons, we can write any letter and thus any word which provides a truly 'open vocabulary' that can be used in most any life situation."
"Just think about how much of your day is spent on a computer or communicating with another person," Shenoy said. "Restoring the ability of people who have lost their independence to interact with computers and others is extremely important, and that is what is bringing projects like this one front and center."
The system has been demonstrated as a proof of concept in just a single patient so far. It builds off previous work done by Shenoy's team in which participants implanted with neural sensors attempted arm movements to move a cursor on a screen to point and click on letters and spell out words and sentences. This method allowed people to "type" around 40 characters per minute, whereas the new handwriting method saw rates of about 90 characters per minute, believed to be a new record for typing via BCI.
The researchers note that rate is about the same as someone of a similar age as the 65-year-old participant typing on a smartphone.
"Communication is central to how we function in society," said Debara Tucci, director of the National Institute on Deafness and Other Communication Disorders, one of the funders for the study. "In today's world of internet-based communication, people with severe speech and physical impairments can face significant communication barriers and, potentially, isolation. We hope these findings will encourage commercial development of this latest BCI technology."
Shenoy would like to see handwriting integrated into a more complex BCI-based system that also allows for decoding speech and point and click navigation like that of current smartphone interfaces.
"Having those two or three modes and switching between them is something we naturally do," he said.
Next the team aims to work with a participant who is unable to speak or suffers from a degenerative neurological disorder like amyotrophic lateral sclerosis that can rob sufferers of their speech.