Man with ALS speaks again thanks to AI

Helping people with ALS return to speaking fluently and naturally: this is the goal achieved by the new human-machine interface BrainGate2, created by the research group at the University of California at Davis. In practice, these are microelectrodes implanted in the brain that have allowed a man to return to speaking fluently, translating his brain signals in near real time. The method uses Artificial Intelligence to identify correlations between written words and neuron signals.

The study, coordinated by Sergey Stavisky , was published in Nature .

The device has been developed and tested at the moment on a single patient, a 45-year-old man, Casey Harrell, suffering from amyotrophic lateral sclerosis, who was implanted with 4 electrodes in his brain. Harrell spoke very badly due to the disease.

The man was then asked to try to pronounce sentences that appeared on a screen and the electrical signals produced by his brain were used to train the AI, which identified correlations between the written words and the signals from the neurons. The data was then transformed into words.

Harrell used the system in both prompted and spontaneous conversational settings. In both cases, the speech decoding occurred in real time, with the system continually updated to keep it accurate. The decoded words were displayed on a screen. Incredibly, they were then read out in a voice that sounded just like Harrell before he had ALS.

The study

Thus, the patient went from understanding just 4% of the words spoken to understanding 60% thanks to the synthesized voice, to the ability to even sing simple melodies. "The results of this research - said David Brandman , one of the authors of the study - offer hope to those who would like to speak but cannot". At the same time, however, the researchers themselves underline, the work done is still too limited, on a single patient, to be able to quickly arrive at a technology available on a large scale.

A vast vocabulary

The system is notable for its speed. In the first session, it took 30 minutes to reach 99.6% accuracy with a vocabulary of 50 words. In the second session, the vocabulary reached 125,000 words. With just 1.4 hours of additional training, the system reached 90.2% accuracy, rising to 97.5%. “The first time we tried the system, it cried with joy when the words the ALS patient was trying to say correctly appeared on the screen,” Stavisky explained.

Brain computer interfaces have long been studied on patients with ALS or stroke, and progress is rapid, although still tied to experimental studies. AI is now bringing a lot to the process of translating brain activity into real speech in real time. In this case, the system's algorithms were trained with data collected while the participant was asked to try to pronounce sentences shown on a computer screen.