One participant, "T5," a 63-year-old man who had never used a BCI before, needed only 37 seconds of calibration time before he could control a computer cursor to reach targets on a screen, just by imagining using his hand to move a joystick.
Dr. David Brandman, lead author of the study and an engineering postdoctoral researcher at Brown University, said that while additional innovations will help to move implantable BCIs like BrainGate toward clinical availability for patients, this advance of rapid, intuitive calibration is a key one. It could allow future users and their caregivers to use the system much more quickly and to keep it calibrated over the long term.
"Up until now, getting going with a BCI has required oversight from a trained technician," said Brandman, whose research engaged a team of scientists, engineers and physicians from across the BrainGate collaboration, which includes Brown, the Providence Veterans Affairs Medical Center (PVAMC), Massachusetts General Hospital (MGH), Case Western Reserve University and Stanford University.
"Our results suggest that once we begin to record from our users' motor cortex, we can push the 'calibrate' button and allow the user to rapidly develop intuitive and high-quality cursor control," Brandman added. "Our new approach removes the need for technician oversight during calibration. This brings us one step closer to getting this system to people who we hope will benefit from it."
Senior author Dr. Leigh Hochberg - director of the BrainGate consortium and clinical trial, professor of engineering at Brown, critical care neurologist at MGH and director of the V.A. Rehabilitation Research and Development Center for Neurorestoration and Neurotechnology at the PVAMC - agreed that the new paper is a key advance.
"In the past few years, our team has demonstrated that people with tetraplegia can use the investigational BrainGate BCI to gain multidimensional control of a robotic arm, to point-and-click on a computer screen to type 39-plus correct characters per minute, and even to move their own arm and hand again - all simply by thinking about that movement," Hochberg said.
"But each of these achievements required time - not to train the user, but to train the computer how to interpret the neural activity that reflects our users' intuitively generated intended hand movements. Traditionally, this had required a many-minutes-long, exceedingly boring creation of a neural decoder. Here, the user is immediately engaged in building that decoder and can see cursor control evolve sometimes even in the first 30 seconds and continue to improve in just three minutes. Watching our participants do this has been really exciting."
Rapid calibration
The paper shows that without any compromise of cursor control performance, the new calibration process does away with several steps and about eight minutes of time compared to the traditional calibration method. In that older approach, trial participants worked iteratively and explicitly through target acquisition attempts while a trained clinical research technician monitored and updated the software.The new decoder software, Brandman said, employs statistical learning algorithms. These allow the BCI system to more quickly learn what the neural signals recorded by implants in the motor cortex of the brain are communicating about the users' intention to move their arms and hands. This more rapid learning allowed the team to streamline calibration.
Essentially, all participants had to do was imagine moving a cursor from the center of a circle of eight targets to whichever target lit up a different color. Lingering over the target for a moment was enough to indicate selection. For about the first minute, study data show, cursor control was erratic as indicated by the wobbly paths from center to target. But by the second minute, the paths the participants traced had become much straighter.
Using one version of the decoder, for example, calibration performance saturated for T5 within about 43 seconds, for "T10," a 34-year-old man, at around 100 seconds and for "T8," a 55-year-old man, within 136 seconds. All three participants had sustained spinal cord injuries resulting in tetraplegia.
Each participant in the research also performed another task in which they selected randomly appearing targets in an on-screen grid. These tests were designed to measure a "bit rate" of information akin to typing on a keyboard and to compare performance after using the traditional calibration method vs. using the new method. The results showed that participants achieved the same bit rate after the new one-step, three-minute method than they could achieve using the old multi-step, technician-supervised, 11-minute method.
Working the very first time
For many other BCIs, Hochberg said, the expectation is that the user and the research team will work together for days to weeks before the user can gain useful control of the BCI. At least in theory, however, an intracortical BCI should work the very first time. With this new calibration method, the BrainGate team set out to test whether a brand-new BCI user - somebody who had never used either the BrainGate BCI or any other - might be able obtain cursor control on his first attempt."Of course, we only get one chance to see whether a BCI user can intuitively gain cursor control their very first time," Hochberg said.
Brandman recalled that exact moment with T5.
"The day before his first attempt at using the intracortical BCI for controlling a computer cursor, I described to T5 that the system was going to be recording from a part of the brain that was responsible for coordinating hand and arm movement," Brandman said. "I then asked him to suggest imagery that would be intuitive for him to use, and he suggested using a joystick.
"The first time he tried to use the system, it didn't work. I was puzzled. But then T5 asked me, 'When should I start?' So I explained to him that he should start using the system with the joystick imagery he had suggested. He then rapidly gained control of the cursor and hit his first target in 37 seconds. Then he said, 'Score one for the guy in the wheelchair!'"
Jaimie Henderson, principal investigator at the Stanford BrainGate site, noted, "It was pretty amazing. We asked T5 to try it out. And it just worked. He was moving the cursor around the screen as well as others who have been using the system for months or years."
Krishna Shenoy, the Lim professor of electrical engineering, bioengineering and neurobiology at Stanford and an investigator of the Howard Hughes Medical Institute, noted, "This substantial reduction in training time is remarkable as it requires the new algorithm to make very efficient use of neural data coming in for so little time, under a minute in some cases, and helps point the way toward further advances of real-world importance."
As a critical care neurologist and director of the Center for Neurotechnology and Neurorecovery at MGH Neurology, Hochberg said he sees "far too often the effects of stroke, neuromuscular failure from ALS, spinal cord injury, and a host of other diseases or injuries of the nervous system that leave people unable to move or unable to communicate. And I want nothing more than to be able to provide a technology that restores that ability to communicate and restores the ability to move. While there's a lot more research and development ahead, watching the video of T5 gain cursor control on his first swing reinforced the potential for neurorestoration that intracranial BCIs may provide in the future."
Imagining intuitive motion
In addition to making the calibration process quicker and easier, the research team also wanted to learn more about which imagery users found most intuitive. This is important, Brandman said, to make the system as easy to use as possible for the user, while capitalizing on the richness and complexity of the neural signals recorded from the motor cortex.So in another series of experiments, T5 performed the calibration task with each of five additional modes of attempted motion after starting out with the joystick imagery that he chose. These covered various scenarios in which some parts of the arm and hand move freely but others remain fixed. For example, in "mouse ball," T5 imagined moving his wrist and elbow as if moving the cursor with a trackball mouse, while in "whole arm" he imagined moving his arm around in free space so that his fixed index finger could point at the on-screen targets.
Performance with each mode varied slightly, but in every case T5 achieved near-peak control within 60 seconds. In the end, after seeing his results, he decided that "mouse ball" was his new favorite.
"One of the beautiful things about using intracortical recording is that we were able to allow T5 to use multiple imageries that were intuitive for him and to allow him to see the benefits of each imagery in real-time," Brandman said.
Brandman said he was also encouraged that the calibration system was robust enough to accommodate all the movement imageries that different participants preferred.
Continuing to develop the most easily usable, reliable and flexible system possible for people with tetraplegia is a main goal of this feasibility trial, Hochberg said.
David M Brandman, Tommy Hosman, Jad Saab, Michael C Burkhart, Benjamin E Shanahan, John G Ciancibello, Anish A Sarma, Daniel J Milstein, Carlos E Vargas-Irwin, Brian Franco, Jessica Kelemen, Christine Blabe, Brian A Murphy, Daniel R Young, Francis R Willett, Chethan Pandarinath, Sergey D Stavisky, Robert F Kirsch, Benjamin L Walter, A Bolu Ajiboye, Sydney S Cash, Emad N Eskandar, Jonathan P Miller, Jennifer A Sweet, Krishna V Shenoy, Jaimie M Henderson, Beata Jarosiewicz, Matthew T Harrison, John D Simeral, Leigh R Hochberg.
Rapid calibration of an intracortical brain–computer interface for people with tetraplegia.
Journal of Neural Engineering, 2018; 15 (2): 026007. doi: 10.1088/1741-2552/aa9ee7.