This article focuses on the research and development work conducted by teams to develop a state-of-the-art brain–machine interface (BMI). BMI is composed of a series of computer chips and electrodes implanted directly into the human brain. It allows individuals with tetraplegia, a type of spinal cord injury that results in a lack of movement and feeling to all four limbs, to operate a robotic arm—and do so by thought alone. In order to create usable systems, designers of brain–machine interfaces also need to incorporate sensory feedback. Experts believe that researchers need to develop medically safe, biocompatible, and cost-effective materials and system components. A working system must be smart enough to self-correct from error states, like a self-driving car. It should also be available off the shelf, and be able to easily and effortlessly calibrate itself to the individual brains that will run it. Engineers need to focus on how a brain-controlled prosthetic might actually be used to promote rehabilitation and activities of daily living.
Nearly 10 years ago, Michael Boninger, a rehabilitative medicine specialist at the University of Pittsburgh Medical Center, met two other members of the Pitt faculty—Douglas Weber, a bioengineer, and Andrew Schwartz, a neurobiologist—at a local watering hole. Schwartz's lab had developed new ways to use electrode arrays to extract movementrelated information from the brains of monkeys. After a beer or two that evening, Schwartz told Weber and Boninger that he was sure, if he could get a computer chip into the brain of a person with a spinal cord injury, he could not only restore their lost abilities to move, he could teach them to play the piano.
Boninger, who had an undergraduate degree in mechanical engineering from Ohio State, said he laughed at first. “I had just started piano lessons for the first time. I was struggling with it—for the record, I’m still struggling with it,” he said. “But in talking to Andy and Doug that night, it seemed like something within the realm of possibility. I told Andy, ‘I’m sure we can make that happen,’ and from there we formed this amazing collaboration.”
Together, the three have developed a state-of-the-art brain-machine interface, or BMI, that's composed of a series of computer chips and electrodes implanted directly into the human brain. It allows individuals with tetraplegia, a type of spinal cord injury that results in a lack of movement and feeling to all four limbs, to operate a robotic arm—and do so by thought alone.
Such mind-controlled devices may seem like something straight out of a science fiction story—or, at the very least, a story that never leaves the bar where it was first concocted. But brain-controlled prosthetics are being developed not just at UPMC, but across the globe. Over the past five years, these prosthetics have enabled paralyzed individuals to grasp a water bottle, pour a cup of coffee, or play the video game Guitar Hero on a computer using their thoughts alone.
But that's a long way from engineering safe, practical brain-controlled assistive devices that can handle the mechanics of everyday living, such as walking on stairs or buttoning a shirt. Can scientists make BMIs both useful and usable outside the confines of the research laboratory?
The answer is still a ways off, but so far researchers are making real progress toward practical brain-controlled prosthetics, which are also called neuroprosthetics. Some new BMIs, like the UPMC model, involve invasive surgical implants that conduct and relay the brain's natural signals to robotic limbs. Others rely on electroencephalography (EEG) caps—systems that can read and relay neural signaling patterns from outside the skull—to operate some form of prosthetic.
Some power limb-like prosthetics like the one being tested at UPMC, while others command specialized exoskeletons that surround and mobilize paralyzed limbs.
The ultimate goal, Boninger said, is to develop new assistive technologies that will help individuals with tetraplegia, as well as those who may experience amputations, stroke, neurodegenerative disease, or other medical issues, maximize their independence.
“The technologies becoming available are incredible,” said Jose Contreras-Vidal, an engineer and neuroscientist at the University of Houston who has been developing an EEG-controlled robotic exoskeleton called Rex. “But at the end of the day, if the patient doesn’t want to use it—if we can’t create something that the patient can use in an independent way—it doesn’t matter.”
Decoding the Brain
Nathan Copeland admits he was a bit anxious to meet President Obama. A participant in the UPMC BMI research program, Copeland, 30, has been wheel-chair-bound for nearly a decade after a terrible car accident. As he and the UPMC team prepared for a big meet-and-greet with the president at the White House Frontiers Conference last October, the researchers were fiddling almost constantly with the BMI set-up attached to Copeland's head. Copeland said he kept telling himself to calm down, that President Obama was “just a guy.”
“At the end of the day, if the patient doesn’t want to use it—if we can’t create something that the patient can use in an independent way—it doesn’t matter.”
—Jose Contreras-Vidal, engineer and neuroscientist at the University of Houston
“He's an incredible guy, sure—but still just a guy,” he says. “But it was such a cool thing to talk to him. He wasn’t just there for some photo op. He was there because he was really interested in the science.”
For a decade leading up to the White House Frontiers Conference, Boninger and his interdisciplinary team, including his Pitt drinking companions, moved step by step to build Copeland's BMI system.
First they worked off Schwartz's animal-model studies and tested whether they could implant recording arrays into the brains of epileptic patients who were scheduled for craniotomies as part of their treatment.
The researchers learned how to successfully record neural signals from the human brain using a specially designed biocompatible neural array. They then learned how to remove the noise and translate those signals into commands for the robotic arm, and then finally they developed a way to provide some crucial sensory feedback to the person from the robotic arm.
Copeland's current system is anything but simple. He first had an array of chips and electrodes surgically implanted into his brain. Those components are linked to a decoder—a device that reads and relays brain signals—that protrudes from Copeland's skull.
Currently, that decoder is linked by thick wires to several computers as well as to the robotic arm. This setup means that Copeland's ability to move the robotic arm is fairly effortless. But there's a lot of sophisticated and expensive computing firepower at work behind the curtain.
The researchers’ work is still far from done. Many people give up on assistive devices because they are too cumbersome, so minimizing the size, weight, and ungainliness of the equipment is still a priority for the team,
Jennifer Collinger, a biomedical engineer on the UPMC team, said, “[Nathan] is literally plugged into the system, which can limit his mobility.”
For that reason, making the system wireless is critical, Boninger said: “The point is to develop a system that could one day help Nathan and others like him do a better job with everyday tasks like brushing his teeth, combing his hair, or holding and drinking from a cup of coffee unassisted. We’re not there yet.”
But technology moves fast. Many BMI system components have already significantly decreased in size, Contreras-Vidal said. The amplifiers that help strengthen the magnitude of neural signals used to take up half a room; now, they can fit in the palm of a hand. They’ve also improved in terms of accuracy and signal-to-noise ratio.
As long as researchers keep users’ needs front and center, they will be able to create BMIs that can benefit them outside the laboratory, Contreras-Vidal said. That's why both the UPMC and University of Houston teams collaborate with computer, electronic, and mechanical engineers to find smaller, more powerful system components, as well as biocompatible implants and light, durable materials for the prosthetics or exoskeletons themselves.
Prosthetics that Sense
To create usable systems, designers of brain-machine interfaces also need to incorporate sensory feedback. After all, our limbs don’t work in a vacuum. Our hands, arms, and legs pick up important haptic information from the environment—information that's processed in the brain or spinal cord to generate signals that tell the limbs the appropriate amount of speed, grip, and force to use. This allows our limbs to move effortlessly through space.
The UPMC BMI is set up so that Copeland can receive sensory feedback directly to his brain when someone touches the robotic arm. While he describes those feelings as “weird,” they could one day help him better calculate the force with which to grasp and pull items. That feedback already improves his performance on many such tasks.
Lewis Wheaton, Director of the Cognitive Motor Control Lab at the Georgia Institute of Technology in Atlanta, says that while BMIs can now let an individual with a spinal cord injury to feel something, we still don’t know about how to optimally “sensorize” a hand.
“There are a lot of things we don’t necessarily understand even within a biological hand very well yet,” Wheaton said. “Where do you need feedback, exactly, to open a jar? To grasp a fragile object? To use appropriate force? These are open questions that will impact the mechanics—and where and what sort of sensors you put on the prosthetic,” he explained.
“We’ve had three decades of basic science that's shown us how to translate neural activity into control signals for the robot,” UPMC's Collinger said. “We’re at the point where we can move the arm in space fairly well and do simple grabs and postures.”
Paralyzed individuals would welcome more freedom and mobility, but they don’t necessarily want to have brain surgery to get it.
But more sophisticated levels of control are needed, Collinger added—and that means figuring out how to actually use sensory feedback to do more dexterous manipulation tasks.
“It's something that is going to take some time,” she said.
A Less Invasive Approach
Though many paralyzed individuals would welcome more freedom and mobility, they don’t necessarily want to have brain surgery to get it. That's why Contreras-Vidal's team has focused on a noninvasive approach using EEG recording, involving a specialized hat that looks like a cross between an aviator's helmet and your grandmother's favorite shower cap.
Like the UPMC team, Contreras-Vidal's team has had to overcome a number of engineering hurdles, including developing the right algorithms to decode the EEG signals. “The algorithms to read the signals are improving quite dramatically,” he said.
The EEG cap, like the implanted array, can track and translate the brain signals required to move a prosthetic—in this case, a robotic leg exoskeleton called Rex. But the EEG cap is not always as effective or accurate as the UPMC team's arrays, Contreras-Vidal said. That said, it may still offer a less risky option, especially for individuals who may have medical issues beyond their tetraplegia or paralysis, he added.
Still, some people will balk at the idea of wearing a large, white wired cap out in the world, Contreras-Vidal admitted. This means there are design challenges to be addressed before such a cap could find widespread use.
“We’d like to come up with different looks for the cap that are based on people's preferences,” Contreras-Vidal said. Alternatively, they could hide the recording electrodes just below the skin with a minor incision.
“That way you won’t see them on top of the head and you can still have an easy wireless network of electrodes,” he said.
There are other engineering problems that need to be solved before any BMI system, internal or external, can see prime time, Contreras-Vidal said.
Researchers need to develop medically safe, biocompatible, and cost-effective materials and system components. A working system must be smart enough to self-correct from error states, like a self-driving car, Contreras-Vidal said.
It should also be available off the shelf, he said, and be able to easily and effortlessly calibrate itself to the individual brains that will run it.
For this reason, engineers in the field must agree about system inputs and outputs to create a plug-and-play system with different options. In that way, individuals who need different kinds of assistance can create a neuroprosthetic that will best work for them.
Toward Everyday Use
There's no doubt that today's BMI systems are cool. But ultimately they must be more than cool. They must be useful, and they must not be so burdensome or unbecoming that people don’t want to wear them, Contreras-Vidal said.
A working system must be smart enough to self-correct, available off the shelf, and able to easily calibrate itself to individual brains.
Wheaton agrees. “While people have tried to integrate sensors and different elements of robotics into prosthetic design, you find that a lot of individuals, particularly those that use upper-limb prostheses, tend to go for simpler, hook-like devices,” he said. They’re easier to learn how to use, and often lighter and better balanced. “And you can’t ignore that they are a lot less expensive than the fancier options,” he added.
Engineers, therefore, need to focus on how a brain-controlled prosthetic might actually be used to promote rehabilitation and activities of daily living, Wheaton said.
According to him, they should consider whether the device will integrate into everyday behavior; whether it's comfortable, appropriate, and maneuverable; whether it feels natural and is also easy to use; and whether it can handle weight or water. “Once you get out of the laboratory, all these questions start to bind together, and you need to find a way to answer them,” he said.
That's why having a multidisciplinary research team is so important, Boninger said. It's also why he, Schwartz, and Weber still meet at the local bar from time to time—but now they include a host of other researchers from a variety of science and engineering disciplines. With so many different perspectives and opinions, he is hopeful that he will one day be able to offer patients a system that can actually maximize their independence.
“This is going to happen by taking small steps,” Boninger said. “We may not get there in 10 years. But we will get there eventually.”