A paralyzed man has become the first human to have electrodes implanted in his brain’s sensory cortex, allowing him to ‘feel’ his robotic arm as it manipulates various objects.
Sensory input is crucial in understanding how to hold an object or if its in danger of slipping.
In a series of tests, Copeland was tasked with picking up cylinders, spheres and cubes and placing them on a box.
No longer required just to rely on sight, Nathan Copeland can complete tasks in about half the time it used to take.
With stimulation of his sensory cortex, Copeland was able to pick up a glass of water, pour it into another cup and casually place it down in under 24 seconds.
Without the senes of ‘touch,’ the glass fell over and it took more than a minute to finish the task.
Nathan Copeland (pictured) has electrodes in his motor and sensory cortex that allow him to control a robotic arm and ‘feel’ what its metal hand is touching
In 2016, bioengineers offered to implant electrodes in both the motor and sensory cortexes of Copeland’s brain to provide him some mobility through a robotic ‘arm.’
Copeland was able to use his robotic arm to shake President Barack Obama’s hand.
The electrodes implanted in his brain ead to ports on the top of Copeland’s head connected to computer cables that relay information.
Sensory input is vital for basic tasks like picking up a coffee mug or telephone. Without it, it can be difficult to tell if you’re grabbing something too tightly or if its slipping
The electrodes placed in the motor cortex track about 200 channels of information heading out from the brain, according to Jennifer Collinger, a biomedical engineer with the University of Pittsburgh’s Department of Physical Medicine and Rehabilitation.
Most brain-computer interfaces (BCI) center on the motor cortex, which guides movement.
Done correctly, they allow a user to signal they want to move a robotic limb.
But without any sensory connection, the movement is much less exact.
With stimulation of his sensory cortex, Copeland was able to pick up a cube and place it on a box in under 4 seconds. Without that stimulation it took more than seven seconds
When we pick up an object—a phone, a pen, a coffee cup—we rely on sensory data as much as visual cues to ensure we’ve got a good grip on it.
‘The sensory piece of what we do as humans to make that happen is really important,’ David Putrino, director of rehabilitation innovation at Mount Sinai’s Icahn School of Medicine told Health Day.
Without understanding where your hand is in space, it can be difficult to tell if you’re grabbing something too tightly or too loosely, Putrino said.
It’s second-nature for most able-bodied people but for a robot it’s a huge obstacle.
‘Before, when I didn’t have stimulation, I would be able to see the robotic hand touching an object but sometimes I would go to lift it off the table and it would fall out or be in the hand awkwardly, so I would spent extra time making sure I actually had hold of something before I started to move it,’ Copeland, 34, told Health Day.
Now bioengineers are stimulating Copeland’s sensory cortex, allowing the system to feed information to him about what his robotic hand is ‘feeling.’
‘Stimulating in the fingertip-related areas in the brain generated sensations that felt like they were coming from the participant’s own hand,’ Collinger told NPR.
In a series of tests, Nathan Copeland (anove) was tasked with picking up cylinders, spheres and cubes and placing them on a box. By stimulating his sensory cortex, engineers were able to allow Copeland to ‘feel’ what the robot’s hand was touching
In a series of tests, Copeland was tasked with picking up cylinders, spheres and cubes and placing them on a box.
In research published this month in the journal Science, engineers report he could deftly guide the object to the target significantly faster with sensory feedback than without.
‘In a sense, this is what we hoped would happen—but perhaps not to the degree that we observed,’ Collinger said in a statement. ‘Sensory feedback from limbs and hands is hugely important for doing normal things in our daily lives, and when that feedback is lacking, people’s performance is impaired.’
On average, Copeland cut down his time performing tasks in half, from about 20 seconds to 10.
Some differences were even more profound: With sensory stimulation, Copeland was able to guide the robot arm to pick up a rock and place it on a box in under 5 seconds.
Without the input, it took more than 15 seconds.
‘The sensation would actually change intensity based on how much force the hand was exerting on the object,’ Copeland told NPR. ‘So I could also tell if I had a firm grip on it or not. The control is so intuitive that I’m basically just thinking about things as if I were moving my own arm,’ he says.
The synthesis of movement and sensation was fairly smooth, Copeland said in a statement.
‘There wasn’t really any point where I felt like stimulation was something I had to get used to,’ he said. ‘Doing the task while receiving the stimulation just went together like PB&J.’
Picking up a glass of water, pouring it into another cup and then safely putting the empty glass down took about 24 seconds with sensory stimulation. Without it, Copeland dropped the glass and took more than a minute and 15 seconds to finish the task.
Picking up a glass of water, pouring it into another glass and then safely putting the glass down took about 24 seconds with sensory stimulation.
Without it, Copeland dropped the glass on the table and took over a minute and 15 seconds to right the cup and finish the task.
‘I could see the hand was touching the object, but I also had that extra reassurance and confidence that I definitely had made contact and I was applying a certain amount of pressure,’ Copeland said.
‘I knew if I went to lift the object off the table, it wouldn’t fall out of my grasp.’
The team acknowledges there’s still a long way to go before their technology can be implemented as a practical aid for the disabled.
‘This is a first step in a technology that’s incredibly invasive. It requires open brain surgery,’ Putrino said. ‘You’re implanting electrodes that have a limited useful life span in the brain tissue,’
Copeland says he wasn’t under any delusion the project would fully restore his mobility.
‘It’s literally the beginning of research,’ he told Health Day. ‘I just hope that through me spending my time trying to push the science forward, someone down the road that has an injury like mine and some sort of condition that limits their function, that this technology can help them in the future.’