Abstract
Millions of people worldwide suffer from diseases that lead to paralysis through disruption of signal pathways between the brain and the muscles. Neuroprosthetic devices are designed to restore lost function and could be used to form an electronic ‘neural bypass’ to circumvent disconnected pathways in the nervous system. It has previously been shown that intracortically recorded signals can be decoded to extract information related to motion, allowing non-human primates and paralysed humans to control computers and robotic arms through imagined movements1,2,3,4,5,6,7,8,9,10,11. In non-human primates, these types of signal have also been used to drive activation of chemically paralysed arm muscles12,13. Here we show that intracortically recorded signals can be linked in real-time to muscle activation to restore movement in a paralysed human. We used a chronically implanted intracortical microelectrode array to record multiunit activity from the motor cortex in a study participant with quadriplegia from cervical spinal cord injury. We applied machine-learning algorithms to decode the neuronal activity and control activation of the participant’s forearm muscles through a custom-built high-resolution neuromuscular electrical stimulation system. The system provided isolated finger movements and the participant achieved continuous cortical control of six different wrist and hand motions. Furthermore, he was able to use the system to complete functional tasks relevant to daily living. Clinical assessment showed that, when using the system, his motor impairment improved from the fifth to the sixth cervical (C5–C6) to the seventh cervical to first thoracic (C7–T1) level unilaterally, conferring on him the critical abilities to grasp, manipulate, and release objects. This is the first demonstration to our knowledge of successful control of muscle activation using intracortically recorded signals in a paralysed human. These results have significant implications in advancing neuroprosthetic technology for people worldwide living with the effects of paralysis.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 51 print issues and online access
$199.00 per year
only $3.90 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Aflalo, T. et al. Decoding motor imagery from the posterior parietal cortex of a tetraplegic human. Science 348, 906–910 (2015)
Bansal, A. K., Truccolo, W., Vargas-Irwin, C. E. & Donoghue, J. P. Decoding 3D reach and grasp from hybrid signals in motor and premotor cortices: spikes, multiunit activity, and local field potentials. J. Neurophysiol. 107, 1337–1355 (2012)
Chapin, J. K., Moxon, K. A., Markowitz, R. S. & Nicolelis, M. A. Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nature Neurosci. 2, 664–670 (1999)
Hochberg, L. R. et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 485, 372–375 (2012)
Hochberg, L. R. et al. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442, 164–171 (2006)
Kennedy, P. R. & Bakay, R. A. Restoration of neural output from a paralyzed patient by a direct brain connection. Neuroreport 9, 1707–1711 (1998)
Santhanam, G., Ryu, S. I., Yu, B. M., Afshar, A. & Shenoy, K. V. A high-performance brain-computer interface. Nature 442, 195–198 (2006)
Serruya, M. D., Hatsopoulos, N. G., Paninski, L., Fellows, M. R. & Donoghue, J. P. Instant neural control of a movement signal. Nature 416, 141–142 (2002)
Taylor, D. M., Tillery, S. I. & Schwartz, A. B. Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829–1832 (2002)
Velliste, M., Perel, S., Spalding, M. C., Whitford, A. S. & Schwartz, A. B. Cortical control of a prosthetic arm for self-feeding. Nature 453, 1098–1101 (2008)
Wessberg, J. et al. Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature 408, 361–365 (2000)
Ethier, C., Oby, E. R., Bauman, M. J. & Miller, L. E. Restoration of grasp following paralysis through brain-controlled stimulation of muscles. Nature 485, 368–371 (2012)
Moritz, C. T., Perlmutter, S. I. & Fetz, E. E. Direct control of paralysed muscles by cortical neurons. Nature 456, 639–642 (2008)
Yousry, T. A. et al. Localization of the motor hand area to a knob on the precentral gyrus. A new landmark. Brain 120, 141–157 (1997)
Humber, C., Ito, K. & Bouton, C. Nonsmooth formulation of the support vector machine for a neural decoding problem. arXiv Preprint at http://arxiv.org/abs/1012.0958v1 (2010)
Ojala, M. & Garriga, G. Permutation tests for studying classifier performance. J. Mach. Learn. Res. 11, 1833–1863 (2010)
Kalsi-Ryan, S., Curt, A., Verrier, M. C. & Fehlings, M. G. Development of the Graded Redefined Assessment of Strength, Sensibility and Prehension (GRASSP): reviewing measurement specific to the upper limb in tetraplegia. J. Neurosurg. Spine 17, 65–76 (2012)
Kilgore, K. L. et al. An implanted upper-extremity neuroprosthesis using myoelectric control. J. Hand Surg. Am . 33, 539–550 (2008)
Peckham, P. H. et al. An advanced neuroprosthesis for restoration of hand and upper arm control using an implantable controller. J. Hand Surg. Am . 27, 265–276 (2002)
Pfurtscheller, G., Muller, G. R., Pfurtscheller, J., Gerner, H. J. & Rupp, R. ‘Thought’–control of functional electrical stimulation to restore hand grasp in a patient with tetraplegia. Neurosci. Lett. 351, 33–36 (2003)
Shaikhouni, A., Donoghue, J. P. & Hochberg, L. R. Somatosensory responses in a human motor cortex. J. Neurophysiol. 109, 2192–2204 (2013)
Fetz, E. E., Finocchio, D. V., Baker, M. A. & Soso, M. J. Sensory and motor responses of precentral cortex cells during comparable passive and active joint movements. J. Neurophysiol. 43, 1070–1089 (1980)
Kirshblum, S. C. et al. International Standards for Neurological Classification of Spinal Cord Injury: cases with classification challenges. J. Spinal Cord Med. 37, 120–127 (2014)
Cox, R. W. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput. Biomed. Res. 29, 162–173 (1996)
Smith, S. M. et al. Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage 23 (Suppl. 1), S208–S219 (2004)
Mallat, S. A Wavelet Tour of Signal Processing (Academic, 1998)
Sharma, G. et al. Time stability of multi-unit, single-unit and LFP neuronal signals in chronically implanted brain electrodes. Bioelectron. Med . 2, 63–71 (2015)
Kennedy, P. Changes in emotional state modulate neuronal firing rates of human speech motor cortex: a case study in long-term recording. Neurocase 17, 381–393 (2011)
Scholkopf, B. et al. Comparing support vector machines with Gaussian kernels to radial basis function classifiers. IEEE Trans. Signal Process. 45, 2758–2765 (1997)
Bradski, G. The OpenCV library. Doctor Dobbs J . 25, 120–126 (2000)
Acknowledgements
We thank the study participant for his dedication and his family for their support. We also thank the development team and management at Battelle Memorial Institute for their support, the surgical support team, C. Majstorovic for assistance with data analysis and equipment during sessions, S. Preston for stereo camera coding and troubleshooting, the clinical study support staff, W. Pease for performing the electromyogram and nerve conduction studies, and M. Zhang for assistance with figure preparation. Financial support for this study came from Battelle Memorial Institute and The Ohio State University Center for Neuromodulation.
Author information
Authors and Affiliations
Contributions
C.E.B., N.V.A., D.A.F., G.S., B.C.G., M.A.B., A.S., A.G.M., D.M.N., P.B.S., W.J.M., and A.R.R. conceived and designed the experiments and fMRI procedure. N.V.A., D.A.F., G.S., B.C.G., M.A.B., and A.G.M. performed the experiments. W.J.M. and M.A.B. were involved in participant recruitment. A.R.R., M.D., and A.S. performed the surgery. All authors contributed to writing the paper.
Corresponding authors
Ethics declarations
Competing interests
During this study, the authors were employed by their respective institutions that provided funding for this work. The custom neuromuscular stimulation sleeve and computer algorithms described in this study are covered by one or more patent applications. The authors associated with these patents are: C.E.B., N.V.A., G.S., D.A.F. and B.C.G., all of which were employed by Battelle at the time of this study.
Extended data figures and tables
Extended Data Figure 1 Neural modulation.
a, b, Examples of rasters and peristimulus histograms from simultaneously recorded units with response to attempted or performed wrist, elbow, and shoulder movements are shown in (a) for channel 68, unit 1 and in (b) for channel 37, unit 1. The participant was presented with cues to attempt wrist flexion (WF), wrist extension (WE), wrist radial deviation (WR), wrist ulnar deviation (WU), elbow extension (EE), or to perform elbow flexion (EF), shoulder flexion (SF), or shoulder abduction (SA). Each cue was presented for a duration of 3 s with a random jitter of 0–2 s followed by a 3 s period with a jitter of 0–2 s. We presented 20 trials of each in random order. The top part of each subpanel is a raster, the black dots represent spikes, each row of spikes represents data from one trial. All trials were aligned on cue presentation (time zero, red dashed line). On the right of each set of panels is an example of 500 randomly selected waveforms from the discriminated unit (blue). The thick yellow line represents the average waveform for the unit. The top panel shows unit 1 on channel 68. This unit responded well to movement around the shoulder and elbow. The bottom panel shows activity from unit 1 on channel 37 that responded to wrist, elbow, and shoulder movements. Because the participant was asked to imagine and hold the movement throughout the cue period, the latent increase of activity after the cue ended (during the rest period) was probably due to the participant imagining the antagonist movement to return to a neutral position.
Extended Data Figure 2 Stimulation artefact removal.
Stimulation artefact was removed from the signal associated with each electrode before further processing (see Methods). a, Example of the voltage data during stimulation for a single electrode shows a period before stimulation and a period during stimulation containing three stimulation artefacts (large amplitude pulses). b, Large artefacts were detected and removed to reduce their effects; however, residual artefacts could remain (causing a twofold to eightfold increase in MWP during stimulation). c, The period before stimulation is shown for reference. d, The period with three stimulation artefacts (after removal) is shown after concatenation/rejoining of the signal segments, and wavelet decomposition is then performed on these shorter, concatenated data (see Methods).
Extended Data Figure 3 Individual movement task performance.
The participant was visually cued to attempt each of the six trained movements. a, Snapshot of each movement. b, The performance (sensitivity, specificity, and accuracy) was measured by automatic evaluation of video frames of hand movements for each of the six moves. The overall accuracy of the NBS was 70.4 ± 1.0% (P < 0.01). Statistics for individual movements are calculated against a movement specific cue vector with two classes: one class for the movement of interest and one class for all other moves plus rest, which we refer to as the non-target class. Sensitivity is the percentage of video frames during cued movement where the correct movement was observed. It captures the ability of the participant to initiate the specific movement in response to the cue and his ability to sustain the movement for the duration of the two second cue. Specificity is the percentage of video frames during non-target cues that were correctly identified. Accuracy is the percentage of video frames where the observed hand position matched the cue vector for that specific movement. It can also be calculated as a weighted average of sensitivity and specificity, with the weights corresponding to the relative frequency of the two classes. As such, the sensitivity and specificity give additional performance information which can be especially useful when the relative frequency of the classes is skewed towards one class. Overall accuracy measures the performance of all six decoders simultaneously by calculating the percentage of video frames where the observed hand position matches the position the participant was asked to achieve. Thus, errors on any of the six movements are incorporated in the overall accuracy, so it is expected to be lower than the accuracy of any individual movement. Error bars, ±2 s.d. (Video taken by D. Friedenberg.)
Extended Data Figure 4 MWP during a combination movement task.
MWP as calculated by the NBS algorithm during a task combining imagined movement, stimulation-induced movement, and non-paralysed muscle movement is shown. The participant’s reaction time and a 1 s boxcar filter used to smooth the neural data creates the delay observed after the cues are presented. The participant’s hand was placed on top of a spoon set on a table and he was cued by the graphical hand to imagine gripping a spoon (solid black line), provided stimulation to evoke the actual gripping of the spoon (red line), and cued by an audible beep to transfer the spoon using the residual movement in his shoulder and elbow (black dashed line). The spoon transfer distance was approximately 45 cm to the left/right, alternating between each trial. Differences in MWP can be seen for each of the three portions of each trial. The increase in MWP between imagined movement and stimulation-induced movement is caused by residual stimulation artefact. During the transfer cue, the utilization of residual (shoulder) movement was associated with a different and consistent MWP pattern, requiring the development of a more robust decoding strategy which combined data from imagined, stimulation-induced, and transfer movements for decoder training (see Methods).
Extended Data Figure 5 Neural decoder outputs for each movement in the individual movement task.
The blue line represents the cue that the participant was trying to match. For a particular movement, when the blue line was at one, the user was prompted to imagine that movement. The black line is the decoder output for that move. The decoded movement is rest if none of the decoder outputs are above zero (dotted line); otherwise, it is determined by the maximum of all the decoder outputs. Green stars are placed above the 29 out of 30 cues in which the neural decoder correctly matched the cue. In this plot the cues have been shifted by 0.8 s to account for reaction and system lag time. Decoder output below −0.5 has been set to −0.5 for visual clarity.
Extended Data Figure 6 Evolution of MWP over time.
At the beginning of each test session, data for all channels were collected over 60 s where the participant was instructed to close his eyes and rest. No stimulation was provided during this period. MWP features with no mean subtraction were calculated to approximate the power in the multiunit wavelet bands corresponding to scales 3–6. After an initial decline the MWP stabilized at 150 days after implantation. Dashed line represents a polynomial fit of order 4.
Extended Data Figure 7 Stimulation electrode pattern.
The stimulator was calibrated to evoke movements in the hand and wrist. a, Representative map of the anode (red) and cathode (black) electrodes in the lower (L) and upper (U) stimulation cuffs used to evoke the ‘hand open’ movement during the individual movement task. b, Stimulation cuffs on the participant’s arm, with the stimulation pattern highlighted. See Supplementary Table 1 for a complete list of electrode patterns used for the individual hand movement task. (Photographs taken by N. Annetta.)
Supplementary information
Supplementary Information
This file contains Supplementary Tables 1-2. (PDF 107 kb)
Participant Attempting the Individual Hand Movement Task without the Use of the Neural Bypass System
This video shows the participant attempting the individual hand movement task without the use of the NBS. The participant was unable to successfully perform any of the movements. The small graphical hand shown on the monitor in the video, along with captions in the video, indicates which movement the participant was being cued to perform at the time. The entire test block is shown in the movie. [Video taken by Nicholas Annetta] (MP4 21121 kb)
Participant Performing Individual Hand Movement Task Using the Neural Bypass System
This video shows the participant performing the individual hand movement task with the use of the NBS. The small graphical hand shown on the monitor in the video, along with captions in the video, indicates which movement the participant was being cued to perform at the time. The entire test block is shown in the movie. [Video taken by Nicholas Annetta] (MP4 15198 kb)
Participant Performing the Functional Movement Task Using the Neural Bypass System
This video shows two representative trials where the participant was successful in performing the functional movement task with the use of the NBS. Captions in the video indicate the movement that was being performed at the time. [Video taken by Nicholas Annetta] (MP4 18630 kb)
Participant Attempting the Functional Movement Task without the Use of the Neural Bypass System
This video shows two representative trials where the participant was not successful in performing the functional movement task without the use of the NBS. He was able to force his hand around the bottle, but without any grip strength, and therefore was unable to lift the bottle and pour the contents. The participant was then asked to attempt the stir stick grasp and stirring portion of the task separately. He was able to wedge the stir stick between his fingers and use friction to lift it out of the jar, but without any grip strength, he was unable to complete the stirring motion without the stir stick falling out of his hand. Captions in the video indicate the movement he was attempting at the time. [Video taken by Nicholas Annetta] (MP4 26303 kb)
Rights and permissions
About this article
Cite this article
Bouton, C., Shaikhouni, A., Annetta, N. et al. Restoring cortical control of functional movement in a human with quadriplegia. Nature 533, 247–250 (2016). https://doi.org/10.1038/nature17435
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nature17435
This article is cited by
-
Brain control of bimanual movement enabled by recurrent neural networks
Scientific Reports (2024)
-
Motion artefact management for soft bioelectronics
Nature Reviews Bioengineering (2024)
-
Boosting brain–computer interfaces with functional electrical stimulation: potential applications in people with locked-in syndrome
Journal of NeuroEngineering and Rehabilitation (2023)
-
Neurotechnologies to restore hand functions
Nature Reviews Bioengineering (2023)
-
An implantable, wireless, battery-free system for tactile pressure sensing
Microsystems & Nanoengineering (2023)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.