Neural encoding & decoding of rapid, dexterous motor sequences to help people with disabilities
Concept of motor and communication brain-computer interfaces (BCIs), also referred to as brain–machine interfaces (BMIs). One or more electrode arrays are implanted in brain regions such as the primary motor cortex, dorsal and ventral premotor cortex, or intraparietal cortex. They record action potentials from hundreds to thousands of neurons and local field potentials. The recorded neural activity is then converted by a decoding algorithm into (1) computer commands for controlling a computer interface or a prosthetic (robotic) arm, or (2) stimulation patterns for functional electrical stimula-tion of muscles in a paralyzed arm. Illustration credit: Sarah Mack, Columbia University for Shenoy KV, Yu BM (2021) Brain Machine Interfaces (entirely new chapter, Chapter 39). Principles of Neural Science, 6th edition. Editors: Kandel ER, Koester JD, Mack SH, Siegelbaum SA. McGraw Hill. pdf_Page_Proofs Amazon.
Neural encoding and decoding of rapid, dexterous attempted handwriting-related motor signals: Toward a single-neuron resolution based "Brain-to-Text" BCI. Illustration credit: Erika Woodrum.
Character Building. Brain–computer interfaces (BCIs) have the potential to restore communication to people who have lost the ability to move or speak. To date, the focus has largely been on motor skills such as reaching or grasping. In this week’s issue, Francis Willett and his colleagues present the results from an intracortical BCI that decodes attempted handwriting movements from neural activity in the motor cortex and translates it to text in real time. The researchers worked with a man who is paralysed from the neck down, asking him to try to write by imagining he was holding a pen on a piece of paper. The BCI used a neural network to translate the neural signals into letters, allowing the man to reach a writing speed of 90 characters per minute with an accuracy of 94.1%. The cover features aggregated images of the alphabet derived from the study participant’s neural activity as he thought about writing. Cover image: K. Krause / Nature adapted from F. R. Willett et al. Nature 593, 249–254 (2021). pdf
Neural encoding and decoding of rapid, dexterous attempted speech-related motor signals: Toward a single-neuron resolution based "Brain-to-Speech" BCI. Illustration credit: Erika Woodrum.
The Stanford Neural Prosthetics Translational Laboratory (NPTL) conducts research aimed at providing clinically useful brain-machine interfaces (also termed brain-computer interfaces) for people with paralysis, and understanding the related underlying human neuroscience (i.e., studying populations of individual neurons). Our goal is to extract signals (information) recorded from electrodes surgically implanted in the brain to provide accurate, high-speed, and robust control of assistive technologies. More specifically, we investigate neural encoding and decodig of rapid, highly-dexterous movement sequences to help people with disabilities.
Current projects include design and validation of high-performance and highly-robust systems that:
- Generate text to help restore communication by decoding attempted handwriting: "Brain-to-Text BCIs" (Willett et al. Nature 2021 pdf).
- Generate speech to help restore communication by decoding attempted speech: "Brain-to-Speech BCIs" (Stavisky et al. eLife 2019 pdf, Stavisky et al. J Neural Eng 2020 pdf, Wilson*, Stavisky* et al. J Neural Eng 2020 pdf).
- Generate full body (both arms, both legs) control signals to help restore arm and leg movements: "Full-body BCIs" (Willett*, Deo* et al. Cell 2020 pdf)
- Control of 2D point-and-click cursors to help restore computer, tablet and phone operation: "2D point-and-click BCIs" (Pandarinath*, Nuyujukian* et al. eLife 2017 pdf; Nuyujukian et al. PLoS One 2018 pdf).
- Fundamental neuroscience investigations of these uniquely human, high-speed and highly-dexterous movement sequences employing single-neuron resolution ensemble recordings.
- Advancing analytical methods: Computation Through Dynamics (CTD; Vyas et al. Ann Rev Neurosci 2020 pdf) and Dynamical Systems Framework (DSF; Shenoy et al. Ann Rev Neurosci 2013 pdf) experiments and analyses.
Projects are NIH NIDCD, NIH BRAIN, NIH NINDS, Simons Foundation and Howard Hughes Medical Institue (HHMI) supported.