EFTA00304368.pdf

Articles in PresS. J Neurophysiol (July 13, 2011). doi:10.1152/0.00104.2011 Neural decoding of treadmill walking from non-invasive, 2 electroencephalographic (EEG) signals 3 4 5 Alessandro Presacco', Ronald Goodmans, Larry Forrester" and Jose Luis Contreras-Vidal 23 6 7 8 University of Maryland, College Park: 9 'Neural Engineering and Smart Prosthetics Research Laboratory, Department of Kinesiology, 10 School of Public Health II 2Fischell Department of Bioengineering 12 3Graduate Program in Neuroscience and Cognitive Science; 13 14 University of Maryland, Baltimore: 15 'Department of Physical Therapy & Rehabilitation Science, University of Maryland School of 16 Medicine 17 'Veteran Affairs Medical Center, Baltimore 18 19 20 Author Contributions: JLCV conceived the research; JLCV designed the experiment with 21 assistance from LF; AP, LF and RG performed the research at the VAMC; AP and JLCV 22 designed the decoders, and analyzed the data at UMCP; AP and JLCV wrote the paper; and LF 23 and RG edited the manuscript. Correspondence to: Alessandro Presacco or Jose L. Contreras-Vidal School of Public Health, Department of Kinesiology SPH Building, University of Maryland, College Park College Park, MD 20742 Tel: (305) 496 5457, Email: (apresacc@umd.edu; pepeum@umd.edul 24 Copyright © 2011 by the American Physiological Society. EFTA00304368 25 26 27 28 Abstract 29 Chronic recordings from ensembles of cortical neurons in primary motor and somatosensory 30 areas in rhesus macaques provide accurate information about bipedal locomotion (Fitzsimmons et 31 al. 2009). Here we show that the linear and angular kinematics of the ankle, knee and hip joints 32 during both normal and precision (attentive) human treadmill walking can be inferred from 33 noninvasive scalp electroencephalography (EEG) with decoding accuracies comparable to those 34 from neural decoders based on multiple single-unit activity (SUAs) recorded in nonhuman 35 primates. Six healthy adults were recorded. Participants were asked to walk on a treadmill at their 36 self-selected comfortable speed while receiving visual feedback of their lower limbs (i.e., 37 precision walking), to repeatedly avoid stepping on a strip drawn on the treadmill belt. Angular 38 kinematics of the left and right hip, knee and ankle joints and EEG were recorded, and neural 39 decoders were designed and optimized using cross-validation procedures. Of note, these decoders 40 were also used to accurately infer gait trajectories in a normal walking task that did not require 41 subjects to control and monitor their foot placement. Our results indicate a high involvement of a 42 fronto-posterior cortical network in the control of both precision and normal walking and suggest 43 that EEG signals can be used to study in real-time the cortical dynamics of walking and to 44 develop brain-machine interfaces aimed at restoring human gait function. 45 46 47 Key Words: BCI; BMI; EEG; neural decoding; treadmill; walking 48 49 50 51 2 EFTA00304369 52 53 54 Introduction 55 Little is known about the organization, neural network mechanisms and computations underlying 56 the control of walking in humans (Choi and Bastian 2007). Although central pattern generators 57 for locomotion are important in the control of walking, supra-spinal networks, including the 58 brainstem, cerebellum and cortex, must be critical as demonstrated by the changing motor and 59 cognitive (i.e., spatial attention) demands imposed by bipedal walking in unknown or cluttered 60 dynamic environments (Choi and Bastian 2007; Grillner et al. 2008; Nielsen 2003; Rossignol et 61 al. 2007). Neuroimaging studies show that rhythmic foot or leg movements recruit primary motor 62 cortex (Christensen et al. 2001; Dobkin et al. 2004; Heuninckx et al. 2005; Heuninckx et al. 2008; 63 Luft et al. 2002; Sahyoun et al. 2004), whereas electrophysiological investigations demonstrate 64 electrocortical potentials related to lower limb movements (Wieser et al. 2010), as well as a 65 greater involvement of human cortex during steady-speed locomotion than previously thought 66 (Gwin et al. 2010a, 2010b). In this regard, studies using functional near-infrared spectroscopy 67 (fN1RS) show involvement of frontal, premotor and supplementary motor areas during walking 68 (Harada et al. 2009; Miyai et al. 2001; Suzuki et al. 2008; Suzuki et al. 2004). That primary 69 sensorimotor cortices carry information about bipedal locomotion has been directly proven by the 70 work of Nicolelis and colleagues (Fitzsimmons et al. 2009), who demonstrated that chronic 71 recordings from ensembles of cortical neurons in primary motor (MI) and primary somatosensory 72 (S1) cortices can be used to predict the kinematics of bipedal walking in rhesus macaques. 73 However, neural decoding of bipedal locomotion in humans has not yet been demonstrated. Here 74 we compare the predictive power of neural decoders based on human scalp (noninvasive) EEG 75 signals during treadmill walking with that reported from multiple single unit activity (SUA) in the 76 rhesus monkey performing bipedal treadmill walking (Fitzsimmons et al. 2009). We demonstrate 77 the feasibility of using scalp EEG to reconstruct the detailed kinematics of human walking, and 78 the potential of the proposed approach as a new tool for inferring the cortical contributions to 79 walking. 3 EFTA00304370 80 81 82 83 84 Materials and Methods 85 86 Experimental setup and procedure. Six healthy adults, aged 18-45 (3 male, 3 female) with no 87 history of neurological disease or lower limb pathology and free of injury participated in the 88 study after giving informed consent. The study was conducted with approved protocols from the 89 Institutional Review Boards at the University of Maryland College Park, the University of 90 Maryland Baltimore, and the Baltimore VA Research and Development Committee. 91 Participants were first asked to walk on a treadmill, to establish their comfortable speed during a 92 5-minute familiarization period that preceded the beginning of the recordings. Next, a 2-minute 93 rest period (baseline) while standing on the treadmill was followed by 5-minutes of precision 94 walking, when subjects were instructed to walk on the treadmill at their comfortable speed while 95 receiving real time visual feedback (30 frames/sec) of their lower limbs through a video monitor 96 in front of them. Subjects were told to avoid stepping on the white stripe (2 inches wide) glued 97 diagonally on the treadmill's belt by using the monitor's video to keep track of foot placement 98 relative to the white stripe. This increased the attentional demands during treadmill walking 99 (Yogev-Seligman et al. 2008), a condition that can be considered to mimic walking in a novel 100 environment or under novel conditions (e.g., after brain injury). Thus, the precision walking 101 paradigm puts us a step closer to the actual application where patients have impaired gait function 102 and therefore would need to rely purely or significantly on effortful attentive conscious control of 103 gait. In an ancillary task, a subset of the participants whose decoders showed the best and worst 104 decoding performance in the precision walking task were also tested under normal walking 105 conditions that did not require precise positioning of the feet nor monitoring of foot placement 106 through a computer monitor (subjects were instructed to direct their gaze straight ahead). 107 108 Limb movement and EEG recordings. The three-dimensional (3D) joint kinematics of the hip, 109 knee and ankle joints were recorded using an infrared optical motion capture system (Optotrak, 110 Northern Digital, Ontario, Canada @ 100 Hz) with foot switch data (Koningsberg 4 EFTA00304371 III Instrumentation, Pasadena, CA, @ 100 Hz). Precision manufactured 5 cm diameter disks 112 (Innovative Sports Training, Chicago, IL), each embedded with three infrared diodes that formed 113 an equilateral triangle (--3 cm sides), were affixed with adhesive and secured with foam wrap at 114 the second sacral vertebra (S-2) and on the thigh, shank, and foot segments of each lower limb. A 115 segmental model of the lower limbs was then determined by digitizing joint centers for the hip, 116 knee and ankle joints of each limb. Gait kinematics were derived from the model using motion 117 analysis software (Motion Monitor, Innovative Sports Training, Chicago, IL) and exported as 118 ascii files containing time histories of the X, Y & Z positions, joint angular positions and joint 119 angular velocities for the hip, knee and ankle joints of the right leg. Whole scalp 60-channel EEG 120 (Neuroscan Synamps2 RT, Compumedics USA, Charlotte, NC, USA) and electro-ocular activity 121 were recorded (sampling rate of 500 Hz; band-pass filtered from 0.1 to 100 Hz; right ear lobe 122 (A2) was used as a reference) and time-locked with the movement kinematics using the 123 footswitch signals. 124 125 Power spectral density analysis. The power spectral density (PSD) for the kinematic data and for 126 each channel of the EEG recorded during rest and during the walking task for the 6 subjects was 127 computed using the adaptive Thompson's multitaper method as implemented in Matlab's pnam 128 function. The time-bandwidth product for the discrete prolate spheroidal sequences used was 4 129 and the frequency resolution 0.1 Hz. The confidence interval was set to 95% and was estimated 130 using a chi-squared approach. In order to account for the variability of the kinematics, and for 131 purposes of cross-validation of the decoders (see the Model performance metrics subsection), 132 during the walking task, the data for each gait parameter (x, y,z,0,0/dt) were divided into 5 133 segments (I minute each one) and the PSD was calculated for each of these 5 segments 134 independently. The segments were then averaged across all the parameters and all the subjects 135 leading to a grand average of the PSD. Frequencies ≤ 3 Hz accounted for > 90% of the total PSD 136 for the kinematics. The same segmentation was applied to each channel of the EEG recorded 137 during rest and walking conditions. The PSD of each segment was averaged across channels and 138 then averaged across subjects leading to a grand average. The grand averages for the kinematics 5 EFTA00304372 139 and the EEG were then smoothed with local regression using weighted linear least squares and a 140 2nd degree polynomial model as implemented in the Matlab's loess function with a span 141 (percentage of the total number of data points) of 10%. 142 143 Signal preprocessing. Figure 1 shows our decoding methodology. All the data analysis, decoder 144 design and cross-validation procedures were performed off-line using custom software written in 145 MATLAB (Mathworks Inc., Natick, MA). The most frontal electrodes (FPI, FP2, FPz) were 146 removed off-line from all the subjects, as they are usually contaminated by eye-blinks. Temporal 147 electrodes were also removed, as they are most susceptible to artifacts from facial and cranial 148 muscle activity (Goncharova et al. 2003). Signals from each EEG electrode were decimated by a 149 factor of 5 (to 100 Hz), then filtered with a zero-phase, 3rd order, band-pass Butterworth filter 150 (0.1 - 2 Hz) and normalized by subtracting their mean and dividing by their standard deviation 151 (Bradberry et al. 2010). Kinematic data were filtered with a zero-phase, 3rd order, band-pass 152 Butterworth filter (0.1 — 3 Hz), as this frequency range accounted for 90% of the signal power. 153 154 Decoding method. A time-embedded (10 lags, corresponding to 100 ms in the past) linear Wiener 155 filter (Bradberry et al. 2010; Carmen et al. 2003; Fitzsimmons et al. 2009) was independently 156 designed, optimized, and cross-validated for each extracted gait parameter. The linear model was 157 given by: 158 N L 159 y(t)= a +EEb aiSn(t — k)+ e(t) st=1 k=0 160 161 where y(t) is the gait parameter measured (x, I dt) time series representing the linear 162 and angular kinematics, and their time derivatives, for the hip, knee and ankle joints; L and 163 N are the number of lags and the number of electrodes, respectively; 5.(r — k) is the 164 standardized voltage measured at EEG electrode n at lag time k, a and b are weights 165 obtained through multiple linear regression and e(t) is the residual error. The parameters of the 6 EFTA00304373 166 model were calculated using the standard GLM functions in MATLAB under the Gaussian 167 distribution using the Matlab's linear link function. 168 169 Model peformance metrics. In order to assess and compare the predictive power of each decoder 170 (neural decoders were trained independently for each subject, and each decoded parameter), a 5- 171 fold cross validation procedure; i.e., 5 distinct sets of test data that were not used to train the 172 decoder were employed for testing purposes. That is, the data recorded during the 5 minutes of 173 the walking task were divided into 5 segments (1 minute each one). Four segments were used for 174 training, while the remaining segment was used for testing the model. This procedure was 175 repeated for all the possible combinations. The Pearson correlation coefficient ( r ) was calculated 176 between the known measured signal and the predicted decoder's output as follows: 177 r(x,1)- cov(x,i) 178 a zo i 179 where x is the actual measured parameter, 1 is the prediction of that parameter and crx and 180 a l are the standard deviations of x and 1 respectively. 181 The SNR (signal to noise ratio) was calculated according to Fitzsimmons et al. (2009). 182 SNR(x,i) =10 logio( Var(x)) 183 MSE(x) 184 185 where the variance (Var) of the actual measured parameter (signal x) was calculated by 186 subtracting out the mean of the signal, then squaring and averaging the amplitude. The noise or 187 error (1 ) was the difference between the predicted and actual measured signal. The mean squared 188 error (MSE) was calculated by squaring the difference, then averaging to get the mean squared 189 error (MSE), or the power of the noise. The ratio between Var(x) and MSE(i) was converted 190 into a decibel (dB) scale. A SNR with a value of "0" means that the signal and the noise are 191 equally present in the reconstructed kinematic parameter. A SNR < 0 (poor prediction) indicates a 192 noisy reconstruction, while a SNR > 0 (good prediction) indicates a high-quality reconstruction of 193 the signal. 7 EFTA00304374 194 195 Sensor dropping analysis. A sensor dropping analysis (SDA) was used to evaluate the 196 importance of groups of sensors of different sizes to decoding accuracy (e.g., Carmen et al. 197 2003; Fitzsimmons et al. 2009). First, decoder models were trained by using each lag of each 198 sensor (one lag at a time) with the above mentioned 5-fold cross validation procedure. In order to 199 rank the sensors, two different methods were then used based on which kinematic parameter was 200 to be decoded. For the joint angle (0) and the angular velocity (d0/ dt) the sensors were ranked 201 based on the maximum value of the correlations calculated at each lag. For the Cartesian 202 positions (x, y, z) reconstructions, the sensors were ranked according to the following sensor 203 sensitivity curve equation (Bradberry et al. 2010): 204 205 — R,, + + c:t , 206 L+1k.o 207 where L is the number of lags, R. is the rank of sensor n and c are the best correlation 208 coefficients for each Cartesian position (x, y, z). These procedures were followed for all the 45 209 sensors used for decoding after removing the most prefrontal and temporal electrodes. The best 210 34 and 16 sensors out of the 45 sensors ranked were then used for training and testing the 211 decoders for each kinematic parameter extracted. 212 213 Decoding kinematics by regions of interest (RO1). In order to assess the contribution to the 214 decoding of each scalp area, the scalp was divided into 5 major ROIs: prefrontal (PF), central 215 (SM), posterior-occipital (PO) and right (RH) and left (LH) hemispheres. The kinematics were 216 decoded using the sensors belonging to each of these ROIs, leading to 5 different decoders for 217 each parameter for each joint and each subject. 218 219 Scalp Maps. To visualize the relative contributions of scalp regions to the reconstruction of the 220 position (x,y,z), joint angle (0) and the angular velocity (O1 dt) of the hip, knee and ankle 221 joints, the squared correlation (i.e., variance) values c for each sensor at each lag were projected EFTA00304375 222 into a time series of scalp maps (-100-0 ms in increments of 10 ms for a total of 11 scalp maps). 223 The topoplot function of EEGLAB [Delorme and Makeig 2004 (http://sccn.ucsd.edukezlab/)] 224 was used to plot the correlation values. The contribution of the reconstruction of each lag, for the 225 Cartesian data, was calculated as follows (Bradberry et al. 2010): 226 EN Ai 2 far + C2 CI nix 227 %Ti =%100* " n=1 L 228 Ejjc2 -Fc2 + c2„,, 229 rt=1 kw() 230 231 for all i from 0 to L, where %Ti is the percentage of reconstruction contribution at time lag i. 232 233 Artifacts. To address the issue of potential mechanical artifacts introduced by motion of the EEG 234 cap wires to the recording amplifiers (due in turn to movement of the subject) the phase-locking 235 value (PLV) (Lachaux et al. 2000, 2002) was computed by using Morlet wavelets (Tallon-Baudry 236 et al. 1997). We made the assumption that if the motion of the EEG wires corrupted in some way 237 the measured EEG signals, this problem should have been observed in all the electrodes as the 238 wires were bundled in a single connector. We were particularly interested in investigating the 239 phase in the 1-2 Hz range, as these were the main frequencies used for decoding. We calculated 240 the PLV between each electrode for the walking task and the corresponding kinematics recorded 241 from the subjects. The averaged values of PLV at 1 and 2 Hz were averaged across the electrodes, 242 leading to a mean value at the two frequencies of interest, and compared with the correlation 243 values of the decoding. 244 Analysis of potential eye movement contributions to decoding. In order to assess a potential 245 contribution of the movement of the eyes to decoding, the decoding process was also carried out 246 by adding the standardized vertical electrooculogram (VEOG) activity to the optimal set of 247 electrodes used for decoding (Bradberry et al. 2011). The r-values and the regression weights 248 were calculated in this new condition. We compared the r-values with and without the VEOG 249 electrode by calculating the difference in % and divided the absolute value of the regression 250 weights of the eye-electrode by the sum of the absolute value of all the regression weights of the 251 best fold. 9 EFTA00304376 252 253 Results 254 Spectral signature of walking kinematics and associated high-density EEG. The power spectral 255 density (PSD) of the gait kinematics (black) in the 0.1 — 5 Hz range along with the 95% 256 confidence intervals (gray) are depicted in Figure 2A. The PSD shows that > 90% of the power is 257 contained in the 0.1 — 3 Hz frequency band with a peak (26.45 dB) at —1.8 Hz. The ratio between 258 upper and lower bounds of the confidence interval throughout all the frequencies was —6.6 dB. 259 Confidence intervals (95%) of the PSD of the EEG at rest (black) and during precision walking 260 (gray) are shown in Figure 2B. Notably, PSD(walking) > PSD(rest) in the delta and theta bands 261 (-0.1 - 7 Hz) and in the low beta range (13 — 18 Hz), whereas for frequencies > 18 Hz the 262 PSD(walking) < PSD(rest). Importantly, the suppression in the mu band observed during upper 263 limb movements (Pfurtscheller et al. 2006) is also present during precision walking in the 8 — 12 264 Hz range. This is clearly depicted in the plot of the ratio of PSD(walking) to PSD(rest) shown in 265 the inset. Of note, the ratio in the 0.1 — 2 Hz range used for decoding was — 1.0 dB implying that 266 walking did not alter the spectral signature in this low frequency band (i.e., low delta) — a finding 267 consistent with the data reported by Gwin et al. (2010). 268 269 Decoding accuracy. Our EEG decoding method was able to reconstruct 3D linear and angular 270 kinematics of the ankle, knee and hip joints with high accuracy. In order to quantify the level of 271 accuracy, we computed the Pearson's r and the SNR between measured and predicted Cartesian 272 positions, joint angles and angular velocities across cross-validation folds. SNR proved to be a 273 more sensitive measure compared to r, which describes the correspondence of signal waveforms, 274 but is insensitive to amplitude scaling and offsets. The average of the correlation values (r) 275 between predicted and recorded kinematics for the six subjects was 0.75 (I0.1) and the signal-to- 276 noise ratio values > 0 (4.13 I 2.03) in all but one measure (subject S6: x axis of the ankle; SNR = 277 -0.35I1.09) confirmed the good quality of the decoded signals. Overall, correlation values across 278 the subjects were slightly higher for joint angle (mean r = 0.78I0.1) and angular velocity (mean r 279 = 0.78&0.09) than for Cartesian positions (mean rx‘,..,= 0.71I0.13). Figures 3(A) and 3(B), show, 10 EFTA00304377 280 respectively, examples of the measured (black) and the reconstructed (gray) kinematics for the 281 best (S4) and worst (S5) subjects in terms of decoding accuracy. As it can be seen, even in the 282 case of the worst case we were able to decode the kinematic parameters with an accuracy r =0.67 283 t 0.09. The quality of the reconstructions of the gait trajectories in 3D space is shown in Figure 4, 284 where an example of the actual and predicted angular velocities and joint angles, and their 285 relative phasing, for the ankle, knee and hip, for subject S4 are depicted in 3D space as well. 286 Table 1 reports the mean and the standard deviation (SD) of the correlation coefficients 287 (r) and of the SNR (dB) values across cross validation folds for all subjects, the best (S4) and 288 worst (55) cases (subjects), and for intra-cortical recordings from rhesus monkey I (Fitzsimmons 289 et al. 2009), while Figure 5 shows the distribution of the correlation coefficients (r) versus SNR 290 (dB) for the 6 subjects and for the 2 rhesus monkeys reported in the Fitzimmons' experiment. All 291 the decoded accuracies resulted in mean r values > 0.5 and high SNR values (all but one > 0), 292 which were comparable with the values reported using recording spikes from rhesus monkeys 293 (Fitzsimmons et al. 2009). In order to rule out the hypothesis that the visual feedback aided 294 decoding, we report in Table 2 the r and SNR values of the best and worst subject decoded under 295 natural walking conditions (no visual feedback and no stripe to step over) from our ancillary task. 296 We used the neural decoders, previously trained using data from the precision walking task, to 297 predict the linear and angular kinematics during normal walking. The decoding accuracies 298 reported for the two conditions were comparable. The avenges of the correlation values (r) 299 between predicted and recorded kinematics for the precision and natural walking task for S4 were 300 respectively 0.8510.08 and 0.7I-0.13, while for S5 were respectively 0.67t0.09 and 0.78&0.12. 301 302 Decoding accuracy by Region of Interest (RO1). Figure 6 depicts the mean decoding accuracy 303 across the three joints for the 5 different ROIs. For both the angular velocity and the joint angle 304 the r and SNR values were higher when all the sensors found during the decoder optimization 305 phase were used to decode. Decoders built based on a subset of electrodes comprising the right 306 (RH) or left (LH) hemispheres scalp regions showed the highest r values among the selected 307 ROIs, while the subset of electrodes spanning the central scalp ROI (SM) showed the lowest r 11 EFTA00304378 308 values. In terms of SNR, the right hemisphere, prefrontal; and posterior-occipital ROIs returned 309 the highest values, while the central scalp ROI returned the lowest values. However, statistically 310 these differences were not significant (Kruskal-Wallis test; all comparisons at p 0.05). 311 312 Topography of the correlation values of the sensors. The topography of the squared correlation 313 (i.e., variance) values of the sensors at the best lag for the best (subject S4) and worst (subject 55) 314 decoded cases is plotted in Figure 7. These scalp maps represent the individual contribution of 315 electrodes to decoding, that is, the spatial distribution of the EEG information about walking 316 contained at each electrode site. From these scalp maps, it can be inferred that neural information 317 about walking is distributed across a sparse cortical network at the macro-scale of EEG. Scalp 318 maps of sensors most relevant to decoding of the right limb suggest that scalp areas from both 319 hemispheres, somewhat lateralized to the right are involved during walking. Although there are 320 some common scalp regions relevant across all the gait parameters, these scalp regions 321 accounting for the highest variance are different across the two subjects S4 and 55. For instance, 322 C6, FZ, 135, and AF4 electrodes are recruited across gait parameters for subject S4, whereas for 323 subject S5 electrode locations at FC6, P6, and PO2 on the right hemisphere seemed to be relevant 324 for decoding walking across all the kinematic parameters. There were also other important 325 differences across subjects. For example, in subject S4 decoding of both Cartesian and angular 326 kinematics recruited anterior scalp areas (electrode locations AF3, FZ and AF4) that in some 327 cases extended to left frontal sites (F5). These scalp areas were absent in subject S5 who showed 328 the lowest decoding accuracies. 329 Of note, the scalp maps of the highest (e.g., r42 > 0.2) electrode contributions to decoding 330 the right limb kinematics were rather sparse, particularly for subject S5, who showed rather focal 331 recruitment of electrodes on the right hemisphere, compared with a more bilateral, but still sparse 332 recruitment of electrodes for subject S4. In summary, a sparse network comprised of right 333 posterior-occipital, right lateral, and bilateral anterior-frontal scalp regions appeared to contain 334 decodable gait information. 335 12 EFTA00304379 336 Minimum number of sensors. Given that the analysis of scalp maps relevant for decoding showed 337 a sparse cortical network for walking, the number of sensors was further optimized using the 338 SDA approach. As shown in Figure 8, the average number of sensors needed to achieve the 339 reported correlations was —27-32, but on average decoding accuracy reached a phase of plateau 340 (i.e., an improvement in DA < 5%) with 14 sensors (Figure 8A). As shown, with an average of 341 27 sensors (i.e., the 'best' sensors), the mean r value across the 6 subjects was 0.75 (&0.06) (black 342 bars), while selecting the best 14 sensors led to a mean r value across the 6 subjects of 0.72 343 (&0.06) (white bars), that is, less than 5% reduction in decoding accuracy (Figure 8B). 344 345 Discussion 346 Gait kinematics can be inferred from scalp EEG signals with high accuracy. This study 347 demonstrates, for the first time, that non-invasive scalp electroencephalographic (EEG) signals 348 can be used to decode kinematic parameters extracted during walking with high accuracy. Of note 349 is the fact that even though we recorded EEG from 60 channels, which some investigators 350 consider to be high-density recordings (Tononi et al. 2010), we showed that as few as 16 sensors 351 were required for decoding with high accuracy. Encouraged by promising results achieved in 352 previous studies carried out in our laboratory (Bradberry et al. 2008, 2009a, 2009b, 2010), we 353 designed neural decoders by using time-domain EEG features extracted solely from the 354 fluctuations in the amplitude (i.e. amplitude modulation or AM) in the EEG signals in the low 355 delta frequency band (0.1 — 2 Hz). 356 Even though Onton et al. (2005) reported significant changes in the theta band (4 — 8 Hz) 357 reflecting increasing cognitive demands, we emphasize that our decoders were designed to use 358 information contained in the delta band only. Moreover, our decoders were able to predict gait 359 kinematics under two different conditions (precision walking and normal walking), which clearly 360 differ in terms of the cognitive demands and task constraints, and thus changes in cognitive 361 demands or modulations in higher frequency bands could not contribute to decoding. 362 Our decoding approach proved to be robust as it prevents over-fitting (i.e., by employing 363 separate training and testing trials) and minimize the effect of artifacts because trials with artifacts l3 EFTA00304380 364 in the training set would contribute minimally to the learning of the optimal decoder weights, and 365 those in the test set could only reduce, not improve, the decoding accuracy (Tsuchiya et al. 2010). 366 The fact that critical information for decoding lower limb kinematics is contained in the smoothed 367 amplitude modulations (AM) in the lower half of the so-called delta band (i.e., 0.1 — 4 Hz) is 368 consistent with recent EEG, electrocorticographic (ECoG), and local field potential (LFP) upper 369 limb movement decoding studies that use the fluctuations in the amplitude of highly smoothed 370 signals for decoding (Walden et al. 2008; Lv et al. 2010; Ball et al. 2009; Acharya et al 2010; 371 Ince et al. 2010; and Zhuang et al 2010). It is also consistent with observations by Gwin et al. 372 (2010a), who showed that meaningful changes during walking or running occur at low 373 frequencies (< 10 Hz) in high-density EEG. 374 Fitzsimmons et al. (2009) were the first to prove that linear decoders could be used to 375 reconstruct locomotion, but their experiments were based on intracortical recordings (spikes) in 376 nonhuman primates. Ferris and colleagues have recently shown electrocortical activity coupled to 377 gait cycle phase during treadmill walking in humans (Gwin et al. 2010b), but their study did not 378 decode gait parameters from the EEG signals. In our experiment, 6 subjects were asked to walk at 379 their preferred speed on a treadmill while receiving visual feedback of their lower limbs (through 380 a video monitor at eye level in front of them), to repeatedly avoid stepping on a strip drawn on the 381 treadmill belt — a condition we called precision walking. Even though angular kinematics were on 382 average slightly better decoded than linear kinematics, we could not identify any parameter that 383 stood out as the best for decoding, except for the Cartesian "x" parameters which showed a lower 384 decoding performance overall. All the kinematic parameters but "x" position were decoded with 385 mean r values > 0.7 (mean r, = 0.67 (a0.16), mean r).= 0.77 (&0.1), = 0.77 (a-0.13), rank = 0.78 386 (a0.09), ran, / = 0.78 (a0.1); and no statistical difference was found among the 5 parameters (p > 387 0.01, ANOVA). Moreover, as shown in Figure 4, the phasing relationship between ankle, knee 388 and hip angular kinematics is preserved in the reconstructed trajectories even though the three 389 joints were decoded independently from each other. Remarkably, as depicted in Figure 6, SNR 390 and r values were comparable to the ones reported by Fitzsimmons et al. (2009), a result that 391 supports the hypothesis that the EEG signals in the low delta frequency band over a large but 14 EFTA00304381 392 sparse cortical network contain decodable information that could be used to design EEG-based 393 brain-machine interface (BMI) systems for restoration of lower limb movement. It cannot be 394 overemphasized that the same decoders calibrated using data from the precision walking task 395 were able to reconstruct the gait kinematics during normal walking, which did not require 396 subjects to monitor and control foot placement and had not access to visual feedback of foot 397 placement, thus demonstrating the robustness of our methods. 398 399 Scalp map analysis. Decoder optimization and scalp maps of correlations for the right limb 400 confirmed that human walking is sub-served by a complex, distributed but sparse cortical 401 network, in which different scalp areas over anterior, right lateral and right anterior-occipital 402 scalp areas seem to equally contribute to the decoding, at least at the macro-scale of EEG. As we 403 decoded the right leg only, it still remains to be seen whether this sparse network that encoded 404 right-side lower limb kinematics would be mirrored in the case of the left leg kinematics. 405 Our best decoded case (subject S4) showed the highest gait-related information in the 406 bilateral anterior, and the lateral and posterior-occipital scalp areas in the right hemisphere. Of 407 note, our worst subject (subject S5) showed a lack of anterior-frontal recruitment for decoding the 408 right limb, which may explain the lower decoding accuracies. In fact, it is plausible that because 409 the precision walking task presumably involves both visual attention and decision making with 410 respect to deciding when or how best to avoid stepping in the white line drawn on the treadmill, 411 this lack of anterior-frontal recruitment for decoding affected the overall performance. The fact 412 that different scalp brain areas could equally contribute to the decoding is supported by the r and 413 SNR values obtained when decoding kinematic parameters using only sensors from specific ROIs. 414 In fact, even though differences in terms of r and SNR were observed between the 5 selected 415 ROIs, statistically these differences were not significant. Our observations are in agreement with 416 the findings by Gwin et al. (2010b), who used source analysis and reported electrocortical sources 417 in the anterior cingulate, posterior parietal and sensorimotor cortex associated with intra-stride 418 changes in spectral power. During the end of stance, they also observed that alpha and beta band 419 spectral power increased in or near the left/right sensorimotor and dorsal anterior cingulated 15 EFTA00304382 420 cortex. However, power increases in the left/right sensorimotor cortices were more pronounced 421 for contralateral limb push-off than for ipsilateral limb push-off. Studies carried out using fNIRS 422 also showed involvement of frontal, premotor and supplementary motor areas during walking 423 (Harada et al. 2009; Miyai et al. 2001; Suzuki et al. 2008; Suzuki et al. 2004). These results 424 support the idea that walking is represented across a plurality of cortical brain areas. 425 426 Minimum number of sensors. An important issue in brain-machine interface design is concerned 427 with the minimum number of sensors necessary to achieve a reasonable decoding accuracy. As it 428 is well-known (Alpaydin 2004), a common occurrence in machine learning is the fact that as the 429 number of input features increases, the decoding accuracy of the predictions increases up to a 430 certain point, after which the model becomes too complicated, over-fitting might occur and as a 431 consequence of this fact performance decreases. Given this, we decided to compare the r values 432 obtained with the number of sensors found in the SDA with the best r values obtained by using up 433 to 16 sensors. Our results indicate that —14 sensors could be sufficient to decode human 434 locomotion using EEG. 435 436 Variability of the kinematics and its relation with decoding accuracy. Spectral analysis of the gait 437 kinematics showed that more than 90% of the power was retained in the 0.1 — 3 Hz range, 438 justifying our choice to band pass filter the kinematic data within this frequency range. The 6.6 439 dB ratio of the upper and lower confidence intervals suggested a significant variability of the 440 kinematic parameters across the 6 subjects. This variability could be due to the fact that each 441 subject chose his/her comfortable pace for the walking task, but also varied his/her gait speed 442 during the task. Consistent with upper limb movement decoding studies (Bradberry et al. 2010), a 443 negative correlation between movement variability and decoding accuracy was found when 444 decoding gait parameters for both angular velocity and joint angle decoding (Figure 9). 445 Specifically, the relationship between the decoding accuracy and gait variability, as measured by 446 the kurtosis (kurtosis = 3 implies normal distribution), for angular velocity and the joint angle 447 was estimated. Low values of the kurtosis (-3) (Figure 9) and high decoding accuracy for both 16 EFTA00304383 448 the angular velocity and the joint angle suggest that a normal distribution is responsible for an 449 increase in decoding accuracy. 450 451 Decoding accuracy was not affected nor corrupted by eye, mechanical or EMG artifacts. The 452 spectral analysis of the EEG showed interesting results. As in the case of the upper limbs 453 (Pfurtscheller et al. 2006), a desynchronization during the walking task was found in the mu band 454 (8 — 12 Hz). As reported by Gwin et al. (2010a), PSD values during walking were generally 455 higher than PSD values during rest (i.e., standing) at low frequencies (0.1 — 7 Hz) and in the 456 middle beta band (13 — 18 Hz). The ratio of PSD(walking) to PSD(rest), albeit small (e.g., — 1dB 457 in the 0.1 — 2 Hz), is consistent with those observations. Moreover, Gwin et al. (2010) reported 458 that gait-related artifacts removed from EEG signals were insubstantial when subjects walked at a 459 slow pace (0.8trils = 2.88 km/h). In our experiments, no subject walked faster than 2.4 km/h, thus 460 reducing further the likelihood of mechanical artifacts. Nevertheless, it could still be argued that 461 EEG signals measured during gross motor tasks like walking are prone to a myriad of 462 physiological, mechanical, and environmental artifacts that would prevent accurate measurement 463 and analysis of cortical dynamics during treadmill walking (Gwin et al. 2010a). However, our 464 proposed method for reconstruction of gait parameters and additional analyses of the potential 465 influence of artifactual components to gait decoding suggest otherwise. 466 First, the decoding accuracies with and without inclusion of the vertical electrooculogram 467 (VEOG) electrode were similar. For all the decoded gait parameters except for the ankle in 468 subject 2 (S2, r.v = 5.1%, r,„ = 9.6%), the addition of the VEOG electrode increased negligibly the 469 decoding accuracy by a maximum of 3.1%. The contribution of VEOG in terms of regression 470 weights was also negligible for all decoded gait parameters except for the reconstruction of limb 471 trajectories in the ankle's z-dimension for subject 2 (S2, r,= 28%). Furthermore, S2 showed the 472 lowest r-value for the ankle (r,= 0.31I-0.19), supporting the notion that eyes movements did not 473 contribute to the high r and SNR values found in this study. Results are reported in Table III. It is 474 also important to point out that in the normal walking condition, subject's gaze was instructed to 475 be maintained straight ahead. This condition is likely to be associated with significant eye 17 EFTA00304384 476 movements due to the compensation of displacements of the head during walking (and neck 477 muscle activity). Indeed, significant eye movements have been reported during standing and 478 walking (Gramann et al., 2010). However, two lines of reasoning argue against the potential 479 contributions of eye movement to decoding: First, the same decoder was used to infer limb 480 kinematics in two conditions (normal walking and precision walking) that differed in the pattern 481 of eye movements (gaze straight ahead vs. monitoring foot placement in a monitor), and second, 482 the correlation analysis showed that eye movements did not assist gait decoding. 483 Second, Goncharova et al. (2003) has shown that electromyographic (EMG) and ocular 484 artifacts do generally occur mainly at frequencies higher than 8 Hz, which is 4 times higher than 485 our frequency cutoff of 2 Hz used for reconstruction. Moreover, Goncharova et al. (2003) 486 reported that EMG activity was localized to the frontal and temporal electrodes in the specific 487 frequency band we used for decoding (delta, < 4 Hz). Therefore, in our study frontal and temporal 488 electrodes were removed from the analysis. 489 Third, correlation values were also calculated between baseline EEG signals band-pass 490 filtered at 0.1 — 2 Hz and gait kinematics (< 3 Hz) and compared with EEG signals acquired 491 during walking, which we hypothesized contained relevant information about gait parameters. 492 Indeed, our results showed that attempting to map baseline EEG signals to gait parameters 493 resulted in extremely low decoding: as a representative example, the r and SNR values for the 494 ankle joint angle for our best decoded subject (S4) were 0.05 I 0.07 and -15.27 I 33.27, 495 respectively, for the baseline EEG signals, whereas decoding accuracies were high (0.87 I 0.01 496 and 6.1 I 0.59 for r and SNR, respectively) when using EEG signals acquired during the walking 497 task, confirming that EEG signals measured during walking contained detailed cortical 498 information about gait parameters. 499 Fourth, to rule out the presence of mechanical artifacts introduced by motion of the EEG 500 cables or walking itself, we computed the phase-locking value (PLV) among sensors. The 501 rationale was that potential motion artifacts due to EEG wires or the subject's motion would 502 affect all sensors equally. To assess the phase-locking value using wavelet analysis, the 503 significance threshold value was set based on the values calculated by Lachaux et al. (2002). In 18 EFTA00304385 504 our case, since we used 6 cycles (tiro ) for the wavelets and 10 cycles (nn,) for the integration 505 window, the significance threshold was estimated to be 0.71. We applied such analysis to both the 506 baseline EEG and the walking EEG conditions. Our results suggest that mechanical artifacts did 507 not play a role in decoding. As a representative example, the mean PLVs across electrodes of our 508 best subject (S4) for the ankle joint angle kinematic during walking were 0.55 a 0.08 at 1 Hz, 509 0.53 a 0.05 at 2 Hz and 0.54 a 0.06 average across 1-2 Hz (the lower bounds for gait-cycles were 510 ≥ 1Hz). Remarkably, when the baseline EEG condition was used, the mean values across 511 electrodes were 0.37 a 0.02 (at 1 Hz), 0.49 a 0.03 (at 2 Hz) and 0.43 a 0.01 (mean of 1 and 2 Hz), 512 which were comparable to those during walking and suggesting lack of mechanical coupling due 513 to concerted wire movement. 514 Fifth, we note that our decoding accuracies were high independently of whether the 515 reconstructed parameters were linear or angular gait kinematics. It is very unlikely that a (global) 516 motion artifact would affect or influence equally both types of gait parameters. For example, 517 mechanical artifacts due to up-down motion would be expected to affect the decoding of vertical 518 trajectories of the hip, ankle and knee joints, but not the decoding of angular joint velocities as 519 they are not linearly related. Nevertheless, the motion of the center of mass (COM), which would 520 be expected to be directly related to that of any upward/downward movement of the EEG wires 521 due to the subject's mechanical motion was very small (sacrum's vertical movement, in meters: 522 SI = -0.01 a 0.015, S2 = 0.0006 a 0.007, S3 = -0.006 a 0.015, S4 = -0.005 a 0.013, S5 = -0.0095 523 a 0.016, S6 = -0.007 a 0.012). In addition to this, decoding of angular velocities (not linearly 524 related with the 3D translational movements of the cables or the sacrum) for the ankle, hip, and 525 knee resulted in high decoding accuracies that were comparable to the ones of the joint angle and 526 Cartesian positions. Furthermore, it is unlikely that the motion artifact would have been the same 527 for both walking conditions; indeed, the fact that the same decoders were used to decode gait in 528 both walking (precision & normal) conditions is a strong argument against the potential influence 529 of movement artifacts to decoding. 530 Finally, we note that the mapping of the spatial distribution of the highest contributing 531 electrodes to decoding resulted in a sparse but distributed network lateralized to the right 19 EFTA00304386 532 hemisphere with a bilateral anterior contribution suggesting specificity of the cortical 533 representation of the right limb's role in walking is contained in the EEG signal. Our scalp maps 534 allowed us to map electrode locations on the scalp surface according to the maximal amount of 535 information that they might carry about each gait parameter. Remarkably, the scalp maps were 536 different across gait parameters; that is, the amount and type of information about gait was 537 different across electrode sites. As noted above, the same network was used for decoding both 538 walking conditions. 539 Overall, these results demonstrate the feasibility of employing a noninvasive EEG-based 540 brain-machine interface (BMI) for the restoration of gait. This view is supported by fMRI studies 541 in which cortical activation was detected when subjects imagined themselves walking (Bakker et 542 al. 2007, 2008; Iseki et al. 2008) and when paraplegic patients imagined foot and leg movements 543 (Alkadhi et al. 2005; Cramer et al. 2005; Hotz-Boendermaker et al. 2008). A cortically EEG- 544 driven BMI for the restoration or rehabilitation of walking could be also used as a strategy to 545 harness or potentiate the remaining functionality and plasticity of spinal cord circuits isolated 546 from the brain (Behnnan et al. 2006; Grasso et al. 2004; Lunenburger et al. 2006), and as a new 547 tool for assessing the cortical contributions to walking in health and disease, or to study the 548 changes in these contributions during learning and adaptation. 549 550 Conclusion. We have shown the feasibility of decoding human walking under precision 551 (attentive, requiring visually-guided foot placement) and normal (subjects's gaze was straight 552 ahead) conditions by using scalp EEG with as few as 16 electrodes. The fact that these two 553 conditions were decoded using the same decoder calibrated in the more complex precision 554 walking task attests to the robustness of the decoding approach. Future studies should investigate 555 the applicability of the present findings to the development of brain-machine interfaces and the 556 suitability of the proposed approach to examine cortical plasticity during gait rehabilitation. 557 558 559 20 EFTA00304387 560 561 Acknowledgements: This research has been supported by the University of Maryland College 562 Park-University of Maryland-Baltimore Seed Grant Program to JLCV and LF. Support by the VA 563 Maryland Exercise & Robotics Center of Excellence (VA RR&D B3688R) and by the University 564 of Maryland's Department of Kinesiology Graduate Student Research Initiative Fund are gladly 565 acknowledged. We thank Dr. Richard F. Macko for his support and valuable discussions during 566 the performance of this study. 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 21 EFTA00304388 588 589 Figure Captions 590 591 Figure 1: Diagram depicting the decoding methodology. The subject was fitted with a 60 channel 592 EEG cap to record brain activity and a plurality of sensors were used to record 3D kinematics and 593 footswitch data. EEG and kinematics were synchronized, preprocessed and saved. The training, 594 testing and optimization of individual neural decoders, for each decoded gait parameter, were 595 performed off-line using cross-validation procedures. 596 597 Figure 2: A. Mean power spectral density (PSD in dB/Hz, in black) and 95% confidence 598 intervals (in gray) of the grand mean of the kinematic parameters across the six subjects. B. 599 Confidence intervals (95%) of the power spectral density (PSD in dB/Hz) of the EEG recorded 600 during rest and walking of the grand mean (not shown) across the six subjects. The black lines 601 represent the PSD at rest, while the gray lines represent the PSD during walking. The inset shows 602 the ratio PSD (walking) to PSD (rest). 603 604 Figure 3: Reconstructed right leg kinematics from EEG for the 'best' (S4, A) and 'worst' (S5, B) 605 decoded subjects. Columns represent ankle, knee and hip joints. Each row represents comparison 606 of reconstructed (gray) and actual (black) measured linear kinematic trajectories for (x, y, z), joint 607 angle and angular velocity time series at the optimal number of sensors. 608 609 Figure 4: Actual and predicted standardized 3D trajectories for angular velocity and joint angle of 610 the ankle for subject S4. Ankle, knee and hip trajectories are plotted respectively in the x, y and z- 611 axes. The letter "S" represent the starting point. A: trajectories of the predicted (black) vs. actual 612 (gray) angular velocities; B: trajectories of the predicted (black) vs. actual (gray) joint angles. 613 22 EFTA00304389 614 Figure 5: Comparison of decoding accuracy (r) vs. SNR (dB) for the current study (N=6) with the 615 nonhuman primate study (monkeys 1 and 2) of Fitzsimons et al. (2009). Stars represent monkeys, 616 while squares represent the 6 subjects of our study. 617 618 Figure 6: Decoding accuracy from different scalp regions of interest (ROIs). The box plots show 619 the r and the SNR values for the angular velocity and the joint angle calculated with electrodes 620 situated across 5 different scalp areas: left hemisphere (LH), right hemisphere (RH), anterior 621 (PF), centro-medial (SM), anterior-occipital (PO), and with all the electrodes (ALL). Both r- 622 values and the SNR values are shown. The scalp map depicts the coverage used for each ROI and 623 the location of the electrodes in each ROI. Right and left hemispheres have been separated by the 624 mid line. Mid-line electrodes (along the line linking FZ and OZ) have been included in neither the 625 right nor the left hemisphere ROIs. 626 627 Figure 7: Spatial distribution of r 2 decoding accuracies across sensors for the 'best' (S4) and 628 'worst' (S5) decoded subjects. Scalp maps represent the spatial distribution of 7'1 across 629 electrodes at the best lag for each parameter resulting from the training of the linear model. From 630 left to right, each column represents the scalp map of the Cartesian positions, joint angles and 631 angular velocities. 632 633 Figure 8: Decoding accuracy with the optimal number of sensors and the lowest number of 634 sensors. A) Mean (&std) Sensors Dropping Analysis (SDA) across the six subjects. B) Decoding 635 accuracy (r) obtained by using the best 34 sensors found by the SDA analysis (black) and by 636 using the highest r among the first best 16 sensors (white) for each subject. Each set of 2 bars 637 (black and white) represents the mean r-values (astd) for each subject. The last set of two bars 638 represents the grand average across the subjects for both the optimal condition (black) and the 639 plateau condition (white). C) Number of sensors used to compute the r-values when the 'best' 640 number of sensors was used (black) and up to 16 sensors were used (white) for each subject. Each 641 set of 2 bars (black and white) represents the r values (astd) of the six subjects. 23 EFTA00304390 642 643 Figure 9: Relationship between gait variability and decoding accuracy for the angular velocity 644 and joint position trajectories. A) Mean (tstd) of the kurtosis of the angular velocity across the 645 three joints (ankle, knee and hip); B) Mean (tstd) of the kurtosis of the joint angle across the 646 three joints (ankle, knee and hip); C) Box plots of the confidence intervals (70%) for the 647 bootstrapped r, kunosis paired values. The horizontal line represents the medians. 648 649 650 651 24 EFTA00304391 652 References 653 654 Acharya S, Fifer MS, Benz IlL, Crone NE, Thakor N. Electrocorticographic amplitude 655 predicts finger positions during slow grasping motions of the hand J Neural Eng 7 046002 656 doi: 10.1088/1741-2560/7/4/046002, 2010 657 658 Alkadhi H, Brugger P, Boendermarker SH, Crelier G, Curt A, Hepp-Reymond MC, Kollias 659 SS. What disconnection tells about motor imagery: evidence from paraplegic patients Cereb 660 Cortex 15: 131-140, 2005 661 662 Alpaydin E. Introduction to machine learning. Cambridge, MA:MIT, 2004, p. 254 663 664 Ball T, Schulze-Bonhage A, Aertsen A, Mehring C. Differential representation of arm 665 movement direction in relation to cortical anatomy and function J Neural Eng 6: 016006, 2009 666 667 Bakker M, de Lange FP, Stevens JA, Toni I Bloem BR. Motor imagery of gait: a quantitative 668 approach Exp Brain Res 179: 497-504, 2007 669 670 Bakker M, de Lange FP, Helmich RC, Scheeringa R, Bloem BR, Toni I. (2008) Cerebral 671 correlates of motor imagery of nonnal and precision gait Neuroimage 41: 998-1010, 2008 672 673 Behrman AL, Bowden MG, Nair PM. Neuroplasticity after spinal cord injury and training: an 674 emerging paradigm shift in rehabilitation and walking recovery Phys Ther 86: 1406-1425, 2006 675 676 Bradberry TJ, Contreras-Vidal JL, Rong F. Decoding hand and cursor kinematics from 677 magnetoencephalographic signals during tool use Conf Proe IEEE Eng Med Biol Soc; 5306:5309, 678 2008 679 25 EFTA00304392 680 Bradberry TJ, Rong F and Contreras-Vidal JL. Decoding center-out hand velocity from MEG 681 signals during visuomotor adaptation Netwoimage 47: 1691:1700, 2009a 682 683 Bradberry TJ, Gentili R, Contreras-Vidal JL. Decoding three dimensional hand kinematics 684 from electroencephalographic signals Conf Proc IEEE Eng Med Biol Soc; 5010:5013, 2009b 685 686 Bradberry T.I. Gentili R.I. Contreras-Vidal JL. Reconstructing Three-Dimensional Iland 687 Movements from Noninvasive Electroencephalographic Signals J Neurosci 30: 3432-3437, 2010 688 689 Bradberry TJ, Gentili RJ, Contreras-Vidal JL. Fast attainment of computer cursor control 690 with noninvasively acquired brain signals J Neural Eng 8(3):036010 691 692 Carmena M. Lebedev MA, Crist RE, O'Doherty JE, Santucci DM, Dimitrov DF, Patil PG, 693 licnriquez CS, Nicolelis MA. Learning to control a brain-machine interface for reaching and 694 grasping by primates PLoS BioL I, E42, 2003 695 696 Choi JT, Bastian AJ. Adaptation reveals independent control networks for human walking 697 Nature Neuroscience 10: 1055 - 1062, 2007 698 699 Christensen LO, Andersen JB, Sinkker T, Nielsen J. (2001) Transcranial magnetic stimulation 700 and stretch reflexes in the tibialis anterior muscle during human walking The Journal of 701 Physiology, 531: 545-557, 2001 702 703 Cramer SC, Lastra L, Lacourse MG, Cohen MJ. Brain motor system function after chronic. 704 complete spinal cord injury Brain 128: 2941-2950, 2005 705 706 Delorme A, Makeig S. EEGLAB: an open source toolbox for analysis of single-trial EEG 707 dynamics including independent component analysis J Neurosci Methods 134: 9-21, 2004 708 26 EFTA00304393 709 Dobkin BH, Firestine A, West M, Saremi K and Woods IL Ankle dorsiflexion as an fMRI 710 paradigm to assay motor control for walking during rehabilitation Neuroimage 32: 370-381, 2004 711 712 Fitzsimmons NA, Lebedev MA, Peikon ID. Nicolesis MA. Extracting kinematic parameters for 713 monkey bipedal walking from cortical neuronal ensemble activity Front. Integr Neurosci. 3:3. 714 doi:10.3389/neuro.07.003.2009, 2009 715 716 Goncharova II, McFarland DJ, Vaughan JR, Wolpaw JR. EMG contamination of EEG: 717 spectral and topographical characteristics CHI; Neurophysiol 114: 1580-1593, 2003 718 719 Gramann K, Gwin JT, Bigdely-Shamlo N, Ferris DP, Makeig S. Visual evoked responses 720 during standing and walking Frontiers in Human Neuroscience, 4:202, 2010 721 722 Grasso R, Ivanenko YP, Zago M, Molinari M, Scivoletto G, Castellano V, Macellari V, 723 Lacquaniti F. Distributed plasticity of locomotor pattern generators in spinal cord injured 724 patients Brain 127: 1019-1034, 2004 725 726 Grillner S, Wall€n P, Saitoh K, Kozlov A, Robertson B. Neural Bases of goal-directed 727 locomotion in vertebrates — An overview Brain Research Reviews 57: 2-12, 2008 728 729 Gwin JT, Grasman K, Makeig S, Ferris DP. Removal of movement artifact from high-density 730 EEG recorded during walking and running J Neurophysiol 103: 3526-3534, 2010a 731 732 Gwin JT, Grasman K, Makeig 5, Ferris DP. Electrocortical activity is coupled to gait cycle 733 phase during treadmill walking J Neuroimage 54(2): 1289-96, 2010b 734 735 Harada T, Miyai I. ,Kubota K. Gait capacity affects cortical activation patterns related to speed 736 control in the elderly Erp Brain Res 193: 445-454, 2009 27 EFTA00304394 737 738 Heuninckx S, Wenderoth N, Debaere F, Peeters R, Swinnen SP. Neural Basis of Aging: The 739 Penetration of Cognition into Action Control J of Neumsci, 25(29): 6787-6796, 2005 740 Heuninckx S, Wenderoth N, Swinnen SP. Systems Neuroplasticity in the Aging Brain: 741 Recruiting Additional Neural Resources for Successful Motor Performance in Elderly Persons J 742 of Neurosci, 28(1): 91-99, 2008 743 744 Iseki K, Hanakawa T, Shinozaki J, Nankaku M, Fukuyama H. Neural mechanisms involved 745 in mental imagery and observation of gait Neuroimage 41: 1021-1031, 2008 746 747 Jerbi K, Lachaux J P, N'Diaye K, Pantazis D, Leahy R M, Garnero L, Baillet S. Coherent 748 neural representation of hand speed in humans revealed by MEG imaging Proc Nall Acad Sci 749 USA 104: 7676-7681, 2007 750 751 Lachaux JP, Rodriguez E, Le Van Quyen M, Lutz A, Martinerie .1, Varela F.I. Studying 752 single-trials of phase synchronous activity in the brain J Bifurcation and Chaos 10(10): 2429- 753 2439, 2000 754 755 Lachaux JP, Lutz A, Rudrauf D, Cosmelli D, Le Van Quyen M, Martinerie J, Varela FJ. 756 Estimating the time-course of coherence between single-trial brain signals: an introduction to 757 wavelet coherence Clin Neurophysiol 32: 157-174, 2002 758 759 Luft A, Smith GV, Forrester L, Whitall G, Macko RF, Hauser TK, Goldberg AP, Hanley 760 DF Comparing brain activation associated with isolated upper and lower limb movement across 761 corresponding joints Human Brain Mapping 17: 131-140, 2002 762 763 Lunenburger L, Bolliger M, Czell D, Muller R, Dietz V. Modulation of locomotor activity in 764 complete spinal cord injury Exp Brain Res 174: 638-646, 2006 28 EFTA00304395 765 766 Lv J, Yuanqing L, Zhenghui G. Decoding hand movement velocity from electroencephalogram 767 signals during a drawing task Biomedical Engineering Online 9: 64 doi:10.1186/1475-925X-9-64, 768 2010 769 770 Nliyai I. 'Fa n a be 11C, Sase I, Eda H, Oda I, Konishi I, Tsunazawa Y, Suzuki T, Yanagida T, 771 Kubota IC. Cortical mapping of gait in humans: a near-infrared spectroscopy topography study 772 Neuroimage 14: 1186-1192, 2001 773 774 Nielsen JB. How we walk: central control of muscle activity during human walking 775 Neuroscientist 9: 195-204, 2003 776 777 Onton J, Delorme A, Makeig S. Frontal midline EEG dynamics during working memory 778 Neurolmage 27: 341-356, 2005 779 780 Pfurtscheller G, Brunner C, Schlogl, Lopes da Silva FH. Mu rhythm (de)synchronization and 781 EEG single-trial classification of different motor imagery tasks Neurolmage 31: 153-159, 2006 782 783 Rosslgnol S, Schwab M, Schwartz AB, Fehlings MG. Spinal cord injury: time to move? J. 784 Neurosc. 27: 11782-11792, 2007 785 786 Sahyoun C, Floyer-Lea A, Johansen-Berg H, Matthews PM. Towards an understanding of 787 gait control: brain activation during the anticipation, preparation and execution of foot 788 movements Neuroimage 21: 568-575, 2004 789 790 Suzuki M, Miyai I, Ono T, Oda I, Konishi I, Kochiyama T, Kubota K. Prefrontal and 791 premotor cortex are involved in adapting walking and running speed on the treadmill: an optical 792 imaging study Neuroimage 23: 1020-1026, 2004 29 EFTA00304396 793 794 Suzuki M, Miyai I, Ono T, Kubota K. Activities in the frontal cortex and gait performance are 795 modulated by preparation. An fNIRS study Nestroimage 39: 600-607, 2008 796 797 Tallon-Baudry C, Bertrand O, Delpuech C, Pernier J. Oscillatory y-Band (30-70 Hz) activity 798 induced by a visual search task in humans. J Neurosci 17(2): 722-734, 1997 799 800 Tononi G, Riedner BA, Hulse BK, Ferrarelli F, Sarasso S. Enhancing sleep slow waves with 801 natural stimuli. Medicamundi 54(2): 82-88, 2010 802 803 Tsuchiya N, Kawasaki H, Oya H, Howard MA 3rd, Adolphs IL Decoding face information in 804 time, frequency and space from direct intracranial recordings of the human brain PloS One 3(12): 805 e3892 doi: 10.1371/journal.pone.0003892, 2008 806 807 Van den Broek SP, Reinders F, Donderwinkel M, Peters ML Volume conduction effects in 808 EEG and MEG Electroenceph. Clin Nenrophysiol 64: 159-170, 1998 809 810 Walden S, Preissl H, Demandt E, Braun C, Birbaumer N, Aertsen E, Mehring C. Hand 811 movement direction decoded from MEG and EEG J Nenrosei 28: 1000-1008, 2008 812 813 Wieser M, Haefeli J, Butler L, Jancke L, Riener R, Koeneke S. Temporal and spatial patterns 814 of cortical activation during assisted lower limb movement Exp Brain Res 203: 181-191, 2010 815 816 Zhuang J, Truccolo W, Vargas-Irwin C, Donoghue JP. Decoding 3-D reach and grasp 817 kinematics from high-frequency local field potentials in primate primary motor cortex. 818 IEEE Trans Monied Eng 57(7):1774-84, 2010 30 EFTA00304397 C 3D Kinematic Footswitches (and -Pass Alter ( Optimized Decoder EFTA00304398 A 40 35 El 30 B .43 20 1p E 15 2 5 50 40 171* 30 -o O 20 a. 10 0 2 3 Frequency (Hz) 4 5 10 15 20 Frequency (Hz) 25 EFTA00304399 A X (m) Y(m) 2 (m) Angle (1 Ang Vel (°/sec) Ankle Knee Hip ()." ‘MAik k 1/4P44C SN't‘ -0.06 0.1m m 0.1 •./NheNP-Nr4—% -60 e‘‘Itst'llselAr‘P 3501 -3001 4 8 1 4 8 1 4 8 B 0.1 Time (sec) Ankle Knee Hip X (m) 0.4 Y (m) -0.4 (m) 0.15 -0.08 Angle (1 Ang Vel (°/sec) 1 01 1 4 8 1 4 8 Time (sec) EFTA00304400 Ang Vel A 0. x B 2 Knee -2 4 .2 Joint Angle Knee -4 4 .1 0 Ankle — Predicted — Actual Ankle EFTA00304401 _.x 0 Decoding Accuracy (r) 0 0 0 4:. en co _1, *• • * ** . . . • g g cn cn cn cn Cl) u) N -& cm ui -P. W n.) —. EFTA00304402 Ang Vel Joint Angle a) co L SNR values 0.9 0.8- 0.7 0.6 LH RH PF 7 5 - 3 1 o SM PO ALL LH RH PF SM PO ALL LH 0.9 0.8 0.7. 0.6. 7 E • LH RH PF SM PO ALL El 3 1 Brain Regions LH RH PF SM PO ALL RH PF •AF3 F7 •FA • • • - •F3 •F1 •FC5 .Fc3 •FC1 •AF4 „ • F8 Fz •F2 •F4 '" FCz •FC2 •FC4 •FC SM •C5 .C3 •C1 •CP3 •CP1 Cz •C2 .C4 •C6 CPz •CP2 •CP4 PO .p5 .P3 •P1 PO1 PO3 ••z PZ •P2 .p4 p6 PO2 PO4 2 EFTA00304403 a) as C Y 0.21 0.12 0 0 0.23 113 0 0 0.12 0.06 0 0.08 0 1 .26 a 0.16 0 0 10.24 0 1.07 0 0 10.03 0 EFTA00304404 0.9 0.8 3 0.7 A !OS 0.6 0.4 0 5 10 15 20 25 30 35 Number of sensors 1 B C 0.6 U U < 0.4 a) C 0.2 0 N Number of sensors o 40 ril S1 S2 S3 S4 S5 S6 Mean ■ Optimal sensors a Plateau sensors S1 S2 S3 S4 S5 S6 Mean EFTA00304405 A B C Kurtosis (Ang Vel) Kurtosis (Joint Angle) Correlation Coefficient (r) 4 3 2 1 0L 81 S2 S3 S4 S5 S6 14 12 10 8 6 4 2 0 0.6 0.4 0.2 0 -0.2 -0.4 -0.6 I S1 S2 S3 S4 S5 S6 Ang Vel Joint Angle EFTA00304406 Table I : Comparison of decoding results in nonhuman primates with the current human study. Spikes EEG Monkey 1 Subject 4 (Best) Subject 6 (Worst) Mean (6 subjects) Ankle r SNR r SNR r SNR r SNR X 0.79 10.09 4.08 11.8 0.81 10.02 4.71 10.67 0.59 10.12 14311.74 0.6810.17 1.9 ± 1.74 Y 026 ± 0.11 626±2.66 0.92 ± 0.009 8.13 1048 0.71±0.17 2.64 13.34 0.8 ± 0.08 424 12.11 Z 0.44 ± 0.16 -0211.48 0.92 ± 0.009 8.0310.4 0.7310.11 3.0412.38 0.76 t 0.1 4.2712.19 Joint Angle NIA N/A 0.87±0.01 6.1 t 0.69 0.68 10.11 2.19 ± 2.44 0.68±0.08 2.81 t 1.16 Ang WI NIA NIA 0.81 ± 0.03 4.54 ± 01 0.67±0.08 2.11±2.04 0.71 t 0.08 326 ± 1.63 Knee r SNR r SNR r SNR r SNR X 026 10.14 1.96±1.84 0.610.06 1.910.66 0.410.07 0.1610.87 0.6710.16 222 t 1.43 Y 0.79 ± 0.13 4.28 t 2.02 0910.01 721±0.6 0.71±0.14 2.64 t 2.61 0.82 t 0.07 6.11 t 1.98 Z 0.39±0.13 .0.62 ± 1.36 021 10.006 7.71±0.61 0.74 10.08 3.12 ± 1.92 0.8±0.07 4.73 t 1.8 Joint Angle 024 ± 0.07 6.29±2.06 0.92 ± 0.01 8.41 ± 0.6 0.76 ± 0.1 329 122 0.85 ± 0.04 6.96 ± 1.36 Ang WI NIA NIA 0910.02 7.16 10.86 0.81 10.08 4.62122 0.84 ± 0.05 6.76 t 1.66 Hip r SNR r SNR r SNR r SNR X 0.610.14 1.16±1.71 0.7610.04 3.6810.68 0.5710.08 1.05115 0.7710.11 3.6411.47 Y 0.66 10.14 1.97 ± 1.92 0.8210.01 4.86 10.31 0.72 ± 0.07 2.84 ± 1.77 0.7±0.1 2.97 ± 1.44 Z 0.6610.13 0.66 11.76 0.8610.02 6.810.72 0.71 I 0.1 2.911.99 0.81±0.06 6 ± 1.43 Joint Angle 0.73 ± 0.11 2.96 ± 1.96 0.9 ± 0.01 729 ± 0.6 0.68 t 0.16 2.11±3.16 0.81 ± 0.07 6.03 ± 1.79 Ang WI NIA N/A 0.88 1 0.006 6.56 1 0.31 0.71 10.13 2.77125 0.8±0.09 4.82 ± 2.3 Correlation coefficient (r) and SNR (dB) for the prediction of different walking parameters for Monkey 1 (Fitzsimmons et al. (2009)), the best (S4) and worst (S5) decoded subjects, and for the mean across the 6 subjects in the current study. The numbers represent mean ± standard deviation. EFTA00304407 Table 2: Comparison of decoding results between precision and natural walking. Subject 4 (Precision walking) Subject 4 (Natural walk ng) Subjed 5 (Precision walking) Subject 5 (Natural walking) Ankle r SNR r SNR r SNR r SNR X 0.01 ± 0.02 4.71 2 0.67 0.47 I 0.17 .1.55 I 4.51 0.59 ± 0.12 1.43: 1.74 0.77:0.03 3.81 I 0.65 Y 0.92 ± 0.009 8.13 I 0.48 0.75 I 0.16 326 t 3.45 031:0.17 2.61 ± 3.34 0.83 ± 0.03 4.99 : 0.84 Z 0.92: 0.009 8.03:0.4 0.01 * 0.11 4.58* 2.99 0.73 * 0.11 3.04 * 2.38 096:0.02 5.69:0.55 Joint Angle 0.87 I 0.01 6.1 I 0.59 0.68 2 0.13 1.79 ± 3 0.68 i 0.11 2.19 I 2.44 0.81 : 0.02 5.43 : 0.64 Ang Vel 021 1 0.03 4.54 ± 0.7 0.75 1 0.07 3.52 2 1.43 0.67: 0.08 2.11 2 2.04 0.82:0.02 4.74 ± 0.68 Knee r SNR r SNR r SNR r SNR X 0.6 I 0.06 1.9 2 0.66 0.37 I 0.11 .1.03 I 3.02 0.4 t 0.07 0.15 I 0.87 0.36 ± 0.04 0.81 s 0.62 Y 0.9 ± 0.01 721 1 0.6 0.74 a 0.07 2.45 i 222 0.71 I 0.14 2.61 i 2.61 022 * 0.04 4.97 : 1.07 Z 0.91 i 0.005 7.71 :0.51 0.76 a 0.09 3.49 t 2.44 0.74:0.08 3.12: 1.92 0.85:0.02 5.63:0.76 Joint Angle 0.92 ± 0.01 8.41 1 0.6 022 s 0.1 4.82 ± 2.82 0.75:0.1 329 s 22 096:0.02 593:099 Ang Vel 09: 0.02 716 1 0.86 024 1 021 523 s 1.42 0.81 ± 0.0B 4.62 s 22 0.87:0.02 6.22: 0.71 Nip I SNR r SNR r SNR r SNR X 0.76 a 0.04 3.68 a 0.68 0.64 a 0.15 0.79:3.54 0.57:0.08 1.05:15 0.67:0.03 238:0.39 Y 0.82 a 0.01 4.86 a 0.31 0.71 a 0.19 2.3 I 4.05 0.72 t 0.07 2.84 I 1.77 0.79 1 022 4.17 ± 0.61 Z 0.85 1 0.02 5.8 1 0.72 0.01 1 0.09 4.64 2 2A2 0.71 ± 0.1 2.9 ± 129 023 ± 0.03 5.1:0.79 Joint Angle 0.9 2 0.01 7.29 2 0.6 022 2 0.07 4.72 t 1.93 0.60 2 0.16 2.11 2 3.16 0.81 ± 021 4.71 2 1.04 Arig Vel 0.88 ± 0.006 6.56 2. 0.31 0.66 I 0.14 1.27 t 3.02 0.71 ± 0.13 2.77:25 0.81 ± 0.03 4.74 s 0.72 Correlation coefficient (r) and SNR (dB) for the prediction of different walking parameters for the best (S4) and worst (S5) decoded subjects under precision and natural walking. The numbers represent mean f standard deviation. EFTA00304408 Table 3: Comparison of decoding accuracy (r) and weights between decoding with and without eye- electrode. Subject 1 Subject 2 Subject 3 Subject 4 Subject 5 Subject 6 Ankle %weight %r %weight %r %weight %r %weight %r %weight %r %weight %r X 0.1 0 0.1 2.9 AM .1.4 0.08 0 0.07 3.5 0.05 3.1 Y 0.03 -12 0.05 5.1 0.07 -1.1 0.09 0 0.06 2.8 0.07 -1.1 Z 0.05 0 28.7 9.6 0.18 0 005 0 0.06 1.3 0.06 0 Joint Angle 0.1 0 0.09 0 024 0 0.12 -121 0.03 -14 0.06 0 Ang Vel 0.04 0 0.09 4.4 0.09 0 0.1 -1.1 0.04 0 0.12 0 Knee %weight iii %weight in %weight in %weight in %weight %r %weight %r X 0.07 0 0.14 -12 0.12 -135 008 -1.6 0.05 0 0.06 1.75 Y 0.07 0 0.11 .6 007 -1 004 -1 0.04 2.8 004 0 2 0.06 0 0.13 2.3 0.06 .13 0.09 0 0.03 1.3 0.06 0 Joint Angle 0.04 12 0.08 4.1 0.04 4.1 0.09 0 0.01 0 0.02 0 Mg WI 0.1 0 0.13 22 0.08 -1.1 0.11 -1 007 135 003 0 Hip %weight %r %weight %r %weight %r %weight %r %weight %r %weight %r X 005 -12 0.13 aO 0.08 0 0.04 -12 006 1.78 004 0 Y 005 -1.4 0.1 0 0.01 .1.23 0.09 0 005 0 0.12 .1.88 Z 007 42 0.11 0 0.01 0 0.01 22 OAS 1.4 0.05 0 Joint Angle 005 0 0.07 -4.7 OM 0 0.01 -1.1 022 1.4 004 0 Mg Val 0.06 0 0.09 -73 0.06 -1 0.06 0 0.05 3 0.04 0 The difference in °A., between correlation coefficient (r) and the ratio between weights for the prediction of different walking parameters for the six subjects decoded under precision walking with and without eye-electrode are shown in table 3. Positive values mean an increase of r and weight with eye-electrode, while negative values mean a decrease of r and weight with eye- electrode. EFTA00304409