OSCR

Decoding intended speech with an intracortical brain-computer interface in a person with long-standing anarthria and locked-in syndrome.

Code ↔ Paper

18 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 18 matches · 4 of them tie a paragraph to a whole file without given lines: weak matches, whose lines are not tinted
  1. [1] § STAR★METHODS › METHOD DETAILS › Mahalanobis distance and hierarchical clustering ↔ 03) Figure 2 a,b,c,d Hierarchical Clustering Dendograms/my_dendrogram_plot.m, the whole file · a weak match · score 0.78 · hierarchical clustering, distance matrix, Dendrogram, Ward, branch, linkage · Report a mismatch
  2. [2] § RESULTS › Sentence speech decoding ↔ 04) Figure 3 b,c Online Speech Decoding Accuracy and Speed/Online-Speech-Performance.ipynb, lines 267 to 341 · score 0.70 · raw phoneme error, word error rate, Phoneme error rate, speech decoding, trial day, online · Report a mismatch
  3. [3] § RESULTS › Sources of error in phoneme decoding › 6v and 55b encode different language units ↔ 06) Figure 5 b,c,d,e,f,g Phased Reading, Internal and Attempted Speech Task/Phased-Task-Tuning-FVAF.ipynb, lines 1 to 48 · score 0.68 · Faith hungry computer, yes feel, cheldgup, zelwog, phase, word · Report a mismatch
  4. [4] § STAR★METHODS › METHOD DETAILS › Mahalanobis distance and hierarchical clustering ↔ 03) Figure 2 a,b,c,d Hierarchical Clustering Dendograms/my_Mahalanobis.m, the whole file · a weak match · score 0.68 · distance matrix, hierarchical clustering, Tikhonov, symmetry, Mahalanobis · Report a mismatch
  5. [5] § RESULTS › Sources of error in phoneme decoding ↔ 05) Figure 4 a,b,c Offline Speech Errors/Offline-Performance-CumulativeSessions.ipynb, the whole file · a weak match · score 0.65 · cumulative sessions, word error rate, phoneme error rate, CL, OL, post · Report a mismatch
  6. [6] § RESULTS › Sources of error in phoneme decoding › 6v and 55b encode different language units ↔ 06) Figure 5 b,c,d,e,f,g Phased Reading, Internal and Attempted Speech Task/Phased-Task-GNB-Phrases.ipynb, lines 47 to 58 · score 0.63 · Faith hungry computer, yes feel, GNB, phase, Figure 5, speech · Report a mismatch
  7. [7] § STAR★METHODS › METHOD DETAILS › Recurrent neural network ↔ utils/brainToText_trainDecoder.py, lines 537 to 622 · score 0.58 · loss function, network, GRU, norm, layer, batches · Report a mismatch
  8. [8] § STAR★METHODS › METHOD DETAILS › Gaussian Naive Bayes ↔ utils/analysis.py, lines 373 to 413 · score 0.57 · Naive Bayes classifier, Gaussian Naive Bayes, predictor, trained · Report a mismatch
  9. [9] § STAR★METHODS › METHOD DETAILS › Gaussian Naive Bayes ↔ utils/analysisT17.py, lines 239 to 279 · score 0.57 · Naive Bayes classifier, Gaussian Naive Bayes, predictor, trained · Report a mismatch
  10. [10] § STAR★METHODS › METHOD DETAILS › Fraction of variance accounted for (FVAF) ↔ utils/analysis.py, lines 118 to 176 · score 0.56 · tuning heatmaps, SSERR, SSTOT, fold, squared, held · Report a mismatch
  11. [11] § STAR★METHODS › METHOD DETAILS › Fraction of variance accounted for (FVAF) ↔ utils/analysisT17.py, lines 115 to 173 · score 0.56 · tuning heatmaps, SSERR, SSTOT, fold, squared, held · Report a mismatch
  12. [12] § STAR★METHODS › METHOD DETAILS › Neural signal processing ↔ utils/mat_to_tfrecord_imagined.py, lines 133 to 179 · score 0.56 · Spike band power, threshold crossing, Redis, sums, bin, neural · Report a mismatch
  13. [13] § STAR★METHODS › METHOD DETAILS › Neural signal processing ↔ utils/mat_to_tfrecord.py, lines 132 to 209 · score 0.54 · standard deviation, threshold crossing, Redis, scored, sums, Spike · Report a mismatch
  14. [14] § STAR★METHODS › METHOD DETAILS › Mahalanobis distance and hierarchical clustering ↔ utils/mat_to_tfrecord.py, lines 132 to 209 · score 0.54 · global standard deviation, threshold crossing, scored, spike, power · Report a mismatch
  15. [15] § STAR★METHODS › METHOD DETAILS › Mahalanobis distance and hierarchical clustering ↔ utils/mat_to_tfrecord_imagined.py, lines 181 to 253 · score 0.54 · global standard deviation, threshold crossing, scored, spike, power · Report a mismatch
  16. [16] § STAR★METHODS › METHOD DETAILS › Recurrent neural network ↔ utils/models.py, the whole file · a weak match · score 0.53 · Dropout, L2, recurrent, weight, GRU, layer · Report a mismatch
  17. [17] § STAR★METHODS › METHOD DETAILS › Sentence speech decoding ↔ utils/brainToText_trainDecoder.py, lines 537 to 622 · score 0.52 · loss function, Error rate, CTC, Network, RNN, decoder · Report a mismatch
  18. [18] § RESULTS › Sentence speech decoding ↔ 04) Figure 3 b,c Online Speech Decoding Accuracy and Speed/Online-Speech-Performance.ipynb, lines 267 to 341 · score 0.52 · raw phoneme error, word error rate, speech, decoded, Figure 3 · Report a mismatch

Paper

Loaded from Europe PMC by your browser; OSCR does not store it: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Jupyter notebook · 405 lines · 15 KB · no license · 2 matches

  1. # %%
  2. PHONE_DEF_SIL = [
  3. 'BLANK','SIL','AA', 'AE', 'AH', 'AO', 'AW',
  4. 'AY', 'B', 'CH', 'D', 'DH',
  5. 'EH', 'ER', 'EY', 'F', 'G',
  6. 'HH', 'IH', 'IY', 'JH', 'K',
  7. 'L', 'M', 'N', 'NG', 'OW',
  8. 'OY', 'P', 'R', 'S', 'SH',
  9. 'T', 'TH', 'UH', 'UW', 'V',
  10. 'W', 'Y', 'Z', 'ZH', ]
  11. import scipy.io
  12. import numpy as np
  13. import matplotlib.pyplot as plt
  14. import os
  15. import datetime
  16. import re
  17. import time
  18. import pickle
  19. import redis
  20. import argparse
  21. import json
  22. from pathlib import Path
  23. import tensorflow as tf
  24. from g2p_en import G2p
  25. g2p = G2p()
  26. def calc_wer(r, h):
  27. # initialisation
  28. import numpy
  29. d = numpy.zeros((len(r)+1)*(len(h)+1), dtype=numpy.uint8)
  30. d = d.reshape((len(r)+1, len(h)+1))
  31. for i in range(len(r)+1):
  32. for j in range(len(h)+1):
  33. if i == 0:
  34. d[0][j] = j
  35. elif j == 0:
  36. d[i][0] = i
  38. # computation
  39. for i in range(1, len(r)+1):
  40. for j in range(1, len(h)+1):
  41. if r[i-1] == h[j-1]:
  42. d[i][j] = d[i-1][j-1]
  43. else:
  44. substitution = d[i-1][j-1] + 1
  45. insertion = d[i][j-1] + 1
  46. deletion = d[i-1][j] + 1
  47. d[i][j] = min(substitution, insertion, deletion)
  49. return d[len(r)][len(h)]
  51. def _cer_and_wer(decodedSentences, trueSentences, returnCI=False):
  52. allCharErr = []
  53. allChar = []
  54. allWordErr = []
  55. allWord = []
  56. for x in range(len(decodedSentences)):
  57. decSent = decodedSentences[x]
  58. trueSent = trueSentences[x]
  60. trueSent = trueSent.strip()
  61. trueSent = trueSent.replace('.', '')
  62. trueSent = trueSent.replace('?', '')
  63. trueSent = trueSent.replace(',', '')
  64. trueSent = trueSent.replace("'", '')
  66. decSent = decSent.strip()
  67. decSent = decSent.replace('.', '')
  68. decSent = decSent.replace('?', '')
  69. decSent = decSent.replace(',', '')
  70. decSent = decSent.replace("'", '')
  72. decSent = decSent.lower()
  73. trueSent = trueSent.lower()
  75. #print(decSent)
  76. #print(trueSent)
  77. nCharErr = calc_wer([c for c in trueSent], [c for c in decSent])
  78. trueWords = trueSent.split(" ")
  79. decWords = decSent.split(" ")
  80. nWordErr = calc_wer(trueWords, decWords)
  82. allCharErr.append(nCharErr)
  83. allWordErr.append(nWordErr)
  84. allChar.append(len(trueSent))
  85. allWord.append(len(trueWords))
  87. cer = np.sum(allCharErr) / np.sum(allChar)
  88. wer = np.sum(allWordErr) / np.sum(allWord)
  90. if not returnCI:
  91. return cer, wer, np.array(allCharErr)/np.array(allChar), np.array(allWordErr)/np.array(allWord)
  92. else:
  93. allChar = np.array(allChar)
  94. allCharErr = np.array(allCharErr)
  95. allWord = np.array(allWord)
  96. allWordErr = np.array(allWordErr)
  98. nResamples = 10000
  99. resampledCER = np.zeros([nResamples,])
  100. resampledWER = np.zeros([nResamples,])
  101. for n in range(nResamples):
  102. resampleIdx = np.random.randint(0, allChar.shape[0], [allChar.shape[0]])
  103. resampledCER[n] = np.sum(allCharErr[resampleIdx]) / np.sum(allChar[resampleIdx])
  104. resampledWER[n] = np.sum(allWordErr[resampleIdx]) / np.sum(allWord[resampleIdx])
  105. cerCI = np.percentile(resampledCER, [2.5, 97.5])
  106. werCI = np.percentile(resampledWER, [2.5, 97.5])
  108. return (cer, cerCI[0], cerCI[1]), (wer, werCI[0], werCI[1])
  110. sessions = ['t17.2024.03.26-Block-27',#Chang
  111. 't17.2024.04.02.50Chang-Block-5',#Chang
  112. 't17.2024.04.02-Block-13',#Open
  113. 't17.2024.04.17-Block-7',#Open
  114. 't17.2024.04.17-Block-8',#Open
  115. 't17.2024.04.17-Block-10',#Open
  116. 't17.2024.05.07-Block-7',#Open
  117. 't17.2024.05.07-Block-8',#Open
  118. 't17.2024.05.07-Block-9',#Open
  119. 't17.2024.05.07-Block-10',#Open
  120. 't17.2024.05.07-Block-11',#Open
  121. 't17.2024.05.14-Block-6',#Open
  122. 't17.2024.05.14-Block-7',#Open
  123. 't17.2024.05.14-Block-8',#Open
  124. 't17.2024.05.14-Block-9',#Open
  125. 't17.2024.05.28-Block-8',#Open
  126. 't17.2024.05.28-Block-9',#Open
  127. 't17.2024.08.12-Block-8',#Chang
  128. 't17.2024.08.12-Block-9',#Chang
  129. 't17.2024.08.12-Block-10',]#Chang
  130. for session in sessions:
  131. session_block = session.split('-')
  132. session = session_block[0]
  133. dataDir = str(Path('../Data', 't17', session, 'brainToText'))
  134. blocks = [int(session_block[2])]
  135. print(dataDir)
  136. all_block_mats = []
  137. for b in blocks:
  138. filename = [str(x) for x in Path(dataDir, 'RedisMat').glob('*('+str(b)+').mat')][0]
  139. mat = scipy.io.loadmat(filename)
  140. all_block_mats.append(mat)
  143. all_trial_times = []
  144. trial_decodes = []
  145. current_logits = []
  146. all_trial_logits = []
  147. logit_index = 0
  148. cue = []
  149. for b in all_block_mats:
  150. logit_index = 0
  151. for trial, (start_time, end_time) in enumerate(zip(b['go_cue_redis_time'][0], b['trial_end_redis_time'][0])):
  152. current_trial = []
  153. current_logits = []
  154. cue.append(b['cue'][trial].strip())
  155. while b['decoder_output_redis_clock'][0][logit_index] < start_time or (np.argmax(b['decoder_logit_output'][logit_index, :],-1) == 0):
  156. logit_index += 1
  157. log_start_time = b['decoder_output_redis_clock'][0][logit_index]
  158. start_index = logit_index
  159. while b['decoder_output_redis_clock'][0][logit_index] < end_time:
  160. l = b['decoder_logit_output'][logit_index, :31]
  161. if np.any(l):
  162. current_logits.append(l)
  163. logit_index += 1
  165. while (np.argmax(b['decoder_logit_output'][logit_index, :],-1) == 0):
  166. logit_index-=1
  167. log_end_time = b['decoder_output_redis_clock'][0][logit_index]
  168. end_index = logit_index
  170. new_logits = current_logits
  171. all_trial_logits.append(np.array(new_logits))
  172. new_logits = np.expand_dims(np.array(new_logits), axis=1)
  173. seq_len = np.expand_dims(new_logits.shape[0], axis=0)
  174. all_trial_times.append((log_end_time-log_start_time)/(1000*60))
  175. decoded_sequence, _ = tf.nn.ctc_greedy_decoder(new_logits, seq_len, merge_repeated=True, blank_index=0)
  176. trial_decodes.append(decoded_sequence[0])
  178. all_decoded_sentences = []
  179. this_sentence = []
  180. trial_decodes_dense = []
  181. trial_remove_dups = []
  182. last_char = -1
  183. for trial, t in enumerate(trial_decodes):
  184. if cue[trial] != '':
  185. pred_ids = np.zeros([500]).astype(np.int32)
  186. trial_dense = tf.cast(tf.sparse.to_dense(t), tf.int32).numpy()[0]
  188. for c in trial_dense:
  189. if PHONE_DEF_SIL[c] not in ['?','.',',',"'"]:
  190. this_sentence.append(PHONE_DEF_SIL[c])
  191. trial_remove_dups.append(c)
  192. last_char = PHONE_DEF_SIL[c]
  193. pred_ids[:len(this_sentence)] = trial_remove_dups
  194. trial_decodes_dense.append(pred_ids)
  195. all_decoded_sentences.append(this_sentence)
  196. this_sentence = []
  197. trial_remove_dups = []
  198. last_char = -1
  200. all_seq_ids = []
  201. all_seq_lens = []
  202. sum_all_lens = 0
  203. for thisTranscription in cue:
  204. if thisTranscription != '':
  205. seqClassIDs = np.zeros([500]).astype(np.int32)
  206. characters = []
  207. if len(thisTranscription) == 0:
  208. characters = 'SIL'
  209. else:
  210. for p in g2p(thisTranscription):
  211. if p == ' ':
  212. characters.append('SIL')
  214. p = re.sub(r'[0-9]', '', p) # Remove stress
  215. if re.match(r'[A-Z]+', p): # Only keep phonemes
  216. characters.append(p)
  217. seqLen = len(characters)
  218. all_seq_lens.append(seqLen)
  219. sum_all_lens += seqLen
  220. seqClassIDs[0:seqLen] = [PHONE_DEF_SIL.index(c) for c in characters]
  221. all_seq_ids.append(seqClassIDs)
  223. # create sparse labels from ground truth labels
  224. batchSize = len(all_seq_ids)
  225. sparseLabels = tf.cast(tf.sparse.from_dense(np.array(all_seq_ids)), dtype=tf.int32)
  226. sparseLabels = tf.sparse.SparseTensor(
  227. indices=sparseLabels.indices,
  228. values=sparseLabels.values,
  229. dense_shape=[batchSize, 500])
  232. sparsePreds = tf.cast(tf.sparse.from_dense(np.array(trial_decodes_dense)), dtype=tf.int32)
  233. sparsePreds = tf.sparse.SparseTensor(
  234. indices=sparsePreds.indices,
  235. values=sparsePreds.values,
  236. dense_shape=[batchSize, 500])
  237. editDistance = tf.edit_distance(tf.cast(sparsePreds, tf.int64), tf.cast(sparseLabels, tf.int64), normalize=False)
  238. all_edit_distances = editDistance.numpy()/np.array(all_seq_lens)
  239. seqErrorRate = tf.cast(tf.reduce_sum(editDistance), dtype=tf.float32)/tf.cast(tf.reduce_sum(sum_all_lens), dtype=tf.float32)
  241. all_trial_cpm = []
  242. for trial_time, current_cue in zip(all_trial_times, cue):
  243. sen_len = len(current_cue.split(' '))
  244. all_trial_cpm.append((sen_len)/(trial_time))
  246. all_final_decoded_sentences = []
  247. final_cue = []
  248. final_decoded = []
  249. for b in all_block_mats:
  250. for decoded in b['ngram_decoder_final_output'][0]:
  251. if decoded[0][0] != 0:
  252. all_final_decoded_sentences.append(decoded[0])
  253. for gt_sentence, pred_sentence in zip(cue, all_final_decoded_sentences):
  254. if gt_sentence != '':
  255. final_cue.append(gt_sentence)
  256. final_decoded.append(pred_sentence)
  258. post_lm_cer, wer, all_post_lm_cer, all_wer = _cer_and_wer(all_final_decoded_sentences, cue)
  259. block = str(blocks[0])
  261. save_string = session + '-Block-' + block
  262. np.save(save_string + '-normEditDistances.npy', all_edit_distances)
  263. np.save(save_string + '-WPM.npy', all_trial_cpm)
  264. np.save(save_string + '-perPostLM.npy', all_post_lm_cer)
  265. np.save(save_string + '-wer.npy', all_wer)
  267. # %%
  268. import numpy as np
  269. import matplotlib.pyplot as plt
  270. import pandas as pd
  271. import seaborn as sns
  272. import matplotlib.ticker as ticker
  273. from datetime import datetime
  274. plt.rcParams["figure.figsize"] = (20,5)
  276. num_train_trials = [1,2,2,3,3,4,5,6]
  277. all_raw_cers = []
  278. all_postLM_cers = []
  279. all_postLM_wers = []
  280. data_frame = []
  281. sessions = ['t17.2024.03.26-Block-27',#Chang
  282. 't17.2024.04.02.50Chang-Block-5',#Chang
  283. 't17.2024.04.02-Block-13',#Open
  284. 't17.2024.05.07-Block-7',#Open
  285. 't17.2024.05.07-Block-8',#Open
  286. 't17.2024.05.07-Block-9',#Open
  287. 't17.2024.05.07-Block-10',#Open
  288. 't17.2024.05.07-Block-11',#Open
  289. 't17.2024.05.14-Block-6',#Open
  290. 't17.2024.05.14-Block-7',#Open
  291. 't17.2024.05.14-Block-8',#Open
  292. 't17.2024.05.14-Block-9',#Open
  293. 't17.2024.05.28-Block-8',#Open
  294. 't17.2024.05.28-Block-9',#Open
  295. 't17.2024.08.12-Block-8',#Chang
  296. 't17.2024.08.12-Block-9',#Chang
  297. 't17.2024.08.12-Block-10']#Chang
  299. last_trial_day = ''
  300. x_ticks = []
  301. sns.set_context("paper", rc={"font.size":20,"axes.titlesize":20,"axes.labelsize":32, 'legend.fontsize': 20, 'lines.markersize':18})
  302. for session in sessions:
  303. raw_cers = np.load(str(session) + '-normEditDistances.npy')
  304. postLM_cers = np.load(str(session) + '-perPostLM.npy')
  305. postLM_wers = np.load(str(session) + '-wer.npy')
  306. raw_cers[raw_cers>1.0] = 1.0
  307. postLM_cers[postLM_cers>1.0] = 1.0
  308. postLM_wers[postLM_wers>1.0] = 1.0
  309. d1 = datetime.strptime('2024.02.29', "%Y.%m.%d")
  310. d2 = datetime.strptime(session.split('-')[0][4:14], "%Y.%m.%d")
  311. trial_day = int(abs((d2 - d1).days))
  312. block = int(session.split('-')[2])
  313. xtick = trial_day + block//10
  314. if trial_day != last_trial_day:
  315. x_ticks.append(trial_day)
  316. else:
  317. x_ticks.append('')
  318. for trial in range(raw_cers.shape[0]):
  319. data_frame.append([session, 'Raw Phoneme Error Rate', int(raw_cers[trial]*100)])
  320. for trial in range(postLM_wers.shape[0]):
  321. data_frame.append([session, 'Word Error Rate', int(postLM_wers[trial]*100)])
  323. last_trial_day = trial_day
  327. df = pd.DataFrame(data_frame, columns=['Trial Day', 'Metric', 'Error Rate (%)'])
  328. print(df)
  329. flatui = ['#ff7f00','#e31a1c','#fb9a99',]
  330. sns.set_palette(flatui)
  331. g = sns.lineplot(x='Trial Day', y='Error Rate (%)', hue='Metric', estimator='mean',marker='o', data=df, linewidth=3, err_style='bars',errorbar=('ci', 95))
  332. sns.despine()
  334. g.set_yticks(np.arange(0,110,10))
  335. g.set_xticklabels(x_ticks)
  336. plt.gca().legend().set_title('')
  337. plt.legend(frameon=False)
  338. plt.xticks(fontsize=20)
  339. plt.yticks(fontsize=25)
  340. plt.savefig('T17SpeechOnlinePerformanceNew.pdf', bbox_inches='tight')
  341. plt.show()
  343. # %%
  344. import numpy as np
  345. import matplotlib.pyplot as plt
  346. import pandas as pd
  347. import seaborn as sns
  348. import matplotlib.ticker as ticker
  349. from datetime import datetime
  350. plt.rcParams["figure.figsize"] = (20,5)
  351. num_train_trials = [1,2,2,3,3,4,5,6]
  352. all_raw_cers = []
  353. all_postLM_cers = []
  354. all_postLM_wers = []
  355. data_frame = []
  356. sessions = ['t17.2024.03.26-Block-27',#Chang
  357. 't17.2024.04.02.50Chang-Block-5',#Chang
  358. 't17.2024.04.02-Block-13',#Open
  359. 't17.2024.05.07-Block-7',#Open
  360. 't17.2024.05.07-Block-8',#Open
  361. 't17.2024.05.07-Block-9',#Open
  362. 't17.2024.05.07-Block-10',#Open
  363. 't17.2024.05.07-Block-11',#Open
  364. 't17.2024.05.14-Block-6',#Open
  365. 't17.2024.05.14-Block-7',#Open
  366. 't17.2024.05.14-Block-8',#Open
  367. 't17.2024.05.14-Block-9',#Open
  368. 't17.2024.05.28-Block-8',#Open
  369. 't17.2024.05.28-Block-9',#Open
  370. 't17.2024.08.12-Block-8',#Chang
  371. 't17.2024.08.12-Block-9',#Chang
  372. 't17.2024.08.12-Block-10']#Chang
  374. last_trial_day = ''
  375. x_ticks = []
  376. sns.set_context("paper", rc={"font.size":20,"axes.titlesize":20,"axes.labelsize":32, 'legend.fontsize': 20, 'lines.markersize':18})
  377. for session in sessions:
  378. cpm = np.load(str(session) + '-WPM.npy')
  379. for trial in range(cpm.shape[0]):
  380. data_frame.append([session, 'T17', cpm[trial]])
  381. d1 = datetime.strptime('2024.02.29', "%Y.%m.%d")
  382. d2 = datetime.strptime(session.split('-')[0][4:14], "%Y.%m.%d")
  383. trial_day = int(abs((d2 - d1).days))
  384. block = int(session.split('-')[2])
  385. xtick = trial_day + block//10
  386. if trial_day != last_trial_day:
  387. x_ticks.append(trial_day)
  388. else:
  389. x_ticks.append('')
  390. last_trial_day = trial_day
  392. df = pd.DataFrame(data_frame, columns=['Trial Day', 'Metric', 'Words per minute'])
  393. print(df)
  394. flatui = ['blue']
  395. sns.set_palette(flatui)
  396. g = sns.lineplot(x='Trial Day', y='Words per minute', hue='Metric', estimator='mean',marker='o', data=df, linewidth=3, err_style='band',errorbar=('ci', 95))
  397. sns.despine()
  398. plt.gca().legend().set_title('')
  399. plt.ylim([0, 80])
  400. g.get_legend().set_visible(False)
  401. g.set_xticklabels(x_ticks)
  402. plt.xticks(fontsize=20)
  403. plt.yticks(fontsize=25)
  404. plt.savefig('T17SpeechOnlinePerformanceWPM.pdf', bbox_inches='tight')
  405. plt.show()

Online-Speech-Performance.ipynb at commit c2085ba, no license · at the source

Overview

Authors: Justin J Jude1,2,3,4, Stephanie Haro3, Hadar Levi-Aharoni1,2,3, Hiroaki Hashimoto1,2, Alexander J Acosta1, Nicholas S Card5, Maitreyee Wairagkar5, David M Brandman5, Sergey D Stavisky5, Ziv M Williams6,7,8, Sydney S Cash1,2, John D Simeral3,9,10, Leigh R Hochberg1,2,3,9,10, Daniel B Rubin1,2
ORCID iDs: Justin J Jude, Alexander J Acosta, Leigh R Hochberg
  1. Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Boston, MA, USA
  2. Department of Neurology, Harvard Medical School, Boston, MA, USA
  3. School of Engineering, Brown University, Providence, RI, USA
  4. Lead contact
  5. Department of Neurological Surgery, University of California, Davis, Davis, CA, USA
  6. Department of Neurosurgery, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA
  7. Harvard-MIT Division of Health Sciences and Technology, Boston, MA, USA
  8. Harvard Medical School, Program in Neuroscience, Boston, MA, USA
  9. VA Center for Neurorestoration and Neurotechnology, VA Providence Healthcare System, Providence, RI, USA
  10. Robert J. and Nancy D. Carney Institute for Brain Science, Brown University, Providence, RI, USA
Institutions: Harvard University (United States); Brown University (United States); Massachusetts General Hospital (United States); University of California, Davis (United States); Harvard–MIT Division of Health Sciences and Technology (United States)
Journal: Cell reports, volume 45, issue 4, article 117162
Dates: published online 30 March 2026; in print March 2026
Type: Research article · Language: English
License: CC BY-NC-ND
Identifiers: DOI 10.1016/j.celrep.2026.117162 · PMID 41920737 · PMCID PMC13335209 · OpenAlex W7144324642
Open access: gold, a free copy (OpenAlex)
Status: code verified (how the code is found and verified)
Categories: human (organism), other condition (population), cognitive (subfield)
Methods: Spectral & time-frequency, Statistics, Machine learning, Preprocessing, Single-unit activity, calcium imaging
Keywords: Amyotrophic Lateral Sclerosis, Locked-in Syndrome, Speech Decoding, Language Encoding, Cp: Neuroscience, Intracortical Brain-computer Interfaces, Bci Communication, Longstanding Anarthria
Topic: EEG and Brain-Computer Interfaces (Cognitive Neuroscience, Neuroscience), according to OpenAlex
Funding: RRD VA (I01 RX004820, I50 RX002864, I01 RX002295); NIDCD NIH HHS (DP2 DC021055, U01 DC017844, K23 DC021297); Burroughs Wellcome Fund; National Institute on Deafness and Other Communication Disorders; National Institute of Neurological Disorders and Stroke; VA Rehabilitation Research &amp; Development Service; A.P. Giannini Foundation; American Heart Association Inc; NINDS NIH HHS (R25 NS065743)
Citations: cited by 2 papers (Europe PMC); 68 references in the paper
Research resources: MATLAB R2026a RRID:SCR_001622, scikit-learn 1.2.2 RRID:SCR_002577, SciPy 1.11.3 RRID:SCR_008058, Python 3.9 RRID:SCR_008394, matplotlib 3.8.4 RRID:SCR_008624, NumPy 1.26.2 RRID:SCR_008633, seaborn 0.13.2 RRID:SCR_018132, Pandas 2.2.2 RRID:SCR_018214

Abstract

The abstract is not reproduced here: the paper's license (CC BY-NC-ND) does not allow it. Read it in the paper, at the publisher or on Europe PMC.

Repositories

Its files are read in the Code ↔ Paper reader above, with 18 matches between paragraphs and lines of code.

justin-jude/anarthria-speech-bci

License: none: the authors keep all their rights
State: the link answers, verified on 30 September 2026
Evidence: files inventoried
Commit: c2085baad13a66a415b269d9977adf24098f370a, 4 August 2025
Languages: Jupyter (15), Python (13), MATLAB (2)
Size: 35 files, 30 scripts
Software Heritage: not archived
Found in: “Data and code availability”
Holds: README, 15 notebooks
Not found: license file, CITATION.cff, environment file, tests, continuous integration, documentation
Tools: NumPy (27 files), Matplotlib (21 files), SciPy (18 files), scikit-learn (12 files), seaborn (10 files), TensorFlow (6 files), Numba (5 files), pandas (3 files), Statistics and Machine Learning Toolbox (1 file)
Availability: 1 check, the latest on 30 September 2026: the link answers
  • 30 September 2026: the link answers
31 files
At the source: github.com/justin-jude/anarthria-speech-bci

Zenodo 18599809

License: CC-BY-4.0
State: the link answers, verified on 30 September 2026
Evidence: files inventoried
Size: 1 file
Software Heritage: not checked
Found in: “Data and code availability”
Not found: README, license file, CITATION.cff, environment file, tests, continuous integration, documentation
Tools: NumPy (27 files), Matplotlib (21 files), SciPy (18 files), scikit-learn (12 files), seaborn (10 files), TensorFlow (6 files), Numba (5 files), pandas (3 files), Statistics and Machine Learning Toolbox (1 file)
Availability: 1 check, the latest on 30 September 2026: the link answers (HTTP 200)
  • 30 September 2026: the link answers (HTTP 200)
31 files
At the source: doi.org/10.5281/zenodo.18599809

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 2 repositories of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 60 scripts, each with its path and the digest of its content;
  • 18 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

Datasets cited

Code and data availability statement

The paper has a code and data availability statement. Its license (CC BY-NC-ND) does not allow reproducing it here; in short, from what the harvester recognized in it:

Read it in the paper: doi.org/10.1016/j.celrep.2026.117162.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 30 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 14 authors, 8 keywords, 9 funders, 54 references, 8 RRIDs.

Cite

This paper

Jude, J. J., Haro, S., Levi-Aharoni, H., Hashimoto, H., Acosta, A. J., Card, N. S., Wairagkar, M., Brandman, D. M., Stavisky, S. D., Williams, Z. M., Cash, S. S., Simeral, J. D., Hochberg, L. R., & Rubin, D. B. (2026). Decoding intended speech with an intracortical brain-computer interface in a person with long-standing anarthria and locked-in syndrome. Cell reports, 45(4), 117162. https://doi.org/10.1016/j.celrep.2026.117162

BibTeX

@article{jude2026decoding,
author = {Jude, Justin J and Haro, Stephanie and Levi-Aharoni, Hadar and Hashimoto, Hiroaki and Acosta, Alexander J and Card, Nicholas S and Wairagkar, Maitreyee and Brandman, David M and Stavisky, Sergey D and Williams, Ziv M and Cash, Sydney S and Simeral, John D and Hochberg, Leigh R and Rubin, Daniel B},
title = {{Decoding intended speech with an intracortical brain-computer interface in a person with long-standing anarthria and locked-in syndrome}},
journal = {Cell reports},
year = {2026},
month = mar,
volume = {45},
number = {4},
pages = {117162},
publisher = {Cell Press},
issn = {2211-1247},
doi = {10.1016/j.celrep.2026.117162},
url = {https://doi.org/10.1016/j.celrep.2026.117162},
pmid = {41920737},
pmcid = {PMC13335209}
}

RIS

TY - JOUR
AU - Jude, Justin J
AU - Haro, Stephanie
AU - Levi-Aharoni, Hadar
AU - Hashimoto, Hiroaki
AU - Acosta, Alexander J
AU - Card, Nicholas S
AU - Wairagkar, Maitreyee
AU - Brandman, David M
AU - Stavisky, Sergey D
AU - Williams, Ziv M
AU - Cash, Sydney S
AU - Simeral, John D
AU - Hochberg, Leigh R
AU - Rubin, Daniel B
TI - Decoding intended speech with an intracortical brain-computer interface in a person with long-standing anarthria and locked-in syndrome
T2 - Cell reports
J2 - Cell Rep
PY - 2026
DA - 2026/03/30
VL - 45
IS - 4
SP - 117162
SN - 2211-1247
PB - Cell Press
DO - 10.1016/j.celrep.2026.117162
UR - https://doi.org/10.1016/j.celrep.2026.117162
LA - en
ER -

CSL-JSON

{
"id": "10.1016/j.celrep.2026.117162",
"type": "article-journal",
"title": "Decoding intended speech with an intracortical brain-computer interface in a person with long-standing anarthria and locked-in syndrome",
"container-title": "Cell reports",
"author": [
{
"family": "Jude",
"given": "Justin J"
},
{
"family": "Haro",
"given": "Stephanie"
},
{
"family": "Levi-Aharoni",
"given": "Hadar"
},
{
"family": "Hashimoto",
"given": "Hiroaki"
},
{
"family": "Acosta",
"given": "Alexander J"
},
{
"family": "Card",
"given": "Nicholas S"
},
{
"family": "Wairagkar",
"given": "Maitreyee"
},
{
"family": "Brandman",
"given": "David M"
},
{
"family": "Stavisky",
"given": "Sergey D"
},
{
"family": "Williams",
"given": "Ziv M"
},
{
"family": "Cash",
"given": "Sydney S"
},
{
"family": "Simeral",
"given": "John D"
},
{
"family": "Hochberg",
"given": "Leigh R"
},
{
"family": "Rubin",
"given": "Daniel B"
}
],
"container-title-short": "Cell Rep",
"volume": "45",
"issue": "4",
"page": "117162",
"DOI": "10.1016/j.celrep.2026.117162",
"PMID": "41920737",
"PMCID": "PMC13335209",
"ISSN": "2211-1247",
"publisher": "Cell Press",
"URL": "https://doi.org/10.1016/j.celrep.2026.117162",
"language": "en",
"issued": {
"date-parts": [
[
2026,
3,
30
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1038/s41591-026-04414-6 [code]
Long-term independent use of an intracortical brain-computer interface for speech and cursor control.
Journal: Nature medicine
In common: pandas, SciPy, Matplotlib, 1 other tool, cognitive, other condition, 14 references, author Leigh R Hochberg
[2] doi:10.1038/s41586-026-10653-x [code]
A mosaic of whole-body representations on the human precentral gyrus.
Journal: Nature
In common: Statistics and Machine Learning Toolbox, scikit-learn, SciPy, 2 other tools, other condition, 10 references, 2 authors
[3] doi:10.1016/j.cell.2025.06.015
Inner speech in motor cortex and implications for speech neuroprostheses.
Journal: Cell
In common: 13 references, 2 authors
[4] doi:10.1088/1741-2552/add0e5
Speech motor cortex enables BCI cursor control and click.
Journal: Journal of neural engineering
In common: other condition, 15 references, author Leigh R Hochberg
[5] doi:10.1088/1741-2552/ad88a4 [code]
Reducing power requirements for high-accuracy decoding in iBCIs.
Journal: Journal of neural engineering
In common: seaborn, scikit-learn, pandas, 3 other tools, 11 references, author Leigh R Hochberg
[6] doi:10.1038/s41467-026-71284-4 [code]
Encoding of speech modes and loudness in ventral precentral gyrus.
Journal: Nature communications
In common: seaborn, scikit-learn, pandas, 3 other tools, cognitive, 10 references, author Leigh R Hochberg
[7] doi:10.1038/s41598-024-51617-3 [code]
Brain control of bimanual movement enabled by recurrent neural networks.
Journal: Scientific reports
In common: Numba, Statistics and Machine Learning Toolbox, scikit-learn, 2 other tools, 10 references, author Leigh R Hochberg
[8] doi:10.1088/1741-2552/ace8be [code]
Direct speech reconstruction from sensorimotor brain activity with optimized deep learning models.
Journal: Journal of neural engineering
In common: TensorFlow, seaborn, scikit-learn, 4 other tools, 11 references
[9] doi:10.1038/s41598-024-60277-2 [code]
Online speech synthesis using a chronically implanted brain-computer interface in an individual with ALS.
Journal: Scientific reports
In common: scikit-learn, pandas, SciPy, 2 other tools, cognitive, other condition, 12 references
[10] doi:10.1038/s41586-023-06377-x [code]
A high-performance speech neuroprosthesis.
Journal: Nature
In common: Numba, TensorFlow, seaborn, 5 other tools, cognitive, other condition, 6 references, author Leigh R Hochberg

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Add the badge to the code's README

OSCR: the badge of this paper

If you are an author or a maintainer of this paper's code, you can add this badge. It says what OSCR knows of the paper (its code, its validated tracing map) and links the code to this page. Paste one of these snippets under the README's title. Keep the two marks around it, so that the badge can be replaced in place and is never added twice. OSCR changes nothing in your repository.

Markdown
<!-- oscr-badge:start -->
[![OSCR](https://openscicode.org/badge/doi/10.1016/j.celrep.2026.117162.svg)](https://openscicode.org/badge/doi/10.1016/j.celrep.2026.117162)
<!-- oscr-badge:end -->
reStructuredText
.. oscr-badge:start

.. image:: https://openscicode.org/badge/doi/10.1016/j.celrep.2026.117162.svg
   :target: https://openscicode.org/badge/doi/10.1016/j.celrep.2026.117162
   :alt: OSCR

.. oscr-badge:end
HTML
<!-- oscr-badge:start -->
<a href="https://openscicode.org/badge/doi/10.1016/j.celrep.2026.117162"><img src="https://openscicode.org/badge/doi/10.1016/j.celrep.2026.117162.svg" alt="OSCR"></a>
<!-- oscr-badge:end -->

On GitHub, open the README, choose Edit (the pencil), paste the snippet under the title, then "Commit changes".

If you choose to, you can have the badge kept up to date for you, with oscr badge --write from the command line or with the GitHub Action in your repository's workflows. Every other form and option (a card, AsciiDoc, LaTeX, shields.io) is on the badge page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: code errors and mismatches between the code and the paper about this paper and its code, from readers and its authors, beside the paper and its tracing map.

Report a mismatch between the code and the paper · Report a code error (signed in)

Reproductions: reports from readers who ran the authors' code and did not get the paper's result, what they ran, with which environment, commit and data.

Report a reproduction failure (signed in)

Versions of its code: the releases of its code tied to a version of this paper (the preprint, the accepted manuscript, the version of record…), each with the tracing map versioned with it.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.