OSCR

A data-driven approach to identifying and evaluating connectivity-based neural correlates of conscious visual perception.

Code ↔ Paper

17 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 17 matches
  1. [1] § Materials and methods › Neuroimaging preprocessing › Anatomical preprocessing ↔ MEG_preprocessing/MEG_preprocessing_driver_script.sh, lines 47–108 · score 0.90 · single shell boundary, Scalp surfaces, FreeSurfer, elements model, forward model, recon
  2. [2] § Materials and methods › Fitting classifiers ↔ barycenter_robustness/barycenter_robustness_checks.ipynb, lines 81–130 · score 0.70 · StratifiedGroupKFold, cross task classification, logistic regression, v1, Irrelevant, predict
  3. [3] § Materials and methods › Neuroimaging preprocessing › MEG preprocessing ↔ MEG_preprocessing/cogitate-msp1/coglib/meeg/qc/QC_processing_eeg.py, lines 179–221 · score 0.69 · muscle artifacts, Maxwell filter, gradiometer, magnetometer, threshold, preprocessed
  4. [4] § Materials and methods › Fitting classifiers ↔ classification/fit_pyspi_classifiers.py, lines 346–422 · score 0.67 · StratifiedGroupKFold, cross task classification, cross validated, fit, SD, train
  5. [5] § Materials and methods › Neuroimaging preprocessing › MEG preprocessing ↔ MEG_preprocessing/MEG_preprocessing_driver_script.sh, lines 47–108 · score 0.67 · event related field, brain region, pipeline, preprocessed, fit, MEG
  6. [6] § Materials and methods › Neuroimaging preprocessing › Anatomical preprocessing ↔ data_visualization/methods.ipynb, lines 26–76 · score 0.65 · intraparietal sulcus, prefrontal cortex, Category selective, inflated, fsaverage, surface
  7. [7] § Materials and methods › Neuroimaging preprocessing › Anatomical preprocessing ↔ data_visualization/methods.ipynb, lines 26–76 · score 0.62 · prefrontal cortex, category selective, parcel, Network, atlas, FreeSurfer
  8. [8] § Materials and methods › Fitting classifiers ↔ MEG_preprocessing/cogitate-msp1/coglib/ieeg/decoding/calibration.py, lines 18–89 · score 0.62 · support vector machine, cross validated, binary, regression, probability, fit
  9. [9] § Materials and methods › Fitting classifiers ↔ barycenter_robustness/barycenter_robustness_checks.ipynb, lines 153–244 · score 0.59 · robustness check, logistic regression, cross validated, fit, classifier, accuracy
  10. [10] § Materials and methods › Neuroimaging data acquisition and task paradigm ↔ MEG_preprocessing/cogitate-msp1/coglib/beh_et/behavior/quality_checks.py, lines 833–903 · score 0.59 · alarm rates, hit rates, behavioral, durations, stimuli
  11. [11] § Materials and methods › Theory-driven neural modeling › Model development and implementation ↔ modeling/CogitateModels.ipynb, lines 66–122 · score 0.56 · strong adaptation, stimulus offset, stimulus onset, simulated, CS, PFC
  12. [12] § Materials and methods › Fitting classifiers ↔ classification/fit_pyspi_classifiers.py, lines 346–422 · score 0.56 · cross task classification, cross validating, pyspi, fit, stratified, folds
  13. [13] § Results ↔ classification/fit_pyspi_classifiers.py, lines 550–631 · score 0.53 · standard deviation, Fitting classifiers, cross validation, pyspi, SD, Irrelevant
  14. [14] § Materials and methods › Neuroimaging data acquisition and task paradigm ↔ MEG_preprocessing/cogitate-msp1/coglib/fmri/logfiles_and_checks/01_exp1_create_events_tsv_file.py, lines 89–223 · score 0.53 · stimulus events, alarm, hit, letters, BIDS, Irrelevant
  15. [15] § Materials and methods › Neuroimaging preprocessing › Anatomical preprocessing ↔ data_visualization/classification_analysis_visualization.ipynb, lines 65–155 · score 0.53 · prefrontal cortex, Category selective, IPS, SPIs, V2, V1
  16. [16] § Materials and methods › Neuroimaging preprocessing › Anatomical preprocessing ↔ data_visualization/classification_analysis_visualization.ipynb, lines 65–155 · score 0.52 · prefrontal cortex, category selective, V2, V1, mapping, CS
  17. [17] § Materials and methods › Theory-driven neural modeling › Quantitative model evaluation ↔ modeling/CogitateModels.ipynb, lines 66–122 · score 0.50 · GNWT models, sweep, smaller, stimulus onset, noise, simulated

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 894 lines · 52 KB · no license · 3 matches

  1. from copy import deepcopy
  2. from glob import glob
  3. import os
  4. from os import path as op
  5. import numpy as np
  6. import pandas as pd
  7. import sys
  8. from sklearn import svm
  9. from sklearn.linear_model import LogisticRegression
  10. from sklearn.metrics import make_scorer, roc_auc_score, accuracy_score, balanced_accuracy_score
  11. from sklearn.model_selection import StratifiedGroupKFold, cross_validate, StratifiedKFold, LeaveOneOut, cross_val_predict
  12. from sklearn.pipeline import Pipeline
  13. from sklearn.utils import resample
  14. import itertools
  15. import argparse
  16. from joblib import Parallel, delayed
  18. # add path to classification analysis functions
  19. from mixed_sigmoid_normalisation import MixedSigmoidScaler
  21. parser=argparse.ArgumentParser()
  22. parser.add_argument('--bids_root',
  23. type=str,
  24. default='/project/hctsa/annie/data/Cogitate_Batch1/MEG_Data/',
  25. help='Path to the BIDS root directory')
  26. parser.add_argument('--n_jobs',
  27. type=int,
  28. default=1,
  29. help='Number of concurrent processing jobs')
  30. parser.add_argument('--SPI_directionality_file',
  31. type=str,
  32. default='/headnode1/abry4213/github/Cogitate_Connectivity_2024/feature_extraction/pyspi_SPI_info.csv',
  33. help='CSV file with SPI directionality info')
  34. parser.add_argument('--subject_ID',
  35. type=str,
  36. default=None,
  37. help='Subject for intra-subject classification [optional]')
  38. parser.add_argument('--classification_type',
  39. type=str,
  40. default='all',
  41. help='Whether to perform average and/or individual classification; default is all')
  42. parser.add_argument('--classifier',
  43. type=str,
  44. default='Logistic_Regression',
  45. help='Which type of classifier to use')
  46. opt=parser.parse_args()
  48. bids_root = opt.bids_root
  49. n_jobs = opt.n_jobs
  50. subject_ID = opt.subject_ID
  51. SPI_directionality_file = opt.SPI_directionality_file
  52. classification_type = opt.classification_type
  53. classifier = opt.classifier
  55. # Read in SPI directionality info
  56. SPI_directionality_info = pd.read_csv(SPI_directionality_file)
  58. # Load data paths
  59. pyspi_res_path = f"{bids_root}/derivatives/time_series_features"
  60. pyspi_res_path_averaged = f"{pyspi_res_path}/averaged_epochs"
  61. pyspi_res_path_individual = f"{pyspi_res_path}/individual_epochs"
  63. classification_res_path = f"{bids_root}/derivatives/classification_results"
  64. classification_res_path_averaged = f"{classification_res_path}/across_participants"
  65. classification_res_path_individual = f"{classification_res_path}/within_participants"
  67. # Make classification result directories
  68. os.makedirs(classification_res_path_averaged, exist_ok=True)
  69. os.makedirs(classification_res_path_individual, exist_ok=True)
  71. # Define classifier
  72. if classifier == "Linear_SVM":
  73. model = svm.SVC(C=1, class_weight='balanced', kernel='linear', random_state=127, probability=True)
  74. elif classifier == "Logistic_Regression":
  75. model = LogisticRegression(penalty='l1', C=1, solver='liblinear', class_weight='balanced', random_state=127)
  76. else:
  77. model = svm.SVC(C=1, class_weight='balanced', kernel='rbf', random_state=127, probability=True)
  79. pipe = Pipeline([('scaler', MixedSigmoidScaler(unit_variance=True)),
  80. ('model', model)])
  82. # Define scoring type
  83. scoring = {'accuracy': 'accuracy',
  84. 'balanced_accuracy': 'balanced_accuracy',
  85. 'AUC': make_scorer(roc_auc_score, response_method='predict_proba')}
  87. # meta-ROI comparisons
  88. meta_ROIs = ["Category_Selective", "IPS", "Prefrontal_Cortex", "V1_V2"]
  90. # Manually define combinations
  91. meta_roi_comparisons = [("Category_Selective", "IPS"),
  92. ("Category_Selective", "Prefrontal_Cortex"),
  93. ("Category_Selective", "V1_V2"),
  94. ("IPS", "Category_Selective"),
  95. ("Prefrontal_Cortex", "Category_Selective"),
  96. ("V1_V2", "Category_Selective")]
  97. # meta_roi_comparisons = list(itertools.permutations(meta_ROIs, 2))
  99. # Relevance type comparisons
  100. relevance_type_comparisons = ["Relevant-non-target", "Irrelevant"]
  102. # Stimulus presentation comparisons
  103. stimulus_presentation_comparisons = ["on", "off"]
  105. # Define all combinations for cross-task classification
  106. all_combos_for_cross_task = list(itertools.product(["relevant_to_irrelevant", "irrelevant_to_relevant"],
  107. ["on", "off"],
  108. meta_roi_comparisons))
  110. # Define cross-validators
  111. group_stratified_CV = StratifiedGroupKFold(n_splits = 10, shuffle = True, random_state=127)
  112. LOOCV = LeaveOneOut()
  113. SKF = StratifiedKFold(n_splits=5, shuffle=True, random_state=127)
  115. # Helper function for cross-task analysis
  116. def cross_task_classifier(direction, meta_roi_comparison, stimulus_presentation, pyspi_data):
  117. ROI_from, ROI_to = meta_roi_comparison
  118. # Filter pyspi data
  119. pyspi_data = (pyspi_data.query("meta_ROI_from == @ROI_from & meta_ROI_to == @ROI_to & stimulus_presentation == @stimulus_presentation")
  120. .reset_index(drop=True)
  121. .drop(columns=['index']))
  123. # All comparisons list
  124. cross_task_classification_results_list = []
  126. for SPI in pyspi_data.SPI.unique():
  127. # Extract this SPI
  128. this_SPI_data = pyspi_data.query(f"SPI == '{SPI}'")
  130. # Find overall number of rows
  131. num_rows = this_SPI_data.shape[0]
  133. # Extract SPI values
  134. this_column_data = this_SPI_data["value"]
  136. # Find number of NaN in this column
  137. num_NaN = this_column_data.isna().sum()
  138. prop_NaN = num_NaN / num_rows
  140. # Find mode and SD
  141. column_mode_max = this_column_data.value_counts().max()
  142. column_SD = this_column_data.std()
  144. # If 0% < num_NaN < 10%, impute by the mean of each component
  145. if 0 < prop_NaN < 0.1:
  146. values_imputed = (this_column_data
  147. .transform(lambda x: x.fillna(x.mean())))
  149. this_column_data = values_imputed
  150. print(f"Imputing column values for {SPI}")
  151. this_SPI_data["value"] = this_column_data
  153. # If there are:
  154. # - more than 10% NaN values;
  155. # - more than 90% of the values are the same; OR
  156. # - the standard deviation is less than 1*10**(-10)
  157. # then remove the column
  158. if prop_NaN > 0.1 or column_mode_max / num_rows > 0.9 or column_SD < 1*10**(-10):
  159. print(f"{SPI} has low SD: {column_SD}, and/or too many mode occurences: {column_mode_max} out of {num_rows}, and/or {100*prop_NaN}% NaN")
  160. continue
  162. # Iterate over stimulus combos
  163. for this_combo in stimulus_type_comparisons:
  165. # Subset data to the corresponding stimulus pairs
  166. final_dataset_for_classification_this_combo = this_SPI_data.query(f"stimulus_type in {this_combo}")
  168. if direction == "relevant_to_irrelevant":
  169. train_df = final_dataset_for_classification_this_combo.query("relevance_type == 'Relevant-non-target'")
  170. test_df = final_dataset_for_classification_this_combo.query("relevance_type == 'Irrelevant'")
  171. else:
  172. train_df = final_dataset_for_classification_this_combo.query("relevance_type == 'Irrelevant'")
  173. test_df = final_dataset_for_classification_this_combo.query("relevance_type == 'Relevant-non-target'")
  175. # Make a deepcopy of the pipeline
  176. this_iter_pipe = deepcopy(pipe)
  178. # Fit classifier
  179. X_train = train_df.value.to_numpy().reshape(-1, 1)
  180. y_train = train_df.stimulus_type.to_numpy().reshape(-1, 1)
  181. X_test = test_df.value.to_numpy().reshape(-1, 1)
  182. y_test = test_df.stimulus_type.to_numpy().reshape(-1, 1)
  184. this_iter_pipe.fit(X_train, y_train)
  185. y_pred = this_iter_pipe.predict(X_test)
  187. # Compute accuracy, balanced accuracy, and AUC
  188. accuracy = accuracy_score(y_test, y_pred)
  189. balanced_accuracy = balanced_accuracy_score(y_test, y_pred)
  191. this_SPI_combo_df = pd.DataFrame({"SPI": [SPI],
  192. "classifier": [classifier],
  193. "meta_ROI_from": [ROI_from],
  194. "meta_ROI_to": [ROI_to],
  195. "cross_task_direction": [direction],
  196. "stimulus_presentation": [stimulus_presentation],
  197. "stimulus_combo": [this_combo],
  198. "accuracy": [accuracy],
  199. "balanced_accuracy": [balanced_accuracy]})
  201. # Append to growing results list
  202. cross_task_classification_results_list.append(this_SPI_combo_df)
  204. # Concatenate all results
  205. cross_task_classification_results_df = pd.concat(cross_task_classification_results_list)
  207. # Return results
  208. return cross_task_classification_results_df
  210. #################################################################################################
  211. # Classification across participants with averaged epochs
  212. #################################################################################################
  214. if classification_type == "averaged":
  215. # Load in pyspi results
  216. all_pyspi_res_list = []
  217. # for pyspi_res_file in os.listdir(pyspi_res_path_averaged):
  218. for pyspi_res_file in glob(f"{pyspi_res_path_averaged}/*all_pyspi_results_1000ms.csv"):
  219. pyspi_res = pd.read_csv(pyspi_res_file)
  220. # Reset index
  221. pyspi_res.reset_index(inplace=True, drop=True)
  222. pyspi_res['stimulus_type'] = pyspi_res['stimulus_type'].replace(False, 'false').replace('False', 'false')
  223. pyspi_res['relevance_type'] = pyspi_res['relevance_type'].replace("Relevant non-target", "Relevant-non-target")
  224. # Rename stimulus to stimulus_presentation if it is present
  225. if 'stimulus' in pyspi_res.columns:
  226. if 'stimulus_presentation' in pyspi_res.columns:
  227. pyspi_res.drop(columns=['stimulus'], inplace=True)
  228. else:
  229. pyspi_res = pyspi_res.rename(columns={'stimulus': 'stimulus_presentation'})
  231. all_pyspi_res_list.append(pyspi_res)
  232. all_pyspi_res = pd.concat(all_pyspi_res_list)
  234. # Stimulus type comparisons
  235. stimulus_types = all_pyspi_res.stimulus_type.unique().tolist()
  236. stimulus_type_comparisons = list(itertools.combinations(stimulus_types, 2))
  238. # Comparing between stimulus types
  239. if not os.path.isfile(f"{classification_res_path_averaged}/comparing_between_stimulus_types_{classifier}_classification_results_AUC.csv"):
  240. # All comparisons list
  241. comparing_between_stimulus_types_classification_results_list = []
  243. for relevance_type in relevance_type_comparisons:
  244. print("Relevance type:" + str(relevance_type))
  245. for stimulus_presentation in stimulus_presentation_comparisons:
  246. print("Stimulus presentation:" + str(stimulus_presentation))
  247. for SPI in all_pyspi_res.SPI.unique():
  248. # First, look at each meta-ROI pair separately
  249. for meta_roi_comparison in meta_roi_comparisons:
  250. print("ROI Comparison:" + str(meta_roi_comparison))
  251. ROI_from, ROI_to = meta_roi_comparison
  252. # Finally, we get to the final dataset
  253. roi_pair_wise_dataset_for_classification = (all_pyspi_res.query("meta_ROI_from == @ROI_from & meta_ROI_to == @ROI_to & relevance_type == @relevance_type & stimulus_presentation == @stimulus_presentation")
  254. .reset_index(drop=True)
  255. .drop(columns=['index']))
  257. # Extract this SPI
  258. this_SPI_data = roi_pair_wise_dataset_for_classification.query(f"SPI == '{SPI}'")
  260. # Find overall number of rows
  261. num_rows = this_SPI_data.shape[0]
  263. # Extract SPI values
  264. this_column_data = this_SPI_data["value"]
  266. # Find number of NaN in this column
  267. num_NaN = this_column_data.isna().sum()
  268. prop_NaN = num_NaN / num_rows
  270. # Find mode and SD
  271. column_mode_max = this_column_data.value_counts().max()
  272. column_SD = this_column_data.std()
  274. # If 0% < num_NaN < 10%, impute by the mean of each component
  275. if 0 < prop_NaN < 0.1:
  276. values_imputed = (this_column_data
  277. .transform(lambda x: x.fillna(x.mean())))
  279. this_column_data = values_imputed
  280. print(f"Imputing column values for {SPI}")
  281. this_SPI_data["value"] = this_column_data
  283. # If there are:
  284. # - more than 10% NaN values;
  285. # - more than 90% of the values are the same; OR
  286. # - the standard deviation is less than 1*10**(-10)
  287. # then remove the column
  288. if prop_NaN > 0.1 or column_mode_max / num_rows > 0.9 or column_SD < 1*10**(-10):
  289. print(f"{SPI} has low SD: {column_SD}, and/or too many mode occurences: {column_mode_max} out of {num_rows}, and/or {100*prop_NaN}% NaN")
  290. continue
  292. # Start an empty list for the classification results
  293. SPI_combo_res_list = []
  295. # Iterate over stimulus combos
  296. for this_combo in stimulus_type_comparisons:
  298. # Subset data to the corresponding stimulus pairs
  299. final_dataset_for_classification_this_combo = this_SPI_data.query(f"stimulus_type in {this_combo}")
  301. # Fit classifier
  302. X = final_dataset_for_classification_this_combo.value.to_numpy().reshape(-1, 1)
  303. y = final_dataset_for_classification_this_combo.stimulus_type.to_numpy().reshape(-1, 1)
  304. groups = final_dataset_for_classification_this_combo.subject_ID.to_numpy().reshape(-1, 1)
  305. groups_flat = np.array([str(item[0]) for item in groups])
  308. # Make a deepcopy of the pipeline
  309. this_iter_pipe = deepcopy(pipe)
  310. this_classifier_res = cross_validate(this_iter_pipe, X, y, groups=groups_flat, cv=group_stratified_CV, scoring=scoring, n_jobs=n_jobs,
  311. return_estimator=False, return_train_score=False)
  313. this_SPI_combo_df = pd.DataFrame({"SPI": [SPI],
  314. "classifier": [classifier],
  315. "meta_ROI_from": [ROI_from],
  316. "meta_ROI_to": [ROI_to],
  317. "relevance_type": [relevance_type],
  318. "stimulus_presentation": [stimulus_presentation],
  319. "stimulus_combo": [this_combo],
  320. "accuracy": [this_classifier_res['test_accuracy'].mean()],
  321. "accuracy_SD": [this_classifier_res['test_accuracy'].std()],
  322. "AUC": [this_classifier_res['test_AUC'].mean()],
  323. "AUC_SD": [this_classifier_res['test_AUC'].std()]})
  325. # Append to growing results list
  326. comparing_between_stimulus_types_classification_results_list.append(this_SPI_combo_df)
  328. comparing_between_stimulus_types_classification_results = pd.concat(comparing_between_stimulus_types_classification_results_list).reset_index(drop=True)
  329. comparing_between_stimulus_types_classification_results.to_csv(f"{classification_res_path_averaged}/comparing_between_stimulus_types_{classifier}_classification_results_AUC.csv", index=False)
  331. # Comparing between relevance types
  332. if not os.path.isfile(f"{classification_res_path_averaged}/comparing_between_relevance_types_{classifier}_classification_results.csv"):
  333. # All comparisons list
  334. comparing_between_relevance_types_classification_results_list = []
  336. for meta_roi_comparison in meta_roi_comparisons:
  337. print("ROI Comparison:" + str(meta_roi_comparison))
  338. ROI_from, ROI_to = meta_roi_comparison
  339. for stimulus_presentation in stimulus_presentation_comparisons:
  340. print("Stimulus presentation:" + str(stimulus_presentation))
  341. # Finally, we get to the final dataset
  342. final_dataset_for_classification = all_pyspi_res.query("meta_ROI_from == @ROI_from & relevance_type in @relevance_type_comparisons and meta_ROI_to == @ROI_to & stimulus_presentation == @stimulus_presentation").reset_index(drop=True).drop(columns=['index'])
  344. for SPI in final_dataset_for_classification.SPI.unique():
  346. # Extract this SPI
  347. this_SPI_data = final_dataset_for_classification.query(f"SPI == '{SPI}'")
  349. # Find overall number of rows
  350. num_rows = this_SPI_data.shape[0]
  352. # Extract SPI values
  353. this_column_data = this_SPI_data["value"]
  355. # Find number of NaN in this column
  356. num_NaN = this_column_data.isna().sum()
  357. prop_NaN = num_NaN / num_rows
  359. # Find mode and SD
  360. column_mode_max = this_column_data.value_counts().max()
  361. column_SD = this_column_data.std()
  363. # If 0% < num_NaN < 10%, impute by the mean of each component
  364. if 0 < prop_NaN < 0.1:
  365. values_imputed = (this_column_data
  366. .transform(lambda x: x.fillna(x.mean())))
  368. this_column_data = values_imputed
  369. print(f"Imputing column values for {SPI}")
  370. this_SPI_data["value"] = this_column_data
  372. # If there are:
  373. # - more than 10% NaN values;
  374. # - more than 90% of the values are the same; OR
  375. # - the standard deviation is less than 1*10**(-10)
  376. # then remove the column
  377. if prop_NaN > 0.1 or column_mode_max / num_rows > 0.9 or column_SD < 1*10**(-10):
  378. print(f"{SPI} has low SD: {column_SD}, and/or too many mode occurences: {column_mode_max} out of {num_rows}, and/or {100*prop_NaN}% NaN")
  379. continue
  381. # Start an empty list for the classification results
  382. SPI_combo_res_list = []
  384. # Fit classifier
  385. X = this_SPI_data.value.to_numpy().reshape(-1, 1)
  386. y = this_SPI_data.relevance_type.to_numpy().reshape(-1, 1)
  387. groups = this_SPI_data.subject_ID.to_numpy().reshape(-1, 1)
  388. groups_flat = np.array([str(item[0]) for item in groups])
  390. group_stratified_CV = StratifiedGroupKFold(n_splits = 10, shuffle = True, random_state=127)
  392. # Make a deepcopy of the pipeline
  393. this_iter_pipe = deepcopy(pipe)
  395. this_classifier_res = cross_validate(this_iter_pipe, X, y, groups=groups_flat, cv=group_stratified_CV, scoring=scoring, n_jobs=n_jobs,
  396. return_estimator=False, return_train_score=False)
  398. this_SPI_relevance_results_df = pd.DataFrame({"SPI": [SPI],
  399. "meta_ROI_from": [ROI_from],
  400. "meta_ROI_to": [ROI_to],
  401. "stimulus_presentation": [stimulus_presentation],
  402. "comparison": ["Relevant non-target vs. Irrelevant"],
  403. "accuracy": [this_classifier_res['test_accuracy'].mean()],
  404. "accuracy_SD": [this_classifier_res['test_accuracy'].std()]})
  406. # Append to growing results list
  407. comparing_between_relevance_types_classification_results_list.append(this_SPI_relevance_results_df)
  409. comparing_between_relevance_types_classification_results = pd.concat(comparing_between_relevance_types_classification_results_list).reset_index(drop=True)
  410. comparing_between_relevance_types_classification_results.to_csv(f"{classification_res_path_averaged }/comparing_between_relevance_types_{classifier}_classification_results.csv", index=False)
  412. # Cross-task learning
  413. if not os.path.isfile(f"{classification_res_path_averaged}/cross_task_{classifier}_classification_results.csv"):
  414. print("Starting cross-task classification")
  415. cross_task_classification_results_list = Parallel(n_jobs=int(n_jobs))(delayed(cross_task_classifier)(direction=direction,
  416. meta_roi_comparison=meta_roi_comparison,
  417. stimulus_presentation=stimulus_presentation,
  418. pyspi_data=all_pyspi_res)
  419. for direction, stimulus_presentation, meta_roi_comparison in all_combos_for_cross_task)
  421. cross_task_classification_results = pd.concat(cross_task_classification_results_list).reset_index(drop=True)
  422. cross_task_classification_results.to_csv(f"{classification_res_path_averaged}/cross_task_{classifier}_classification_results.csv", index=False)
  424. #################################################################################################
  425. # Classification across participants with individual epochs
  426. #################################################################################################
  428. if classification_type == "individual":
  429. # meta-ROI comparisons
  430. meta_ROIs = ["Category_Selective", "IPS", "Prefrontal_Cortex", "V1_V2"]
  431. meta_roi_comparisons = list(itertools.permutations(meta_ROIs, 2))
  433. # BY STIMULUS TYPE
  434. # Load in this subject's pyspi results
  435. if not op.isfile(f"{classification_res_path_individual}/sub-{subject_ID}_comparing_between_stimulus_types_{classifier}_classification_results.csv"):
  437. # Load in results
  438. individual_subject_pyspi_res = pd.read_csv(f"{pyspi_res_path_individual}/sub-{subject_ID}_ses-1_all_pyspi_results_individual_epochs_1000ms.csv")
  440. # Fix stimulus_type where False to 'false'
  441. individual_subject_pyspi_res['stimulus_type'] = individual_subject_pyspi_res['stimulus_type'].replace(False, 'false')
  442. individual_subject_pyspi_res['relevance_type'] = individual_subject_pyspi_res['relevance_type'].replace("Relevant non-target", "Relevant-non-target")
  444. # Relevance type comparisons
  445. relevance_type_comparisons = ["Relevant-non-target", "Irrelevant"]
  447. # Stimulus presentation comparisons
  448. stimulus_presentation_comparisons = individual_subject_pyspi_res.stimulus_presentation.unique().tolist()
  450. # Stimulus type comparisons
  451. stimulus_types = individual_subject_pyspi_res.stimulus_type.unique().tolist()
  452. stimulus_type_comparisons = list(itertools.combinations(stimulus_types, 2))
  454. # All comparisons list
  455. comparing_between_stimulus_types_classification_results_list = []
  457. for meta_roi_comparison in meta_roi_comparisons:
  458. print("ROI Comparison:" + str(meta_roi_comparison))
  459. ROI_from, ROI_to = meta_roi_comparison
  460. for relevance_type in relevance_type_comparisons:
  461. print("Relevance type:" + str(relevance_type))
  462. for stimulus_presentation in stimulus_presentation_comparisons:
  463. print("Stimulus presentation:" + str(stimulus_presentation))
  464. # Finally, we get to the final dataset
  465. final_dataset_for_classification = individual_subject_pyspi_res.query("meta_ROI_from == @ROI_from & meta_ROI_to == @ROI_to & relevance_type == @relevance_type & stimulus_presentation == @stimulus_presentation").reset_index(drop=True).drop(columns=['index'])
  467. for SPI in final_dataset_for_classification.SPI.unique():
  469. # Extract this SPI
  470. this_SPI_data = final_dataset_for_classification.query(f"SPI == '{SPI}'")
  472. # Find overall number of rows
  473. num_rows = this_SPI_data.shape[0]
  475. # Extract SPI values
  476. this_column_data = this_SPI_data["value"]
  478. # Find number of NaN in this column
  479. num_NaN = this_column_data.isna().sum()
  480. prop_NaN = num_NaN / num_rows
  482. # Find mode and SD
  483. column_mode_max = this_column_data.value_counts().max()
  484. column_SD = this_column_data.std()
  486. # If 0% < num_NaN < 10%, impute by the mean of each component
  487. if 0 < prop_NaN < 0.1:
  488. values_imputed = (this_column_data
  489. .transform(lambda x: x.fillna(x.mean())))
  491. this_column_data = values_imputed
  492. print(f"Imputing column values for {SPI}")
  493. this_SPI_data["value"] = this_column_data
  495. # If there are:
  496. # - more than 10% NaN values;
  497. # - more than 90% of the values are the same; OR
  498. # - the standard deviation is less than 1*10**(-10)
  499. # then remove the column
  500. if prop_NaN > 0.1 or column_mode_max / num_rows > 0.9 or column_SD < 1*10**(-10):
  501. print(f"{SPI} has low SD: {column_SD}, and/or too many mode occurences: {column_mode_max} out of {num_rows}, and/or {100*prop_NaN}% NaN")
  502. continue
  504. # Start an empty list for the classification results
  505. SPI_combo_res_list = []
  507. # Iterate over stimulus combos
  508. for this_combo in stimulus_type_comparisons:
  510. # Subset data to the corresponding stimulus pairs
  511. final_dataset_for_classification_this_combo = this_SPI_data.query(f"stimulus_type in {this_combo}")
  513. # Fit classifier
  514. X = final_dataset_for_classification_this_combo.value.to_numpy().reshape(-1, 1)
  515. y = final_dataset_for_classification_this_combo.stimulus_type.to_numpy().reshape(-1, 1)
  517. stimulus_stratified_CV = StratifiedKFold(n_splits = 10, shuffle = True, random_state=127)
  519. # Make a deepcopy of the pipeline
  520. this_iter_pipe = deepcopy(pipe)
  521. this_classifier_res = cross_validate(this_iter_pipe, X, y, cv=stimulus_stratified_CV, scoring=scoring, n_jobs=n_jobs,
  522. return_estimator=False, return_train_score=False)
  524. this_SPI_combo_df = pd.DataFrame({subject_ID: ["sub-" + subject_ID],
  525. "SPI": [SPI],
  526. "meta_ROI_from": [ROI_from],
  527. "meta_ROI_to": [ROI_to],
  528. "relevance_type": [relevance_type],
  529. "stimulus_presentation": [stimulus_presentation],
  530. "stimulus_combo": [this_combo],
  531. "accuracy": [this_classifier_res['test_accuracy'].mean()],
  532. "accuracy_SD": [this_classifier_res['test_accuracy'].std()]})
  534. # Append to growing results list
  535. comparing_between_stimulus_types_classification_results_list.append(this_SPI_combo_df)
  537. comparing_between_stimulus_types_classification_results = pd.concat(comparing_between_stimulus_types_classification_results_list).reset_index(drop=True)
  538. comparing_between_stimulus_types_classification_results.to_csv(f"{classification_res_path_individual}/sub-{subject_ID}_comparing_between_stimulus_types_{classifier}_classification_results.csv", index=False)
  540. # Comparing between relevance types
  541. # BY RELEVANCE TYPE
  542. if not os.path.isfile(f"{classification_res_path_individual}/sub-{subject_ID}_comparing_between_relevance_types_{classifier}_classification_results.csv"):
  544. # Load in results
  545. individual_subject_pyspi_res = pd.read_csv(f"{pyspi_res_path_individual}/sub-{subject_ID}_ses-1_all_pyspi_results_individual_epochs_1000ms.csv")
  547. # Fix stimulus_type where False to 'false'
  548. individual_subject_pyspi_res['stimulus_type'] = individual_subject_pyspi_res['stimulus_type'].replace(False, 'false')
  550. # Relevance type comparisons
  551. relevance_type_comparisons = ["Relevant-non-target", "Irrelevant"]
  553. # Stimulus presentation comparisons
  554. stimulus_presentation_comparisons = individual_subject_pyspi_res.stimulus_presentation.unique().tolist()
  556. # All comparisons list
  557. comparing_between_relevance_types_classification_results_list = []
  559. for meta_roi_comparison in meta_roi_comparisons:
  560. print("ROI Comparison:" + str(meta_roi_comparison))
  561. ROI_from, ROI_to = meta_roi_comparison
  562. for stimulus_presentation in stimulus_presentation_comparisons:
  563. print("Stimulus presentation:" + str(stimulus_presentation))
  564. # Finally, we get to the final dataset
  565. final_dataset_for_classification = individual_subject_pyspi_res.query("meta_ROI_from == @ROI_from & relevance_type in @relevance_type_comparisons and meta_ROI_to == @ROI_to & stimulus_presentation == @stimulus_presentation").reset_index(drop=True).drop(columns=['index'])
  567. for SPI in final_dataset_for_classification.SPI.unique():
  569. # Extract this SPI
  570. this_SPI_data = final_dataset_for_classification.query(f"SPI == '{SPI}'")
  572. # Find overall number of rows
  573. num_rows = this_SPI_data.shape[0]
  575. # Extract SPI values
  576. this_column_data = this_SPI_data["value"]
  578. # Find number of NaN in this column
  579. num_NaN = this_column_data.isna().sum()
  580. prop_NaN = num_NaN / num_rows
  582. # Find mode and SD
  583. column_mode_max = this_column_data.value_counts().max()
  584. column_SD = this_column_data.std()
  586. # If 0% < num_NaN < 10%, impute by the mean of each component
  587. if 0 < prop_NaN < 0.1:
  588. values_imputed = (this_column_data
  589. .transform(lambda x: x.fillna(x.mean())))
  591. this_column_data = values_imputed
  592. print(f"Imputing column values for {SPI}")
  593. this_SPI_data["value"] = this_column_data
  595. # If there are:
  596. # - more than 10% NaN values;
  597. # - more than 90% of the values are the same; OR
  598. # - the standard deviation is less than 1*10**(-10)
  599. # then remove the column
  600. if prop_NaN > 0.1 or column_mode_max / num_rows > 0.9 or column_SD < 1*10**(-10):
  601. print(f"{SPI} has low SD: {column_SD}, and/or too many mode occurences: {column_mode_max} out of {num_rows}, and/or {100*prop_NaN}% NaN")
  602. continue
  604. # Start an empty list for the classification results
  605. SPI_combo_res_list = []
  607. # Fit classifier
  608. X = this_SPI_data.value.to_numpy().reshape(-1, 1)
  609. y = this_SPI_data.relevance_type.to_numpy().reshape(-1, 1)
  611. stimulus_stratified_CV = StratifiedKFold(n_splits = 10, shuffle = True, random_state=127)
  613. # Make a deepcopy of the pipeline
  614. this_iter_pipe = deepcopy(pipe)
  615. this_classifier_res = cross_validate(this_iter_pipe, X, y, cv=stimulus_stratified_CV, scoring=scoring, n_jobs=n_jobs,
  616. return_estimator=False, return_train_score=False)
  618. this_SPI_relevance_results_df = pd.DataFrame({subject_ID: ["sub-" + subject_ID],
  619. "SPI": [SPI],
  620. "meta_ROI_from": [ROI_from],
  621. "meta_ROI_to": [ROI_to],
  622. "stimulus_presentation": [stimulus_presentation],
  623. "comparison": ["Relevant non-target vs. Irrelevant"],
  624. "accuracy": [this_classifier_res['test_accuracy'].mean()],
  625. "accuracy_SD": [this_classifier_res['test_accuracy'].std()]})
  627. # Append to growing results list
  628. comparing_between_relevance_types_classification_results_list.append(this_SPI_relevance_results_df)
  630. comparing_between_relevance_types_classification_results = pd.concat(comparing_between_relevance_types_classification_results_list).reset_index(drop=True)
  631. comparing_between_relevance_types_classification_results.to_csv(f"{classification_res_path_individual}/sub-{subject_ID}_comparing_between_relevance_types_{classifier}_classification_results.csv", index=False)
  633. #################################################################################################
  634. # Classification across participants with individual epochs
  635. #################################################################################################
  637. if classification_type == "individual_subsampled":
  638. # meta-ROI comparisons
  639. meta_ROIs = ["Category_Selective", "IPS", "Prefrontal_Cortex", "V1_V2"]
  640. meta_roi_comparisons = list(itertools.permutations(meta_ROIs, 2))
  642. # BY STIMULUS TYPE
  643. # Load in this subject's pyspi results
  644. if not op.isfile(f"{classification_res_path_individual}/sub-{subject_ID}_subsampled_comparing_between_stimulus_types_{classifier}_classification_results.csv"):
  646. # Load in results
  647. individual_subject_pyspi_res = pd.read_csv(f"{pyspi_res_path_individual}/sub-{subject_ID}_ses-1_all_pyspi_results_individual_epochs_1000ms.csv")
  649. # Fix stimulus_type where False to 'false'
  650. individual_subject_pyspi_res['stimulus_type'] = individual_subject_pyspi_res['stimulus_type'].replace(False, 'false')
  651. individual_subject_pyspi_res['relevance_type'] = individual_subject_pyspi_res['relevance_type'].replace("Relevant non-target", "Relevant-non-target")
  653. # Relevance type comparisons
  654. relevance_type_comparisons = ["Relevant-non-target", "Irrelevant"]
  656. # Stimulus presentation comparisons
  657. stimulus_presentation_comparisons = individual_subject_pyspi_res.stimulus_presentation.unique().tolist()
  659. # Stimulus type comparisons
  660. stimulus_types = individual_subject_pyspi_res.stimulus_type.unique().tolist()
  661. stimulus_type_comparisons = list(itertools.combinations(stimulus_types, 2))
  663. # All comparisons list
  664. comparing_between_stimulus_types_classification_results_list = []
  666. for meta_roi_comparison in meta_roi_comparisons:
  667. print("ROI Comparison:" + str(meta_roi_comparison))
  668. ROI_from, ROI_to = meta_roi_comparison
  669. for relevance_type in relevance_type_comparisons:
  670. print("Relevance type:" + str(relevance_type))
  671. for stimulus_presentation in stimulus_presentation_comparisons:
  672. print("Stimulus presentation:" + str(stimulus_presentation))
  673. # Finally, we get to the final dataset
  674. final_dataset_for_classification = individual_subject_pyspi_res.query("meta_ROI_from == @ROI_from & meta_ROI_to == @ROI_to & relevance_type == @relevance_type & stimulus_presentation == @stimulus_presentation").reset_index(drop=True).drop(columns=['index'])
  676. for SPI in final_dataset_for_classification.SPI.unique():
  678. # Extract this SPI
  679. this_SPI_data = final_dataset_for_classification.query(f"SPI == '{SPI}'")
  681. # Find overall number of rows
  682. num_rows = this_SPI_data.shape[0]
  684. # Extract SPI values
  685. this_column_data = this_SPI_data["value"]
  687. # Find number of NaN in this column
  688. num_NaN = this_column_data.isna().sum()
  689. prop_NaN = num_NaN / num_rows
  691. # Find mode and SD
  692. column_mode_max = this_column_data.value_counts().max()
  693. column_SD = this_column_data.std()
  695. # If 0% < num_NaN < 10%, impute by the mean of each component
  696. if 0 < prop_NaN < 0.1:
  697. values_imputed = (this_column_data
  698. .transform(lambda x: x.fillna(x.mean())))
  700. this_column_data = values_imputed
  701. print(f"Imputing column values for {SPI}")
  702. this_SPI_data["value"] = this_column_data
  704. # If there are:
  705. # - more than 10% NaN values;
  706. # - more than 90% of the values are the same; OR
  707. # - the standard deviation is less than 1*10**(-10)
  708. # then remove the column
  709. if prop_NaN > 0.1 or column_mode_max / num_rows > 0.9 or column_SD < 1*10**(-10):
  710. print(f"{SPI} has low SD: {column_SD}, and/or too many mode occurences: {column_mode_max} out of {num_rows}, and/or {100*prop_NaN}% NaN")
  711. continue
  713. # Start an empty list for the classification results
  714. SPI_combo_res_list = []
  716. # Iterate over stimulus combos
  717. for this_combo in stimulus_type_comparisons:
  719. # Subset data to the corresponding stimulus pairs
  720. final_dataset_for_classification_this_combo = this_SPI_data.query(f"stimulus_type in {this_combo}")
  722. # Fit classifier
  723. X = final_dataset_for_classification_this_combo.value.to_numpy().reshape(-1, 1)
  724. y = final_dataset_for_classification_this_combo.stimulus_type.to_numpy().reshape(-1, 1)
  726. # Check if there are >20 samples in each class of y, without hard-coding the values of y
  727. if np.unique(y, return_counts=True)[1].min() > 20:
  728. print(f"Running subsampled classification for {this_combo}")
  729. classification_across_iters_list = []
  730. # For 50 iterations, randomly sample 20 samples from each class for leave-one-out classification
  731. for iter_num in range(100):
  732. # Randomly sample 20 samples from each class for leave-one-out classification
  733. X_resampled = []
  734. y_resampled = []
  735. for this_class in np.unique(y):
  736. X_resampled_class, y_resampled_class = resample(X[y == this_class], y[y == this_class], n_samples=20, replace=False, random_state=iter_num)
  737. X_resampled.append(X_resampled_class)
  738. y_resampled.append(y_resampled_class)
  740. X_resampled = np.concatenate(X_resampled).reshape(-1, 1)
  741. y_resampled = np.concatenate(y_resampled)
  743. y_pred = cross_val_predict(deepcopy(pipe), X_resampled, y_resampled, cv=LOOCV, n_jobs=n_jobs)
  744. y_pred_proba = cross_val_predict(deepcopy(pipe), X_resampled, y_resampled, cv=LOOCV, n_jobs=n_jobs, method='predict_proba')
  746. # Calculate classification results
  747. accuracy = accuracy_score(y_resampled, y_pred)
  748. balanced_accuracy = balanced_accuracy_score(y_resampled, y_pred)
  749. AUC = roc_auc_score(y_resampled, y_pred_proba[:, 1])
  751. # Combine into df
  752. iter_df = pd.DataFrame({"iter_num": iter_num + 1, "accuracy": accuracy, "balanced_accuracy": balanced_accuracy, "AUC": AUC}, index=[0])
  753. classification_across_iters_list.append(iter_df)
  755. # Combine all iterations
  756. classification_across_iters_df = pd.concat(classification_across_iters_list, ignore_index=True)
  758. this_SPI_combo_df = pd.DataFrame({subject_ID: ["sub-" + subject_ID],
  759. "SPI": [SPI],
  760. "meta_ROI_from": [ROI_from],
  761. "meta_ROI_to": [ROI_to],
  762. "relevance_type": [relevance_type],
  763. "stimulus_presentation": [stimulus_presentation],
  764. "stimulus_combo": [this_combo],
  765. "accuracy": [classification_across_iters_df['accuracy'].mean()],
  766. "balanced_accuracy": [classification_across_iters_df['balanced_accuracy'].mean()],
  767. "AUC": [classification_across_iters_df['AUC'].mean()]})
  770. # Append to growing results list
  771. comparing_between_stimulus_types_classification_results_list.append(this_SPI_combo_df)
  773. if len(comparing_between_stimulus_types_classification_results_list) > 0:
  774. comparing_between_stimulus_types_classification_results = pd.concat(comparing_between_stimulus_types_classification_results_list).reset_index(drop=True)
  775. comparing_between_stimulus_types_classification_results.to_csv(f"{classification_res_path_individual}/sub-{subject_ID}_subsampled_comparing_between_stimulus_types_{classifier}_classification_results.csv", index=False)
  777. # Comparing between relevance types
  778. # BY RELEVANCE TYPE
  779. if not os.path.isfile(f"{classification_res_path_individual}/sub-{subject_ID}_subsampled_comparing_between_relevance_types_{classifier}_classification_results.csv"):
  781. # Load in results
  782. individual_subject_pyspi_res = pd.read_csv(f"{pyspi_res_path_individual}/sub-{subject_ID}_ses-1_all_pyspi_results_individual_epochs_1000ms.csv")
  784. # Fix stimulus_type where False to 'false'
  785. individual_subject_pyspi_res['stimulus_type'] = individual_subject_pyspi_res['stimulus_type'].replace(False, 'false')
  787. # Relevance type comparisons
  788. relevance_type_comparisons = ["Relevant-non-target", "Irrelevant"]
  790. # Stimulus presentation comparisons
  791. stimulus_presentation_comparisons = individual_subject_pyspi_res.stimulus_presentation.unique().tolist()
  793. # All comparisons list
  794. comparing_between_relevance_types_classification_results_list = []
  796. for meta_roi_comparison in meta_roi_comparisons:
  797. print("ROI Comparison:" + str(meta_roi_comparison))
  798. ROI_from, ROI_to = meta_roi_comparison
  799. for stimulus_presentation in stimulus_presentation_comparisons:
  800. print("Stimulus presentation:" + str(stimulus_presentation))
  801. # Finally, we get to the final dataset
  802. final_dataset_for_classification = individual_subject_pyspi_res.query("meta_ROI_from == @ROI_from & relevance_type in @relevance_type_comparisons and meta_ROI_to == @ROI_to & stimulus_presentation == @stimulus_presentation").reset_index(drop=True).drop(columns=['index'])
  804. for SPI in final_dataset_for_classification.SPI.unique():
  806. # Extract this SPI
  807. this_SPI_data = final_dataset_for_classification.query(f"SPI == '{SPI}'")
  809. # Find overall number of rows
  810. num_rows = this_SPI_data.shape[0]
  812. # Extract SPI values
  813. this_column_data = this_SPI_data["value"]
  815. # Find number of NaN in this column
  816. num_NaN = this_column_data.isna().sum()
  817. prop_NaN = num_NaN / num_rows
  819. # Find mode and SD
  820. column_mode_max = this_column_data.value_counts().max()
  821. column_SD = this_column_data.std()
  823. # If 0% < num_NaN < 10%, impute by the mean of each component
  824. if 0 < prop_NaN < 0.1:
  825. values_imputed = (this_column_data
  826. .transform(lambda x: x.fillna(x.mean())))
  828. this_column_data = values_imputed
  829. print(f"Imputing column values for {SPI}")
  830. this_SPI_data["value"] = this_column_data
  832. # If there are:
  833. # - more than 10% NaN values;
  834. # - more than 90% of the values are the same; OR
  835. # - the standard deviation is less than 1*10**(-10)
  836. # then remove the column
  837. if prop_NaN > 0.1 or column_mode_max / num_rows > 0.9 or column_SD < 1*10**(-10):
  838. print(f"{SPI} has low SD: {column_SD}, and/or too many mode occurences: {column_mode_max} out of {num_rows}, and/or {100*prop_NaN}% NaN")
  839. continue
  841. # Start an empty list for the classification results
  842. SPI_combo_res_list = []
  844. # Fit classifier
  845. X = this_SPI_data.value.to_numpy().reshape(-1, 1)
  846. y = this_SPI_data.relevance_type.to_numpy().reshape(-1, 1)
  848. # Check if there are >20 samples in each class of y, without hard-coding the values of y
  849. if np.unique(y, return_counts=True)[1].min() > 20:
  850. print(f"Running subsampled classification for {this_combo}")
  851. classification_across_iters_list = []
  852. # For 100 iterations, randomly sample 20 samples from each class for leave-one-out classification
  853. for iter_num in range(100):
  854. # Randomly sample 20 samples from each class for leave-one-out classification
  855. X_resampled = []
  856. y_resampled = []
  857. for this_class in np.unique(y):
  858. X_resampled_class, y_resampled_class = resample(X[y == this_class], y[y == this_class], n_samples=20, replace=False, random_state=iter_num)
  859. X_resampled.append(X_resampled_class)
  860. y_resampled.append(y_resampled_class)
  862. X_resampled = np.concatenate(X_resampled).reshape(-1, 1)
  863. y_resampled = np.concatenate(y_resampled)
  865. y_pred = cross_val_predict(deepcopy(pipe), X_resampled, y_resampled, cv=LOOCV, n_jobs=n_jobs)
  866. y_pred_proba = cross_val_predict(deepcopy(pipe), X_resampled, y_resampled, cv=LOOCV, n_jobs=n_jobs, method='predict_proba')
  868. # Calculate classification results
  869. accuracy = accuracy_score(y_resampled, y_pred)
  870. balanced_accuracy = balanced_accuracy_score(y_resampled, y_pred)
  871. AUC = roc_auc_score(y_resampled, y_pred_proba[:, 1])
  873. # Combine into df
  874. iter_df = pd.DataFrame({"iter_num": iter_num + 1, "accuracy": accuracy, "balanced_accuracy": balanced_accuracy, "AUC": AUC}, index=[0])
  875. classification_across_iters_list.append(iter_df)
  877. # Combine all iterations
  878. classification_across_iters_df = pd.concat(classification_across_iters_list, ignore_index=True)
  879. this_SPI_relevance_results_df = pd.DataFrame({subject_ID: ["sub-" + subject_ID],
  880. "SPI": [SPI],
  881. "meta_ROI_from": [ROI_from],
  882. "meta_ROI_to": [ROI_to],
  883. "stimulus_presentation": [stimulus_presentation],
  884. "comparison": ["Relevant non-target vs. Irrelevant"],
  885. "accuracy": [classification_across_iters_df['accuracy'].mean()],
  886. "balanced_accuracy": [classification_across_iters_df['balanced_accuracy'].mean()],
  887. "AUC": [classification_across_iters_df['AUC'].mean()]})
  889. # Append to growing results list
  890. comparing_between_relevance_types_classification_results_list.append(this_SPI_relevance_results_df)
  892. if len(comparing_between_relevance_types_classification_results_list) > 0:
  893. comparing_between_relevance_types_classification_results = pd.concat(comparing_between_relevance_types_classification_results_list).reset_index(drop=True)
  894. comparing_between_relevance_types_classification_results.to_csv(f"{classification_res_path_individual}/sub-{subject_ID}_subsampled_comparing_between_relevance_types_{classifier}_classification_results.csv", index=False)

fit_pyspi_classifiers.py at commit cab648b, no license · at the source

Overview

Authors: Annie G Bryant1,2, Christopher J Whyte2,3
ORCID iDs: Annie G Bryant, Christopher J Whyte
  1. School of Physics, The University of Sydney, Camperdown, NSW, Australia
  2. Centre for Complex Systems, The University of Sydney, Camperdown, NSW, Australia
  3. Neuroscience Research Theme, School of Medical Sciences, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia
Institutions: The University of Sydney (Australia)
Journal: Neuroscience of consciousness, volume 2026, issue 1, article niag029
Dates: received 7 April 2025; accepted 17 May 2026; published online 7 July 2026
Type: Research article · Language: English
License: CC BY-NC
Identifiers: DOI 10.1093/nc/niag029 · PMID 42415873 · PMCID PMC13338905 · OpenAlex W7167585600
Open access: gold, a free copy (OpenAlex)
Status: code verified
Categories: MEG (modality), none (in silico) (organism), cognitive (subfield)
Methods: Connectivity, Smoothing, state filtering, decompositions, Machine learning, Statistics, Preprocessing
Keywords: complex systems, consciousness, visual perception, functional connectivity
Topic: Face Recognition and Perception (Cognitive Neuroscience, Neuroscience), according to OpenAlex
Citations: not cited yet (Europe PMC); 80 references in the paper

Abstract

Identifying the neural correlates of conscious visual perception remains a major challenge in neuroscience, requiring theories that bridge between subjective experience and measurable neural correlates. However, theoretical interpretation of empirical evidence is often post hoc and susceptible to confirmation bias. Building upon the adversarial collaboration mediated by the COGITATE Consortium, we present a generalizable approach for the data-driven identification, evaluation, and theoretical modeling of connectivity-based neural correlates of conscious visual perception. Using the same magnetoencephalography (MEG) dataset and accompanying pre-registered hypotheses from the COGITATE Consortium, we systematically compared 246 functional connectivity (FC) measures between regions predicted to underlie conscious vision by Integrated Information Theory (IIT) and/or Global Neuronal Workspace Theory (GNWT). We identified a family of FC measures based on the barycenter—tracking the ‘center of mass’ between two signals—as the top-performing stimulus decoding measures that generalize across regions central to predictions of both IIT and GNWT. To interpret these findings within a theoretical framework, we developed neural mass models that recapitulate the neural dynamics hypothesized to underlie conscious perception by each theory. Comparing simulated barycenter values from these models against empirically measured MEG data revealed that both the GNWT-based model, featuring delayed ignition dynamics, and the IIT-based model, which relied on synchronous sensory dynamics, captured the observed connectivity patterns. These results lend tentative support to GNWT, as the presence of ignition dynamics independent of task-demand conditions contradicts the predictions of IIT. Beyond dataset-specific conclusions and limitations, we introduce a framework for systematically identifying and testing candidate neural correlates of conscious visual perception in an unbiased and interpretable manner.

Reproduced under the paper's license (CC BY-NC), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 17 matches between paragraphs and lines of code.

anniegbryant/MEG_functional_connectivity

License: none: the authors keep all their rights
State: the link answers, verified on 27 September 2026
Evidence: files inventoried
Commit: cab648bbe2eab7093f9ab7b9a5fcddabf28e60a0, 28 January 2026
Languages: Python (223), MATLAB (31), Shell (19), Jupyter (12), R (3)
Size: 945 files, 288 scripts
Software Heritage: not archived
Found in: “Code and data availability”
Holds: README, environment (requirements.txt, MEG_preprocessing/cogitate-msp1/coglib/bayesFactor/requirements.txt, MEG_preprocessing/cogitate-msp1/coglib/xnat/requirements_xnat.txt, MEG_preprocessing/cogitate-msp1/coglib/ieeg/plotting_uniformization/requirements.txt), tests, 12 notebooks
Not found: license file, CITATION.cff, continuous integration, documentation
Tools: NumPy (149 files), pandas (101 files), Matplotlib (89 files), MNE-Python (81 files), SciPy (66 files), MNE-BIDS (39 files), scikit-learn (29 files), seaborn (28 files), NiBabel (15 files), Nilearn (12 files), statsmodels (12 files), tidyverse (11 files), cowplot (9 files), easystats (9 files), broom (8 files), patchwork (8 files), scikit-image (8 files), Pingouin (6 files), FreeSurfer (5 files), MNE-Connectivity (5 files), autoreject (4 files), ggpubr (4 files), PyPREP (4 files), SPM (4 files), afex (2 files), BayesFactor (2 files), emmeans (2 files), FSL (2 files), ggplot2 (2 files), lmerTest (2 files), xarray (2 files), AFNI (1 file), car (1 file), ggseg (1 file), Statistics and Machine Learning Toolbox (1 file), NetworkX (1 file)
Availability: 1 check, the latest on 27 September 2026: the link answers
  • 27 September 2026: the link answers
289 files
At the source: github.com/anniegbryant/MEG_functional_connectivity

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 288 scripts, each with its path and the digest of its content;
  • 17 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

Datasets cited

Code and data availability

All MEG data analyzed in this study are openly available upon registration at https://doi.org/10.17617/1.wqa3-wk71 (http://dx.doi.org/10.17617/1.wqa3-wk71) (Liu et al. 2024). All code needed to reproduce our analyses and visuals are freely available in our GitHub repository at https://github.com/anniegbryant/MEG_functional_connectivity. Preprocessed empirical data are also shared as a Zenodo repository at https://doi.org/10.5281/zenodo.18294156 (http://dx.doi.org/10.5281/zenodo.18294156).

Reproduced under the paper's license (CC BY-NC), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 27 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 2 authors, 4 keywords, 68 references.

Cite

This paper

Bryant, A. G., & Whyte, C. J. (2026). A data-driven approach to identifying and evaluating connectivity-based neural correlates of conscious visual perception. Neuroscience of consciousness, 2026(1), niag029. https://doi.org/10.1093/nc/niag029

BibTeX

@article{bryant2026data,
author = {Bryant, Annie G and Whyte, Christopher J},
title = {{A data-driven approach to identifying and evaluating connectivity-based neural correlates of conscious visual perception}},
journal = {Neuroscience of consciousness},
year = {2026},
month = jul,
volume = {2026},
number = {1},
pages = {niag029},
publisher = {Oxford University Press},
issn = {2057-2107},
doi = {10.1093/nc/niag029},
url = {https://doi.org/10.1093/nc/niag029},
pmid = {42415873},
pmcid = {PMC13338905}
}

RIS

TY - JOUR
AU - Bryant, Annie G
AU - Whyte, Christopher J
TI - A data-driven approach to identifying and evaluating connectivity-based neural correlates of conscious visual perception
T2 - Neuroscience of consciousness
J2 - Neurosci Conscious
PY - 2026
DA - 2026/07/07
VL - 2026
IS - 1
SP - niag029
SN - 2057-2107
PB - Oxford University Press
DO - 10.1093/nc/niag029
UR - https://doi.org/10.1093/nc/niag029
LA - en
ER -

CSL-JSON

{
"id": "10.1093/nc/niag029",
"type": "article-journal",
"title": "A data-driven approach to identifying and evaluating connectivity-based neural correlates of conscious visual perception",
"container-title": "Neuroscience of consciousness",
"author": [
{
"family": "Bryant",
"given": "Annie G"
},
{
"family": "Whyte",
"given": "Christopher J"
}
],
"container-title-short": "Neurosci Conscious",
"volume": "2026",
"issue": "1",
"page": "niag029",
"DOI": "10.1093/nc/niag029",
"PMID": "42415873",
"PMCID": "PMC13338905",
"ISSN": "2057-2107",
"publisher": "Oxford University Press",
"URL": "https://doi.org/10.1093/nc/niag029",
"language": "en",
"issued": {
"date-parts": [
[
2026,
7,
7
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1038/s41597-026-07350-9 [code]
An open multi-center MEG-EEG dataset for studying conscious visual perception.
Journal: Scientific data
In common: PyPREP, MNE-Connectivity, MNE-BIDS, 27 other tools, MEG, 7 references
[2] doi:10.1038/s41597-026-07377-y [code]
An open-access multi-site fMRI dataset for investigating conscious visual perception.
Journal: Scientific data
In common: PyPREP, MNE-Connectivity, MNE-BIDS, 27 other tools, 5 references
[3] doi:10.1162/imag.a.1245 [code]
Towards precision EEG connectomics: Evaluating the benefits of dense sampling.
Journal: Imaging neuroscience (Cambridge, Mass.)
In common: MNE-Connectivity, autoreject, AFNI, 19 other tools, 2 references
[4] doi:10.1162/imag.a.1321 [code]
Phase similarity between similar objects indicates representational merging across retrieval training but not sleep.
Journal: Imaging neuroscience (Cambridge, Mass.)
In common: autoreject, Pingouin, easystats, 15 other tools, cognitive, 2 references
[5] doi:10.1002/hbm.70605 [code]
BrainEnrich: Revealing Biological Insights for Imaging-Derived Phenotypes Through Transcriptomic Enrichment.
Journal: Human brain mapping
In common: ggseg, easystats, broom, 17 other tools, 1 reference
[6] doi:10.1016/j.nicl.2026.104012 [code]
Structural-functional multilayer brain network properties and outcome of combined repetitive transcranial magnetic stimulation and psychotherapy for obsessive-compulsive disorder.
Journal: NeuroImage. Clinical
In common: afex, Pingouin, car, 16 other tools, 2 references
[7] doi:10.1038/s41467-026-74824-0 [code]
Learning regularities in noise engages both neural predictive activity and representational changes.
Journal: Nature communications
In common: MNE-BIDS, autoreject, Pingouin, 12 other tools, MEG, cognitive, 3 references
[8] doi:10.1038/s41467-026-71830-0 [code]
Predicting individual differences of fear and cognitive learning and extinction.
Journal: Nature communications
In common: AFNI, car, broom, 14 other tools, cognitive, 1 reference
[9] doi:10.1093/cercor/bhag113 [code]
Long-term reliability and stability of parameterized resting state EEG: evidence from a five-year follow-up.
Journal: Cerebral cortex (New York, N.Y. : 1991)
In common: PyPREP, MNE-BIDS, easystats, 14 other tools, 1 reference
[10] doi:10.1038/s41467-026-73865-9 [code]
Histamine shapes the neurocomputational dynamics of human learning.
Journal: Nature communications
In common: BayesFactor, easystats, car, 13 other tools, cognitive

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.