OSCR

Benchmarking speech biomarkers of Alzheimer's against cognitive and neural measures.

Code ↔ Paper

1 match between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 1 match
  1. [1] § METHODS › Machine learning ↔ Scripts/train_models_bayes.py, lines 30–65 · score 0.89 · Bayesian optimization, outer loop, inner loop, parallel processing, cross validation, iterations

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 503 lines · 28 KB · no license · 1 match

  1. import numpy as np
  2. import pandas as pd
  3. from pathlib import Path
  5. from sklearn.model_selection import StratifiedKFold, KFold, LeaveOneOut, StratifiedShuffleSplit, ShuffleSplit
  6. from sklearn.linear_model import LogisticRegression as LR
  7. from sklearn.svm import SVC, SVR
  8. from sklearn.neighbors import KNeighborsClassifier as KNNC
  9. from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA
  10. from sklearn.discriminant_analysis import QuadraticDiscriminantAnalysis as QDA
  11. from sklearn.linear_model import Lasso, Ridge, ElasticNet
  12. from sklearn.neighbors import KNeighborsRegressor as KNNR
  13. from sklearn.naive_bayes import GaussianNB
  14. from sklearn.model_selection import train_test_split
  15. from sklearn.preprocessing import MinMaxScaler,StandardScaler
  16. from sklearn.impute import KNNImputer
  17. from xgboost import XGBClassifier as xgboost
  18. from xgboost import XGBRegressor as xgboostr
  19. import itertools,pickle,sys, json
  20. import logging,sys,os,argparse
  21. from psrcal.calibration import AffineCalLogLoss
  22. from sklearn.preprocessing import LabelEncoder
  24. sys.path.append(str(Path(Path.home(),'scripts_generales'))) if 'Users/gp' in str(Path.home()) else sys.path.append(str(Path(Path.home(),'user','scripts_generales')))
  26. import utils
  28. from expected_cost.ec import CostMatrix
  30. ##---------------------------------PARAMETERS---------------------------------##
  31. def parse_args():
  32. parser = argparse.ArgumentParser(
  33. description='Train models with hyperparameter optimization and feature selection'
  34. )
  35. parser.add_argument('--project_name', default='crossling_mci',type=str,help='Project name')
  36. parser.add_argument('--stats', type=str, default='', help='Stats to be considered (default = all)')
  37. parser.add_argument('--shuffle_labels', type=int, default=0, help='Shuffle labels flag (1 or 0)')
  38. parser.add_argument('--stratify', type=int, default=1, help='Stratification flag (1 or 0)')
  39. parser.add_argument('--calibrate', type=int, default=0, help='Whether to calibrate models')
  40. parser.add_argument('--n_folds_outer', type=float, default=5, help='Number of folds for cross validation (outer loop)')
  41. parser.add_argument('--n_folds_inner', type=float, default=5, help='Number of folds for cross validation (inner loop)')
  42. parser.add_argument('--n_iter', type=int, default=20, help='Number of hyperparameter iterations')
  43. parser.add_argument('--init_points', type=int, default=20, help='Number of random initial points to test during Bayesian optimization')
  44. parser.add_argument('--feature_selection',type=int,default=1,help='Whether to perform feature selection with RFE or not')
  45. parser.add_argument('--fill_na',type=int,default=0,help='Values to fill nan with. Default (=0) means no filling (imputing instead)')
  46. parser.add_argument('--n_seeds_train',type=int,default=10,help='Number of seeds for cross-validation training')
  47. parser.add_argument('--n_seeds_shuffle',type=int,default=1,help='Number of seeds for shuffling')
  48. parser.add_argument('--scaler_name', type=str, default='StandardScaler', help='Scaler name')
  49. parser.add_argument('--id_col', type=str, default='id', help='ID column name')
  50. parser.add_argument('--n_boot',type=int,default=1000,help='Number of bootstrap iterations')
  51. parser.add_argument('--n_boot_train',type=int,default=0,help='Number of bootstrap iterations for training')
  52. parser.add_argument('--n_boot_test',type=int,default=1000,help='Number of bootstrap iterations for testing')
  53. parser.add_argument('--shuffle_all',type=int,default=1,help='Whether to shuffle all models or only the best ones')
  54. parser.add_argument('--filter_outliers',type=int,default=0,help='Whether to filter outliers in regression problems')
  55. parser.add_argument('--early_fusion',type=int,default=1,help='Whether to perform early fusion')
  56. parser.add_argument('--overwrite',type=int,default=0,help='Whether to overwrite past results or not')
  57. parser.add_argument('--parallel',type=int,default=0,help='Whether to parallelize processes or not')
  58. parser.add_argument('--n_seeds_test',type=int,default=1,help='Number of seeds for testing')
  59. parser.add_argument('--bootstrap_method',type=str,default='bca',help='Bootstrap method [bca, percentile, basic]')
  60. parser.add_argument('--round_values',type=int,default=0,help='Whether to round predicted values for regression or not')
  61. parser.add_argument('--add_dem',type=int,default=0,help='Whether to add demographic features or not')
  62. parser.add_argument('--cut_values',type=float,default=-1,help='Cut values above a given threshold')
  63. parser.add_argument('--regress_out',type=str,default='',help='List of demographic variables to regress out from target variable, separated by "_"')
  64. parser.add_argument('--regress_out_method',type=str,default='linear',help='Whether to perform linear or non-linear regress-out')
  65. return parser.parse_args()
  67. def load_configuration(args):
  68. # Global configuration dictionaries
  69. config = dict(
  70. project_name = args.project_name,
  71. stats = str(args.stats),
  72. shuffle_labels = bool(args.shuffle_labels),
  73. shuffle_all = bool(args.shuffle_all),
  74. stratify = bool(args.stratify),
  75. calibrate = bool(args.calibrate),
  76. n_folds_outer = float(args.n_folds_outer),
  77. n_folds_inner = float(args.n_folds_inner),
  78. n_iter = float(args.n_iter),
  79. feature_selection = bool(args.feature_selection),
  80. fill_na = int(args.fill_na),
  81. init_points = float(args.init_points),
  82. n_seeds_train = float(args.n_seeds_train) if args.n_folds_outer!= -1 else float(1),
  83. n_seeds_shuffle = float(args.n_seeds_shuffle) if args.shuffle_labels else float(0),
  84. scaler_name = args.scaler_name,
  85. id_col = args.id_col,
  86. n_boot = float(args.n_boot),
  87. n_boot_test = float(args.n_boot_test),
  88. n_boot_train = float(args.n_boot_train),
  89. filter_outliers = bool(args.filter_outliers),
  90. early_fusion = bool(args.early_fusion),
  91. overwrite = bool(args.overwrite),
  92. parallel = bool(args.parallel),
  93. n_seeds_test = float(args.n_seeds_test) if args.n_folds_outer!= -1 else float(0),
  94. bootstrap_method = args.bootstrap_method,
  95. round_values = bool(args.round_values),
  96. add_dem = bool(args.add_dem),
  97. cut_values = float(args.cut_values),
  98. regress_out = sorted(list(args.regress_out.split('_'))),
  99. regress_out_method = str(args.regress_out_method)
  100. )
  102. return config
  104. args = parse_args()
  105. config = load_configuration(args)
  106. project_name = config['project_name']
  107. add_dem = config['add_dem']
  108. round_values = config['round_values']
  109. cut_values = config['cut_values']
  110. regress_out = config['regress_out']
  111. fill_na = config['fill_na'] if config['fill_na'] != 0 else None
  113. logging.info('Configuration loaded. Starting training...')
  114. logging.info('Training completed.')
  116. ##------------------ Configuration and Parameter Parsing ------------------##
  117. home = Path(os.environ.get('HOME', Path.home()))
  118. if 'Users/gp' in str(home):
  119. data_dir = home / 'data' / project_name
  120. else:
  121. data_dir = Path('D:/path/data', project_name)
  123. results_dir = Path(str(data_dir).replace('data', 'results'))
  125. main_config = json.load(Path(Path(__file__).parent,'main_config.json').open())
  127. y_labels = main_config['y_labels'][project_name]
  128. tasks = main_config['tasks'][project_name]
  129. single_dimensions = main_config['single_dimensions'][project_name]
  130. data_file = main_config['data_file'][project_name]
  132. try:
  133. test_size = main_config['test_size'][project_name]
  134. except:
  135. test_size = 0
  137. try:
  138. thresholds = main_config['thresholds'][project_name]
  139. except:
  140. thresholds = [None]
  142. try:
  143. cmatrix = CostMatrix(np.array(main_config["cmatrix"][project_name])) if main_config["cmatrix"][project_name] is not None else None
  144. except:
  145. cmatrix = None
  147. config['test_size'] = float(test_size)
  148. config['data_file'] = data_file
  149. config['tasks'] = tasks
  150. config['single_dimensions'] = single_dimensions
  152. config['y_labels'] = y_labels
  153. config['avoid_stats'] = list(set(['min','max','median','skewness','kurtosis','std','mean','stddev']) - set(config['stats'].split('_'))) if config['stats'] != '' else []
  154. config['stat_folder'] = '_'.join(sorted(config['stats'].split('_')))
  156. config['random_seeds_train'] = [int(3**x) for x in np.arange(1, config['n_seeds_train']+1)]
  157. config['random_seeds_test'] = [int(3**x) for x in np.arange(1, config['n_seeds_test']+1)] if config['test_size'] > 0 else ['']
  158. config['random_seeds_shuffle'] = [float(3**x) for x in np.arange(1, config['n_seeds_shuffle']+1)] if config['shuffle_labels'] else ['']
  159. config['bayes'] = True
  161. if config['calibrate']:
  162. calmethod = AffineCalLogLoss
  163. calparams = {'bias':True, 'priors':None}
  164. else:
  165. calmethod = None
  166. calparams = None
  168. models_dict = {'clf':{
  169. 'lr':LR,
  170. 'knnc':KNNC,
  171. 'xgb':xgboost,
  172. #'qda':QDA,
  173. #'lda': LDA
  174. },
  176. 'reg':{'lasso':Lasso,
  177. 'ridge':Ridge,
  178. 'elastic':ElasticNet,
  179. #'knnr':KNNR,
  180. #'svr':SVR,
  181. #'xgb':xgboostr
  182. }
  183. }
  185. hyperp = json.load(Path(Path(__file__).parent,'hyperparameters.json').open())
  187. for task in tasks:
  188. if isinstance(y_labels,dict):
  189. y_labels_ = y_labels[task]
  190. else:
  191. y_labels_ = y_labels
  193. for y_label in y_labels_:
  194. dimensions = list()
  195. if isinstance(single_dimensions,dict):
  196. single_dimensions_ = single_dimensions[task]
  197. else:
  198. single_dimensions_ = single_dimensions
  200. if isinstance(single_dimensions_,list) and config["early_fusion"]:
  201. for ndim in range(len(single_dimensions_)):
  202. for dimension in itertools.combinations(single_dimensions_,ndim+1):
  203. dimensions.append('__'.join(dimension))
  204. elif isinstance(single_dimensions_,list) and not config["early_fusion"]:
  205. dimensions = single_dimensions_
  206. else:
  207. dimensions = [single_dimensions_]
  209. for dimension in dimensions:
  210. try:
  211. all_data = pd.read_csv(Path(data_dir,data_file))
  212. except:
  213. all_data = pd.read_csv(Path(data_dir,data_file),encoding='latin1')
  215. all_data = all_data.loc[:, ~all_data.columns.str.match(r'^Unnamed')]
  217. features = [col for col in all_data.columns if any(f'{x}__{y}__' in col for x,y in itertools.product(task.split('__'),dimension.split('__'))) and 'timestamp' not in col]
  219. if len(config["avoid_stats"]) > 0:
  220. features = [col for col in features if all(f'_{x}' not in col for x in config['avoid_stats'])]
  222. covariates = list(set(regress_out).intersection(set(all_data.columns)))
  223. config['covariates'] = covariates
  225. if config['add_dem']:
  226. for col in set(['sex','age','education','handedness']).intersection(set(all_data.columns)):
  228. all_data[f'{task}__dem__{col}'] = all_data[col]
  230. demographic_features = [f'{task}__dem__{col}' for col in all_data.columns if col in ['sex','age','education','handedness']]
  232. features.extend(demographic_features)
  233. dimension = dimension + '__dem' if dimension != '' else 'dem'
  235. print(task,dimension)
  236. data = all_data[features + [y_label, config['id_col']]]
  237. data.dropna(axis=1,how='all',inplace=True)
  238. data.dropna(subset=y_label,inplace=True)
  239. if cut_values > 0:
  240. data = data[data[y_label] <= cut_values]
  242. data = data.reset_index(drop=True)
  244. if len(np.unique(data[y_label])) > 3:
  245. config['problem_type'] = 'reg'
  246. scoring_metric = 'r2'
  247. else:
  248. config['problem_type'] = 'clf'
  249. scoring_metric = 'roc_auc' if len(np.unique(data[y_label])) == 2 else 'norm_expected_cost'
  251. if config['project_name'] == 'crossling_mci':
  252. config['problem_type'] = 'reg'
  253. scoring_metric = 'r2'
  255. if config['problem_type'] == 'reg' and config['filter_outliers']:
  256. all_data = all_data[np.abs((all_data[y_label] - all_data[y_label].mean()) / all_data[y_label].std()) < 2]
  258. #convert y_label to categories
  259. y = data.pop(y_label)
  261. y = pd.Series(LabelEncoder().fit_transform(y) if config['problem_type'] == 'clf' else y,name=y_label)
  263. if data.shape[0] == 0:
  264. continue
  266. if config['shuffle_labels'] and config['problem_type'] == 'clf':
  267. np.random.seed(42)
  268. zero_indices = np.where(y == 0)[0]
  269. one_indices = np.where(y == 1)[0]
  271. # Shuffle and select half of the indices for flipping
  272. zero_to_flip = np.random.choice(zero_indices, size=len(zero_indices) // 2, replace=False)
  273. one_to_flip = np.random.choice(one_indices, size=len(one_indices) // 2, replace=False)
  275. # Flip the values at the selected indices
  276. y[zero_to_flip] = 1
  277. y[one_to_flip] = 0
  279. elif config['shuffle_labels']:
  280. np.random.seed(42)
  281. #Perform random permutations of the labels
  282. y = np.random.permutation(y)
  284. ID = data.pop(config['id_col'])
  286. if (config['problem_type'] == 'reg') & ('group' in data.columns) & (config['stratify']):
  287. strat_col = data.pop('group')
  288. elif (config['problem_type'] == 'clf') & (config['stratify']):
  289. strat_col = y
  290. else:
  291. strat_col = None
  293. for model_key, model_class in models_dict[config['problem_type']].items():
  294. print(model_key)
  296. held_out = float(config["test_size"]) > 0
  297. n_folds_outer = config['n_folds_outer']
  298. n_folds_inner = config['n_folds_inner']
  300. if held_out:
  301. n_samples_dev = int(data.shape[0] * (1 - config['test_size']))
  302. random_seeds_test = config['random_seeds_test']
  303. else:
  304. n_samples_dev = data.shape[0]
  305. random_seeds_test = ['']
  306. config["n_seeds_test"] = 0
  307. config["random_seeds_test"] = ['']
  309. if n_folds_outer== 0:
  310. n_folds_outer= int(n_samples_dev / np.unique(y).shape[0])
  311. CV_outer = (StratifiedKFold(n_splits=n_folds_outer, shuffle=True)
  312. if strat_col is not None
  313. else KFold(n_splits=n_folds_outer, shuffle=True))
  314. config["kfold_folder"] = f'l{np.unique(y).shape[0]}out'
  315. n_samples_outer = n_samples_dev - np.unique(y).shape[0]
  316. elif n_folds_outer== -1:
  317. CV_outer = LeaveOneOut()
  318. n_samples_outer = n_samples_dev - 1
  319. config["kfold_folder"] = 'loocv'
  320. elif n_folds_outer < 1:
  321. CV_outer = (StratifiedShuffleSplit(n_splits=1,test_size=n_folds_outer)
  322. if strat_col is not None
  323. else ShuffleSplit(n_splits=1,test_size=n_folds_outer))
  324. n_samples_outer = int(n_samples_dev*(1-n_folds_outer))
  325. config['kfold_folder'] = f'{int(n_folds_outer*100)}pct'
  327. else:
  328. n_folds_outer = int(n_folds_outer)
  329. CV_outer = (StratifiedKFold(n_splits=n_folds_outer, shuffle=True)
  330. if strat_col is not None
  331. else KFold(n_splits=n_folds_outer, shuffle=True))
  332. n_samples_outer = int(n_samples_dev*(1-1/n_folds_outer))
  333. config['kfold_folder'] = f'{n_folds_outer}_folds'
  335. if n_folds_inner == 0:
  336. n_folds_inner = int(n_samples_outer / np.unique(y).shape[0])
  337. CV_inner = (StratifiedKFold(n_splits=n_folds_inner, shuffle=True)
  338. if strat_col is not None
  339. else KFold(n_splits=n_folds_inner, shuffle=True))
  340. config["kfold_folder"] += f'_l{np.unique(y).shape[0]}ocv'
  341. n_max = n_samples_outer - np.unique(y).shape[0]
  342. elif n_folds_inner == -1:
  343. CV_inner = LeaveOneOut()
  344. config["kfold_folder"] += '_loocv'
  345. n_max = n_samples_outer - 1
  346. elif n_folds_inner < 1:
  347. CV_inner = (StratifiedShuffleSplit(n_splits=1,test_size=n_folds_inner)
  348. if strat_col is not None
  349. else ShuffleSplit(n_splits=1,test_size=n_folds_inner))
  350. n_max = int(n_samples_outer*(1-n_folds_inner))
  351. config["kfold_folder"] += f'_{int(n_folds_inner*100)}pct'
  352. else:
  353. n_folds_inner = int(n_folds_inner)
  354. CV_inner = (StratifiedKFold(n_splits=n_folds_inner, shuffle=True)
  355. if strat_col is not None
  356. else KFold(n_splits=n_folds_inner, shuffle=True))
  357. n_max = int(n_samples_outer*(1-1/n_folds_inner))
  358. config["kfold_folder"] += f'_{n_folds_inner}_folds'
  360. with open(Path(__file__).parent/'config.json', 'w') as f:
  361. json.dump(config, f, indent=4)
  363. subfolders = [
  364. task, dimension,
  365. config['kfold_folder'], f'{y_label}_res' if len(covariates) > 0 else y_label, config['stat_folder'],scoring_metric,
  366. 'hyp_opt' if config['n_iter'] > 0 else '','feature_selection' if config['feature_selection'] else '',
  367. 'filter_outliers' if config['filter_outliers'] and config['problem_type'] == 'reg' else '','rounded' if round_values else '','cut' if cut_values > 0 else '',
  368. 'shuffle' if config['shuffle_labels'] else ''
  369. ]
  371. path_to_save = results_dir.joinpath(*[str(s) for s in subfolders if s])
  373. if Path(path_to_save,'config.json').exists():
  374. with open(Path(path_to_save,'config.json'), 'rb') as f:
  375. old_config = json.load(f)
  376. old_config['n_boot'] = config['n_boot']
  378. if (not config['overwrite']) & (any(old_config[x] != config[x] for x in ['n_iter','init_points','n_seeds_train','n_boot'])):
  379. for key in ['n_iter','init_points','n_seeds_train','n_boot']:
  380. print(f'Warning: {key} has changed from {old_config[key]} to {config[key]}. Overwriting previous results.')
  381. config[key] = old_config[key]
  382. with open(Path(path_to_save,'config.json'),'w') as f:
  383. json.dump(config, f, indent=4)
  385. if 'scoring_metric' not in list(config.keys()):
  386. config['scoring_metric'] = scoring_metric
  387. config['add_dem'] = add_dem
  388. config['round_values'] = round_values
  390. with open(Path(Path(__file__).parent,'config.json'),'w') as f:
  391. json.dump(config, f, indent=4)
  393. for random_seed_test in random_seeds_test:
  394. Path(path_to_save,f'random_seed_{int(random_seed_test)}' if config['test_size'] else '').mkdir(exist_ok=True,parents=True)
  395. if test_size > 0:
  396. X_train_, X_test_, y_train_, y_test_, ID_train_, ID_test_ = train_test_split(
  397. data, y, ID,
  398. test_size=config['test_size'],
  399. random_state=int(random_seed_test),
  400. shuffle=True,
  401. stratify=strat_col)
  403. if strat_col is not None:
  404. strat_col_train_, strat_col_test_ = train_test_split(
  405. strat_col,
  406. test_size=config['test_size'],
  407. random_state=int(random_seed_test),
  408. shuffle=True,
  409. stratify=strat_col)
  411. # Reset indexes after split.
  412. X_train_.reset_index(drop=True, inplace=True)
  413. X_test_.reset_index(drop=True, inplace=True)
  414. y_train_ = y_train_.reset_index(drop=True)
  415. y_test_ = y_test_.reset_index(drop=True)
  416. ID_train_ = ID_train_.reset_index(drop=True)
  417. ID_test_ = ID_test_.reset_index(drop=True)
  418. if strat_col is not None:
  419. strat_col_train_ = strat_col_train_.reset_index(drop=True)
  420. else:
  421. strat_col_train_ = None
  423. else:
  424. X_train_, y_train_, ID_train_ = data.reset_index(drop=True), y.reset_index(drop=True), ID.reset_index(drop=True)
  425. if strat_col is not None:
  426. strat_col_train_ = strat_col.reset_index(drop=True)
  427. else:
  428. strat_col_train_ = None
  430. X_test_, y_test_, ID_test_ = pd.DataFrame(), pd.Series(), pd.Series()
  432. data_train = pd.concat((X_train_,y_train_,ID_train_),axis=1)
  434. data_test = pd.concat((X_test_,y_test_,ID_test_),axis=1)
  436. data_train.to_csv(Path(path_to_save,f'random_seed_{int(random_seed_test)}' if config['test_size'] else '', 'data_train.csv'),index=False)
  437. data_test.to_csv(Path(path_to_save,f'random_seed_{int(random_seed_test)}' if config['test_size'] else '', 'data_test.csv'),index=False)
  439. hyperp['knnc']['n_neighbors'] = (1,n_max)
  440. hyperp['knnr']['n_neighbors'] = (1,n_max)
  442. # Check for data leakage.
  443. assert set(ID_train_).isdisjoint(set(ID_test_)), 'Data leakage detected between train and test sets!'
  445. if (Path(path_to_save,f'random_seed_{int(random_seed_test)}' if config['test_size'] else '', f'all_models_{model_key}.csv').exists() and config['calibrate'] == False) or (Path(path_to_save,f'random_seed_{int(random_seed_test)}' if config['test_size'] else '', f'cal_outputs_{model_key}.pkl').exists() and config['calibrate']):
  446. if not bool(config['overwrite']):
  447. print(f'Results already exist for {task} - {y_label} - {model_key}. Skipping...')
  448. continue
  450. print(f'Training model: {model_key}')
  452. all_models,outputs_best,y_dev,y_pred_best,IDs_dev = utils.nestedCVT(model_class=models_dict[config['problem_type']][model_key],
  453. scaler=StandardScaler if config['scaler_name'] == 'StandardScaler' else MinMaxScaler,
  454. imputer=KNNImputer,
  455. X=X_train_,
  456. y=y_train_.values if isinstance(y_train_, pd.Series) else y_train_,
  457. n_iter=int(config['n_iter']),
  458. iterator_outer=CV_outer,
  459. iterator_inner=CV_inner,
  460. strat_col=strat_col_train_,
  461. random_seeds_outer=config['random_seeds_train'],
  462. hyperp_space=hyperp[model_key],
  463. IDs=ID_train_,
  464. init_points=int(config['init_points']),
  465. scoring=scoring_metric,
  466. problem_type=config['problem_type'],
  467. cmatrix=cmatrix,priors=None,
  468. threshold=thresholds,
  469. parallel=bool(config['parallel']),
  470. feature_selection=bool(config['feature_selection']),
  471. calmethod=calmethod,
  472. calparams=calparams,
  473. round_values=round_values,
  474. covariates=covariates if isinstance(covariates,pd.DataFrame) else None,
  475. fill_na = fill_na,
  476. regress_out_method = config['regress_out_method']
  477. )
  479. Path(path_to_save,f'random_seed_{int(random_seed_test)}' if config['test_size'] else '').mkdir(parents=True, exist_ok=True)
  480. with open(Path(path_to_save,f'random_seed_{int(random_seed_test)}' if config['test_size'] else '','config.json'),'w') as f:
  481. json.dump(config,f)
  483. all_models.to_csv(Path(path_to_save,f'random_seed_{int(random_seed_test)}' if config['test_size'] else '',f'all_models_{model_key}.csv'),index=False)
  484. result_files = {
  485. 'X_train.npy': X_train_,
  486. 'y_train.npy': y_train_,
  487. 'IDs_train.npy': ID_train_,
  488. 'y_dev.npy': y_dev,
  489. 'IDs_dev.npy': IDs_dev,
  490. f'outputs_{model_key}.npy': outputs_best}
  492. if test_size > 0:
  493. result_files.update({
  494. 'X_test.npy': X_test_,
  495. 'y_test.npy': y_test_,
  496. 'IDs_test.npy': ID_test_,
  497. })
  498. for fname, obj in result_files.items():
  499. with open(Path(path_to_save,f'random_seed_{int(random_seed_test)}' if config['test_size'] else '', fname), 'wb') as f:
  500. np.save(f, obj)
  502. with open(Path(path_to_save,f'random_seed_{int(random_seed_test)}' if config['test_size'] else '', 'config.json'), 'w') as f:
  503. json.dump(config, f, indent=4)

train_models_bayes.py, no license · at the source

Overview

Authors: Ivan Caro1,2, Gonzalo Pérez1,2,3, Joaquín Valdés Bize4, Joaquín Ponferrada1, Franco J Ferrante1,2,3, Alejandro Sosa Welford1, Lara Gauder5,6, Loreto Olavarría7,8,9,10, Fernando Henríquez9,11,12, Teresita Ramos7, Cristina Besnier7, Luciana Ferrer5, Maria Luisa Gorno‐Tempini13, Andrea Slachevsky7,9,11,14, Agustín Ibañez1,15,16,17,18,19, Adolfo M García1,16,20,19
ORCID iDs: Ivan Caro, Franco J Ferrante
20 affiliations
  1. Cognitive Neuroscience Center (CNC), Department of Life and Behavioral Sciences, Universidad de San Andrés, Buenos Aires, Argentina
  2. National Scientific and Technical Research Council (CONICET), Buenos Aires, Argentina
  3. School of Engineering, University of Buenos Aires, Buenos Aires, Argentina
  4. Department of Psychiatry, School of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile
  5. Instituto de Investigación en Ciencias de la Computación (ICC), CONICET‐UBA, Buenos Aires, Argentina
  6. Departamento de Computación, Faculty of Exact and Natural Sciences, University of Buenos Aires (UBA), Buenos Aires, Argentina
  7. Memory and Neuropsychiatric Center (CMYN) Neurology Department, Hospital del Salvador & Faculty of Medicine, University of Chile, Santiago, Chile
  8. Departamento de Psiquiatría Oriente, Facultad de Medicina, Universidad de Chile, Santiago, Chile
  9. Neuropsychology and Clinical Neuroscience Laboratory (LANNEC), Physiopathology Program – Institute of Biomedical Sciences (ICBM), Neuroscience and East Neuroscience Departments, Faculty of Medicine, University of Chile, Santiago, Chile
  10. Servicio de Neurología, Clínica Las Condes, Santiago, Chile
  11. Gerosciences Center for Brain Health and Metabolism, Santiago, Chile
  12. Interdisciplinary Center for Neuroscience (NeuroUC), Laboratory for Cognitive and Evolutionary Neuroscience (LANCE), Department of Psychiatry, Faculty of Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile
  13. Memory and Aging Center, University of California, San Francisco, California, USA
  14. Servicio de Neurología, Departamento de Medicina, Clínica Alemana‐Universidad del Desarrollo, Santiago, Chile
  15. Latin American Brain Health Institute (BrainLat), Universidad Adolfo Ibáñez, Santiago, Chile
  16. Global Brain Health Institute, University of California San Francisco, San Francisco, California, USA
  17. Department of Biophysics, School of Medicine, Istanbul Medipol University, Istanbul, Türkiye
  18. Barcelonaβeta Brain Research Center (BBRC), Pasqual Maragall Foundation, Barcelona, Spain
  19. Trinity College Dublin, Dublin, Ireland
  20. Departamento de Lingüística y Literatura, Facultad de Humanidades, Universidad de Santiago de Chile, Santiago, Chile
Institutions: Consejo Nacional de Investigaciones Científicas y Técnicas (Argentina); University of San Andrés (Argentina); Universidad de Buenos Aires (Argentina); Pontificia Universidad Católica de Chile (Chile); Clínica Las Condes (Chile); Hospital del Salvador (Chile); University of Chile (Chile); University of California, San Francisco (United States); Universidad del Desarrollo (Chile); Clínica Alemana (Chile); Trinity College Dublin (Ireland); Adolfo Ibáñez University (Chile); Istanbul Medipol University (Türkiye); Global Brain Health Institute (United States); Universidad de Santiago de Chile (Chile)
Journal: Alzheimer's & dementia : the journal of the Alzheimer's Association, volume 22, issue 4, article e71365
Dates: received 21 December 2025; accepted 11 March 2026; published online 14 April 2026; in print April 2026
Type: Research article · Language: English
License: CC BY-NC-ND
Identifiers: DOI 10.1002/alz.71365 · PMID 41979006 · PMCID PMC13077448 · OpenAlex W7154239291
Open access: hybrid, a free copy (OpenAlex)
Status: code verified
Categories: fMRI (modality), human (organism), Alzheimer's / dementia (population), cognitive (subfield)
Methods: Spectral & time-frequency, Connectivity, Statistics, Machine learning, fMRI & imaging, Preprocessing
Keywords: Alzheimer's disease, automated speech and language analysis, cognitive assessments, digital biomarkers, machine learning, magnetic resonance imaging
MeSH: Alzheimer Disease*, Biomarkers*, Cognition*, Speech*, Aged, Brain, Executive Function, Female, Humans, Machine Learning, Magnetic Resonance Imaging, Male, Memory, Episodic, Neuropsychological Tests (* major topic)
Topic: Neurobiology of Language and Bilingualism (Cognitive Neuroscience, Neuroscience), according to OpenAlex
Funding: Fondecyt Regular (1231839); National Institute on Aging (2P01AG019724, R01 AG057234, R01 AG21051, CARDS‐NIH, R01 AG083799, R01 AG075775); NIA NIH HHS (CARDS-NIH, R01 AG075775, R01 AG21051, 2P01AG019724, R01 AG083799, P01 AG019724, R01 AG057234); Alzheimer&apos;s Association (SG‐20‐725707); Global Brain Health Institute; The Bluefield project to cure FTD; ANID Fondecyt regular (1250091, 1220995, 1250317); ANID/FONDAP (15150012); BRAIN-CLIMA: Investigating the Combined Impact of Heat and Air Pollution on Blood-Brain Barrier Integrity and Brain Aging in Latin America (335293/Z/25/Z); ANID/PIA/ANILLOS (ACT210096); ANID (FONDAP ID15150012, FONDECYT Regular 1250317 1250091); Agencia Nacional de Promoción Científica y Tecnológica (01-PICTE-2022-05-00103, 01‐PICTE‐2022‐05‐00103); Wellcome Leap CARE Program (CARE-2025-0883490149); Fondo de Fomento al Desarrollo Científico y Tecnológico (ID20I10152); FIC NIH HHS; Marie Skłodowska-Curie Actions - MSCA; Agencia Nacional de Investigación y Desarrollo (FONDECYT Regular 1250091); Rainwater Charitable Foundation—The Bluefield project to cure FTD; Alzheimer's Association (SG-20-725707); Rainwater Charitable Foundation - Tau Consortium, and Global Brain Health Institute; Tau Consortium; Advancing Female-Specific Predictive Models and Risk Assessment Tools for Alzheimer's Disease in the US and Latin America, and the CliCBrain (101236426); NIA of the NIH (2P01AG019724, R01AG075775, R01AG083799); JPI JPND-Care (DISCeRN 2025); FONDEF (ID20I10152); Fogarty International Center (FIC); Health and Social Care Research with a Focus on the Moderate and Late Stages of Neurodegenerative Diseases; NIH; Rainwater Charitable Foundation; Fogarty International Center; National Institutes of Health; Wellcome Trust; PIA Anillos (ACT210096)
Citations: cited by 2 papers (Europe PMC); 75 references in the paper

Abstract

The abstract is not reproduced here: the paper's license (CC BY-NC-ND) does not allow it. Read it in the paper, at the publisher or on Europe PMC.

Repository

Its files are read in the Code ↔ Paper reader above, with 1 match between paragraphs and lines of code.

OSF ad8jb

License: none: the authors keep all their rights
State: the link answers, verified on 28 September 2026
Evidence: files inventoried
Languages: Python (4)
Size: 7 files, 4 scripts
Software Heritage: not checked
Found in: “DATA AVAILABILITY STATEMENT”
Not found: README, license file, CITATION.cff, environment file, tests, continuous integration, documentation
Tools: NumPy (4 files), pandas (4 files), SciPy (3 files), Matplotlib (2 files), scikit-learn (2 files), seaborn (2 files), XGBoost (2 files), Pingouin (1 file), statsmodels (1 file)
Availability: 1 check, the latest on 28 September 2026: the link answers (HTTP 200)
  • 28 September 2026: the link answers (HTTP 200)
4 files
At the source: osf.io/ad8jb/overview?view_only=e69e33184e334282ade387b1c4da32a9

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 4 scripts, each with its path and the digest of its content;
  • 1 match between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Code and data availability statement

The paper has a code and data availability statement. Its license (CC BY-NC-ND) does not allow reproducing it here; in short, from what the harvester recognized in it:

Read it in the paper: doi.org/10.1002/alz.71365.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 28 September 2026: the first record

Recorded: type, language, journal, volume, issue, pages, dates, 16 authors, 6 keywords, 14 MeSH terms, 33 funders, 70 references.

Cite

This paper

Caro, I., Pérez, G., Bize, J. V., Ponferrada, J., Ferrante, F. J., Welford, A. S., Gauder, L., Olavarría, L., Henríquez, F., Ramos, T., Besnier, C., Ferrer, L., Gorno‐Tempini, M. L., Slachevsky, A., Ibañez, A., & García, A. M. (2026). Benchmarking speech biomarkers of Alzheimer's against cognitive and neural measures. Alzheimer's & dementia : the journal of the Alzheimer's Association, 22(4), e71365. https://doi.org/10.1002/alz.71365

BibTeX

@article{caro2026benchmarking,
author = {Caro, Ivan and Pérez, Gonzalo and Bize, Joaquín Valdés and Ponferrada, Joaquín and Ferrante, Franco J and Welford, Alejandro Sosa and Gauder, Lara and Olavarría, Loreto and Henríquez, Fernando and Ramos, Teresita and Besnier, Cristina and Ferrer, Luciana and Gorno‐Tempini, Maria Luisa and Slachevsky, Andrea and Ibañez, Agustín and García, Adolfo M},
title = {{Benchmarking speech biomarkers of Alzheimer's against cognitive and neural measures}},
journal = {Alzheimer's \& dementia : the journal of the Alzheimer's Association},
year = {2026},
month = apr,
volume = {22},
number = {4},
pages = {e71365},
publisher = {Wiley},
issn = {1552-5260},
doi = {10.1002/alz.71365},
url = {https://doi.org/10.1002/alz.71365},
pmid = {41979006},
pmcid = {PMC13077448}
}

RIS

TY - JOUR
AU - Caro, Ivan
AU - Pérez, Gonzalo
AU - Bize, Joaquín Valdés
AU - Ponferrada, Joaquín
AU - Ferrante, Franco J
AU - Welford, Alejandro Sosa
AU - Gauder, Lara
AU - Olavarría, Loreto
AU - Henríquez, Fernando
AU - Ramos, Teresita
AU - Besnier, Cristina
AU - Ferrer, Luciana
AU - Gorno‐Tempini, Maria Luisa
AU - Slachevsky, Andrea
AU - Ibañez, Agustín
AU - García, Adolfo M
TI - Benchmarking speech biomarkers of Alzheimer's against cognitive and neural measures
T2 - Alzheimer's & dementia : the journal of the Alzheimer's Association
J2 - Alzheimers Dement
PY - 2026
DA - 2026/04/01
VL - 22
IS - 4
SP - e71365
SN - 1552-5260
PB - Wiley
DO - 10.1002/alz.71365
UR - https://doi.org/10.1002/alz.71365
LA - en
ER -

CSL-JSON

{
"id": "10.1002/alz.71365",
"type": "article-journal",
"title": "Benchmarking speech biomarkers of Alzheimer's against cognitive and neural measures",
"container-title": "Alzheimer's & dementia : the journal of the Alzheimer's Association",
"author": [
{
"family": "Caro",
"given": "Ivan"
},
{
"family": "Pérez",
"given": "Gonzalo"
},
{
"family": "Bize",
"given": "Joaquín Valdés"
},
{
"family": "Ponferrada",
"given": "Joaquín"
},
{
"family": "Ferrante",
"given": "Franco J"
},
{
"family": "Welford",
"given": "Alejandro Sosa"
},
{
"family": "Gauder",
"given": "Lara"
},
{
"family": "Olavarría",
"given": "Loreto"
},
{
"family": "Henríquez",
"given": "Fernando"
},
{
"family": "Ramos",
"given": "Teresita"
},
{
"family": "Besnier",
"given": "Cristina"
},
{
"family": "Ferrer",
"given": "Luciana"
},
{
"family": "Gorno‐Tempini",
"given": "Maria Luisa"
},
{
"family": "Slachevsky",
"given": "Andrea"
},
{
"family": "Ibañez",
"given": "Agustín"
},
{
"family": "García",
"given": "Adolfo M"
}
],
"container-title-short": "Alzheimers Dement",
"volume": "22",
"issue": "4",
"page": "e71365",
"DOI": "10.1002/alz.71365",
"PMID": "41979006",
"PMCID": "PMC13077448",
"ISSN": "1552-5260",
"publisher": "Wiley",
"URL": "https://doi.org/10.1002/alz.71365",
"language": "en",
"issued": {
"date-parts": [
[
2026,
4,
1
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1002/alz.71232
Neural embedding of frailty in cognitively unimpaired aging and dementia across Latin America.
Journal: Alzheimer's & dementia : the journal of the Alzheimer's Association
In common: Alzheimer's / dementia, 7 references
[2] doi:10.1016/j.nicl.2026.104012 [code]
Structural-functional multilayer brain network properties and outcome of combined repetitive transcranial magnetic stimulation and psychotherapy for obsessive-compulsive disorder.
Journal: NeuroImage. Clinical
In common: XGBoost, Pingouin, statsmodels, 6 other tools, fMRI, 1 reference
[3] doi:10.1002/agm2.70073 [code]
Foreign Language Learning in Older Adults Modifies Resting-State Functional Connectivity Between the Subcortical Structures and the Cortex.
Journal: Aging medicine (Milton (N.S.W))
In common: statsmodels, seaborn, scikit-learn, 4 other tools, fMRI, 3 references
[4] doi:10.1162/imag.a.1208 [code]
Brain network analysis in Alzheimer's disease and mild cognitive impairment using high-density diffuse optical tomography.
Journal: Imaging neuroscience (Cambridge, Mass.)
In common: statsmodels, seaborn, scikit-learn, 4 other tools, Alzheimer's / dementia, 3 references
[5] doi:10.1038/s41598-026-56688-y [code]
On the value of radiomics in addition to clinical measures in emotional conflict fMRI for predicting sertraline response in major depressive disorder.
Journal: Scientific reports
In common: XGBoost, statsmodels, seaborn, 5 other tools, fMRI, 1 reference
[6] doi:10.1002/acn3.70396 [code]
Memory and Resting-State Connectivity in Acute Transient Global Amnesia: A Case-Control fMRI Study.
Journal: Annals of clinical and translational neurology
In common: Pingouin, statsmodels, seaborn, 4 other tools, fMRI, cognitive, 2 references
[7] doi:10.1002/dad2.70366 [code]
Redefining the norms: an interdisciplinary perspective on language testing in multilinguals with acquired and progressive neurogenic disorders.
Journal: Alzheimer's & dementia (Amsterdam, Netherlands)
In common: pandas, SciPy, Matplotlib, 1 other tool, cognitive, 4 references
[8] doi:10.1523/eneuro.0041-26.2026 [code]
Ocular Speech Tracking Persists in Blindness, but Its Dynamics and Oculo-Cerebral Connectivity Depend on Visual Status.
Journal: eNeuro
In common: Pingouin, statsmodels, seaborn, 5 other tools, cognitive, 1 reference
[9] doi:10.1093/sleep/zsag030 [code]
Divergent disruption of brain networks following total and chronic sleep loss: a longitudinal fMRI study.
Journal: Sleep
In common: statsmodels, seaborn, scikit-learn, 4 other tools, fMRI, 3 references
[10] doi:10.1038/s41514-026-00456-9 [code]
Exploring the link between body physiology and cognition: the role of the brain and aging.
Journal: npj aging
In common: XGBoost, statsmodels, seaborn, 5 other tools, cognitive, 1 reference

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.