OSCR

Effect of vascular lesion preprocessing on Brain Intensity AbNormality Classification Algorithm (BIANCA) white matter hyperintensity segmentation.

Code ↔ Paper

28 matches between paragraphs of the paper and lines of its authors' code, computed by the harvester (lexical-v1). Click a colored paragraph or line to see its counterpart.

The 28 matches
  1. [1] § Methods › Statistical analysis ↔ Phase_2_a/1_2a_bland_altman_test_temp.py, lines 919–984 · score 0.94 · 0.75–0.9, Intraclass Correlation Coefficient, Bland Altman, 0.5–0.75, equivalence margin, way mixed
  2. [2] § Methods › LOCATE adaptive thresholding ↔ 1_preprare_locate.py, lines 4–85 · score 0.91 · unbiased lesion probability, LOCally Adaptive Threshold, fold cross validation, LOCATE training, training subjects, leakage
  3. [3] § Methods › Lesion delineation, removal, and inpainting ↔ 6_train_bianca.py, lines 1–59 · score 0.85 · NAWM intensities, lesion_filling, FLAIR images, lesion voxel, intensity distribution, FSL
  4. [4] § Results › Phase II-B: Volume-based agreement (n = 211) ↔ Phase_2_b/2_2b_volume_difference_lesion_bonf.py, lines 1–61 · score 0.81 · 0.14 mL, 0.07 mL, 0.29 mL, 0.06 mL, marginal, 0.04 mL
  5. [5] § Methods › Statistical analysis ↔ Phase_2_a/5_2a_shapley_values.py, lines 1–59 · score 0.79 · TreeExplainer, max depth, Random Forest, bag, SHAP, trees
  6. [6] § Methods › Phase I: Parameter optimization › Cluster-based analysis ↔ 4_Analyze_cluster_metrics.py, lines 43–116 · score 0.77 · ground truth masks, Minimum cluster, Dice coefficient, complementing, recall, detection
  7. [7] § Methods › Preprocessing pipeline ↔ 1_preprare_locate.py, lines 4–85 · score 0.75 · ventricle distance map, white matter mask, adaptive thresholding, pipeline, LOCATE, segmentation
  8. [8] § Methods › Statistical analysis ↔ Phase_2_b/5_2b_shapley_values.py, lines 1–63 · score 0.75 · TreeExplainer, max depth, Random Forest, SHAP, trees, MAE
  9. [9] § Results › Population characteristics ↔ Phase_2_b/0_2b_population.py, lines 1–79 · score 0.74 · Prisma fit, 3–10, Scanner distribution, BeLOVE, lesion volume, ARWMC
  10. [10] § Methods › Datasets and scanner distribution ↔ Phase_2_a/0_2a_population.py, lines 1–85 · score 0.72 · full cohort, ground truth WMH, Prisma fit, WMH masks, BeLOVE, Trio
  11. [11] § Methods › Lesion delineation, removal, and inpainting ↔ 3_threshold_analysis_parallel_temp.py, lines 1–78 · score 0.70 · lesion_filling, lesion voxel, zero filled, FSL, physiologically, NAWM
  12. [12] § Methods › Phase I: Parameter optimization › Cluster-based analysis ↔ 4_plot_cluster_grid_search.py, lines 1–48 · score 0.69 · Optimal minimum cluster, ground truth masks, recall, filtering, predicted, voxel
  13. [13] § Methods › Study design and datasets ↔ 3_threshold_analysis_parallel_temp.py, lines 1–78 · score 0.69 · lesion_filling, lesion voxels, zero filled, cross validation, FSL, NAWM
  14. [14] § Results › Population characteristics ↔ Phase_2_a/0_2a_population.py, lines 1–85 · score 0.69 · LOCATE training pool, ground truth WMH, Prisma fit, scanner balanced, BeLOVE, Population
  15. [15] § Results › Phase II-B: Volume-based agreement (n = 211) ↔ Phase_2_b/3_2b_volume_difference_scanner_bonf.py, lines 1–59 · score 0.69 · Prisma fit, 14.67 mL, 15.58 mL, largest, partly, baseline
  16. [16] § Methods › Study design and datasets ↔ 5_create_5_fold_cross_validation_sets.py, lines 1–63 · score 0.67 · lesion_filling, cross validation, zero filled, FSL, NAWM, CV
  17. [17] § Methods › Preprocessing pipeline › GE scanner exclusion ↔ Phase_1/GE_COMPARE/2_GE_analysis.py, lines 1–86 · score 0.64 · GE inclusion, GE subjects, medium, Philips, stratified, negligible
  18. [18] § Results › Phase II-A: preprocessing effect assessment (n = 89) ↔ Phase_2_a/5_2a_shapley_values.py, lines 61–120 · score 0.64 · profiles, SHAP, explanatory, Forest, lesion volume, MAE
  19. [19] § Methods › Phase I: Parameter optimization › Training condition comparison ↔ 4_Analyze_cluster_metrics.py, lines 407–555 · score 0.64 · preprocessing affects, Wilcoxon signed rank, training preprocessing, bootstrapped, Bonferroni, Cliff
  20. [20] § Methods › Cross-validation strategy ↔ 4_cluster_size_grid_search.py, lines 1–84 · score 0.63 · LOO CV, cross validation, leakage, mixed, seeds, stratified
  21. [21] § Results › Phase I: parameter optimization (n = 89) › Threshold analysis ↔ Phase_1/5FCV_SET/1_voxel_treshhold_analysis.py, lines 1–61 · score 0.63 · disadvantage, b0, medium, adaptive, Pairwise, Wilcoxon
  22. [22] § Methods › Preprocessing pipeline ↔ Phase_2_b/0_2b_population.py, lines 1–79 · score 0.63 · HD BET, fsl_anat, Brain, mask, segmentation, Preprocessing
  23. [23] § Methods › Cross-validation strategy ↔ 5_create_5_fold_cross_validation_sets.py, lines 1–63 · score 0.62 · fold cross validation, LOO CV, seeds, stratified, severity, scanner
  24. [24] § Results › Phase II-A: preprocessing effect assessment (n = 89) ↔ Phase_2_a/1_2a_bland_altman_test_temp.py, lines 1–64 · score 0.60 · variance, attributable, TOST, opposing, marginally, shifts
  25. [25] § Results › Population characteristics ↔ Phase_1/1_phase_1_belove_k_bianca_population.py, lines 62–125 · score 0.60 · Prisma fit, ground truth WMH, 6.96 mL, BeLOVE, cutoffs, IQR
  26. [26] § Results › Phase I: parameter optimization (n = 89) › Cluster-based analysis ↔ 4_Analyze_cluster_metrics.py, lines 43–116 · score 0.58 · complemented voxel, minimum cluster, detection, precision, Cliff, metrics
  27. [27] § Methods › Statistical analysis ↔ 4_Analyze_cluster_metrics.py, lines 670–767 · score 0.54 · Mann Whitney, Wilcoxon signed rank, BeLOVE, Cliff, Delta
  28. [28] § Methods › Lesion delineation, removal, and inpainting ↔ Phase_2_a/8_2a_correlation_dice_2.py, lines 203–232 · score 0.52 · intracranial hemorrhage, ischemic infarcts, lacunes, inpainting, lesions

Paper

Loaded from Europe PMC by your browser, not stored by OSCR: doi.org · Europe PMC

The paper is loaded when this pane is shown.

The authors' code

Python · 771 lines · 33 KB · no license · 4 matches

  1. #!/usr/bin/env python3
  2. # -*- coding: utf-8 -*-
  3. """
  4. Cluster-Level Analysis of BIANCA WMH Segmentation
  5. ===================================================
  6. @author: temuuleu
  8. # =============================================================
  9. # REVIEWER RESPONSE (R1, Comment 2; R5, #17)
  10. # =============================================================
  11. #
  12. # R1 raised the concern that global Dice alone cannot distinguish
  13. # genuine sensitivity gains from spatially distributed false positives.
  14. # R5 (#17) noted that precision values were not contextualized.
  15. #
  16. # This script produces all cluster-level analyses for the manuscript:
  17. #
  18. # MAIN TEXT:
  19. # 1. Voxel vs Cluster dissociation (Figure: Dice vs Cluster F1 scatter)
  20. # 2. Cluster Precision vs Recall (Figure: bar chart)
  21. # 3. R vs NR condition comparison (Figure: boxplot + Kruskal-Wallis)
  22. # 4. Cliff's Delta for R vs NR on cluster-level
  23. #
  24. # SUPPLEMENTAL:
  25. # 5. Post-hoc Wilcoxon with Bonferroni correction
  26. # 6. Per-dataset breakdown (BeLOVE vs Challenge, Mann-Whitney U)
  27. # 7. GT cluster count distribution
  28. # 8. Summary tables (mean, SD, median per condition x threshold)
  29. #
  30. # NOT INCLUDED (not requested by reviewers):
  31. # - Per-scanner stratification (already covered in voxel-level analysis)
  32. # - Per-severity stratification (already covered in voxel-level analysis)
  33. # - Full 9x3 condition matrix (identical values, no added information)
  34. #
  35. # Methodology:
  36. # - Connected-component labeling: 26-connectivity
  37. # - Minimum cluster size: threshold- and dataset-specific (grid search)
  38. # - Overlap criterion: any-overlap (>=1 voxel), MICCAI 2017
  39. # - Filtering: prediction only; GT retained without filtering
  40. # - Evaluated: 10 seeds x 5-fold stratified CV x 89 subjects
  41. # =============================================================
  43. # =============================================================
  44. # PAPER TEXT: METHODS
  45. # =============================================================
  46. #
  47. # 2.X Lesion-level evaluation
  48. #
  49. # To complement voxel-level Dice coefficients, we computed lesion-level
  50. # precision, recall, and F1 score using connected-component analysis.
  51. # Binary segmentation masks were decomposed into spatially contiguous
  52. # clusters using 26-connectivity (scipy.ndimage.label). Predicted
  53. # clusters smaller than a threshold-specific minimum cluster size
  54. # (determined via grid search; Supplemental Table X) were removed to
  55. # exclude spurious components; ground truth masks were not filtered.
  56. # Consistent with the MICCAI 2017 WMH Segmentation Challenge (Kuijf
  57. # et al., 2019), a predicted cluster was classified as a true positive
  58. # if it overlapped with any ground truth cluster by at least one voxel
  59. # (any-overlap criterion). Lesion-level precision was defined as the
  60. # proportion of predicted clusters classified as true positives, and
  61. # lesion-level recall as the proportion of ground truth clusters
  62. # detected. Lesion-level F1 was computed as the harmonic mean of
  63. # precision and recall.
  64. #
  65. # To assess the effect of training preprocessing on cluster-level
  66. # detection, Kruskal-Wallis H-tests were performed for each threshold
  67. # and metric. Where the omnibus test was significant (alpha = 0.05),
  68. # post-hoc pairwise Wilcoxon signed-rank tests were conducted with
  69. # Bonferroni correction (corrected alpha = 0.05/3 = 0.0167). Effect
  70. # sizes were quantified using Cliff's Delta, with |delta| >= 0.28 as
  71. # the threshold for a meaningful difference.
  72. # =============================================================
  74. # =============================================================
  75. # PAPER TEXT: RESULTS
  76. # =============================================================
  77. #
  78. # 3.X Lesion-level evaluation
  79. #
  80. # Cluster-level F1 scores (0.855-0.877) substantially exceeded
  81. # voxel-level Dice coefficients (0.540-0.567), indicating that BIANCA
  82. # detected the majority of WMH clusters but delineated their boundaries
  83. # imprecisely (Supplemental Figure X). Lesion-level precision was
  84. # consistently high across all thresholding approaches (0.881-0.928),
  85. # confirming that the majority of predicted clusters corresponded to
  86. # genuine WMH regions. Lesion-level recall ranged from 0.849 to 0.905,
  87. # with B+L achieving the highest detection rate.
  88. #
  89. # Kruskal-Wallis tests revealed no significant differences between
  90. # training preprocessing conditions on any cluster-level metric
  91. # (all p > 0.92; Supplemental Table X), confirming that lesion removal
  92. # does not affect lesion detection performance. This finding, combined
  93. # with the high and stable cluster-level precision (0.881-0.928),
  94. # addresses the concern that higher WMH volumes after lesion removal
  95. # might reflect spatially distributed false positives rather than
  96. # genuine sensitivity gains.
  97. # =============================================================
  98. """
  100. import matplotlib
  101. matplotlib.use('Agg')
  103. import pandas as pd
  104. import numpy as np
  105. import matplotlib.pyplot as plt
  106. import seaborn as sns
  107. import os
  108. import glob
  109. from scipy.stats import wilcoxon, mannwhitneyu, kruskal
  111. try:
  112. from cliffs_delta import cliffs_delta
  113. HAS_CLIFFS = True
  114. except ImportError:
  115. HAS_CLIFFS = False
  116. print("WARNING: cliffs_delta not installed. Install with: pip install cliffs_delta")
  118. # =============================================================
  119. # CONFIG
  120. # =============================================================
  121. SEED_DIR = "Phase_1/5FCV_SET/bianca_result_dir"
  122. OUTPUT_DIR = "Phase_1/Cluster_analysis_results"
  123. PLOT_DIR = os.path.join(OUTPUT_DIR, "plots")
  125. # Labels for plots (LaTeX subscript) vs Excel (plain text)
  126. TH_PLOT = {'85': r'B$_{0.85}$', '90': r'B$_{0.90}$', 'locate': 'B+L'}
  127. TH_PLAIN = {'85': 'B_0.85', '90': 'B_0.90', 'locate': 'B+L'}
  128. COND_LABELS = {
  129. 'non_removed': 'Train Non Removed',
  130. 'removed': 'Train Removed',
  131. 'filled': 'Train Inpainted',
  132. }
  134. # All metrics: voxel-level + cluster-level
  135. VOXEL_METRICS = ['dice_score', 'sensitivity', 'precision']
  136. CLUSTER_METRICS = ['lesion_f1', 'lesion_precision', 'lesion_recall']
  137. ALL_METRICS = VOXEL_METRICS + CLUSTER_METRICS
  139. # Bootstrap CI parameters (consistent with manuscript: Efron & Tibshirani, 1993)
  140. N_BOOTSTRAP = 1000
  141. CI_ALPHA = 0.05 # 95% CI
  142. RANDOM_SEED = 42
  144. os.makedirs(OUTPUT_DIR, exist_ok=True)
  145. os.makedirs(PLOT_DIR, exist_ok=True)
  148. # =============================================================
  149. # HELPER: Excel export with metadata
  150. # =============================================================
  151. def save_excel(filepath, dataframe, title, description):
  152. """Save DataFrame to Excel with descriptive metadata header."""
  153. from openpyxl import Workbook
  154. from openpyxl.styles import Font, PatternFill, Alignment, Border, Side
  155. from openpyxl.utils import get_column_letter
  157. wb = Workbook()
  158. ws = wb.active
  159. n_cols = len(dataframe.columns)
  160. merge_width = max(n_cols, 6)
  162. thin = Border(*(Side(style='thin', color='B0B0B0'),) * 4)
  164. # Title
  165. row = 1
  166. ws.merge_cells(start_row=row, start_column=1, end_row=row, end_column=merge_width)
  167. c = ws.cell(row=row, column=1, value=title)
  168. c.font = Font(name='Arial', bold=True, size=11, color='FFFFFF')
  169. c.fill = PatternFill('solid', fgColor='2F5496')
  170. c.alignment = Alignment(horizontal='left', vertical='center', wrap_text=True)
  172. # Description
  173. for line in description:
  174. row += 1
  175. ws.merge_cells(start_row=row, start_column=1, end_row=row, end_column=merge_width)
  176. c = ws.cell(row=row, column=1, value=line)
  177. c.font = Font(name='Arial', size=9, italic=True, color='333333')
  178. c.fill = PatternFill('solid', fgColor='F2F2F2')
  179. c.alignment = Alignment(horizontal='left', vertical='center', wrap_text=True)
  181. row += 2 # empty separator
  183. # Headers
  184. for ci, col in enumerate(dataframe.columns, 1):
  185. c = ws.cell(row=row, column=ci, value=col)
  186. c.font = Font(name='Arial', bold=True, size=10)
  187. c.fill = PatternFill('solid', fgColor='D6E4F0')
  188. c.alignment = Alignment(horizontal='center', vertical='center')
  189. c.border = thin
  191. # Data
  192. for _, data_row in dataframe.iterrows():
  193. row += 1
  194. for ci, val in enumerate(data_row, 1):
  195. c = ws.cell(row=row, column=ci)
  196. if isinstance(val, (np.bool_, bool)):
  197. c.value = str(val)
  198. elif isinstance(val, float) and not np.isnan(val):
  199. c.value = val
  200. c.number_format = '0.0000'
  201. else:
  202. c.value = val
  203. c.font = Font(name='Arial', size=10)
  204. c.alignment = Alignment(horizontal='center')
  205. c.border = thin
  207. # Auto-width
  208. for ci in range(1, n_cols + 1):
  209. max_len = max(len(str(dataframe.columns[ci-1])),
  210. dataframe.iloc[:, ci-1].astype(str).str.len().max() if len(dataframe) > 0 else 0)
  211. ws.column_dimensions[get_column_letter(ci)].width = min(max_len + 3, 35)
  213. wb.save(filepath)
  216. # =============================================================
  217. # HELPER: Bootstrap 95% confidence intervals
  218. # =============================================================
  219. def bootstrap_ci(values, n_boot=N_BOOTSTRAP, alpha=CI_ALPHA, seed=RANDOM_SEED):
  220. """
  221. Compute bootstrap 95% CI for the mean.
  223. Same-size resampling with replacement, 1000 iterations,
  224. consistent with manuscript methodology (Efron & Tibshirani, 1993).
  226. Returns (mean, ci_lower, ci_upper)
  227. """
  228. rng = np.random.default_rng(seed)
  229. values = np.asarray(values)
  230. n = len(values)
  231. boot_means = np.array([
  232. rng.choice(values, size=n, replace=True).mean()
  233. for _ in range(n_boot)
  234. ])
  235. ci_lo = np.percentile(boot_means, 100 * alpha / 2)
  236. ci_hi = np.percentile(boot_means, 100 * (1 - alpha / 2))
  237. return round(values.mean(), 4), round(ci_lo, 4), round(ci_hi, 4)
  240. def fmt_ci(mean, lo, hi):
  241. """Format as 'mean [lo, hi]' for tables."""
  242. return f"{mean:.3f} [{lo:.3f}, {hi:.3f}]"
  245. # =============================================================
  246. # 1. LOAD DATA
  247. # =============================================================
  248. def load_data(seed_dir=SEED_DIR):
  249. files = sorted(glob.glob(os.path.join(seed_dir, "bianca_metrics_seed_*.xlsx")))
  250. if not files:
  251. files = sorted(glob.glob("bianca_metrics_seed_*.xlsx"))
  252. if not files:
  253. raise FileNotFoundError("No bianca_metrics_seed_*.xlsx files found.")
  254. df = pd.concat([pd.read_excel(f) for f in files], ignore_index=True)
  255. df['threshold'] = df['threshold'].astype(str)
  256. df['dataset'] = df['subject'].apply(
  257. lambda x: 'BeLOVE' if x.startswith('belove') else 'Challenge')
  258. print(f"Loaded {len(files)} seeds: {len(df)} rows, "
  259. f"{df['subject'].nunique()} subjects")
  260. return df
  263. # =============================================================
  264. # 2. VOXEL vs CLUSTER DISSOCIATION (Main Text Figure + Table)
  265. # =============================================================
  266. def voxel_vs_cluster(df):
  267. """
  268. Core analysis addressing R1 Comment 2:
  269. Cluster F1 >> Dice demonstrates that BIANCA detects most lesions
  270. but delineates boundaries imprecisely. High cluster precision
  271. confirms that detected regions are genuine WMH, not distributed FP.
  273. Reports both voxel-level (Dice, sensitivity, precision) and
  274. cluster-level (F1, precision, recall) with bootstrap 95% CIs.
  275. """
  276. filled = df[(df['train_condition'] == 'filled') & (df['test_condition'] == 'filled')]
  277. per_sub = filled.groupby(['subject', 'threshold', 'dataset']).agg(
  278. {m: 'mean' for m in ALL_METRICS}).reset_index()
  280. # Summary table with bootstrap CIs
  281. rows = []
  282. print("\n" + "=" * 70)
  283. print("VOXEL vs CLUSTER PERFORMANCE (train=filled, test=filled)")
  284. print("Bootstrap 95% CI (1000 iterations)")
  285. print("=" * 70)
  287. for th in ['85', '90', 'locate']:
  288. sub = per_sub[per_sub['threshold'] == th]
  289. row = {'threshold': TH_PLAIN[th]}
  291. # Voxel-level metrics with CIs
  292. for m, label in [('dice_score', 'Dice'),
  293. ('sensitivity', 'Vox Sens'),
  294. ('precision', 'Vox Prec')]:
  295. mean, lo, hi = bootstrap_ci(sub[m].values)
  296. row[f'{m}_mean'] = mean
  297. row[f'{m}_sd'] = round(sub[m].std(), 4)
  298. row[f'{m}_ci_lower'] = lo
  299. row[f'{m}_ci_upper'] = hi
  301. # Cluster-level metrics with CIs
  302. for m, label in [('lesion_f1', 'Clust F1'),
  303. ('lesion_precision', 'Clust P'),
  304. ('lesion_recall', 'Clust R')]:
  305. mean, lo, hi = bootstrap_ci(sub[m].values)
  306. row[f'{m}_mean'] = mean
  307. row[f'{m}_sd'] = round(sub[m].std(), 4)
  308. row[f'{m}_ci_lower'] = lo
  309. row[f'{m}_ci_upper'] = hi
  311. row['f1_minus_dice'] = round(
  312. row['lesion_f1_mean'] - row['dice_score_mean'], 4)
  313. rows.append(row)
  315. print(f"\n {TH_PLOT[th]}:")
  316. print(f" Voxel: Dice={fmt_ci(row['dice_score_mean'], row['dice_score_ci_lower'], row['dice_score_ci_upper'])} "
  317. f"Sens={fmt_ci(row['sensitivity_mean'], row['sensitivity_ci_lower'], row['sensitivity_ci_upper'])} "
  318. f"Prec={fmt_ci(row['precision_mean'], row['precision_ci_lower'], row['precision_ci_upper'])}")
  319. print(f" Cluster: F1={fmt_ci(row['lesion_f1_mean'], row['lesion_f1_ci_lower'], row['lesion_f1_ci_upper'])} "
  320. f"P={fmt_ci(row['lesion_precision_mean'], row['lesion_precision_ci_lower'], row['lesion_precision_ci_upper'])} "
  321. f"R={fmt_ci(row['lesion_recall_mean'], row['lesion_recall_ci_lower'], row['lesion_recall_ci_upper'])}")
  322. print(f" Gap (cF1 - Dice) = {row['f1_minus_dice']:+.3f}")
  324. # Scatter plot: Dice vs Cluster F1
  325. colors = {'85': '#1f77b4', '90': '#ff7f0e', 'locate': '#2ca02c'}
  326. fig, ax = plt.subplots(figsize=(8, 7))
  327. for th in ['85', '90', 'locate']:
  328. sub = per_sub[per_sub['threshold'] == th]
  329. ax.scatter(sub['dice_score'], sub['lesion_f1'], alpha=0.5, s=30,
  330. color=colors[th], label=TH_PLOT[th])
  331. ax.plot([0, 1], [0, 1], 'k--', alpha=0.3, lw=1)
  332. ax.set_xlabel('Voxel-level Dice', fontsize=12)
  333. ax.set_ylabel('Cluster-level F1', fontsize=12)
  334. ax.legend(fontsize=10)
  335. ax.grid(True, alpha=0.2)
  336. ax.set_xlim(0, 1); ax.set_ylim(0, 1)
  337. fig.tight_layout()
  338. fig.savefig(os.path.join(PLOT_DIR, "dice_vs_cluster_f1_scatter.png"), dpi=300)
  339. plt.close()
  340. print(" Saved: dice_vs_cluster_f1_scatter.png")
  342. return pd.DataFrame(rows)
  345. # =============================================================
  346. # 3. PRECISION vs RECALL: Voxel + Cluster (Main Text Figure)
  347. # =============================================================
  348. def precision_recall_bars(df):
  349. """
  350. Addresses R5 #17: Precision values contextualized.
  351. Shows voxel-level (Dice, Sens, Prec) vs cluster-level (F1, P, R)
  352. side by side, demonstrating that moderate voxel precision coexists
  353. with high cluster precision.
  354. """
  355. filled = df[(df['train_condition'] == 'filled') & (df['test_condition'] == 'filled')]
  357. thresholds = ['85', '90', 'locate']
  358. fig, axes = plt.subplots(1, 2, figsize=(14, 5.5))
  360. # Panel A: Voxel-level
  361. vox_metrics = [('dice_score', 'Dice'), ('sensitivity', 'Sensitivity'), ('precision', 'Precision')]
  362. x = np.arange(len(thresholds))
  363. w = 0.25
  364. vox_colors = ['#5B9BD5', '#70AD47', '#ED7D31']
  365. for i, (m, label) in enumerate(vox_metrics):
  366. means = [filled[filled['threshold'] == t][m].mean() for t in thresholds]
  367. sds = [filled[filled['threshold'] == t][m].std() for t in thresholds]
  368. bars = axes[0].bar(x + i * w - w, means, w, yerr=sds, label=label,
  369. color=vox_colors[i], capsize=3, alpha=0.85)
  370. for bar in bars:
  371. axes[0].text(bar.get_x() + bar.get_width()/2., bar.get_height() + 0.02,
  372. f'{bar.get_height():.2f}', ha='center', va='bottom', fontsize=8)
  374. axes[0].set_xticks(x)
  375. axes[0].set_xticklabels([TH_PLOT[t] for t in thresholds], fontsize=11)
  376. axes[0].set_ylabel('Score', fontsize=11)
  377. axes[0].set_title('Voxel-level', fontsize=12, fontweight='bold')
  378. axes[0].legend(fontsize=9)
  379. axes[0].set_ylim(0, 1.15)
  380. axes[0].grid(True, alpha=0.2, axis='y')
  382. # Panel B: Cluster-level
  383. clu_metrics = [('lesion_f1', 'F1'), ('lesion_precision', 'Precision'), ('lesion_recall', 'Recall')]
  384. clu_colors = ['#5B9BD5', '#70AD47', '#ED7D31']
  385. for i, (m, label) in enumerate(clu_metrics):
  386. means = [filled[filled['threshold'] == t][m].mean() for t in thresholds]
  387. sds = [filled[filled['threshold'] == t][m].std() for t in thresholds]
  388. bars = axes[1].bar(x + i * w - w, means, w, yerr=sds, label=label,
  389. color=clu_colors[i], capsize=3, alpha=0.85)
  390. for bar in bars:
  391. axes[1].text(bar.get_x() + bar.get_width()/2., bar.get_height() + 0.02,
  392. f'{bar.get_height():.2f}', ha='center', va='bottom', fontsize=8)
  394. axes[1].set_xticks(x)
  395. axes[1].set_xticklabels([TH_PLOT[t] for t in thresholds], fontsize=11)
  396. axes[1].set_title('Cluster-level', fontsize=12, fontweight='bold')
  397. axes[1].legend(fontsize=9)
  398. axes[1].set_ylim(0, 1.15)
  399. axes[1].grid(True, alpha=0.2, axis='y')
  401. fig.tight_layout()
  402. fig.savefig(os.path.join(PLOT_DIR, "voxel_vs_cluster_bars.png"), dpi=300)
  403. plt.close()
  404. print(" Saved: voxel_vs_cluster_bars.png")
  407. # =============================================================
  408. # 4. CONDITION COMPARISON: Kruskal-Wallis + Wilcoxon + Cliff's Delta
  409. # (Main Text Results + Supplemental Tables)
  410. # =============================================================
  411. def condition_comparison(df):
  412. """
  413. Core analysis: Does training preprocessing affect cluster-level detection?
  415. 1. Kruskal-Wallis H-test (omnibus, 3 training conditions)
  416. 2. Post-hoc Wilcoxon signed-rank (Bonferroni alpha = 0.05/3)
  417. 3. Cliff's Delta (|delta| >= 0.28 for meaningful difference)
  419. Test condition fixed to 'filled' (inpainted).
  420. Per-subject means averaged across seeds/folds for independence.
  421. """
  422. COMPARISONS = [
  423. ('non_removed', 'removed', 'Train Non Removed vs Train Removed'),
  424. ('non_removed', 'filled', 'Train Non Removed vs Train Inpainted'),
  425. ('removed', 'filled', 'Train Removed vs Train Inpainted'),
  426. ]
  427. BONF_ALPHA = 0.05 / len(COMPARISONS)
  428. train_conds = ['non_removed', 'removed', 'filled']
  430. omnibus_rows = []
  431. posthoc_rows = []
  433. print("\n" + "=" * 70)
  434. print("TRAIN CONDITION COMPARISON (cluster-level, test=filled)")
  435. print(f"Omnibus: Kruskal-Wallis | Post-hoc: Wilcoxon (Bonferroni alpha={BONF_ALPHA:.4f})")
  436. print("=" * 70)
  438. for th in ['85', '90', 'locate']:
  439. # Per-subject averages per condition
  440. cond_data = {}
  441. for tc in train_conds:
  442. cond_data[tc] = df[
  443. (df['train_condition'] == tc) &
  444. (df['test_condition'] == 'filled') &
  445. (df['threshold'] == th)
  446. ].groupby("subject")[ALL_METRICS].mean()
  448. print(f"\n --- {TH_PLOT[th]} ---")
  450. for metric in ALL_METRICS:
  451. groups = [cond_data[tc][metric].values for tc in train_conds]
  452. try:
  453. h_stat, kw_p = kruskal(*groups)
  454. except ValueError:
  455. h_stat, kw_p = 0.0, 1.0
  457. omnibus_rows.append({
  458. 'threshold': TH_PLAIN[th],
  459. 'metric': metric,
  460. 'groups': 'Non Removed, Removed, Inpainted',
  461. 'H_statistic': round(h_stat, 4),
  462. 'p_kruskal_wallis': round(kw_p, 4),
  463. 'significant_alpha_0.05': kw_p < 0.05,
  464. 'n_per_group': len(groups[0]),
  465. })
  467. if metric == 'lesion_f1':
  468. sig = "SIGNIFICANT" if kw_p < 0.05 else "n.s."
  469. print(f" Kruskal-Wallis (F1): H={h_stat:.4f}, p={kw_p:.4f} ({sig})")
  471. # Post-hoc pairwise
  472. for cond_a, cond_b, label in COMPARISONS:
  473. merged = cond_data[cond_a].merge(
  474. cond_data[cond_b], on='subject', suffixes=('_a', '_b'))
  475. vals_a = merged[f'{metric}_a'].values
  476. vals_b = merged[f'{metric}_b'].values
  478. try:
  479. _, p_w = wilcoxon(vals_a, vals_b)
  480. except ValueError:
  481. p_w = 1.0
  483. d_val, d_size = (np.nan, 'N/A')
  484. if HAS_CLIFFS:
  485. d_val, d_size = cliffs_delta(vals_b, vals_a)
  487. # Bootstrap CIs for each condition mean
  488. m_a, lo_a, hi_a = bootstrap_ci(vals_a)
  489. m_b, lo_b, hi_b = bootstrap_ci(vals_b)
  490. diff_vals = vals_b - vals_a
  491. m_d, lo_d, hi_d = bootstrap_ci(diff_vals)
  493. posthoc_rows.append({
  494. 'threshold': TH_PLAIN[th],
  495. 'comparison': label,
  496. 'metric': metric,
  497. 'mean_a': m_a,
  498. 'ci_lower_a': lo_a,
  499. 'ci_upper_a': hi_a,
  500. 'mean_b': m_b,
  501. 'ci_lower_b': lo_b,
  502. 'ci_upper_b': hi_b,
  503. 'mean_diff': m_d,
  504. 'diff_ci_lower': lo_d,
  505. 'diff_ci_upper': hi_d,
  506. 'kruskal_wallis_p': round(kw_p, 4),
  507. 'omnibus_significant': kw_p < 0.05,
  508. 'p_wilcoxon': round(p_w, 4),
  509. 'bonferroni_alpha': round(BONF_ALPHA, 4),
  510. 'significant_bonferroni': (kw_p < 0.05) and (p_w < BONF_ALPHA),
  511. 'cliffs_delta': round(d_val, 4) if not np.isnan(d_val) else np.nan,
  512. 'effect_size_category': d_size,
  513. 'n_subjects': len(merged),
  514. })
  516. # Print post-hoc F1 summary
  517. for cond_a, cond_b, label in COMPARISONS:
  518. merged = cond_data[cond_a].merge(
  519. cond_data[cond_b], on='subject', suffixes=('_a', '_b'))
  520. f1_diff = merged['lesion_f1_b'].mean() - merged['lesion_f1_a'].mean()
  521. try:
  522. _, pw = wilcoxon(merged['lesion_f1_a'].values, merged['lesion_f1_b'].values)
  523. except ValueError:
  524. pw = 1.0
  525. d_str = ""
  526. if HAS_CLIFFS:
  527. d, s = cliffs_delta(merged['lesion_f1_b'].values, merged['lesion_f1_a'].values)
  528. d_str = f" delta={d:.4f} ({s})"
  529. print(f" {label}: dF1={f1_diff:+.4f} p={pw:.4f}{d_str}")
  531. # Boxplot (Main Text Figure)
  532. filled_test = df[df['test_condition'] == 'filled']
  533. per_sub = filled_test.groupby(
  534. ['subject', 'train_condition', 'threshold']).agg({'lesion_f1': 'mean'}).reset_index()
  535. per_sub['train_label'] = per_sub['train_condition'].map(COND_LABELS)
  537. fig, axes = plt.subplots(1, 3, figsize=(15, 5), sharey=True)
  538. order = list(COND_LABELS.values())
  539. for i, th in enumerate(['85', '90', 'locate']):
  540. sub = per_sub[per_sub['threshold'] == th]
  541. sns.boxplot(data=sub, x='train_label', y='lesion_f1', order=order,
  542. hue='train_label', hue_order=order, legend=False,
  543. ax=axes[i], palette='Set2', width=0.6)
  544. axes[i].set_title(TH_PLOT[th], fontsize=12, fontweight='bold')
  545. axes[i].set_xlabel('')
  546. axes[i].set_ylim(0, 1.05)
  547. axes[i].grid(True, alpha=0.2, axis='y')
  548. axes[i].tick_params(axis='x', rotation=15)
  549. axes[0].set_ylabel('Cluster-level F1', fontsize=11)
  550. fig.tight_layout()
  551. fig.savefig(os.path.join(PLOT_DIR, "cluster_f1_condition_boxplot.png"), dpi=300)
  552. plt.close()
  553. print(" Saved: cluster_f1_condition_boxplot.png")
  555. return pd.DataFrame(omnibus_rows), pd.DataFrame(posthoc_rows)
  558. # =============================================================
  559. # 5. PER-DATASET: BeLOVE vs Challenge (Supplemental)
  560. # =============================================================
  561. def per_dataset(df):
  562. """
  563. Supplemental analysis: in-distribution (BeLOVE) vs
  564. out-of-distribution (Challenge) performance.
  565. Mann-Whitney U for unpaired comparison.
  566. """
  567. filled = df[(df['train_condition'] == 'filled') & (df['test_condition'] == 'filled')]
  569. # Descriptive
  570. desc_rows = []
  571. for ds in ['BeLOVE', 'Challenge']:
  572. dsdf = filled[filled['dataset'] == ds]
  573. n = dsdf['subject'].nunique()
  574. for th in ['85', '90', 'locate']:
  575. sub = dsdf[dsdf['threshold'] == th]
  576. per_sub = sub.groupby('subject')[ALL_METRICS].mean()
  577. row = {'dataset': ds, 'n': n, 'threshold': TH_PLAIN[th]}
  578. for m in ALL_METRICS:
  579. mean, lo, hi = bootstrap_ci(per_sub[m].values)
  580. row[f'{m}_mean'] = mean
  581. row[f'{m}_sd'] = round(per_sub[m].std(), 4)
  582. row[f'{m}_ci_lower'] = lo
  583. row[f'{m}_ci_upper'] = hi
  584. row['n_pred_mean'] = round(sub['n_pred_clusters'].mean(), 1)
  585. row['n_gt_mean'] = round(sub['n_gt_clusters'].mean(), 1)
  586. desc_rows.append(row)
  588. # Statistical comparison
  589. stat_rows = []
  590. print("\n" + "=" * 70)
  591. print("PER-DATASET (Mann-Whitney U: BeLOVE vs Challenge)")
  592. print("=" * 70)
  593. for th in ['85', '90', 'locate']:
  594. bel = filled[(filled['dataset'] == 'BeLOVE') & (filled['threshold'] == th)].groupby("subject")[ALL_METRICS].mean()
  595. cha = filled[(filled['dataset'] == 'Challenge') & (filled['threshold'] == th)].groupby("subject")[ALL_METRICS].mean()
  596. for m in ALL_METRICS:
  597. _, p = mannwhitneyu(bel[m].values, cha[m].values, alternative='two-sided')
  598. d_val, d_size = (np.nan, 'N/A')
  599. if HAS_CLIFFS:
  600. d_val, d_size = cliffs_delta(bel[m].values, cha[m].values)
  601. m_bel, lo_bel, hi_bel = bootstrap_ci(bel[m].values)
  602. m_cha, lo_cha, hi_cha = bootstrap_ci(cha[m].values)
  603. stat_rows.append({
  604. 'threshold': TH_PLAIN[th], 'metric': m,
  605. 'mean_belove': m_bel,
  606. 'ci_belove': f'[{lo_bel:.4f}, {hi_bel:.4f}]',
  607. 'mean_challenge': m_cha,
  608. 'ci_challenge': f'[{lo_cha:.4f}, {hi_cha:.4f}]',
  609. 'diff': round(m_bel - m_cha, 4),
  610. 'p_mann_whitney': round(p, 4),
  611. 'cliffs_delta': round(d_val, 4) if not np.isnan(d_val) else np.nan,
  612. 'effect_size': d_size,
  613. })
  614. if m == 'lesion_f1':
  615. print(f" {TH_PLAIN[th]} F1: BeLOVE={bel[m].mean():.4f} vs "
  616. f"Challenge={cha[m].mean():.4f} p={p:.4f}")
  618. return pd.DataFrame(desc_rows), pd.DataFrame(stat_rows)
  621. # =============================================================
  622. # 6. GT CLUSTER DISTRIBUTION (Supplemental)
  623. # =============================================================
  624. def gt_distribution(df):
  625. """Descriptive statistics of GT cluster counts (26-connectivity)."""
  626. filled = df[(df['train_condition'] == 'filled') &
  627. (df['test_condition'] == 'filled') &
  628. (df['threshold'] == '85')]
  629. gt = filled.groupby('subject')['n_gt_clusters'].mean()
  631. rows = []
  632. for label, data in [('All', gt),
  633. ('BeLOVE', filled[filled['dataset'] == 'BeLOVE'].groupby('subject')['n_gt_clusters'].mean()),
  634. ('Challenge', filled[filled['dataset'] == 'Challenge'].groupby('subject')['n_gt_clusters'].mean())]:
  635. rows.append({
  636. 'group': label, 'n': len(data),
  637. 'mean': round(data.mean(), 1), 'sd': round(data.std(), 1),
  638. 'median': round(data.median(), 1),
  639. 'min': int(data.min()), 'max': int(data.max()),
  640. 'q25': int(data.quantile(0.25)), 'q75': int(data.quantile(0.75)),
  641. })
  643. print("\n" + "=" * 70)
  644. print("GT CLUSTER DISTRIBUTION (26-connectivity)")
  645. print("=" * 70)
  646. for r in rows:
  647. print(f" {r['group']:>10s}: mean={r['mean']}, median={r['median']}, "
  648. f"range={r['min']}-{r['max']}")
  650. return pd.DataFrame(rows)
  653. # =============================================================
  654. # 7. SUMMARY TABLE (Supplemental)
  655. # =============================================================
  656. def summary_table(df):
  657. """Mean, SD, median per train x test x threshold (all 27 combos)."""
  658. metric_cols = ['dice_score', 'sensitivity', 'precision',
  659. 'lesion_f1', 'lesion_precision', 'lesion_recall',
  660. 'n_pred_clusters', 'n_gt_clusters']
  661. group_cols = ['train_condition', 'test_condition', 'threshold']
  663. summary = df.groupby(group_cols)[metric_cols].agg(['mean', 'std', 'median']).round(4)
  664. # Flatten MultiIndex columns
  665. flat = summary.reset_index()
  666. flat.columns = ['_'.join(str(c) for c in col).rstrip('_') for col in flat.columns]
  667. return flat
  670. # =============================================================
  671. # MAIN
  672. # =============================================================
  673. def main():
  674. print("=" * 70)
  675. print("CLUSTER-LEVEL ANALYSIS OF BIANCA WMH SEGMENTATION")
  676. print("=" * 70)
  678. df = load_data()
  680. # --- MAIN TEXT ---
  681. dissoc_df = voxel_vs_cluster(df)
  682. precision_recall_bars(df)
  683. omnibus_df, posthoc_df = condition_comparison(df)
  685. # --- SUPPLEMENTAL ---
  686. dataset_desc_df, dataset_stat_df = per_dataset(df)
  687. gt_df = gt_distribution(df)
  688. summary_df = summary_table(df)
  690. # =============================================================
  691. # SAVE
  692. # =============================================================
  693. print(f"\nSaving to {OUTPUT_DIR}/...")
  695. # Main Text tables
  696. save_excel(
  697. os.path.join(OUTPUT_DIR, "voxel_vs_cluster_dissociation.xlsx"),
  698. dissoc_df,
  699. "Voxel-Level vs Cluster-Level Performance Dissociation",
  700. ["Addresses R1 Comment 2: Cluster F1 substantially exceeds Dice, indicating that BIANCA detects most",
  701. "WMH clusters but delineates boundaries imprecisely. High cluster precision confirms detected regions",
  702. "are genuine WMH, not distributed false positives.",
  703. "Condition: train=filled, test=filled. Per-subject means across 10 seeds x 5-fold CV (n=89).",
  704. "Cluster parameters: 26-connectivity, any-overlap (MICCAI 2017), prediction-only MCS filtering."])
  706. save_excel(
  707. os.path.join(OUTPUT_DIR, "kruskal_wallis_train_condition.xlsx"),
  708. omnibus_df,
  709. "Kruskal-Wallis H-Test: Effect of Training Preprocessing on Cluster-Level Metrics",
  710. ["Question: Does training with Non Removed vs Removed vs Inpainted images affect lesion detection?",
  711. "Design: 3 groups (train conditions), test condition fixed to 'filled' (inpainted).",
  712. "Test: Kruskal-Wallis H-test (non-parametric omnibus, k=3 independent groups).",
  713. "Null hypothesis: All three training conditions produce equal cluster-level metrics.",
  714. "Significance: alpha = 0.05.",
  715. "Data: Per-subject means averaged across 10 seeds x 5-fold stratified CV (n=89 per group).",
  716. "If significant: see posthoc_wilcoxon_bonferroni.xlsx for pairwise comparisons."])
  718. save_excel(
  719. os.path.join(OUTPUT_DIR, "posthoc_wilcoxon_bonferroni.xlsx"),
  720. posthoc_df,
  721. "Post-hoc Wilcoxon Signed-Rank Tests with Bonferroni Correction",
  722. ["Pairwise comparisons of training conditions (paired by subject).",
  723. "Test: Wilcoxon signed-rank (non-parametric, paired).",
  724. "Correction: Bonferroni (3 comparisons, corrected alpha = 0.05/3 = 0.0167).",
  725. "Effect size: Cliff's Delta (|d| >= 0.28 = meaningful, per manuscript conventions).",
  726. "Column 'omnibus_significant': Kruskal-Wallis was significant for this metric.",
  727. "Column 'significant_bonferroni': True only if BOTH omnibus AND post-hoc significant.",
  728. "Note: Post-hoc tests interpretable only when omnibus is significant."])
  730. # Supplemental tables
  731. save_excel(
  732. os.path.join(OUTPUT_DIR, "per_dataset_descriptive.xlsx"),
  733. dataset_desc_df,
  734. "Cluster-Level Metrics by Dataset: BeLOVE vs WMH Segmentation Challenge",
  735. ["BeLOVE (n=58): in-distribution (included in cross-validated training).",
  736. "Challenge (n=31): out-of-distribution (external validation dataset).",
  737. "Condition: train=filled, test=filled."])
  739. save_excel(
  740. os.path.join(OUTPUT_DIR, "per_dataset_mann_whitney.xlsx"),
  741. dataset_stat_df,
  742. "Mann-Whitney U Test: BeLOVE vs Challenge Comparison",
  743. ["Test: Mann-Whitney U (non-parametric, unpaired, 2 groups).",
  744. "Significance: alpha = 0.05.",
  745. "Effect size: Cliff's Delta.",
  746. "Per-subject means (BeLOVE n=58, Challenge n=31)."])
  748. save_excel(
  749. os.path.join(OUTPUT_DIR, "gt_cluster_distribution.xlsx"),
  750. gt_df,
  751. "Ground Truth Cluster Count Distribution (26-Connectivity)",
  752. ["Descriptive statistics of the number of GT clusters per subject.",
  753. "26-connectivity used (consistent with all cluster-level analyses).",
  754. "No size filtering applied to ground truth (expert-delineated lesions)."])
  756. save_excel(
  757. os.path.join(OUTPUT_DIR, "summary_all_conditions.xlsx"),
  758. summary_df,
  759. "Full Summary: Mean, SD, Median per Train x Test x Threshold",
  760. ["All 3 train x 3 test x 3 threshold = 27 combinations.",
  761. "Statistics: mean, standard deviation, median for each metric.",
  762. "Averaged across 10 seeds x 5-fold stratified CV x 89 subjects."])
  764. # Raw data
  765. df.to_excel(os.path.join(OUTPUT_DIR, "all_seeds_raw.xlsx"), index=False)
  767. print(f"\nDone. Results: {OUTPUT_DIR}/ Plots: {PLOT_DIR}/")
  770. if __name__ == "__main__":
  771. main()

4_Analyze_cluster_metrics.py at commit 1f27300, no license · at the source

Overview

Authors: Uchralt Temuulen1, Ralf Mekle1, Ivana Galinovic1, Joachim E Weber1,2,3,4, Ulf Landmesser2,3,5, Sebastian Kelle3, Matthias Endres1,3,4,6,7, Christian Stehning8, Kersten Villringer1
ORCID iDs: Uchralt Temuulen, Ralf Mekle, Ivana Galinovic, Joachim E Weber, Sebastian Kelle, Matthias Endres, Kersten Villringer
  1. Center for Stroke Research Berlin, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
  2. Berlin Institute of Health (BIH) at Charité-Universitätsmedizin Berlin, Berlin, Germany
  3. German Centre for Cardiovascular Research (DZHK), Partner site Berlin, Berlin, Germany
  4. Department of Neurology, Charité-Universitätsmedizin Berlin, Berlin, Germany
  5. Department of Cardiology, Charité-Universitätsmedizin Berlin, Berlin, Germany
  6. German Center for Neurodegenerative Diseases (DZNE), Partner site Berlin, Berlin, Germany
  7. German Center for Mental Health (DZPG), Partner site Berlin, Berlin, Germany
  8. Philips Clinical Science, Hamburg, Germany
Institutions: Humboldt-Universität zu Berlin (Germany); Center for Stroke Research Berlin (Germany); Charité - Universitätsmedizin Berlin (Germany); Freie Universität Berlin (Germany); German Centre for Cardiovascular Research (Germany); Berlin Institute of Health at Charité - Universitätsmedizin Berlin (Germany); Philips (Germany) (Germany)
Journal: NeuroImage. Clinical, volume 50, article 104001
Dates: received 15 January 2026; accepted 1 May 2026; published online 11 May 2026; in print 2026
Type: Research article · Language: English
License: CC BY
Identifiers: DOI 10.1016/j.nicl.2026.104001 · PMID 42119523 · PMCID PMC13195355 · OpenAlex W7161065280
Open access: gold, a free copy (OpenAlex)
Status: code verified
Categories: structural MRI / diffusion (modality), human (organism), stroke (population), methods / tools (subfield)
Methods: Statistics, Machine learning, Connectivity, Single-unit activity, calcium imaging
Keywords: White matterhyperintensities, BIANCA, LOCATE, Automated segmentation, Stroke, Cerebral small vessel disease, FLAIR
MeSH: Algorithms*, Brain*, Image Interpretation, Computer-Assisted*, Image Processing, Computer-Assisted*, Magnetic Resonance Imaging*, Stroke*, White Matter*, Female, Humans, Male (* major topic)
Topic: Acute Ischemic Stroke Management (Epidemiology, Medicine), according to OpenAlex
Funding: German Research Foundation; BMBF Bonn; Deutsches Zentrum für Herz-Kreislauf-Forschung eV
Citations: not cited yet (Europe PMC); 60 references in the paper

Abstract

Background: White matter hyperintensity (WMH) segmentation using BIANCA (Brain Intensity AbNormality Classification Algorithm) in stroke populations is complicated by vascular lesions that share T2-hyperintense signal characteristics with WMH. Whether preprocessing decisions in the treatment of lesions affect segmentation accuracy has not yet been systematically evaluated in cerebrovascular cohorts involving multiple scanners.

Methods: We compared fixed probability thresholds and locally adaptive thresholding with LOCATE (LOCally Adaptive Threshold Estimation), and three lesion-handling approaches: lesions present (non removed), replaced with zero intensities (removed), and replaced with normal-appearing white matter intensities (NAWM; inpainted), using the BeLOVE cohort (Berlin Longterm Observation of Vascular Events) and the WMH Segmentation Challenge dataset. Phase I (n = 89) optimized thresholding via stratified 5-fold cross-validation. Phase II assessed preprocessing effects on segmentation accuracy (Phase II-A, n = 89) and volume agreement (Phase II-B, n = 211).

Results: BIANCA with LOCATE adaptive thresholding achieved moderate segmentation overlap (mean Dice 0.567), with lesion-level detection exceeding an F1 of 0.85. Preprocessing effects were statistically detectable but negligible in magnitude, with near-perfect agreement between all conditions. Stroke lesion volume was the highest-ranked predictor of volume differences between conditions; these scaled with lesion size but remained negligible in magnitude across all subgroups.

Conclusions: BIANCA with LOCATE achieved moderate WMH segmentation performance with the best sensitivity-precision trade-off in this multi-scanner cerebrovascular cohort. Preprocessing effects were negligible at the group level. However, large lesions distort the FLAIR intensity distribution on which BIANCA relies for classification, which justifies lesion removal as a recommended preprocessing step.

Reproduced under the paper's license (CC BY), from the paper cited above.

Repository

Its files are read in the Code ↔ Paper reader above, with 28 matches between paragraphs and lines of code.

mendeltem/Bianca_Evaluation

License: none: the authors keep all their rights
State: the link answers, verified on 28 September 2026
Evidence: files inventoried
Commit: 1f27300aa772a82509608f194a7558f4d79b4b70, 20 March 2026
Languages: Python (33), Shell (1)
Size: 40 files, 34 scripts
Software Heritage: not archived
Found in: “Data availability”
Holds: README, environment (requirements.txt)
Not found: license file, CITATION.cff, tests, continuous integration, documentation
Tools: pandas (32 files), NumPy (31 files), SciPy (21 files), Matplotlib (20 files), FSL (7 files), Pillow (6 files), scikit-learn (6 files), seaborn (4 files), NiBabel (3 files), SHAP (2 files)
Availability: 1 check, the latest on 28 September 2026: the link answers
  • 28 September 2026: the link answers
35 files
At the source: github.com/mendeltem/Bianca_Evaluation

The paper's code and data availability statement is in the Data section.

Tracing map

Proposed by the machine: these links were found in the paper and verified at the source, without human review. The map will receive a Zenodo DOI once one of the paper's authors has validated it with their ORCID.

What the map holds:

  • 1 repository of the authors' code, each at its verified commit, with its license and how the link was found in the paper;
  • 34 scripts, each with its path and the digest of its content;
  • 28 matches between paragraphs of the paper and lines of the code (method lexical-v1);
  • neither the text of the paper nor the code itself.

Its JSON (tracing-map.json) is deposited on Zenodo with its DOI once the map is validated.

Data

No dataset and no data link were found in the paper.

Data availability

Analysis scripts for WMH segmentation, preprocessing, statistical testing and figure generation are available at https://github.com/mendeltem/Bianca_Evaluation under the MIT License.

Source imaging data cannot be shared publicly due to participant privacy restrictions under the European General Data Protection Regulation (GDPR). Processed, de-identified derivative data can be obtained from the corresponding author upon reasonable request and completion of appropriate data sharing agreements with the Center for Stroke Research Berlin.

Reproduced under the paper's license (CC BY), from the paper cited above.

Versions

The history of this record: each version stored by the harvester or made by a correction of its authors or of the maintainers of its code, and what changed in its facts. The texts of the paper (its abstract, its availability statements) are not part of it; versions that changed only those are not listed.

Version 1, 28 September 2026: the first record

Recorded: type, language, journal, volume, pages, dates, 9 authors, 7 keywords, 10 MeSH terms, 3 funders, 47 references.

Cite

This paper

Temuulen, U., Mekle, R., Galinovic, I., Weber, J. E., Landmesser, U., Kelle, S., Endres, M., Stehning, C., & Villringer, K. (2026). Effect of vascular lesion preprocessing on Brain Intensity AbNormality Classification Algorithm (BIANCA) white matter hyperintensity segmentation. NeuroImage. Clinical, 50, 104001. https://doi.org/10.1016/j.nicl.2026.104001

BibTeX

@article{temuulen2026effect,
author = {Temuulen, Uchralt and Mekle, Ralf and Galinovic, Ivana and Weber, Joachim E and Landmesser, Ulf and Kelle, Sebastian and Endres, Matthias and Stehning, Christian and Villringer, Kersten},
title = {{Effect of vascular lesion preprocessing on Brain Intensity AbNormality Classification Algorithm (BIANCA) white matter hyperintensity segmentation}},
journal = {NeuroImage. Clinical},
year = {2026},
month = may,
volume = {50},
pages = {104001},
publisher = {Elsevier},
issn = {2213-1582},
doi = {10.1016/j.nicl.2026.104001},
url = {https://doi.org/10.1016/j.nicl.2026.104001},
pmid = {42119523},
pmcid = {PMC13195355}
}

RIS

TY - JOUR
AU - Temuulen, Uchralt
AU - Mekle, Ralf
AU - Galinovic, Ivana
AU - Weber, Joachim E
AU - Landmesser, Ulf
AU - Kelle, Sebastian
AU - Endres, Matthias
AU - Stehning, Christian
AU - Villringer, Kersten
TI - Effect of vascular lesion preprocessing on Brain Intensity AbNormality Classification Algorithm (BIANCA) white matter hyperintensity segmentation
T2 - NeuroImage. Clinical
J2 - Neuroimage Clin
PY - 2026
DA - 2026/05/11
VL - 50
SP - 104001
SN - 2213-1582
PB - Elsevier
DO - 10.1016/j.nicl.2026.104001
UR - https://doi.org/10.1016/j.nicl.2026.104001
LA - en
ER -

CSL-JSON

{
"id": "10.1016/j.nicl.2026.104001",
"type": "article-journal",
"title": "Effect of vascular lesion preprocessing on Brain Intensity AbNormality Classification Algorithm (BIANCA) white matter hyperintensity segmentation",
"container-title": "NeuroImage. Clinical",
"author": [
{
"family": "Temuulen",
"given": "Uchralt"
},
{
"family": "Mekle",
"given": "Ralf"
},
{
"family": "Galinovic",
"given": "Ivana"
},
{
"family": "Weber",
"given": "Joachim E"
},
{
"family": "Landmesser",
"given": "Ulf"
},
{
"family": "Kelle",
"given": "Sebastian"
},
{
"family": "Endres",
"given": "Matthias"
},
{
"family": "Stehning",
"given": "Christian"
},
{
"family": "Villringer",
"given": "Kersten"
}
],
"container-title-short": "Neuroimage Clin",
"volume": "50",
"page": "104001",
"DOI": "10.1016/j.nicl.2026.104001",
"PMID": "42119523",
"PMCID": "PMC13195355",
"ISSN": "2213-1582",
"publisher": "Elsevier",
"URL": "https://doi.org/10.1016/j.nicl.2026.104001",
"language": "en",
"issued": {
"date-parts": [
[
2026,
5,
11
]
]
}
}

The tracing map gets a citation of its own once an author has validated it and it has a DOI.

Similar papers

The papers with a page that share the most with this one: the tools found in their code, their categories, datasets, cited references and authors, the rarest counting most.

[1] doi:10.1186/s12938-026-01555-0 [code]
Incorporating normal periventricular changes for enhanced pathological white matter hyperintensity segmentation: on multiclass deep learning approaches.
Journal: Biomedical engineering online
In common: seaborn, scikit-learn, pandas, 3 other tools, stroke, methods / tools, structural MRI / diffusion, 5 references
[2] doi:10.1212/wnl.0000000000218472 [code]
Lesion-Level Subtypes of White Matter Hyperintensity Evolution Beyond Spatial Location.
Journal: Neurology
In common: SHAP, NiBabel, scikit-learn, 4 other tools, stroke, structural MRI / diffusion, 3 references
[3] doi:10.1111/ene.70678 [code]
Who Falls After a Stroke? Evidence From a Prospective Stroke Cohort.
Journal: European journal of neurology
In common: NiBabel, seaborn, pandas, 3 other tools, stroke, 1 reference, author Matthias Endres
[4] doi:10.1371/journal.pbio.3003856 [code]
Aging and metabolism contribute separately to brain-body health.
Journal: PLoS biology
In common: FSL, NiBabel, seaborn, 5 other tools, structural MRI / diffusion, 3 references
[5] doi:10.1038/s41467-026-71555-0 [code]
A deep representation learning model to predict response to vagus nerve stimulation.
Journal: Nature communications
In common: SHAP, FSL, NiBabel, 6 other tools, structural MRI / diffusion, 1 reference
[6] doi:10.1162/imag.a.1252 [code]
Does the brain's E:I balance really shape long-range temporal correlations? Lessons learned from 3T MRI.
Journal: Imaging neuroscience (Cambridge, Mass.)
In common: FSL, NiBabel, seaborn, 5 other tools, structural MRI / diffusion, 3 references
[7] doi:10.1002/hbm.70602 [code]
Neuroimaging Correlates of Post-Stroke Pain After Ischemic Stroke: Secondary Analysis of the INSPiRE-TMS Trial.
Journal: Human brain mapping
In common: NiBabel, seaborn, pandas, 3 other tools, stroke, author Matthias Endres
[8] doi:10.1186/s12880-026-02481-2 [code]
Deep learning-based neuroanatomical profiling reveals population-specific brain changes in multiple sclerosis: a large-scale Middle Eastern study.
Journal: BMC medical imaging
In common: FSL, Pillow, NiBabel, 6 other tools, structural MRI / diffusion, 1 reference
[9] doi:10.1002/ana.78203 [code]
AI-Driven Mapping of Seizure Spread Patterns.
Journal: Annals of neurology
In common: FSL, Pillow, NiBabel, 6 other tools, structural MRI / diffusion, 1 reference
[10] doi:10.3389/fnins.2026.1870124 [code]
An end-to-end pipeline for automated fetal brain segmentation and biometry from 3D SSFP MRI.
Journal: Frontiers in neuroscience
In common: FSL, NiBabel, seaborn, 5 other tools, structural MRI / diffusion, 2 references

Contribute

The authors of this paper can claim it, correct its record and validate its tracing map, and the maintainers of its code (its owner, or a public member of its organization) correct what it says of their repository; anyone signed in can ask for its removal. Every request goes to OSCR's own machine, which answers it; your account page follows them.

Sign in with ORCID to claim this paper as one of its authors, correct its record or validate its tracing map: when the paper's metadata lists your ORCID iD, you are recognized at once. Maintainers of its code: sign in with GitHub, then claim the repository on your account page.

Request its removal

To ask OSCR to remove this record, the copies of its authors' scripts or its tracing map, use the removal request page: signed in, you say who you are, what to remove and why, then review and confirm the request. Published rules decide every request (how).

Discussion, reproductions, activity

Discussion: questions and error reports about this paper and its code, from signed-in readers and its authors. It opens with sign-in.

Reproductions: reports from readers who ran the authors' code: what they reproduced, with which environment, commit and data. It opens with sign-in.

Activity: what happens around this paper: new versions of its record, its map's validation, discussions and reproductions. It opens with sign-in.